Few Shot Prompting
While large models are really smart and demonstrate remarkable zero-shot capabilities, they fall short for complex tasks, few shot prompting can be used for contextual learning and steering the Large Language Model (LLM) to generate our desired responses, such as demonstrations and examples.
Some example prompts would be:
A "whatpu" is a small, furry animal native to Tanzania. An example of a sentence that uses the word whatpu is:
We were traveling in Africa and we saw these very cute whatpus.
To do a "farduddle" means to jump up and down really fast. An example of a sentence that uses the word farduddle is:Here in the next example we teach the LLM the sentiment of the statement and ask it to predict the sentiment
This is awesome! // Positive
This is bad! // Negative
Wow that movie was rad! // Positive
What a horrible show! //We get the answer negative as expected, the newer LLM are becoming more robust to even random formats. Example
Positive This is awesome!
This is bad! Negative
Wow that movie was rad!
Positive
What a horrible show! --Although the labels now are random the LLM was still able to predict the answer.
Limitations of Few Shot Prompting
Standard few-shot prompting works well for many tasks but is not a perfect technique, especially when dealing with more complex reasoning tasks.
Example prompt
The odd numbers in this group add up to an even number: 4, 8, 9, 15, 12, 2, 1.
A: The answer is False.
The odd numbers in this group add up to an even number: 17, 10, 19, 4, 8, 12, 24.
A: The answer is True.
The odd numbers in this group add up to an even number: 16, 11, 14, 4, 8, 13, 24.
A: The answer is True.
The odd numbers in this group add up to an even number: 17, 9, 10, 12, 13, 4, 2.
A: The answer is False.
The odd numbers in this group add up to an even number: 15, 32, 5, 13, 82, 7, 1.
A: Output:
The answer is True.This didn’t seem to work as the few-shot prompting is not reliable for reasoning problems, this can be solved with Chain Of Thought (COT) prompting technique in Prompt Engineering, in this technique we break the problem down in to a series of smaller tasks and demonstrate it to the model. This has been popularised for arithmetic commonsense and symbolic reasoning tasks.
Overall, it seems that providing examples is useful for solving some tasks. When zero-shot prompting and few-shot prompting are not sufficient, it might mean that whatever was learned by the model isn’t enough to do well at the task. From here it is recommended to start thinking about fine-tuning your models or experimenting with more advanced prompting techniques.
Source : promptingguide.ai
References