Prompt Chaining
To improve the reliability and performance of the Large Language Model (LLM) application, one of the most common ways is to break the tasks in multiple sub tasks, which was one of the ideas for Chain Of Thought (COT) prompting. In prompt chaining we identity the sub tasks and prompt it to the LLM and then use the response as input in another prompt, the idea is to chain the prompts, hence the name.
This technique is ideal when you want to accomplish a complex task and the LLM model struggles with a single detailed prompt. this method is used in transformations or additional processes on the generated responses before reaching the final output.
Apart from the performance aspect we are also able to see all the steps in achieving the end goal, hence increased transparency and control in each step, which helps us in debugging the application, helping in the positive feedback loop.
This is useful when you are building LLM based applications which focus on conversation, helping us in improving the personalized user experience of our applications.
Use Cases for Prompt Chaining - Document QA
One of the most common use cases for prompt chaining is using to answer questions on a large text document. In this we would have two steps:
- Summarize / Extract features/ data from the large text
- Use the summarized data to answer the original question
Prompt 1
You are a helpful assistant. Your task is to help answer a question given in a document. The first step is to extract quotes relevant to the question from the document, delimited by ####. Please output the list of quotes using <quotes></quotes>. Respond with "No relevant quotes found!" if no relevant quotes were found.
####
{{document}}
####This prompt passes the document information to the LLM to summarize the document information from Prompt Engineering website
Output:
<quotes>
- Chain-of-thought (CoT) prompting[27]
- Generated knowledge prompting[37]
- Least-to-most prompting[38]
- Self-consistency decoding[39]
- Complexity-based prompting[41]
- Self-refine[42]
- Tree-of-thought prompting[43]
- Maieutic prompting[45]
- Directional-stimulus prompting[46]
- Textual inversion and embeddings[59]
- Using gradient descent to search for prompts[61][62][63][64]
- Prompt injection[65][66][67]
</quotes>Prompt 2:
Given a set of relevant quotes (delimited by <quotes></quotes>) extracted from a document and the original document (delimited by ####), please compose an answer to the question. Ensure that the answer is accurate, has a friendly tone, and sounds helpful.
####
{{document}}
####
<quotes>
- Chain-of-thought (CoT) prompting[27]
- Generated knowledge prompting[37]
- Least-to-most prompting[38]
- Self-consistency decoding[39]
- Complexity-based prompting[41]
- Self-refine[42]
- Tree-of-thought prompting[43]
- Maieutic prompting[45]
- Directional-stimulus prompting[46]
- Textual inversion and embeddings[59]
- Using gradient descent to search for prompts[61][62][63][64]
- Prompt injection[65][66][67]
</quotes>Output Prompt 2:
The prompting techniques mentioned in the document include:
1. Chain-of-thought (CoT) prompting[27]
2. Generated knowledge prompting[37]
3. Least-to-most prompting[38]
4. Self-consistency decoding[39]
5. Complexity-based prompting[41]
6. Self-refine[42]
7. Tree-of-thought prompting[43]
8. Maieutic prompting[45]
9. Directional-stimulus prompting[46]
10. Textual inversion and embeddings[59]
11. Using gradient descent to search for prompts[61][62][63][64]
12. Prompt injection[65][66][67]
Each of these techniques employs unique strategies to enhance or specify the interactions with large language models to produce the desired outcomes.As you can see simplifying and creating a prompt chains is a useful prompting approach where the responses need to undergo several operations or transformations.
References