Memory
Large Language Models are stateless and each transaction is unique/independent, they have no information of any past interactions and conversations.
These chat bots appear to have memory when they are provided with the context, i.e. conversation
LangChain provides several kinds of memory to store and accumulate the conversation such as:
- Conversational Buffer Memory
- Conversational Buffer Memory Window
- Conversation Token Buffer Memory
- Conversation Summary Buffer Memory
LangChain also provide a different kind of memory called as Entity Memory
This/These memory/memories of the application can be stored in a conventional database such as Postgres or a key value pair such as MongoDB and can also be stored as Embeddings in a Vector Stores.
Conversational Buffer Memory
In this type of memory all the conversation is sent to the LLM for more contextual understanding but as the conversation goes on and gets longer, the conversation starts getting expensive real quick.
Conversational Buffer Memory Window
This is similar to Conversational Buffer Memory but we can define how many exchange windows of the conversation should be sent to the LLM, although context can be lost as per the window size k decreases, this helps in minimizing cost
Conversation Token Buffer Memory
Similar to the Conversational Buffer Memory and Conversational Buffer Memory Window this works in the same way, but this limits on the amount of tokens used in the per interaction with the LLM which completely disregards the complete exchange and makes sure that the last n tokens as mentioned in the limit are used for the following conversation and interactions it is the best for minimizing cost of the conversation as billings are based on tokens. Each LLM has it’s own way of calculating tokens.
Conversation Summary Buffer Memory
Conversation Summary Buffer Memory is used when you want to have the entire context of the conversation, as this is not economically feasible in most cases, we can summarize the entire conversation and use this summary as context, in this process we are able to retain the context and also reduce on token usage too as we can limit the summary as per tokens too.