Caching

Caching is a default strategy used to reduce the load times and the response times from the applications, for # as every call is a billable unit it makes caching a very highly desired feature for applications where responses are limited or when certain requests are very frequently asked.

For the ChatBuddy Application that I am developing the chatbot in of itself doesn’t generate any revenue so cost is a very sensitive element, hence caching is the need of the hour with some decent TTL (Time To Live) such that when inactive cache doesn’t die out and doesn’t rebuild everyday

There are many cache providers for LangChain which are text match and semantic search for the most closet matches.

Look into Cache from the official documentation

A Caching project is GPTCache [Not Updated Since 2023 November]