What is Retrieval -Augmented Generation?

• LLMs produce “hallucinations” when handling queries beyond their training data or requiring current information
• Retrieval-Augmented Generation (RAG) enhances LLMs by retrieving relevant document chunks from external knowledge base through semantic similarity calculation.

Source: https://assets-global.website-files.com/618399cd49d125734c8dec95/64f96acdb54fef43d33ef1ad_rag.png

• RAG for LLMs works because of the ability of LLMs to perform in-context learning.

Why do we need RAG?

• It can be used to add “context” information from a data store that is then included with the user-entered prompt to build a richer prompt.
• RAG effectively reduces the problem of generating factually incorrect content
• Example of use-cases:

  • Real-time context (the weather, your location, etc);

    

    Output obtained from Pi.ai

  • User-specific information (orders the user has made at this website, actions the user has taken on the website, the user’s status, etc);

  • Relevant factual information (documents not included in the LLM’s training data - either because they are private or they were updated after the LLM was trained).

RAG is not Fine-Tuning!