<aside> 📘 LLM RAG Security Serials:
Prompt injection: What’s the worst that can happen?
Mitigating Security Risks in RAG LLM Applications | CSA
</aside>
Written by Ken Huang, CEO of DistributedApps.ai and VP of Research at CSA GCR.
Retrieval augmented generation (RAG) is an effective technique used by AI engineers to develop large language model (LLM) powered applications. However, the lack of security controls in RAG-based LLM applications can pose risks if not addressed properly.
In this post, we will analyze the RAG architecture, identify potential security risks at each stage, and recommend techniques to mitigate those risks when developing RAG-based LLM applications. Our goal is to provide developers with practical guidance on building more secure LLM applications using the RAG pattern. By understanding the security implications of RAG and implementing appropriate controls, we can harness the power of LLMs while safeguarding against potential vulnerabilities.
<aside> 📘 这其实只探讨了 inference RAG，不过也只有 inference RAG 有暴露面
</aside>
Figure 1: RAG Components and Workflow
Figure 1: RAG Components and Workflow
A RAG system comprises various components that work together to provide contextual responses to queries. The key components are the knowledge source, indexer, vector database, retriever, and generator. The workflow involves indexing the knowledge source, storing embeddings in a vector database, retrieving relevant context for a query, and using a language model to generate a response.
Knowledge Source: The knowledge source is the foundation of the RAG system. It comprises textual documents, databases, and knowledge graphs that collectively form a comprehensive knowledge base.
Indexing and Embedding Generation: The information from the knowledge source undergoes indexing, which organizes the data to make it easily searchable and retrievable. This involves categorizing the information and creating indexes.
As part of indexing, vector embeddings are also generated from the indexed data. These embeddings capture the semantic meaning of the text. They allow the retrieval system to find relevant information based on query context.
Vector Database: The generated embeddings are stored in a vector database optimized for vector data. This enables efficient retrieval based on semantic similarity.
Retriever: The retriever uses semantic search and approximate nearest neighbors (ANN) to fetch contextually relevant data from the vector database for a given query or prompt. It understands query semantics to retrieve information beyond just keywords.
Generator: The generator uses an LLM like GPT-4 or Claud2. It takes the context from the retriever and generates a coherent, relevant response. The LLM understands complex language to produce contextual text.
Workflow: