Retrieval-Augmented Generation (RAG): Architecture and Implementation
Who this is for:
Architecture / Concept Overview: Retrieval-Augmented Generation (RAG): Architecture and Implementation
A RAG pipeline has three stages: ingestion (chunk and embed), retrieval (similarity search), and generation (LLM with context).
%%{init: {"theme":"base","themeVariables":{"background":"#0B0E14","primaryTextColor":"#E0E6ED","lineColor":"#5D6470","darkMode":true,"primaryColor":"#2E4A4A","secondaryColor":"#374151","secondaryTextColor":"#E0E6ED","tertiaryColor":"#111827","tertiaryTextColor":"#E0E6ED","edgeLabelBackground":"#1f2937"}}}%%
flowchart LR
classDef source fill:#3F4B59,stroke:#9CA3AF,stroke-width:2px,rx:8,ry:8,color:#E0E6ED
classDef ingestion fill:#5A4B36,stroke:#C9A86B,stroke-width:2px,rx:8,ry:8,color:#E0E6ED
classDef processing fill:#535072,stroke:#8E82B4,stroke-width:2px,rx:8,ry:8,color:#E0E6ED
classDef storage fill:#2E4A4A,stroke:#5FAFA8,stroke-width:2px,rx:8,ry:8,color:#E0E6ED
classDef serving fill:#3D5550,stroke:#6BB7AA,stroke-width:2px,rx:8,ry:8,color:#E0E6ED
classDef governance fill:#5A3F52,stroke:#C28BB0,stroke-width:2px,rx:8,ry:8,color:#E0E6ED
DOCS[Source Documents] -->|Parse| CHUNK[Chunking Pipeline]
CHUNK -->|Embed| EMB[Embedding Model]
EMB -->|Store| VS[Vector Search Index]
QUERY[User Query] -->|Embed| Q_EMB[Query Embedding]
Q_EMB -->|Search| VS
VS -->|Top-K Docs| PROMPT[Prompt Assembly]
PROMPT -->|Generate| LLM[Foundation Model]
LLM -->|Response| USER[User]
DOCS:::source
CHUNK:::ingestion
EMB:::processing
VS:::storage
QUERY:::source
Q_EMB:::processing
PROMPT:::serving
LLM:::serving
USER:::governance
*RAG pipeline: documents are chunked and embedded into a vector index; user queries retrieve relevant chunks that ground the LLM response.*
%%{init: {"theme":"base","themeVariables":{"background":"#0B0E14","primaryTextColor":"#E0E6ED","lineColor":"#5D6470","darkMode":true,"primaryColor":"#2E4A4A","secondaryColor":"#374151","secondaryTextColor":"#E0E6ED","tertiaryColor":"#111827","tertiaryTextColor":"#E0E6ED","edgeLabelBackground":"#1f2937"}}}%%
graph TD
classDef source fill:#3F4B59,stroke:#9CA3AF,stroke-width:2px,rx:8,ry:8,color:#E0E6ED
classDef ingestion fill:#5A4B36,stroke:#C9A86B,stroke-width:2px,rx:8,ry:8,color:#E0E6ED
classDef processing fill:#535072,stroke:#8E82B4,stroke-width:2px,rx:8,ry:8,color:#E0E6ED
classDef storage fill:#2E4A4A,stroke:#5FAFA8,stroke-width:2px,rx:8,ry:8,color:#E0E6ED
classDef serving fill:#3D5550,stroke:#6BB7AA,stroke-width:2px,rx:8,ry:8,color:#E0E6ED
classDef governance fill:#5A3F52,stroke:#C28BB0,stroke-width:2px,rx:8,ry:8,color:#E0E6ED
RAG[RAG Components] --> INGEST[Ingestion Layer]
RAG --> RETRIEVAL[Retrieval Layer]
RAG --> GENERATION[Generation Layer]
RAG --> EVAL_L[Evaluation Layer]
INGEST --> PARSE[Document Parsing]
INGEST --> CHUNK_S[Chunking Strategy]
INGEST --> EMBED_S[Embedding Model]
RETRIEVAL --> VECTOR[Vector Similarity]
RETRIEVAL --> HYBRID[Hybrid Search]
RETRIEVAL --> RERANK[Re-Ranking]
GENERATION --> PROMPT_T[Prompt Template]
GENERATION --> LLM_S[LLM Selection]
GENERATION --> GUARD[Guardrails]
RAG:::governance
INGEST:::ingestion
RETRIEVAL:::storage
GENERATION:::serving
EVAL_L:::processing
PARSE:::source
CHUNK_S:::source
EMBED_S:::source
VECTOR:::source
HYBRID:::source
RERANK:::source
PROMPT_T:::source
LLM_S:::source
GUARD:::source
*Four-layer RAG architecture: ingestion, retrieval, generation, and evaluation.*
Key Terms
Prerequisites and Setup
- Unity Catalog with a catalog and schema for document tables.
- A Vector Search endpoint created in your workspace.
- Foundation Model APIs enabled for embedding and generation.
- Databricks Runtime for ML or serverless compute.
Step-by-Step Implementation
Configuration Reference
| Parameter | Default | Description |
|---|---|---|
chunk_size | 500 | Maximum characters per chunk |
chunk_overlap | 50 | Characters overlapping between consecutive chunks |
num_results (top_k) | 10 | Number of documents retrieved per query |
embedding_model_endpoint_name | — | Model used for automatic embedding |
pipeline_type | TRIGGERED | Sync mode: TRIGGERED (manual) or CONTINUOUS |
temperature | 0.1 | Low temperature for factual RAG responses |