Session 9: Building the RAG Pipeline

📊 View Lecture Slides

Full-screen presentation with navigation

Session 9: Building the RAG Pipeline

Session Duration: 2 Hours

Block: 2 — AI-Assisted Engineering & Integration

Session clock

Minutes	Mode	Focus
0–50	Lecture	Theoretical Foundation & Concepts
50–110	Core lab	Implement Retrieve & Augment logic
110–120	Checkpoint	Pair share / show artifact

Note: Stretch work starts only after the Core checkpoint is completed.

---

Learning Objectives

By the end of this session, students will be able to:

• Describe the complete data flow of a Retrieval-Augmented Generation (RAG) pipeline from user query to grounded answer.
• Implement the “Retrieve” and “Augment” steps in JavaScript, performing semantic search against a local JSON vector store.
• Integrate the complete RAG function into the Express /query route to serve the frontend UI.
• Evaluate RAG answers against ungrounded AI answers to measure the reduction in hallucination.

---

Part 1: Theoretical Foundation — RAG: The Complete Picture

1.1 Why RAG is the Industry Standard

In Session 7, we connected our app directly to the Gemini API. If we asked it a highly specific question about our custom product idea or a private document, it either guessed (hallucinated) or gave a generic, unhelpful answer.

In Session 8, we learned how to turn our private knowledge base (our Obsidian vault) into searchable mathematical vectors (Embeddings).

Today, we combine them. Retrieval-Augmented Generation (RAG) is the dominant architectural pattern for building AI applications today. It solves the knowledge cutoff problem and drastically reduces hallucination without the immense cost of retraining or fine-tuning a foundational model.

• Retrieval: Search your private database for facts relevant to the user’s question.
• Augmented Generation: Give those facts to the LLM and command it to write an answer based exclusively on what you just provided.

1.2 The Complete RAG Data Flow

When a user clicks “Submit” in your UI, the following sequence must execute synchronously on your server in a matter of seconds:

1. User Query: The server receives the text (e.g., “What is our refund policy?”).
2. Embed Query: The server calls the embedding API to turn that question into a vector array ([0.12, -0.45, ...]).
3. Semantic Search (Retrieve): The server compares the query vector against every chunk vector in embeddings.json using Cosine Similarity.
4. Top K Selection: The server sorts the chunks by similarity score and extracts the Top 3 (or Top K) most relevant chunks of text.
5. Augment Prompt: The server concatenates the Top 3 chunks of text into a massive prompt string, alongside the user’s original question.
6. Generate: The server sends this augmented prompt to the text generation API (gemini-2.5-flash).
7. Return: The server sends the generated text—plus the file names of the sources used—back to the user’s browser.

1.3 The Critical Prompt Design for RAG

The magic of RAG relies heavily on a highly restrictive System Prompt. You must explicitly forbid the model from using its pre-trained “world knowledge” to answer the question.

export const RAG_SYSTEM_PROMPT = `You are a precise, highly accurate knowledge base assistant.
Your primary directive is to answer the user's question using ONLY the information provided in the CONTEXT blocks below.

If the provided context does not contain enough information to confidently answer the question, you MUST output exactly:
"I don't have enough information in my knowledge base to answer this."

Do not use your general training knowledge. Do not invent facts, dates, or prices.
When answering, explicitly cite the source document name if relevant.`;

Without this strict perimeter, the model will “leak” its general training data into your proprietary answers, leading to subtle and dangerous hallucinations.

1.4 Evaluating Retrieval Quality

Retrieval is the bottleneck of RAG. If your semantic search returns the wrong chunks of text (e.g., pulling a note on “Marketing Strategy” instead of “Refund Policy”), no amount of prompt engineering will save you. The LLM will simply say “I don’t know” or hallucinate.

During development, you must vigorously log the file name and the score of the chunks your system retrieves before it passes them to the LLM. If the retrieval is failing, you need to revisit Session 8 and adjust your chunking strategy or improve the quality of your Markdown context (Session 4).

---

Part 2: Practical Labs — Build and Integrate the RAG Pipeline

Lab 9.0 — The Quality Gate (5 min)

Before writing the RAG logic, confirm that your embeddings.json from Session 8 actually exists and contains data.

Run this quick check in your terminal (ensure you are in the project root):

node -e "import('./embeddings.json',{with:{type:'json'}}).then(m=>console.log('Indexed chunks:', m.default.length))"

If it returns an error or 0, you must go back to Session 8 and run npm run build-embeddings successfully before continuing. Do not attempt to build a RAG pipeline without a database.

Lab 9.1 — Implement Semantic Retrieval (Core)

Open lib/rag.js in your starter kit. Your first task is to implement the retrieveRelevantChunks function.

You need to:

1. Import and load embeddings.json.
2. Embed the user’s query using the embedText function you wrote in Session 8. (Set taskType: 'RETRIEVAL_QUERY').
3. Map over your loaded embeddings, calculating the cosine similarity between the query vector and each stored chunk vector.
4. Sort the results descending by score.
5. Slice and return the top k chunks.

Hint on array sorting in JavaScript:

// Scored items should look like: { file, content, score }
return scoredArray.sort((a, b) => b.score - a.score).slice(0, topK);

Lab 9.2 — Assemble the Augmented Prompt (Core)

Locate the buildRagPrompt helper function in the kit. This function’s job is simple but vital: take the user’s question, take the array of chunks returned by retrieveRelevantChunks, and concatenate them into a single, clean string.

Ensure the output looks structurally clear to the LLM, for example:

CONTEXT:
---
Source: refund-policy.md
[chunk text here]
---
Source: terms-of-service.md
[chunk text here]

USER QUESTION: What is the refund policy?

Lab 9.3 — Wire the Full ragQuery Function (Core)

Now, bring it all together in the main ragQuery exported function.

export async function ragQuery(aiClient, userQuestion) {
  // 1. Retrieve
  const chunks = await retrieveRelevantChunks(aiClient, userQuestion, 3); // Top 3

  // 2. Augment
  const augmentedPrompt = buildRagPrompt(userQuestion, chunks);

  // 3. Generate
  const response = await aiClient.models.generateContent({
    model: 'gemini-2.5-flash',
    contents: augmentedPrompt,
    config: {
      systemInstruction: RAG_SYSTEM_PROMPT, // From Section 1.3
      temperature: 0.1, // Keep it low for strict factual retrieval
    },
  });

  // 4. Format Return
  return {
    answer: response.text,
    sources: chunks.map(c => c.file), // Send the file names to the UI
    topScore: chunks[0]?.score ?? 0,
  };
}

Lab 9.4 — Integrate with Express and Test (Core)

Finally, open server.js. Update your POST /query route to call your new ragQuery function instead of the basic AI call from Session 7. Return the { answer, sources } payload to the frontend. The starter UI is already coded to render a list of sources if you provide them in the JSON!

Testing Protocol: Test five distinct questions through your browser UI. Time-box this to 10 minutes.

1. In-KB: A question directly answered by a note in your vault. (Does it get it right and cite the source?)
2. Partial: A question where only half the answer is in the vault.
3. Not-in-KB: Ask about the weather or sports. (It MUST trigger your fallback: “I don’t have enough information…”)
4. Misleading (Training Data Bait): Ask a generic technical question that is heavily featured in its pre-training but contradicts your vault. (Does it stick to the vault?)
5. Real Use Case: A complex question your actual target user would ask.

Document the results in your prompts.md or a new testing.md file.

Stretch Goal: Modify your Express response and Frontend UI to display the cosine similarity score of the top source, allowing the user to see the “confidence” of the retrieval.

---

Key Takeaways

• RAG architecture completely bridges the gap between massive general intelligence and proprietary, local knowledge.
• The pipeline executes sequentially: Embed Query → Retrieve Vectors → Augment Prompt → Generate Answer.
• The System Prompt is the security guard. It must explicitly command the LLM to ignore its general training and rely only on the injected context.
• Retrieval accuracy is paramount. If the vector search fails to pull the right document, the LLM will fail to answer the question.
• Your application is now functionally grounded in your curated Markdown exocortex.

---

Further Reading & Resources

• Pinecone / ChromaDB: For production systems with millions of documents, local JSON files are too slow. Research dedicated Vector Databases (not required for this course’s Core deliverables).
• Review lib/rag.js in the course kit thoroughly to ensure you understand the flow of data across the pipeline.