Session 9 Slides: Building the RAG Pipeline

--:-- --
↓ Scroll for more

Session 9

Building the RAG Pipeline

AI Product Engineering

Block 2: AI-Assisted Engineering & Integration

Today’s Agenda

  • Why RAG?
  • The RAG Pipeline
  • The Critical RAG System Prompt
  • Evaluate Retrieval First
  • Interactive: Retrieve or Generate? (3–4 min)
  • Interactive: Retrieval Quality (3 min)
  • Lab: Build the RAG Pipeline (Core)
  • Lab: Five Test Queries

Why RAG?

Without RAG

LLM answers from training data → hallucination, knowledge cutoff, no domain specificity

With RAG

LLM answers from your curated knowledge → grounded, controllable, up-to-date

The RAG Pipeline

1. Embed the user query → query vector
2. Compare to knowledge base → find top K similar chunks
3. Build augmented prompt: system + context + question
4. LLM generates answer from context only
5. Return answer + sources to user

Interactive: Retrieve or Generate? (3–4 min)

Put each step in its phase.

The Critical RAG System Prompt

You are a knowledge base assistant.
Answer using ONLY the information in the context below.
If the context lacks the answer, say:
"I don't have enough information."
Do not use general training knowledge.
Do not invent facts.

Without this, the model will "help" by hallucinating from training data.

Evaluate Retrieval First

Before worrying about answer quality:

  • Are the top-K chunks actually relevant to the query?
  • Is the answer in the knowledge base at all?
  • Are chunks the right size?

Log retrieved chunks during development. Bad retrieval cannot be fixed by prompt engineering.

Interactive: Retrieval Quality (1/3)

True or false — cold-call, then tap.

Interactive: Retrieval Quality (2/3)

True or false — cold-call, then tap.

Interactive: Retrieval Quality (3/3)

True or false — cold-call, then tap.

Lab: Build the RAG Pipeline (1/2)

Gate: embeddings.json must exist — otherwise finish Session 8 first.

  1. 9.1: retrieveRelevantChunks — embed query, cosine-sort, top 3
  2. 9.2: Use kit buildRagPrompt

Lab: Build the RAG Pipeline (2/2)

  1. 9.3: Implement ragQuery → { answer, sources, topScore }
  2. 9.4: Wire POST /query → { response, sources }

Lab: Five Test Queries

  1. Clearly in knowledge base
  2. Partially in knowledge base
  3. Not in KB → expect "I don't know"
  4. Misleading / training-data bait → refuse hallucination
  5. Real use-case question

Time-box notes to bullets. Stretch: show scores in UI.

Session 9 Summary

  • RAG: embed → retrieve → augment → generate
  • System prompt must restrict model to context only
  • Retrieval quality is the most important variable
  • Log everything during development
  • Your app is now a knowledge-grounded AI system

Next Session: Introduction to AI Agents