Session 9 Slides: Building the RAG Pipeline

--:-- --
↓ Scroll for more

Session 9

Building the RAG Pipeline

AI Product Engineering

Block 2: AI-Assisted Engineering & Integration

Why RAG?

Without RAG

LLM answers from training data → hallucination, knowledge cutoff, no domain specificity

With RAG

LLM answers from your curated knowledge → grounded, controllable, up-to-date

The RAG Pipeline

1. Embed the user query → query vector
2. Compare to knowledge base → find top K similar chunks
3. Build augmented prompt: system + context + question
4. LLM generates answer from context only
5. Return answer + sources to user

The Critical RAG System Prompt

You are a knowledge base assistant.
Answer using ONLY the information in the context below.
If the context lacks the answer, say:
"I don't have enough information."
Do not use general training knowledge.
Do not invent facts.

Without this, the model will "help" by hallucinating from training data.

Evaluate Retrieval First

Before worrying about answer quality:

  • Are the top-K chunks actually relevant to the query?
  • Is the answer in the knowledge base at all?
  • Are chunks the right size?

Log retrieved chunks during development. Bad retrieval cannot be fixed by prompt engineering.

Lab: Five Test Queries

  1. Question clearly in your knowledge base
  2. Question partially in knowledge base
  3. Question NOT in knowledge base → expect "I don't know"
  4. Misleading question → expect refusal to hallucinate
  5. Your real-world use case

Session 9 Summary

  • RAG: embed → retrieve → augment → generate
  • System prompt must restrict model to context only
  • Retrieval quality is the most important variable
  • Log everything during development
  • Your app is now a knowledge-grounded AI system

Next Session: Introduction to AI Agents