Session 8 Slides: Grounding AI — Embeddings Basics

--:-- --
↓ Scroll for more

Session 8

Grounding AI

Embeddings Basics

AI Product Engineering

Block 2: AI-Assisted Engineering & Integration

Two Problems with Static AI

Knowledge Cutoff

Model knows nothing after its training date

Specificity Gap

Model knows nothing about your domain, your documents, your data

Grounding = feed relevant information at query time, not from training data.

What Is a Vector Embedding?

A list of numbers that represents the meaning of text.

Key property: similar meaning → similar numbers.

  • "dogs" → [1.0, 0.1, 0.0]
  • "puppies" → [0.9, 0.2, 0.1] (similar!)
  • "algebra" → [-0.1, 0.0, 1.0] (very different)

Real models: hundreds to thousands of dimensions.

Cosine Similarity

ScoreMeaning
1.0Identical meaning
0.7–0.9Very similar
0.5–0.7Related but different
0.0Completely unrelated

Find relevant documents = find highest similarity to query embedding.

The Embedding Workflow

Offline (Once)

Read each document → call embedding API → store text + vector

Online (Per Query)

Embed query → find most similar vectors → feed matching text to LLM

Chunking Strategy

  • Too large: embedding averages over too many topics — loses specificity
  • Too small: single sentences are often ambiguous without context
  • Target: 200–500 words with 50–100 word overlap at boundaries
  • Chunk on: paragraph breaks, headings, or section dividers

Session 8 Summary

  • Embeddings = numerical representations of meaning
  • Cosine similarity = how semantically related two texts are
  • Two-phase workflow: offline indexing + online retrieval
  • Chunk size (200–500 words) determines retrieval granularity
  • Your vault is now embedded — ready for RAG

Next Session: Building the RAG Pipeline