Session 8 Slides: Grounding AI — Embeddings Basics

--:-- --
↓ Scroll for more

Session 8

Grounding AI

Embeddings Basics

AI Product Engineering

Block 2: AI-Assisted Engineering & Integration

Today’s Agenda

  • Two Problems with Static AI
  • What Is a Vector Embedding?
  • Cosine Similarity
  • The Embedding Workflow
  • Chunking Strategy
  • Interactive: Similarity Drill (3–4 min)
  • API gotcha: Embedding 2 prefixes (not taskType)
  • Interactive: Offline vs Online (3 min)
  • Lab: Lab checklist (Core)

Two Problems with Static AI

Knowledge Cutoff

Model knows nothing after its training date

Specificity Gap

Model knows nothing about your domain, your documents, your data

Grounding = feed relevant information at query time, not from training data.

What Is a Vector Embedding?

A list of numbers that represents the meaning of text.

Key property: similar meaning → similar numbers.

  • "dogs" → [1.0, 0.1, 0.0]
  • "puppies" → [0.9, 0.2, 0.1] (similar!)
  • "algebra" → [-0.1, 0.0, 1.0] (very different)

Real models: hundreds to thousands of dimensions.

Cosine Similarity

ScoreMeaning
1.0Identical meaning
0.7–0.9Very similar
0.5–0.7Related but different
0.0Completely unrelated

Find relevant documents = find highest similarity to query embedding.

Interactive: Similarity Drill (1/3)

Which pair should score highest on cosine similarity?

Interactive: Similarity Drill (2/3)

Choose the best definition.

Interactive: Similarity Drill (3/3)

Choose the best interpretation of the score.

The Embedding Workflow

Offline (Once)

Read each document → call embedding API → store text + vector

Online (Per Query)

Embed query → find most similar vectors → feed matching text to LLM

Interactive: Offline vs Online (1/3)

True or false.

Interactive: Offline vs Online (2/3)

True or false.

Interactive: Offline vs Online (3/3)

True or false.

Chunking Strategy

  • Too large: embedding averages over too many topics — loses specificity
  • Too small: single sentences are often ambiguous without context
  • Target: 200–500 words; overlap is Stretch in JavaScript (no NumPy)
  • Chunk on: paragraph breaks, headings, or section dividers

API gotcha: gemini-embedding-2

  • Do not pass config.taskType — not supported on Embedding 2
  • Do prefix text: docs → title: none | text: …
  • Queries (Session 9) → task: search result | query: …
  • Return response.embeddings[0].values as a number[]

Lab checklist (1/2)

  1. Before: git checkout track-NN · GEMINI_API_KEY in .env · npm install
  2. 8.1: Implement embedText in lib/embeddings.js (prefix format, no taskType)
  3. 8.2: Three-sentence cosine check (two AI sentences vs coffee)

Lab checklist (2/2)

  1. 8.3: npm run build-embeddings (default sample-vault/, max 10 files)
  2. Confirm embeddings.json has { file, content, embedding } — Session 9 gates on this

Stretch: heading split + overlap chunking. Seeds are ~7–8 notes; grow to ≥10 before showcase.

Session 8 Summary

  • Embeddings = numerical representations of meaning
  • Cosine similarity = how semantically related two texts are
  • Two-phase workflow: offline indexing + online retrieval
  • Embedding 2: task hints in text prefixes, not config.taskType
  • embeddings.json ready — Session 9 builds RAG on top

Next Session: Building the RAG Pipeline