📊 View Lecture Slides Full-screen presentation with navigation

Session 8: Grounding AI — Embeddings Basics

Session Duration: 2 Hours     Block: 2 — AI-Assisted Engineering & Integration

Learning Objectives

By the end of this session, students will be able to:

  • Explain what vector embeddings are and how they represent semantic meaning
  • Generate embeddings from text using a cloud embedding API
  • Calculate cosine similarity to measure semantic relatedness between texts
  • Describe how embeddings form the foundation of a RAG retrieval system

Hour 1: How AI Represents Meaning (Instructor-Led — 60 minutes)

1.1 The Problem with Static Knowledge

Every large language model has a knowledge cutoff date — a point in time beyond which it has no information. For a student asking about a news event from last week, or a company asking about their internal processes, the model’s training data is useless.

Beyond the cutoff problem, there is a specificity problem: even within the model’s knowledge window, it knows nothing about your particular domain, your specific documents, or your personal knowledge base.

The solution is grounding: feeding the model relevant information at query time, so that it answers from that information rather than from its general training. This is what RAG (Retrieval-Augmented Generation) provides — and embeddings are the mechanism that makes the retrieval part work.

1.2 What Is a Vector Embedding?

A vector embedding is a list of numbers that represents the meaning of a piece of text. The key property is: similar meaning → similar numbers.

Imagine a 3-dimensional space where:

  • [1.0, 0.1, 0.0] represents “dogs”
  • [0.9, 0.2, 0.1] represents “puppies” (similar to dogs → similar vector)
  • [-0.1, 0.0, 1.0] represents “algebra” (different meaning → very different vector)

Real embedding models produce vectors with hundreds or thousands of dimensions, capturing far more nuanced semantic relationships. But the principle is identical.

1.3 Cosine Similarity

To compare two embeddings, we use cosine similarity — a measure of the angle between two vectors:

  • 1.0: Identical meaning
  • 0.7–0.9: Very similar meaning
  • 0.5–0.7: Related but different topics
  • 0.0: Completely unrelated

This is how a retrieval system finds relevant documents: it embeds the user’s query, then finds the documents in the knowledge base whose embeddings are closest to the query embedding.

1.4 The Embedding Workflow

Offline (once, when building the knowledge base):
  For each document chunk:
    1. Read the text
    2. Call the embedding API → get a vector
    3. Store the text + vector in a vector store

Online (at query time):
    1. Receive user query
    2. Embed the query → get a vector
    3. Find the most similar vectors in the store
    4. Return the corresponding text chunks
    5. Feed those chunks as context to the LLM
    6. LLM answers from the retrieved context

1.5 Chunking Strategy

Embeddings are computed per chunk. A chunk that is too large loses specificity (the embedding averages over too many topics). A chunk that is too small loses context (individual sentences are often ambiguous without surrounding text).

General guidelines:

  • Target chunk size: 200–500 words
  • Include overlap between chunks: 50–100 words (prevents missing content at chunk boundaries)
  • Chunk on natural boundaries: paragraph breaks, headings, or section dividers

Hour 2: Practical — Generate and Explore Embeddings (60 minutes)

Lab 8.1 — Generate Your First Embeddings

Install the required library and generate embeddings from a small test set:

import os
from dotenv import load_dotenv
import google.generativeai as genai

load_dotenv()
genai.configure(api_key=os.environ.get('GEMINI_API_KEY'))

# Generate an embedding
def embed_text(text: str) -> list[float]:
    result = genai.embed_content(
        model="models/text-embedding-004",
        content=text,
        task_type="retrieval_document"
    )
    return result['embedding']

# Test with a few sentences
texts = [
    "Artificial intelligence is transforming software development.",
    "Machine learning enables computers to learn from data.",
    "The price of coffee has increased this month.",
]

embeddings = [embed_text(t) for t in texts]
print(f"Embedding dimension: {len(embeddings[0])}")

Lab 8.2 — Compute Similarity

import numpy as np

def cosine_similarity(a: list[float], b: list[float]) -> float:
    a, b = np.array(a), np.array(b)
    return float(np.dot(a, b) / (np.linalg.norm(a) * np.linalg.norm(b)))

# Compare each pair
for i, (t1, e1) in enumerate(zip(texts, embeddings)):
    for j, (t2, e2) in enumerate(zip(texts, embeddings)):
        if i < j:
            sim = cosine_similarity(e1, e2)
            print(f"Similarity({i+1},{j+1}): {sim:.3f}")
            print(f"  '{t1[:50]}...'")
            print(f"  '{t2[:50]}...'")

Do the similarity scores match your intuition? The first two sentences should score much higher than either compared to the coffee sentence.

Lab 8.3 — Embed Your Vault Notes

Load the Markdown notes from your Obsidian vault and generate embeddings for each:

import os
import json

def load_vault_notes(vault_path: str) -> list[dict]:
    notes = []
    for root, dirs, files in os.walk(vault_path):
        for file in files:
            if file.endswith('.md'):
                path = os.path.join(root, file)
                with open(path, 'r', encoding='utf-8') as f:
                    content = f.read()
                notes.append({'file': file, 'content': content})
    return notes

notes = load_vault_notes('./my-vault')  # adjust path
print(f"Loaded {len(notes)} notes")

# Generate embeddings (be mindful of API rate limits)
embeddings_data = []
for note in notes[:10]:  # start with first 10
    embedding = embed_text(note['content'][:2000])  # truncate if needed
    embeddings_data.append({
        'file': note['file'],
        'content': note['content'],
        'embedding': embedding
    })

# Save to JSON for use in Session 9
with open('embeddings.json', 'w') as f:
    json.dump(embeddings_data, f)
print("Embeddings saved to embeddings.json")

Key Takeaways

  • Vector embeddings represent text meaning as lists of numbers — similar meaning produces similar vectors
  • Cosine similarity measures how semantically related two texts are
  • The embedding workflow has two phases: offline indexing and online retrieval
  • Chunk size (200–500 words with overlap) determines retrieval granularity
  • You now have embeddings of your knowledge base, ready for the RAG pipeline in Session 9

Further Reading

  • Google embedding models documentation: ai.google.dev
  • “Understanding Embeddings” — Weaviate blog
  • NumPy documentation for vector operations