00_how_rag_works/

🎯 Why This Component

This lesson exists because RAG tutorials that start with "and now embeddings" leave readers with a vague sense that embeddings are better without a concrete sense of why. By implementing the dumb thing first, the reader builds intuition for the gap. When lesson 02 introduces semantic similarity and asks the same query against the same documents, the reader sees synonyms work and has the "aha" moment that justifies the rest of the curriculum.

It's also fast. No model downloads, no GGUF, no GPU; node 00_how_rag_works/01_basic_search.js runs in milliseconds. This is the "kick the tires" lesson — the one a reader can complete on a phone tethered to coffee-shop wifi.

🏗️ Design Rationale

Architecture

flowchart LR
    Q[query string] --> N[normalize: lowercase + tokenize]
    N --> S[remove stop-words]
    S --> M[match query terms in documents]
    M --> R[rank by hit count or TF-IDF-like score]
    R --> O[print top-k]
    D[hardcoded docs[]] --> M
    style M fill:#3fb95033,stroke:#3fb950

Lifecycle — Why a synonym query fails

sequenceDiagram
    participant U as User
    participant L as 00_how_rag_works
    participant Idx as in-memory docs[]
    U->>L: query = "puppy training"
    L->>L: tokens = ['puppy','training']
    L->>Idx: for each doc, count token hits
    Note over Idx: doc1: "Dogs need consistent training"
'training' hits [ok]
'puppy' miss [no] (says 'dogs')
score = 1
    Note over Idx: doc2: "Cats are independent"
both miss
score = 0
    Note over Idx: doc3: "How to housetrain a puppy"
'puppy' hits [ok]
'training' miss [no] (says 'housetrain')
score = 1
    Idx-->>L: tied scores; arbitrary order
    L-->>U: prints unhelpful ranking
    Note over U: "puppy training" should clearly match
doc1 + doc3 above doc2 — but the
scoring can't see the synonym

Interactions

📝 Annotated Code Walkthrough — 01_basic_search.js

const documents = [
    "Dogs need consistent training and patience.",
    "Cats are independent and aloof.",
    "How to housetrain a puppy: a beginner's guide.",
    "Reinforcement learning trains agents through rewards.",
    // ...
];

function tokenize(s) {
    return s.toLowerCase().split(/\s+/).filter(Boolean);
}

function score(query, doc) {
    const qTokens = tokenize(query);
    const dTokens = tokenize(doc);
    let hits = 0;
    for (const t of qTokens) {
        if (dTokens.includes(t)) hits++;
    }
    return hits;
}

function search(query, docs) {
    return docs
        .map((d, i) => ({ doc: d, idx: i, score: score(query, d) }))
        .sort((a, b) => b.score - a.score)
        .slice(0, 5);
}

const query = "puppy training";
console.log(`Query: "${query}"`);
search(query, documents).forEach((r, rank) => {
    console.log(`  ${rank + 1}. (score=${r.score}) ${r.doc}`);
});

Deep-Dive Highlights

⚠ Edge Cases & Boundaries

Directory Map

examples/00_how_rag_works/
├── 01_basic_search.js          ← Pure keyword search; demonstrates synonym failure
├── 02_failure_modes.js         ← Adds harder paraphrase queries
└── 03_stop_words.js            ← Stop-word filter; shows it's not enough

Configuration Reference

Code Examples

Run lesson 00

cd examples/00_how_rag_works
node 01_basic_search.js
node 02_failure_modes.js
node 03_stop_words.js

Sample output (synonym failure)

Query: "puppy training"
  1. (score=1) Reinforcement learning trains agents through rewards.
  2. (score=1) Dogs need consistent training and patience.
  3. (score=1) How to housetrain a puppy: a beginner's guide.
  4. (score=0) Cats are independent and aloof.
  5. (score=0) Embeddings turn text into vectors.

What lesson 02 will produce (semantic, for contrast)

Query: "puppy training"
  1. (sim=0.82) How to housetrain a puppy: a beginner's guide.
  2. (sim=0.74) Dogs need consistent training and patience.
  3. (sim=0.41) Reinforcement learning trains agents through rewards.
  4. (sim=0.18) Cats are independent and aloof.
  5. (sim=0.12) Embeddings turn text into vectors.