06_retrieval_strategies/

Constant k = 60

const RRF_K = 60;

Standard from Cormack, Clarke, Büttcher (2009). Smaller k over-rewards #1; larger k flattens the score — effectively making rank-1 and rank-50 indistinguishable. 60 is the published sweet spot.

Index by document id

function rrf(rankedLists, k = RRF_K) {
    const scores = new Map();
    for (const list of rankedLists) {
        list.forEach((doc, rank) => {
            const id = doc.id || doc.metadata?.id || doc.pageContent;
            scores.set(id, (scores.get(id) || 0) + 1 / (k + rank + 1));
        });
    }

Identity is critical — the fusion needs to recognise the same document in two lists. Falls back through id → metadata.id → raw text. The +1 accounts for 0-based ranks (RRF formula uses 1-based).

Sort by fused score

    const fused = [];
    for (const [id, score] of scores) fused.push({ id, score });
    fused.sort((a, b) => b.score - a.score);

Each document's score is the sum across all retrieval lists. Documents that appear high in any list score well; those that appear high in multiple lists score best.

Map ids back to documents

    const docById = new Map();
    for (const list of rankedLists) for (const d of list)
        docById.set(d.id || d.metadata?.id || d.pageContent, d);
    return fused.map(f => ({ ...docById.get(f.id), rrfScore: f.score }));
}

Reattaches the document object — we lost it when we kept only ids. Tagging rrfScore on the result lets debug output show why each doc made the cut.

Fetch a candidate pool

const candidates = await store.search(qVec, fetchK);  // e.g., 20

MMR re-ranks among a pre-fetched pool. fetchK (~4× the desired output) trades a few extra cosine ops for diversity headroom.

Greedy selection loop

const selected = [];
const remaining = [...candidates];
while (selected.length < k && remaining.length) {
    let bestIdx = -1, bestScore = -Infinity;
    for (let i = 0; i < remaining.length; i++) {
        const c = remaining[i];
        const relevance = cosine(qVec, c.vector);
        const maxSimToSelected = Math.max(
            0, ...selected.map(s => cosine(s.vector, c.vector))
        );
        const mmr = lambda * relevance - (1 - lambda) * maxSimToSelected;
        if (mmr > bestScore) { bestScore = mmr; bestIdx = i; }
    }
    selected.push(remaining.splice(bestIdx, 1)[0]);
}

The maxSimToSelected is the diversity penalty — if a candidate is very similar to one we've already picked, its MMR score takes a hit. λ tunes the relevance/diversity balance: 1.0 = ignore diversity (pure top-k), 0.0 = ignore relevance (pure spread).

Why O(k·n) is fine

For typical k=5, n=20: 100 cosine ops. Trivial. MMR's expense scales with desired output size, not the corpus — the corpus filtering already happened in the vector store. The greedy approach trades optimal selection (NP-hard) for tractable runtime.