— LLM Provider Abstraction

Multi-provider chat, embedding, and specialized models

Chat models, embedding models, reranking, OCR, vision, sequence-to-text, text-to-speech.

Classes

Files

~N k

Lines

Key Metric

❓ What is this & why?

The llm module abstracts multiple LLM providers (OpenAI, Claude, local models) behind a unified interface. This prevents vendor lock-in: swapping from OpenAI to Claude requires changing one config line, not refactoring code.

💡 Real-world analogy

Like a translator agency. You don't need to hire a specific translator; the agency routes your request to whoever is available. If your usual translator is busy or expensive, the agency can swap in another without you noticing.

🤔 Why Is It Designed This Way?

{{DESIGN_RATIONALE}} — 150+ words explaining the design choices: what pattern does this follow and why? What constraints shaped the implementation? What trade-offs were made and what was sacrificed?

Design Choice	What Was Chosen	What Was Rejected	Why
Example: data layout	Array of structs	Struct of arrays	Random access pattern dominates; cache locality for single items matters more than SIMD sweeps

Architecture

flowchart TD
    subgraph Chat["Chat Models"]
        C1["OpenAI"]
        C2["Claude"]
        C3["Local LLM"]
    end
    subgraph Embed["Embedding Models"]
        E1["BGE"]
        E2["OpenAI Embed"]
        E3["Local ONNX"]
    end
    subgraph Other["Specialized"]
        O1["Reranker"]
        O2["OCR"]
        O3["TTS"]
    end

Key Classes

Class / Function	File	Purpose
`ClassName`	file.h	Brief description

Data Structures

Type	Name	Description
enum	`EnumName`	Brief description

Interactions

Direction	Partner	Interface	Description
→	Other Module	`methodName()`	Brief description

Configuration

Knob / Parameter	Default	Description
`param_name`	value	What it controls

Directory Map

Source tree for llm

submod_name/
├── file_a.ext                 ← Role: brief annotation
├── file_b.ext                 ← Role: brief annotation
├── file_c.ext                 ← Role: brief annotation
└── index.ext                  ← Role: entry point

📑 Annotated Code Walkthrough

The most important code in this sub-module, annotated with WHY each piece exists:

// Replace with actual code from the source
void key_function() {
    // ← WHY: Explain the purpose of this line
    // ← WHY: What constraint or requirement drives this?
}

Line-by-line:

Line N: What this does and WHY — what would break if removed
Line M: The design choice here — why this approach over the obvious alternative

⚠️ Edge Cases & Boundaries

Edge Case	What Happens	Where in Code	Risk Level
Empty input	How it's handled	function_name:line	🟢 Low
Concurrent access	How it's handled	function_name:line	🔴 High

Deep-Dive Highlights

Core mechanisms from this sub-module have been extracted into dedicated focus pages for in-depth exploration. Each page covers one algorithm, protocol, or subsystem with diagrams, code walks, and configuration details.

Deep-Dive Pages

Provider Dispatch

sequenceDiagram
    participant Code as Application
    participant Factory as Factory
    participant OpenAI as OpenAI
    participant Claude as Claude
    Code->>Factory: create("openai")
    Factory->>OpenAI: Initialize
    OpenAI->>Code: Provider
    Code->>Code: Switch config
    Code->>Factory: create("claude")
    Factory->>Claude: Initialize
    Claude->>Code: Provider

Code Examples

Example: Typical Usage

// Replace with actual code example
void example() {
    // ...
}