agent — Core Agent Engine

🎯 Why This Module Exists

The agent/ module was extracted from a monolithic 3,600-line run_agent.py that had grown to mix orchestration logic with utility functions, making testing, maintenance, and extension increasingly painful. Without this separation, every change to error handling would risk breaking context compression; every new memory backend would require modifying the core conversation loop; and unit testing any individual subsystem would require spinning up the entire agent stack. By extracting pure, self-contained utilities into agent/, the project achieves clean separation of concerns: run_agent.py focuses solely on the conversation orchestration loop, while agent/ provides the pluggable engines and services it consults. This modular architecture enables independent testing of compression, error classification, and credential management; allows third parties to swap implementations via abstract base classes (ContextEngine, MemoryProvider); and ensures the same utilities work across CLI, gateway, and cron contexts. When Anthropic releases a new caching strategy or a provider changes its error signatures, updates localize to a single file rather than scattering across the entire codebase.

🚶 Step-by-Step Narrated Walkthrough

Follow a single user query through the agent pipeline as it flows through the entire agent module:

1. User Input Received — The user types a message. SubdirectoryHintTracker checks any recent tool calls for new directory paths and lazily loads AGENTS.md / .cursorrules hints if the agent has navigated to a new directory.
2. Memory Prefetch — MemoryManager.prefetch_all() fires across all registered providers (built-in + at most one external). Each provider retrieves relevant context from its backend. Results are merged into a fenced memory context block via build_memory_context_block().
3. System Prompt Assembly — prompt_builder._build_system_prompt() constructs the system prompt by combining identity text, platform hints, the skills index (parsed from skill frontmatter via skill_commands.py), injected context files, and the memory block. Prompt injection detection scans AGENTS.md content for suspicious patterns.
4. Preflight Token Check — model_metadata.estimate_messages_tokens_rough() uses a 4-chars/token heuristic with provider-specific adjustments to estimate whether the assembled messages fit within the model's context window. If the estimate exceeds the threshold, ContextEngine.should_compress_preflight() signals compression is needed.
5. Context Compression (if triggered) — ContextCompressor.compress() prunes old tool results, protects head and tail messages, serializes middle turns, and calls the auxiliary LLM to generate a structured summary. If a previous summary exists, it uses iterative update mode to preserve accumulated facts.
6. Smart Routing Decision — SmartModelRouting evaluates whether the user message is "simple" (under 160 chars, under 28 words, no code blocks or complex keywords). If simple, it routes to a configured cheap model to save ~80% on token cost. Otherwise, the primary model is used.
7. Credential Selection — CredentialPool selects the next available API credential via round-robin or least-used strategy. Exhausted credentials (those with recent 429/402 errors) are skipped until their cooldown expires.
8. Prompt Caching — For Anthropic models, apply_anthropic_cache_control() injects up to 4 cache_control breakpoints (system prompt + last 3 non-system messages) to achieve ~75% input cost reduction on cache hits.
9. API Call & Response — The request is sent to the selected LLM provider. Response headers (x-ratelimit-*) are parsed by RateLimitTracker to update RPM/TPM bucket state. ContextEngine.update_from_response(usage) records actual token counts.
10. Error Classification & Recovery — If the API call throws an exception, classify_api_error() runs the error through a priority-ordered pipeline: provider-specific patterns, HTTP status codes, message pattern matching, transport heuristics, and context overflow detection. The resulting ClassifiedError determines whether to retry with backoff, rotate credentials, trigger compression, fall back to another model, or abort.
11. Trajectory Capture — On success, TrajectoryCapture records the full turn (input messages, output, tool calls, token usage, timing) for reinforcement learning training data. This creates structured datasets that can train future model versions.
12. Display Formatting — display.py formats the assistant response for terminal output with syntax highlighting, markdown rendering, and token usage statistics. The final formatted response is presented to the user, and MemoryManager.sync_all() writes the exchange to all registered memory providers in the background.

Context Compression with Iterative Summary Updates

The Problem: Long agent conversations can exceed the model's context window (e.g., 200K or 1M tokens). Naively dropping old messages loses critical decision context — file paths, design choices, error resolutions — that the agent needs to continue working effectively.

The Approach: When last_prompt_tokens exceeds a configurable threshold (default: 75% of max context), the compressor divides the conversation into three zones: a protected head (system prompt + opening exchange), a middle (summarization target), and a protected tail (recent messages by token budget, ~20K tokens). The middle is serialized with truncation (6K chars per message), then sent to an auxiliary LLM with a structured prompt requesting a handoff summary. On subsequent compressions, the previous summary is passed to the LLM with instructions to update it — moving items from "In Progress" to "Done", adding new facts, and preserving unresolved questions.

Why This Algorithm: Iterative updates prevent the cascading information loss that would occur if each compression started from scratch. A fresh summarization of the most recent middle chunk would forget facts from the previous compression. The iterative approach creates a durable, ever-growing handoff artifact. The summary budget scales with input size: max(2000, min(0.20 * content_tokens, 12000)), capped to avoid the summary itself consuming too much context.

graph TD
    A["Conversation exceeds threshold"] --> B{"Previous summary exists?"}
    B -->|"No"| C["Fresh summarization prompt"]
    B -->|"Yes"| D["Iterative update prompt"]
    C --> E["Prune old tool results"]
    D --> E
    E --> F["Protect head messages"]
    F --> G["Protect tail by token budget"]
    G --> H["Serialize middle turns"]
    H --> I["Call auxiliary LLM"]
    I --> J{"LLM call succeeded?"}
    J -->|"Yes"| K["Insert summary + tail"]
    J -->|"No"| L["Drop middle, enter 600s cooldown"]
    K --> M["Store as _previous_summary"]
    L --> M

Hot Paths

The most performance-sensitive code paths in this module:

• Token estimation (estimate_messages_tokens_rough): Called on every turn for preflight checks — O(n) over message content using a 4-chars/token heuristic. Must be fast since it runs before every API call; avoids tokenizer imports for speed.
• Context compression: O(n) message traversal for pruning + one LLM API call. The LLM call dominates (2-10 seconds); the local pruning and serialization are negligible. Triggered only when threshold is crossed, typically every 15-30 turns.
• Error classification (classify_api_error): O(k) where k is the number of pattern matchers (~50). Runs only on exceptions. Priority ordering means common cases (429, 402) match early and short-circuit.
• Memory prefetch: Runs in parallel across providers; latency is max(provider_latencies). Built-in provider is local file I/O (~1ms); external providers add network latency (50-500ms).
• Prompt caching injection (apply_anthropic_cache_control): O(n) deep copy of messages + O(1) breakpoint insertion. Runs on every Anthropic API call but is fast compared to the API latency itself.

Resource Usage

• Memory: Message history is the primary memory consumer, growing linearly with conversation length. Compression resets this to head + summary + tail (~28K tokens typical). _previous_summary is a single string retained across compressions. CredentialPool and RateLimitTracker maintain small fixed-size state per credential.
• I/O: External I/O is dominated by LLM API calls (1-30 seconds per call). Memory providers add network I/O during prefetch/sync (background). TrajectoryCapture writes structured JSON to disk asynchronously. InsightsEngine queries the local SQLite session database on demand.
• CPU: Minimal — the module is I/O-bound. The only CPU-intensive local operation is regex-based secret redaction in redact.py, which runs O(n) over output text with ~15 compiled patterns.