backtesting/engine/ — Simulation Engine & Controller

Orchestrates day-by-day historical simulation with the full agent pipeline

The BacktestEngine orchestrates the full backtest run — date range iteration, agent invocation, trade execution, and metric aggregation. The AgentController normalizes agent I/O for each simulation step. The TradeExecutor translates Portfolio Tracking & Performance Metrics" href="../portfolio/index.html">portfolio-manager decisions (buy/sell/short/cover) into actual position changes. Together they create a realistic historical simulation with no look-ahead bias.

3
Classes
4
Files
~0.5 k
Lines
5
Actions

📚 What is this & why does it exist?

The engine is the core loop of the backtesting system. It iterates through each trading day in a historical range, running the complete LangGraph agent pipeline as if it were live — agents can only see data up to that day, preventing future-data leakage. The controller manages agent invocation and output normalization, the engine manages the day loop (fetch prices, call agents, execute trades, compute value), and the trader translates decisions into portfolio changes. This three-layer separation keeps each concern clean, testable, and independently replaceable.

💡 Real-world analogy

Think of a time machine for trading. The engine "travels" to each historical date, runs the AI agents as if they were making real-time decisions, then "travels" forward to the next business day. The BacktestEngine is the time machine itself — it transports you to each date, sets the scene (fetches prices, prepares the portfolio snapshot), and records what happened. The AgentController is the machine operator — it programs the destination parameters (which tickers, which analysts, which date range) and ensures the crew follows protocol. The TradeExecutor is the crew member who actually pulls the trading lever at each stop, converting abstract decisions ("buy 50 shares of AAPL") into concrete portfolio mutations.

🤔 Why Is It Designed This Way?

The engine follows a pipeline-of-components pattern where each class owns exactly one responsibility. The BacktestEngine could have been a single monolithic class that fetches data, calls agents, executes trades, and computes metrics all in one loop body. Instead, it delegates to specialized components — AgentController for agent I/O normalization, TradeExecutor for position mutations, PerformanceMetricsCalculator for analytics, and OutputBuilder for display. This decomposition was driven by testability: each component can be unit-tested in isolation with mock inputs. The trade-off is more indirection (you must trace across 4+ files to understand one day's flow), but this is acceptable because the flow is strictly linear — no callbacks, no branching pipelines.

Design ChoiceWhat Was ChosenWhat Was RejectedWhy
Loop granularityDay-by-day iteration with pd.date_range(freq="B")Event-driven / tick-levelAgent pipeline is expensive (LLM calls); daily resolution matches typical hedge fund rebalancing frequency
Data prefetchBulk prefetch all tickers before loopLazy per-day fetchingAPI calls are cached; prefetching warms the cache so the loop body never blocks on network I/O
Agent normalizationDedicated AgentController classInline normalization in engine loopAgents can return inconsistent outputs (missing tickers, string quantities); a dedicated normalizer keeps the engine loop clean
Trade execution modelSynchronous, sequential per-tickerBatch / atomic multi-ticker executionSimulates real-world order-by-order execution; partial fills are handled naturally via cash checks

Contents

Architecture

flowchart TD
  CLI["cli.py -- CLI Entry Point"] --> Engine["BacktestEngine -- Day Loop Orchestrator"]
  Engine --> Prefetch["_prefetch_data -- Warm API Cache"]
  Engine --> DateLoop["pd.date_range -- Business Day Iterator"]
  DateLoop --> PriceFetch["get_price_data -- Current Day Prices"]
  PriceFetch --> Controller["AgentController.run_agent -- Invoke and Normalize Agent"]
  Controller --> AgentFn["agent callable -- LangGraph Pipeline"]
  AgentFn --> Controller
  Controller --> Executor["TradeExecutor.execute_trade -- Position Mutations"]
  Executor --> Portfolio["Portfolio -- State Manager"]
  Portfolio --> Valuation["calculate_portfolio_value -- Daily NAV"]
  Valuation --> Metrics["PerformanceMetricsCalculator -- Sharpe, Drawdown"]
  Valuation --> Output["OutputBuilder -- Table Rows and Display"]
  Metrics --> Engine

Data Flow

sequenceDiagram
  participant CLI as cli.py
  participant Eng as BacktestEngine
  participant AC as AgentController
  participant Agent as LangGraph Agent
  participant TE as TradeExecutor
  participant PF as Portfolio
  participant Val as Valuation
  participant Perf as MetricsCalc

  CLI->>Eng: run_backtest()
  Eng->>Eng: _prefetch_data() for all tickers + SPY
  loop Each business day
    Eng->>Eng: Fetch current_prices via get_price_data
    Eng->>AC: run_agent(agent, tickers, dates, portfolio)
    AC->>Agent: agent(tickers, start, end, snapshot)
    Agent-->>AC: AgentOutput (decisions + signals)
    AC-->>Eng: Normalized AgentOutput
    loop Each ticker
      Eng->>TE: execute_trade(ticker, action, qty, price, portfolio)
      TE->>PF: apply_long_buy / apply_long_sell / apply_short_open / apply_short_cover
      PF-->>TE: executed quantity
    end
    Eng->>Val: calculate_portfolio_value(portfolio, prices)
    Val-->>Eng: total_value
    Eng->>Perf: compute_metrics(portfolio_values)
    Perf-->>Eng: PerformanceMetrics
  end

Key Classes & Functions

Class / FunctionFilePurpose
BacktestEngineengine.pyMain orchestrator: initializes components, runs the day-by-day loop, aggregates results
BacktestEngine.__init__engine.pyAccepts agent callable, tickers, dates, capital; creates Portfolio, TradeExecutor, AgentController, MetricsCalculator, OutputBuilder, BenchmarkCalculator
BacktestEngine.run_backtestengine.pyCore loop: prefetches data, iterates business days, calls agent, executes trades, computes value and metrics
BacktestEngine._prefetch_dataengine.pyWarms API cache for all tickers (prices, financials, insider trades, news) and SPY benchmark
BacktestEngine.get_portfolio_valuesengine.pyReturns the historical equity curve as list of PortfolioValuePoint
AgentControllercontroller.pyInvokes the trading agent and normalizes its output into consistent AgentDecisions
AgentController.run_agentcontroller.pyConverts Portfolio to snapshot, calls agent, normalizes decisions (coerces action to Action enum, quantity to float, fills missing tickers with hold/0)
TradeExecutortrader.pyStateless trade dispatcher — routes action to the correct Portfolio method
TradeExecutor.execute_tradetrader.pyDispatches buy/sell/short/cover to Portfolio; returns 0 for hold or invalid actions
Actiontypes.pyEnum with values: buy, sell, short, cover, hold — used for input validation
AgentOutputtypes.pyTypedDict with decisions (per-ticker action+quantity) and analyst_signals
PortfolioValuePointtypes.pyTypedDict for a single equity-curve data point (date, value, exposures)
PerformanceMetricstypes.pyTypedDict for computed metrics: sharpe, sortino, max drawdown, exposures
maincli.pyCLI entry point: parses args, interactive prompts for analysts/model, constructs and runs BacktestEngine

Data Structures

TypeNameDescription
EnumAction5-value string enum: buy, sell, short, cover, hold
LiteralActionLiteralType alias for action string literals — backward-compatible with pre-enum code
TypedDictAgentDecisionPer-ticker decision: {action: ActionLiteral, quantity: float}
DictAgentDecisionsMapping of ticker to AgentDecision
TypedDictAgentOutputComplete agent response: {decisions, analyst_signals}
TypedDictPortfolioValuePointDaily snapshot: Date, Portfolio Value, Long/Short/Gross/Net Exposure, L/S Ratio
TypedDictPerformanceMetricsAggregated metrics: sharpe_ratio, sortino_ratio, max_drawdown, exposures
TypedDictPortfolioSnapshotFull portfolio state passed to agents: cash, margin, positions, realized gains

Interactions

DirectionPartnerInterfaceDescription
→PortfolioPortfolio.get_snapshot()AgentController obtains a serializable snapshot to pass to the agent callable
→PortfolioPortfolio.apply_long_buy() etc.TradeExecutor dispatches trade actions to mutate portfolio state
→Valuationcalculate_portfolio_value()Engine computes total NAV after each day's trades
→Valuationcompute_exposures()Engine records long/short/gross/net exposure for each day
→MetricsCalculatorcompute_metrics()Engine updates performance metrics when 3+ data points exist
→OutputBuilderbuild_day_rows()Engine generates display rows for each day's results
→BenchmarkCalculatorget_return_pct()Engine fetches SPY buy-and-hold return for comparison
→API layerget_prices(), get_price_data()Engine and prefetch fetch historical price data via the tools API
←cli.pyBacktestEngine() constructorCLI constructs engine with parsed args (tickers, dates, model, analysts)

Configuration

Knob / ParameterDefaultDescription
initial_capital100000Starting cash for the simulated portfolio
margin_requirement0.0Margin ratio for short positions (0 = no margin required)
start_date1 month agoFirst date of simulation (YYYY-MM-DD)
end_datetodayLast date of simulation (YYYY-MM-DD)
tickers(required)Comma-separated list of stock symbols to trade
selected_analystsinteractiveWhich analyst agents to include in the pipeline
model_name / model_providerinteractiveLLM model for agent inference (supports OpenAI, Anthropic, Ollama)
freq="B"Business daysDate range frequency; hardcoded to business days (no weekends/holidays)

Directory Map

Source tree for engine
backtesting/
├── engine.py              ← BacktestEngine: day-loop orchestrator
├── controller.py          ← AgentController: agent invocation + output normalization
├── trader.py              ← TradeExecutor: action dispatch to Portfolio
├── types.py               ← Action enum, TypedDicts (AgentOutput, PerformanceMetrics, etc.)
└── cli.py                 ← CLI entry point: arg parsing, interactive prompts, engine launch

📑 Annotated Code Walkthrough

The most important code in this sub-module — the core simulation loop inside BacktestEngine.run_backtest():

BacktestEngine.run_backtest() — The Day Loop

def run_backtest(self) -> PerformanceMetrics:
    self._prefetch_data()                                          # [1]

    dates = pd.date_range(self._start_date, self._end_date, freq="B")  # [2]
    if len(dates) > 0:
        self._portfolio_values = [
            {"Date": dates[0], "Portfolio Value": self._initial_capital}  # [3]
        ]

    for current_date in dates:                                     # [4]
        lookback_start = (current_date - relativedelta(months=1)).strftime("%Y-%m-%d")
        current_date_str = current_date.strftime("%Y-%m-%d")
        previous_date_str = (current_date - relativedelta(days=1)).strftime("%Y-%m-%d")

        # Fetch current prices for all tickers
        current_prices: Dict[str, float] = {}
        missing_data = False
        for ticker in self._tickers:
            try:
                price_data = get_price_data(ticker, previous_date_str, current_date_str)
                if price_data.empty:
                    missing_data = True; break
                current_prices[ticker] = float(price_data.iloc[-1]["close"])  # [5]
            except Exception:
                missing_data = True; break
        if missing_data:
            continue                                               # [6]

        agent_output = self._agent_controller.run_agent(           # [7]
            self._agent, tickers=self._tickers,
            start_date=lookback_start, end_date=current_date_str,
            portfolio=self._portfolio, ...)

        for ticker in self._tickers:                               # [8]
            d = agent_output["decisions"].get(ticker, {"action": "hold", "quantity": 0})
            executed_qty = self._executor.execute_trade(
                ticker, d["action"], d["quantity"], current_prices[ticker], self._portfolio)

        total_value = calculate_portfolio_value(self._portfolio, current_prices)  # [9]

        if len(self._portfolio_values) > 3:                        # [10]
            computed = self._perf.compute_metrics(self._portfolio_values)
            if computed:
                self._performance_metrics.update(computed)

    return self._performance_metrics

Line-by-line:

  • [1] Prefetch: Warms the API cache for all tickers and SPY. Without this, every day in the loop would trigger network requests, making the backtest orders of magnitude slower.
  • [2] Business-day range: freq="B" skips weekends and generates only trading days. This ensures the simulation never tries to trade on non-market days.
  • [3] Initial equity point: Seeds the equity curve with starting capital at day 0. Required so that return calculations have a baseline reference point.
  • [4] Day loop: Each iteration represents one simulated trading day. The lookback window is 1 month — agents see only this window of historical data to avoid look-ahead bias.
  • [5] Close price: Uses iloc[-1]["close"] to get the most recent closing price. The query range is [previous_day, current_day] to handle weekends/holidays gracefully.
  • [6] Skip missing data: If any ticker lacks price data for this day, the entire day is skipped rather than trading with partial information. This prevents mismatched position valuations.
  • [7] Agent invocation: The controller normalizes agent output — coercing actions to the Action enum, quantities to floats, and filling missing tickers with hold/0.
  • [8] Trade execution: Each ticker is executed sequentially. The executor dispatches to the correct Portfolio method (buy/sell/short/cover) and returns the actual executed quantity (may be less than requested if insufficient cash).
  • [9] Valuation: After all trades, the portfolio's total value is computed: cash + long positions - short positions. This is appended to the equity curve.
  • [10] Metrics gate: Performance metrics are only computed when 3+ data points exist. Computing Sharpe or Sortino with fewer points would produce meaningless results.

⚠️ Edge Cases & Boundaries

Edge CaseWhat HappensWhere in CodeRisk Level
Missing price data for a tickerEntire day is skipped (continue); no trades executedengine.py:run_backtest L116-130🟡 Medium — could skip many days for illiquid stocks
Agent returns unknown action stringAgentController coerces to Action.HOLD; TradeExecutor returns 0controller.py:run_agent L56-57🟢 Low
Agent returns negative or None quantityCoerced to 0.0; no trade executedcontroller.py:run_agent L50-53, trader.py:execute_trade L18-19🟢 Low
Insufficient cash for buy orderPortfolio buys max affordable quantity (partial fill)portfolio.py:apply_long_buy L99-111🟡 Medium — silent partial fills may surprise users
Empty date range (start == end)Loop body never executes; returns default empty metricsengine.py:run_backtest L99-105🟢 Low
lookback_start equals current_dateDay is skipped to avoid zero-length lookback windowengine.py:run_backtest L111-112🟢 Low

Code Examples

Example: Launching a Backtest from CLI

# Construct and run the engine programmatically
from src.backtesting.engine import BacktestEngine
from src.main import run_hedge_fund

engine = BacktestEngine(
    agent=run_hedge_fund,
    tickers=["AAPL", "GOOGL", "MSFT"],
    start_date="2025-01-01",
    end_date="2025-06-01",
    initial_capital=100000,
    model_name="gpt-4o",
    model_provider="OpenAI",
    selected_analysts=["technical_analyst", "fundamentals_analyst"],
    initial_margin_requirement=0.5,
)

metrics = engine.run_backtest()
print(f"Sharpe: {metrics.get('sharpe_ratio', 'N/A')}")
print(f"Max DD: {metrics.get('max_drawdown', 'N/A')}%")

# Access the full equity curve
values = engine.get_portfolio_values()
for point in values[-3:]:
    print(f"{point['Date']}: ${point['Portfolio Value']:,.2f}")

Example: TradeExecutor Dispatch

from src.backtesting.trader import TradeExecutor
from src.backtesting.portfolio import Portfolio

portfolio = Portfolio(tickers=["AAPL"], initial_cash=50000, margin_requirement=0.5)
executor = TradeExecutor()

# Buy 100 shares at $150
bought = executor.execute_trade("AAPL", "buy", 100, 150.0, portfolio)
# bought = 100 (cash: 50000 - 15000 = 35000)

# Try to short 200 shares at $150 — margin limits apply
shorted = executor.execute_trade("AAPL", "short", 200, 150.0, portfolio)
# margin_required = 200 * 150 * 0.5 = 15000
# available_cash = 35000 - 0 = 35000 (enough for full order)

# Hold action returns 0 with no side effects
held = executor.execute_trade("AAPL", "hold", 50, 150.0, portfolio)
# held = 0

Example: AgentController Normalization

# The controller ensures every ticker has a valid decision,
# even if the agent omits some or returns malformed data
controller = AgentController()
raw_output = agent(tickers=["AAPL", "TSLA"], ...)
# raw_output might be: {"decisions": {"AAPL": {"action": "buy", "quantity": "50"}}}
# Note: TSLA is missing, quantity is a string

normalized = controller.run_agent(agent, tickers=["AAPL", "TSLA"], ...)
# normalized["decisions"]["AAPL"] = {"action": "buy", "quantity": 50.0}
# normalized["decisions"]["TSLA"] = {"action": "hold", "quantity": 0.0}