v2 — Next-Generation Quantitative Pipeline

Next-Generation Quantitative Pipeline

A work-in-progress quantitative finance pipeline that adds traditional numerical methods to complement the LLM-based agents. Introduces pure-math signal generation, Portfolio Tracking & Performance Metrics" href="../backtesting_doc/portfolio/index.html">portfolio optimization (mean-variance, Black-Litterman, risk parity), systematic risk management, and optimal execution modeling.

🎯 Why This Module Exists

While the v1 system (src/) relies entirely on LLM-based analysis, it has fundamental limitations: LLM calls are expensive ($0.01–$0.10 per agent per stock), non-deterministic (same input can yield different outputs), and slow (seconds per call). The v2 module exists to add a complementary quantitative layer using traditional finance methods. Pure numerical signals—momentum, mean-reversion, volatility—run in milliseconds with deterministic results. Portfolio optimization using the Markowitz mean-variance framework produces mathematically optimal weight allocations. Risk management with Value-at-Risk and drawdown controls provides systematic safeguards. The Almgren-Chriss execution model accounts for market impact—something the v1 system ignores entirely. v2 is designed to eventually either replace or augment v1: LLM agents could generate qualitative signals that feed into v2's quantitative optimizer for the best of both worlds. Currently a work in progress with interfaces defined and partial implementations.

💡 Real-world analogy

If v1 is like asking a panel of human experts for investment advice—subjective, expensive, and slow—then v2 is like building a spreadsheet model with formulas: objective, cheap, and fast. Each expert (LLM agent) charges a consulting fee and takes time to think. A spreadsheet, once set up, recalculates instantly for free. The ideal system uses both: human judgment for qualitative factors (management quality, competitive moats, market sentiment) and math for quantitative optimization (optimal portfolio weights, risk constraints, execution timing).

Files

Sub-packages

~870

Lines of Code

Python

Language (WIP)

Quantitative Finance Portfolio Optimization Work in Progress

Architecture

The v2 pipeline follows a linear flow: raw market data enters through the DataClient protocol, gets transformed into numerical signals, optimized into portfolio weights, checked against risk limits, and finally translated into executable trade orders. Each stage is a separate sub-package with clean interfaces.

flowchart LR
  subgraph Data["data/"]
    DC["DataClient Protocol"]
    FD["FDClient"]
  end
  subgraph Signals["signals/"]
    BS["BaseSignal ABC"]
    SR["SignalResult"]
  end
  subgraph Portfolio["portfolio/"]
    PO["Optimizer"]
    PT["PortfolioTarget"]
  end
  subgraph Risk["risk/"]
    RM["RiskManager"]
  end
  subgraph Execution["pipeline/"]
    EX["Execution"]
    TO["TradeOrder"]
  end
  subgraph BT["backtesting/"]
    BE["Engine"]
  end
  DC --> BS
  FD -.->|implements| DC
  BS --> SR --> PO
  PO --> PT --> RM
  RM --> EX --> TO
  TO --> BE

Data Flow

flowchart TD
  API["Financial Datasets API"] --> FDC["FDClient"]
  FDC --> Prices["OHLC Prices"]
  FDC --> Metrics["Financial Metrics"]
  FDC --> News["Company News"]
  Prices --> SIG["Signal Computation"]
  Metrics --> SIG
  SIG --> QS["QuantSignals per ticker"]
  QS --> OPT["Mean-Variance Optimizer"]
  OPT --> TGT["PortfolioTarget weights"]
  TGT --> RISK["Risk Manager: VaR + Drawdown"]
  RISK -->|approved| EXEC["Execution: Almgren-Chriss"]
  RISK -->|rejected| SCALE["Scale Down Positions"]
  SCALE --> EXEC
  EXEC --> ORDERS["TradeOrder list"]
  ORDERS --> BACK["Backtesting Engine"]

🚶 Step-by-Step Narrated Walkthrough

Follow a single portfolio rebalance for AAPL, GOOGL, MSFT as it flows through the v2 pipeline:

Data Fetch — FDClient.get_prices() pulls 1 year of OHLC bars from the Financial Datasets API. Retries on 429 with exponential backoff (5s, 15s, 30s).
Metrics Fetch — get_financial_metrics() retrieves TTM fundamentals: P/E, EV/EBITDA, margins. Returns Pydantic models, never raises.
Signal Computation — Each BaseSignal subclass (momentum, value, etc.) calls compute(ticker, end_date) and returns a SignalResult with a score in [-1, +1]. Helpers like _compute_rsi() and _sigmoid() handle the math.
Signal Aggregation — Individual signals are collected into a QuantSignals object per ticker, with an optional composite_score.
Portfolio Optimization — The optimizer takes expected returns (from signals) and a covariance matrix, then solves for weights that maximize the Sharpe ratio subject to constraints.
Risk Check — The risk manager validates: position-level concentration, portfolio VaR, max drawdown. Positions exceeding limits get scaled down.
Order Generation — Target weights are compared to current holdings. Differences become TradeOrder objects with action, shares, estimated cost.
Execution Simulation — Almgren-Chriss model estimates market impact and optimal execution schedule. Large orders are sliced to minimize price impact.
Backtesting — The vectorized backtester replays the strategy historically with a 10bps fixed cost model and point-in-time data constraints.

Key Data Models

All data structures are Pydantic BaseModel subclasses defined in v2/models.py—a single source of truth for the entire pipeline.

Model	Purpose	Key Fields
`SignalResult`	Output of one quantitative signal	`signal_name`, `value` (-1 to +1), `z_score`, `percentile`, `components`
`QuantSignals`	All signals for one ticker on one date	`ticker`, `date`, `signals` dict, `composite_score`
`PortfolioTarget`	Optimizer output — target weights	`weights` (ticker → float), `expected_return`, `expected_risk`
`TradeOrder`	A single trade to execute	`ticker`, `action` (buy/sell/short/cover), `shares`, `price`, `estimated_cost`
`ExecutionResult`	Batch execution output	`orders` list, `total_cost`

Model Code Example

class SignalResult(BaseModel):
    """Output of a single quantitative signal."""
    signal_name: str = Field(description="e.g. 'momentum', 'earnings_surprise'")
    value: float = Field(description="Signal strength from -1.0 (bearish) to +1.0 (bullish)")
    z_score: float | None = None
    percentile: float | None = None  # 0-100
    components: dict[str, float] = Field(default_factory=dict)
    metadata: dict[str, Any] = Field(default_factory=dict)

class PortfolioTarget(BaseModel):
    """Output of the portfolio optimizer -- target weights."""
    weights: dict[str, float] = Field(
        default_factory=dict, description="ticker -> target weight (-1 to +1)"
    )
    expected_return: float | None = None
    expected_risk: float | None = None

🏗️ Design Decisions

Decision	Choice Made	Alternatives Considered	Why This Approach
Data provider interface	`Protocol` (structural typing)	ABC inheritance, dependency injection	Any class with the right methods works automatically—no need to inherit from a base class. Easier to swap `FDClient` for `YFinanceClient` without touching pipeline code.
Signal output range	Normalized [-1, +1]	Raw scores, percentile ranks, z-scores	Uniform range makes signals directly comparable and composable. Raw z-scores can blow up; percentiles lose magnitude information.
Error handling in data layer	Return empty list / None, never raise	Raise exceptions, Result monad	Pipeline must be resilient to partial data failures. One API error shouldn't crash the entire portfolio rebalance. Missing data gets default-scored rather than causing exceptions.
Models layer	Single `models.py` file	Per-package model files	Single source of truth prevents circular imports and makes the full schema visible in one place. At ~70 LOC, it doesn't warrant splitting.
API client retry strategy	Fixed delays (5s, 15s, 30s)	Exponential backoff, jitter	Financial Datasets API rate limits are predictable. Fixed delays are simpler to reason about and debug. Three retries cover transient 429s without excessive waiting.

🔬 Algorithm Spotlight: Mean-Variance Optimization

Markowitz Mean-Variance (Planned)

The Problem: Given N stocks with expected returns and a covariance matrix, find the portfolio weights that maximize risk-adjusted return (Sharpe ratio).

The Approach: Formulate as a quadratic optimization: minimize w'*Sigma*w (portfolio variance) subject to w'*mu >= target_return and sum(w) = 1. The efficient frontier traces the set of optimal portfolios across all target returns. The tangency portfolio (max Sharpe) is the point where a line from the risk-free rate is tangent to the frontier.

Why This Algorithm: Mean-variance is the foundation of Modern Portfolio Theory. Black-Litterman and risk parity (also planned for v2) are extensions that address its sensitivity to input estimates. Plain mean-variance tends to produce extreme weights when expected return estimates are noisy—which is why v2 plans to add eigenvalue cleaning via the Marchenko-Pastur distribution to denoise the covariance matrix before optimization.

graph TD
    R["Expected Returns mu"] --> QP["Quadratic Program"]
    C["Covariance Matrix Sigma"] --> CLEAN["Eigenvalue Cleaning"]
    CLEAN --> QP
    QP --> EF["Efficient Frontier"]
    EF --> TP["Tangency Portfolio: max Sharpe"]
    TP --> W["Optimal Weights w"]
    W --> CONS["Apply Constraints"]
    CONS --> FINAL["Final PortfolioTarget"]

Trace through an example

Given: 3 stocks (AAPL, GOOGL, MSFT) with expected returns [0.12, 0.10, 0.08] and a 3x3 covariance matrix.

Step 1: Clean the covariance matrix—remove eigenvalues below the Marchenko-Pastur threshold (noise from finite sample size).

Step 2: Solve the QP: minimize w'*Sigma*w subject to w'*mu >= 0.10, sum(w) = 1, 0 <= w_i <= 0.4 (40% max position).

Step 3: Result: w = [0.40, 0.35, 0.25] — AAPL gets max weight due to highest Sharpe. Without the max-position constraint, naive MVO would put 100% in AAPL.

Result: PortfolioTarget(weights={"AAPL": 0.40, "GOOGL": 0.35, "MSFT": 0.25}, expected_return=0.104, expected_risk=0.18)

Data Client & Protocol

The data layer uses Python's Protocol for structural typing—any class implementing the right method signatures is automatically a valid DataClient, no inheritance needed.

@runtime_checkable
class DataClient(Protocol):
    """Protocol that all data providers must satisfy.
    Methods return empty lists or None on failure -- never raise."""

    def get_prices(self, ticker: str, start_date: str, end_date: str, **kwargs) -> list[Price]: ...
    def get_financial_metrics(self, ticker: str, end_date: str, ...) -> list[FinancialMetrics]: ...
    def get_news(self, ticker: str, end_date: str, ...) -> list[CompanyNews]: ...
    def get_insider_trades(self, ticker: str, end_date: str, ...) -> list[InsiderTrade]: ...
    def get_company_facts(self, ticker: str) -> CompanyFacts | None: ...
    def get_earnings(self, ticker: str) -> Earnings | None: ...

The concrete FDClient implements this protocol with retry logic (5s/15s/30s on HTTP 429), session-based connection pooling, and context-manager support.

Signal Framework

All quantitative signals extend BaseSignal, an abstract base class that enforces a consistent interface and provides shared math utilities:

class BaseSignal(ABC):
    @property
    @abstractmethod
    def name(self) -> str: ...

    @abstractmethod
    def compute(self, ticker: str, end_date: str, *, api_key: str | None = None) -> SignalResult: ...

    # Shared helpers available to all signals:
    @staticmethod
    def _safe_float(value, default=0.0) -> float: ...     # NaN/None guard
    @staticmethod
    def _percentile_rank(value, values) -> float: ...      # 0-100 rank
    @staticmethod
    def _normalize_to_signal(raw, low=-1.0, high=1.0): ... # clamp to range
    @staticmethod
    def _sigmoid(x, scale=5.0) -> float: ...               # tanh mapping
    @staticmethod
    def _compute_rsi(prices, period=14) -> float: ...      # RSI indicator

Configuration & Knobs

Parameter	Location	Default	Description
`FINANCIAL_DATASETS_API_KEY`	Environment variable	none	API key for Financial Datasets. Falls back to empty string.
`timeout`	`FDClient.__init__`	`30.0`	HTTP request timeout in seconds.
`_RETRY_DELAYS`	`FDClient`	`(5, 15, 30)`	Fixed retry delays on 429 rate limit responses.
Signal range	`BaseSignal._normalize_to_signal`	`[-1.0, +1.0]`	All signals normalized to this range.
RSI period	`BaseSignal._compute_rsi`	`14`	Lookback period for RSI calculation.
Sigmoid scale	`BaseSignal._sigmoid`	`5.0`	Controls steepness of tanh mapping.
Backtest cost model	`backtesting/engine.py`	`10 bps`	Fixed transaction cost per trade.

⚠️ What Could Go Wrong

Failure Mode	Trigger	Detection	Recovery / Mitigation
API rate limiting	Burst of requests exceeding plan limits	HTTP 429 response	`FDClient._request()` retries 3 times with fixed delays (5s, 15s, 30s). After exhaustion, returns `None` and logs a warning.
Missing price data	New IPO, delisted stock, API outage	Empty list from `get_prices()`	Signal returns default score (0.0). `_safe_float()` guards against NaN propagation throughout the pipeline.
Singular covariance matrix	Too few observations, perfectly correlated assets	Optimization solver failure	Planned: eigenvalue cleaning removes noise eigenvalues. Fallback to equal-weight portfolio.
Extreme signal values	Market crash, data anomaly	Raw values outside expected range	`_normalize_to_signal()` clamps to [-1, +1]. `_sigmoid()` provides smooth compression via tanh.

Directory Map

Source tree for v2

v2/
├── __init__.py                  ← Package docstring, top-level exports
├── models.py                   ← All Pydantic models (single source of truth)
├── data/                       ← Data provider layer
│   ├── protocol.py             ← DataClient Protocol (structural typing)
│   ├── client.py               ← FDClient: Financial Datasets API wrapper
│   └── models.py               ← Price, FinancialMetrics, CompanyNews, etc.
├── signals/                    ← Quantitative signal generation
│   ├── base.py                 ← BaseSignal ABC + shared math helpers
│   └── ...                     ← Concrete signals (momentum, value, etc.)
├── portfolio/                  ← Portfolio construction
│   └── optimizer.py            ← Mean-variance, Black-Litterman, risk parity
├── risk/                       ← Risk management
│   └── manager.py              ← VaR, drawdown controls, position sizing
├── pipeline/                   ← Orchestration and execution
│   └── execution.py            ← Almgren-Chriss optimal execution
└── backtesting/                ← Strategy validation
    └── engine.py               ← Vectorized backtester, 10bps cost model