Factor & Strategy Lifecycle

This chapter is generated from the canonical platform doc, which lives in docs/ (single source of truth).

Factor & Strategy Lifecycle

How a factor is born, earns evidence, gets validated, becomes part of a strategy, and how a strategy is created and promoted to live trading.

Last updated: 2026-06-02

There are two parallel lifecycles — the factor lifecycle and the strategy lifecycle. A strategy is a thin wrapper that points at one validated factor and applies a generic portfolio mechanic to it. The factor lifecycle gates the strategy lifecycle: a strategy may not promote to production until its underlying factor is validated.

1. A factor is born — a YAML declaration

Everything starts as a file: config/factors/<factor_id>.yaml. A real example (config/factors/ma_50d.yaml):

factor_id: ma_50d
status: experimental        # ← lifecycle state
kind: atomic                # atomic (one _f column) | composite (recipe over factors)
nightly_enabled: false
source_column: ma_50d_f     # lifts straight from gold.fact_eod_features
source_table: fact_eod
style: momentum
research:
  ic_horizons: [1, 5, 10, 21, 63]
  promotion_mcpt_threshold: 0.05   # ← the gate knobs live here

Two kinds of factor:

  • Atomic — lifts a single pre-computed _f column. (Feature columns end in _f, computed in DuckDB SQL by sbfactors.) ma_50d → ma_50d_f.
  • Composite — declares source_factors + a recipe (linear_blend, ic_weighted_blend, or ols_residualize) that combines other factors.

FactorConfigService loads and validates these. The register_factors step writes one row per factor into the ops.factor registry table with status='experimental'.

Key idea: a factor is first-class and separate from any strategy. It exists, earns evidence, and gets validated entirely on its own — before any strategy references it.

2. The factor lifecycle: experimental → validated → deprecated

The status lives in ops.factor:

Status Meaning
experimental Newly registered. Not yet trusted. Every factor sits here today.
validated Passed the research gates. Usable by strategies in production.
deprecated Past its usefulness; existing strategies grandfathered.

How a factor earns its evidence (nightly research flow)

Each night the research sub-flow (research.py) runs the factor block against every active factor and writes evidence to the ops.research_* tables:

compute_factor_diagnostics  → ops.research_factor_diagnostics  (is it stationary? degenerate?)
compute_ic                  → ops.research_ic_summary          (IC IR: does it predict returns?)
compute_factor_mcpt         → ops.research_factor_mcpt         (is the IC statistically real, not luck?)
+ alphalens, contribution, (research-day) FWER / selection-bias / superiority
  • Diagnostics — stationarity, entropy, dispersion stability, panel unit-root (CIPS). Answers: is this factor well-behaved and still discriminating?
  • IC (Information Coefficient) — cross-sectional Spearman rank correlation between factor values and forward returns; the ic_ir (IC / std of IC) is the signal-to-noise ratio. Answers: does ranking by this factor sort future winners from losers?
  • MCPT (Monte-Carlo Permutation Test) — shuffles factor values across symbols and recomputes IC thousands of times to build a null distribution. The p-value answers: is the observed IC real, or could random noise produce it?

How the status actually advances — F-139 (not yet built)

Today nothing reads the evidence back into the lifecycle, so every factor is stuck at experimental. F-139 — Factor Lifecycle Promotion Automation (design brief) is the service that closes this loop. Each night its FactorPromotionGate will:

  • experimental → validated when diagnostics are healthy AND MCPT p < 0.05 AND |IC IR| > 0.2 at the 21-day horizon.
  • → deprecated when a factor degenerates (or a validated one falls out of band) for 3 consecutive nightlies.

Every verdict is recorded in a new ops.factor_lifecycle_event audit table; every actual transition also writes to the ops.factor_promotion event log.

F-139 is the automated bridge from “we have evidence” to “the status reflects it.” It is the user-visible payoff of the entire research-phase stack.

3. How a factor becomes part of a strategy

A strategy is a (factor_id, mechanic, params) triple. It does not invent its own signal — it applies a generic portfolio mechanic to a factor’s values. A real example (config/strategies/classic-momentum.yaml):

factor_id: momentum_12m_1m        # ← the factor it rides
mechanic: long_short_quantiles    # ← how to turn factor values into positions
params:
  long_q: 10                      # long the top decile
  short_q: 10                     # short the bottom decile
  holding_days: 21
universe_id: us_large_cap
benchmark: SPY

strategy:
  name: classic-momentum
  version: "1.0.0"

backtest:    { from_date: 2024-01-02, capital: 500000, long_only: false, ... }
universe:    { size: 500, min_market_cap_billions: 1.0, exchanges: [NYSE, NASDAQ] }
promotion_gates: { min_elapsed_days: 30, min_green_nightlies: 5, sharpe_min: 1.1, max_drawdown_max: -0.20 }

The mechanic comes from the StrategyMechanic enum (sbcontracts.strategy_config) — v1 ships long_short_quantiles, long_only_top_n, dollar_neutral_rank. The factor supplies the ranking signal; the mechanic supplies the recipe for turning ranks into a portfolio. Swap factor_id, keep the mechanic → a different strategy with zero new signal code.

The hard rule: a strategy may only promote past incubating once its underlying factor’s status is validated. The factor lifecycle gates the strategy lifecycle — validated factors are the only legal inputs to production strategies.

4. How a strategy is created and its own lifecycle

Creating a strategy = write the YAML. Per CLAUDE.md §2 constraint 13, every strategy must have config/strategies/<name>.yaml, or StrategyConfigService raises StrategyConfigNotFoundError. Gate thresholds, backtest parameters, universe filters, and benchmarks are declared in that YAML — never hardcoded in Python.

The strategy then enters its own state machine, enforced by StrategyRegistry:

registered → incubating → paper → live → retired
                  │           ╲
                  └── retired   └── retired       (rollback → reverts to prior version)
Stage What it means
registered Strategy exists in ops.strategy.
incubating Accruing a track record. Every nightly tick_promotion adds a tick event.
paper Promoted to simulated execution (paper-trading backend).
live Real execution backend.
retired Done; or rolled back to a prior version.

What drives strategy promotion

The gates in gates.py, AND-composed and evaluated nightly by tick_promotion:

  • ElapsedMarketDaysGate — accrued ≥ min_elapsed_days tick events.
  • ConsecutiveGreenNightliesGate — the last N nightlies all succeeded.
  • BacktestMetricBandGate — Sharpe ≥ sharpe_min, max drawdown ≤ max_drawdown_max (read from the YAML promotion_gates block).
  • MCPTSignificanceGate — the strategy’s own MCPT p-value. This is a single time-series test on the backtest P&L, distinct from the factor’s cross-sectional MCPT in §2.

Every register/promote/tick/rollback is a row in ops.strategy_promotion, and each transition runs in one DuckDB transaction so the headline ops.strategy row and its ledger event commit together.

5. The whole arc, end to end

config/factors/X.yaml                         config/strategies/S.yaml
   │ register_factors                            │ register
   ▼                                             ▼
ops.factor: status=experimental             ops.strategy: status=registered
   │                                             │
   │  nightly research flow                      │  must wait for factor X = validated
   │  → ops.research_{diagnostics,ic,mcpt}       │
   ▼                                             ▼
[F-139 FactorPromotionGate]  ──validated──►  incubating ─tick×N─► paper ─► live
   │                          (legal input)     │  strategy gates (Sharpe, MCPT, green nightlies)
   ▼                                             ▼
status=validated / deprecated                ops.strategy_promotion ledger

One-line summary: factors are validated cross-sectionally (do their values rank winners vs losers, provably?) and live/die in ops.factor; strategies are validated as portfolios (does this mechanic on this validated factor make money with acceptable risk?) and live/die in ops.strategy. F-139 is the missing automation that moves a factor from experimental to validated so strategies have something legal to ride.

Reference map

Concern Location
Factor declaration config/factors/<id>.yaml, FactorConfigService
Factor registry / lifecycle ops.factor
Factor evidence ops.research_factor_diagnostics, ops.research_ic_summary, ops.research_factor_mcpt
Factor promotion automation F-139 — design brief (planned)
Feature _f column compute sbfactors (DuckDB SQL)
Strategy declaration config/strategies/<name>.yaml, StrategyConfigService
Strategy mechanics StrategyMechanic enum in sbcontracts.strategy_config
Strategy registry / lifecycle StrategyRegistry, ops.strategy, ops.strategy_promotion
Strategy promotion gates gates.py, tick_promotion
Research flow orchestration research.py
Backtest fan-out (F-120) sbbacktest (StrategyBacktestService)

See also: CLAUDE.md §1 (Architecture, factor-centric model), docs/research_spec.md (factor/strategy separation D10), and docs/factors.md (per-factor evidence read).