10 The Multiplex Data Substrate
Author: Todd B. Adams Reinforces: Proposal §5 — Data Requirements & Sourcing · Reading order: doc 03 of the research-platform package
The claim of this doc. The proposal names three data layers (§5). One of them runs in production here today; the other two are gaps — but gaps with real adjacent machinery already in the codebase, not blank slates. This is the honest map.
10.1 The three proposal layers, mapped to platform reality
| Proposal layer (§5) | Target variables | Platform reality today | Status |
|---|---|---|---|
| Asset pricing & factors (Layer 1) | OHLCV, Fama-French 3/5, momentum, volatility | Fully sourced: point-in-time EOD (gold.fact_eod), a governed factor library (gold.fact_factor_value), FF5+UMD return series (silver.kenneth_french_factors), and an AQR factor-return reference shelf |
BUILT |
| Supply-chain links (Layer 2) | Customer–supplier directionality, revenue % | Not ingested. But the platform already runs a stdlib SEC EDGAR fetch/parse harness (F-319 PIT-integrity audit) — the 10-K extraction path the proposal names | PROPOSED (adjacent: EDGAR machinery BUILT) |
| Institutional ownership (Layer 3) | 13F holdings, share counts, AUM | Not ingested. But FINRA short-interest is sourced for positioning factors (F-299), and sbcrowding computes a return-correlation crowding proxy — both approximations of the holdings-based ground truth |
PROPOSED (adjacent: FINRA + crowding BUILT) |
10.2 Layer 1 — the built layer
This is the load-bearing point of the whole package: the proposal’s hardest data layer to get right — clean, point-in-time, survivorship-corrected factor loadings — is the one the platform already delivers. Details in doc 04. The proposal generates this layer synthetically (§6.1, structural SDEs); here it is empirical and audited.
Beyond raw factors, the platform carries the academic reference series the proposal’s benchmarking needs: - Fama-French 5 + Carhart UMD, daily and monthly, in silver.kenneth_french_factors (F-316) — the exact “French Data Library” source the proposal cites, already ingested. - AQR factor returns (QMJ, BAB, TSMOM) on a committed reference shelf (F-318) for cross-checking constructed factors against published ones. - FRED macro series (term/credit/vol spreads, risk-free curve) in silver.fred_* (F-317) — regime covariates for a phase-transition model.
10.3 Layer 2 — supply chain (PROPOSED)
The proposal’s free-sourcing path is “SEC Edgar Parsing (Form 10-K extraction via NLP).” The platform already has an EDGAR client — a stdlib urllib fetcher with CIK resolution, filings-index parsing, and XBRL companyfacts extraction — built for the F-319 point-in-time integrity audit (EDGAR PIT findings). Extending it from fundamentals verification to customer–supplier edge extraction is a scoped ingestion task, not new infrastructure. This is the least-mature of the three layers and is honestly the largest single data-engineering item in the roadmap (doc 07).
10.4 Layer 3 — institutional ownership (PROPOSED)
The proposal wants 13F holdings to model ownership crowding. The platform approaches the same construct from two directions today: - FINRA short interest (source: finra) is ingested and drives positioning-style factors (days_to_cover, short_pct_float, F-299) — the short side of ownership positioning, point-in-time-snapped to settlement + dissemination lag. - sbcrowding comomentum infers crowding from return correlations rather than holdings — a proxy that the 13F layer would replace with ground truth (doc 05).
13F ingestion reuses the same EDGAR client as Layer 2, so the two gaps share a sourcing path.
10.5 Why the gaps are credible, not hand-waving
Every PROPOSED layer above attaches to a BUILT mechanism: EDGAR ingestion, FINRA sourcing, crowding computation, and the keymap-driven dataset framework (dataset reference) that makes adding a new dataset a declarative config change, not a rewrite. The research adds datasets and a model; it does not add a data platform.
Cross-links: dataset reference · EDGAR PIT findings · proposal §5 · factor layer