Strategy Audit Summary — findtorontoevents.ca/audit

Strategies Tracked

in strategy_summary table

Total Picks Resolved

7,753

in pick_dimension_snapshot

Documented Strategies

410

in docs/ALL_STRATEGIES.md

Unique Strategies (DB)

702

in trading_picks.strategy

Strategies Sized

T1/T2/T3 (all shadow)

Incidents Open

20 P0, 12 P1, 5 P2

1. Database Tables (6 Dedicated to Strategies)

Table	Rows	Purpose
`strategy_summary`	88	Canonical catalog: PF/WR/DSR/PBO/time-windows per strategy
`pick_dimension_snapshot`	7,753	ALL resolved picks with Score/Trust/AGV/Regime/Edge sub-tags
`pick_funnel_views`	7	Performance by nav-surface (Smart Picks button vs tab, etc.)
`edge_discovery`	23	Pre-computed edge significance (Bonferroni-corrected)
`metric_dimensions`	41	Dictionary of all Score/Trust/AGV/Regime/Edge dimension values
`view_definition_catalog`	10	Documents every dashboard button/filter with its rules

2. What "Shadow" Status Means

Every strategy is currently at "shadow" sizing — meaning zero are approved for real-money allocation.

Shadow means the strategy is tracked and monitored but not trusted for forward capital allocation. A strategy must pass ALL of these thresholds to graduate from shadow:

Tier	Min PF	Min WR	Min n	Min DSR	Max PBO	Max MDD	Description
T1	> 2.0	> 55%	≥ 30	> 0.95	< 0.05	< 10%	Renaissance-grade
T2	> 1.5	> 50%	≥ 30	> 0.90	< 0.10	< 20%	Institutional
T3	> 1.2	> 48%	≥ 20	> 0.80	< 0.20	< 30%	Retail-OK
shadow	Does not meet T3 thresholds — not approved for sizing						Monitor only

DSR (Deflated Sharpe Ratio): Adjusts observed Sharpe for the number of trials tested. Negative DSR means the strategy's in-sample performance doesn't survive statistical adjustment for multiple testing.
PBO (Probability of Backtest Overfitting): Fraction of times the best in-sample strategy ranks in the bottom half out-of-sample. > 0.20 = high overfitting risk.
Key blocker: 62% of trading_picks are TIME_EXIT phantom-closes (exit=entry, pnl=0), which dilute WR/PF metrics and prevent any strategy from meeting thresholds.

3. Top Strategies Per Asset Class

CRYPTO (4,204 picks)

Strategy	PF	WR	n	DSR	PBO	Status
crypto_momentum_high_confidence	0.759	42.4%	2,425	-40.35	0.505	shadow
unknown (unclassified picks)	1.310	51.9%	408	15.34	0.416	—
prediction_market_consensus	1.290	43.6%	94	1.99	0.438	—
ml_enhanced_DYDXUSDT_15m_D_ensemble_stack	1.200	55.9%	34	5.18	0.987	shadow

FOREX (2,335 picks)

Strategy	PF	WR	n	DSR	PBO	Status
forex_carry_trend	0.198	30.1%	675	-23.23	0.450	shadow
fx_smart_carry_trade_momentum	6.213	100%	53	6.88	0.593	—

COMMODITY (929 picks)

Strategy	PF	WR	n	DSR	PBO	Status
commodity_term_structure	1.064	31.6%	247	-1.06	0.300	shadow

EQUITY (88 picks)

Strategy	PF	WR	n	DSR	PBO	Status
stocks_rsi2_pullback	21.508	100%	64	7.69	0.645	shadow
equity_quality_momentum	0.110	34.4%	61	-12.46	0.274	shadow

FUTURES (18 picks), BOND (13 picks), ETF (16 picks)

Insufficient resolved picks (< 20) for statistical validation. These classes need more forward-tested data before DSR/PBO can be meaningfully computed.

4. Pick Sources & Coverage

trading_picks (7,753 resolved)

702 unique strategy names. Top sources by pick count:

Source Type	Picks	Examples
Copy Trader (Polymarket/Bybit)	3,129+	non_crypto_consensus, copy_pm_comtruise, copy_hl_whale_*
Prediction Markets	2,897+	prediction_market_consensus, kalshi_mtf_consensus
AI Tournament Models	3,615	42 unique models (claude_opus_4_7, deepseek_v4, grok3, etc.)
Alpha Engine	~1,000	crypto_, forex_, equity_* strategies
Unknown/Unclassified	2,117	strategy field is NULL or empty

Documentation Gaps

docs/ALL_STRATEGIES.md documents 410 strategies — only 14 match what's in the DB
702 unique strategy names in trading_picks — most are undocumented
2,117 picks have no strategy assigned (UNKNOWN category)
pick_dimension_snapshot now covers 100% of resolved picks (7,753/7,753)

5. AI Tournament (findtorontoevents.ca/audit/ai-tournament.html)

3,615 picks from 42 AI models tracked in tournament_picks table.

Top models by pick count: deepseek_v4 (89), grok3 (89), cursor_agent (59), llama4_scout (57), llm7_qwen (55), minimax_m2_5 (54).

Note: Tournament picks are tracked in a separate table (tournament_picks) with model_id, provider, persona_id, asset_class fields. They are NOT yet linked to the strategy_summary table. This is a known gap — tournament models should be documented as strategies too.

6. Incidents & Enhancements Tracking

Incidents are stored in 20 per-asset-class tables (INCIDENT_* and ENHANCEMENT_*) plus views vw_all_incidents and vw_all_enhancements.

View/Table	Rows	Status Breakdown
`vw_all_incidents`	45	37 OPEN, 1 RESOLVED, 7 TRIAGED
`vw_all_enhancements`	73	Mostly BACKLOG
`INCIDENT_OVERALL`	22	20 P0 OPEN
`ENHANCEMENT_OVERALL`	50	System-wide enhancements

Live page: incidents.html — rendered from DB tables, auto-refreshed nightly.

7. Rigorous Backtest Harness

File: alpha_engine/rigorous_backtest_harness.py

Purged walk-forward (8-fold, 5% purge + 2% embargo)
Deflated Sharpe Ratio (DSR) — adjusts for multiple testing
Probability of Backtest Overfitting (PBO) — measures IS/OS divergence
Cost/slippage modeling per asset class
69 existing strategies backtested + 7 world-class designs tested

8. Next Steps to Reach World-Class

Fix TIME_EXIT phantom-closes — 62% of picks are zero-PnL exits diluting metrics
Fix EXPIRED→WON mislabels — inflates WR artificially (tracked as P0 incident)
Classify unknown picks — 2,117 picks have no strategy assigned
Link tournament models to strategies — 42 AI models need strategy documentation
Document all 702 strategies — only 14 of 410 documented strategies match DB
Increase sample sizes — ETF/FUTURES/BOND have < 20 resolved picks each
Re-run backtests — after data fixes, re-test through rigorous harness

9. Key Files

File	Purpose
`docs/ALL_STRATEGIES.md`	Central strategy repository (410 strategies documented)
`alpha_engine/rigorous_backtest_harness.py`	Backtest harness (purged WF + DSR + PBO + costs)
`alpha_engine/new_strategies/world_class_strategies.py`	7 world-class strategy designs (one per asset class)
`tools/build_metric_dimension_tracking.py`	Populates all 6 tracking tables from DB
`reports/STRATEGY_ROADMAP_COMPREHENSIVE_2026-05-29.md`	Path to investable strategies
`.github/workflows/strategy-funnel-hourly.yml`	Hourly refresh of strategy_funnel_data.json

📊 Strategy Audit Summary