gap intelligence: model #18 vs market (consensus with #17)

← lab · 21,280 model-vs-market gaps dissected · survival AUC 0.6664 · ran 7/5/2026

What this is: Dissects every disagreement between our models and the market: who was right in which situations, whether two engines agreeing means anything, and whether a classifier can predict when OUR side of the gap wins.

Every model-vs-market disagreement dissected: who was right, where, and can we predict it? model advantage = market log-loss minus model log-loss in the bucket (positive = our number was closer to reality there).

Gap size	Direction	n	Model advantage	Reality	Model said	Market said	Stable
large	model_lower	1,363	-0.0372	0.4754	0.3597	0.4882	⏳ yes
large	model_higher	1,604	-0.03655	0.3834	0.5084	0.3765	⏳ yes
medium	model_lower	3,332	-0.01431	0.3965	0.3238	0.3949	⏳ yes
medium	model_higher	2,921	-0.0116	0.3858	0.4561	0.3846	⏳ yes
small	model_lower	3,574	-0.00584	0.3646	0.3207	0.3553	⏳ yes
small	model_higher	3,056	-0.00321	0.359	0.397	0.3626	⏳ yes
tiny	model_higher	2,546	-0.00095	0.328	0.3531	0.3433	⏳ yes
tiny	model_lower	2,687	-0.00031	0.3502	0.3363	0.346	no

Gap-survival classifier holdout AUC: 0.6664 · top features: rel_gap, prob_market, price_open, gap

Hurricane test over 4,560 shared predictions — Brier: engine A 0.19485, engine B 0.19611, engine mean 0.19346, market 0.19016. Averaging wrong models beats both members, but the market still beats the ensemble.

Reading the columnswhat each number actually means

Model advantage	market log-loss minus model log-loss in the bucket; positive = our probability was closer to what actually happened
Gap size	\|model p − market p\|: tiny <2pp, small <5pp, medium <10pp, large <20pp, huge beyond
Survival AUC	can gap features predict when our side of the gap is right? >0.5 = learnable
Hurricane test	are individually-wrong engines nearly-right on average? Brier: lower = better

Spec · the reproducible recipe

{
  "kind": "gap_intel",
  "model_run": 18,
  "second_run": 17
}