gap intelligence: model #20 vs market (consensus with #17)

← lab · 21,280 model-vs-market gaps dissected · survival AUC 0.6778 · ran 7/6/2026

What this is: Dissects every disagreement between our models and the market: who was right in which situations, whether two engines agreeing means anything, and whether a classifier can predict when OUR side of the gap wins.

Every model-vs-market disagreement dissected: who was right, where, and can we predict it? model advantage = market log-loss minus model log-loss in the bucket (positive = our number was closer to reality there).

Gap size	Direction	n	Model advantage	Reality	Model said	Market said	Stable
large	model_lower	1,586	-0.04565	0.483	0.3504	0.4813	⏳ yes
large	model_higher	1,777	-0.04481	0.3748	0.5171	0.3831	⏳ yes
medium	model_lower	3,515	-0.02048	0.3991	0.3165	0.3878	⏳ yes
medium	model_higher	3,109	-0.01393	0.377	0.4519	0.3802	⏳ yes
small	model_higher	2,874	-0.00375	0.3619	0.3976	0.3632	⏳ yes
small	model_lower	3,286	-0.00215	0.3478	0.3198	0.3543	⏳ yes
tiny	model_higher	2,339	-0.00069	0.3442	0.3573	0.3476	⏳ yes
tiny	model_lower	2,532	-0.00067	0.3424	0.3263	0.3362	no

Gap-survival classifier holdout AUC: 0.6778 · top features: rel_gap, price_open, prob_market, gap

Hurricane test over 4,560 shared predictions — Brier: engine A 0.19805, engine B 0.19611, engine mean 0.19499, market 0.19016. Averaging wrong models beats both members, but the market still beats the ensemble.

Reading the columnswhat each number actually means

Model advantage	market log-loss minus model log-loss in the bucket; positive = our probability was closer to what actually happened
Gap size	\|model p − market p\|: tiny <2pp, small <5pp, medium <10pp, large <20pp, huge beyond
Survival AUC	can gap features predict when our side of the gap is right? >0.5 = learnable
Hurricane test	are individually-wrong engines nearly-right on average? Brier: lower = better

Spec · the reproducible recipe

{
  "kind": "gap_intel",
  "model_run": 20,
  "second_run": 17
}