Hey.

A Prior-Guided LLM Agent for Virtual Biddingv3
What changed in v3: the language model's contribution is reported as position sizing, not directional corroboration. v2 led with corroboration and treated sizing as a second axis. Testing both channels on data published after the evaluation window closed reverses that ordering.
Live scoreboard · the two channels scored forward since 2026-08-06
v2 as traded
 
always-INC × σ^1.5
 
v3 · 5-voter ensemble
 
confidence effect
weights only
direction effect
direction only
v2 as traded always-INC × σ^1.5 v3 · mean-vote × confidence
Loading the live ledger…
Leaderboard · every direction rule, same universe and weights
Each row swaps only the direction. Universe, daily budget and the σ^1.5 × confidence weights are identical, so the column is a clean read on direction alone. bandit* is the Thompson bandit rebuilt around a Normal posterior over the spread rather than a Beta over a win rate: same arms, but it learns the expected payoff instead of which side wins more often. That one change moves it from −$2,519 to −$204.

The voter set was chosen on redundancy, not on this column. Over 91 days v3's daily standard deviation is $243, so a 91-day total carries a standard deviation of $2,317, and every drop-or-add variant tried spanned about $1,500, which is 0.65σ. What separates the priors is how much they agree: bandit* matches mean14 93% of the time, mean10 92% and mean7 89%, because a recency-weighted posterior mean is a trailing mean. Voting four near-duplicates is one voter with extra confidence. So mean10 was dropped as a 92% duplicate, and persistence added as the only prior that genuinely disagrees with the rest, at 56–66% agreement, despite being weak alone.
Day by day
datev2priorv3 cum v2cum priorcum v3
Every settled day since the live deployment began. Daily P&L on the left, running total on the right.
Where the result moves
v2 · what was trading v3 · what trades now VOTERS LLM · Thompson bandit · supervised LR 3 voters LLM · bandit* · mean7 · mean14 · persistence 5 voters WHAT EACH VOTER ESTIMATES which side wins more often — a sign, graded on hit rate the wrong objective the expected spread in dollars, which is what the book is paid the change that matters HOW THEY COMBINE majority vote, ties flat unchanged majority vote, ties flat unchanged POSITION WEIGHT σ^1.5 trailing volatility confidence unused σ^1.5 × LLM confidence the LLM's real job 91 SETTLED DAYS, LIVE worst single day −$2,962 −$4,376 worst single day −$719 +$466
The vote itself is untouched. What changes is what the voters are trying to be right about. v2's three voters all predict a sign and are graded on how often they are right, and on this record the most accurate rule is also the one that loses the most money, because the book is paid in dollars, not in hit rate. v3 keeps the LLM and the bandit but has each statistical voter estimate the expected spread instead, and lets the LLM's confidence set position weight, which is the one thing it measurably does well. bandit* is the same two arms rebuilt around a Normal posterior over the spread instead of a Beta over a win rate.
Channel 1 · corroboration
in sampleout of sample
direction margin+$871+$18
longs retained1,35114
Not a mis-specified veto. The 415 rejected proposals realize a positive-spread rate of 0.443 and would have lost $421 if held. The screen works; there is no directional edge left to keep.
Channel 2 · confidence allocation
in sampleout of sample
effect on always-INC+$5,769+$964
per MWh+$0.092+$0.467
Concentrated where it matters: −$427 over Jun 3 – Jul 31, then +$1,391 over Aug 1 – Sep 6, the stretch in which the premium inverted and the unweighted book lost $7,390.
The benchmark is one episode
always-INC, 63,148 hub-hoursP&Lshare
full evaluation window$108,553100%
January 2026 alone$79,22973.0%
top five days$107,13198.7%
excluding January$29,32327.0%
Every margin in the paper, ours included, is measured against a level that a handful of days determines. v3 states this in the abstract rather than leaving it implicit.
After the window closed · 2026-04-23 to 2026-08-25
strategyevaluation windowafter
always-INC+76,147−2,150
always-INC × σ^1.5+56,455−3,159
supervised LR+74,567−1,990
Thompson TS+68,390−1,879
BTZ-DPDS+62,603+3,255
The forward premium inverts, volatility weighting makes the loss larger rather than smaller, and both statistical corroborators turn negative. Only DPDS, which sizes its own capital, survives.
Out-of-sample LLM figures use the production deployment's logged directions and confidence at HB_NORTH, 2,064 hub-hours over 2026-06-03 to 2026-09-06. That deployment uses a larger model and a different prompt than the archived runs, and covers one hub and one run, so it is a proxy for the evaluated agent rather than that agent.