Roman Prigodskii

HomeTwo papers and three school projects

Research

On this page

Two sole-authored papers, both grown out of evaluating Vertex MMA. One audits its market-efficiency hypotheses with e-values and was accepted as a poster at two NeurIPS 2026 workshops. The other measures the smallest improvement its evaluation can detect, and is a working paper.

author, on both papers
Sole
paper one, at two NeurIPS 2026 workshops
Accepted
paper two
Working paper
walk-forward bouts in paper one, 1,787 of them priced
4,141
walk-forward rows in paper two, under five seeds
3,087

Accepted at two workshops

NewInML, NeurIPS 2026 workshop, poster.

E-Values: From Statistics to ML, NeurIPS 2026 workshop, poster for the shorter version.

Both non-archival.

An e-value audit of 84 pre-registered market-efficiency hypotheses

Read the paperPDF, 287 KBShort versionPDF, 257 KB

At fair prices, six of the 84 clear 20, the bar for one hypothesis.

The 84 pre-registered hypotheses, by e-valueEach mark is one hypothesis on a logarithmic e-value axis from 0.01 to 1,680. At fair prices six reach 20, the bar for a single hypothesis; with the market's margin charged, one does, on the most favourable of five seeds. None reaches 1,680, the bar for all 84 at once. The best reaches 151.the best of the 84: 1510.1110100201,680under0.0120, the bar for one hypothesis1,680, the bar for all 84 at once
The 84 pre-registered hypotheses, by e-valueEach mark is one hypothesis on a logarithmic e-value axis from 0.01 to 1,680. At fair prices six reach 20, the bar for a single hypothesis; with the market's margin charged, one does, on the most favourable of five seeds. None reaches 1,680, the bar for all 84 at once. The best reaches 151.best: 1510.1110100201,680← 10 more under 0.01, off the scalebar for onefor all 84

Point at a mark, or use the arrow keys.

Each mark is one hypothesis, placed at its e-value: its evidence against the market’s price, where 1 is none. Blue marks clear the bar. The one that clears 20 with the margin does so on the most favourable of five seeds; its five-seed median is 12.3. The hatched band runs from the best of the 84, 151, to the bar for all 84 at once, 1,680.

In brief

The null hypothesis is market efficiency: the market’s closing price for a fight, with its margin taken out, is already the right probability. Each of the 84 names a segment where a good market might still be wrong, with the reason it might be. Each is tested with an e-value, a measure of evidence whose expected value stays at 1 or below while the price is right, and read twice: at fair prices, with the margin taken out, and at the prices actually posted. The audit covers 4,141 walk-forward out-of-fold bouts scored by models refit at 32 quarterly origins, 1,787 of which carry an archived closing price, first analysed with paired log-loss and Benjamini-Hochberg and then re-tested with e-values.

All 84 hypotheses, at fair prices and with the margin.

All 84 hypotheses, at fair prices and with the marginEach mark is one pre-registered hypothesis. Horizontally, its e-value at fair prices on a logarithmic axis from 0.01 to 1,680; vertically, the same test with the market's margin charged, on a logarithmic axis from 0.001 to 100. Mark size grows with the number of bouts. Six marks clear 20 at fair prices; one also clears 20 with the margin, at 21.2, on the most favourable of five seeds. None comes near 1,680. Ten marks are under 0.01 at fair prices and stack left of the axis.20201,680, the bar for all 84 at oncee with the margin1001010.10.010.0010.010.1110100201,680e at fair pricesunder0.01
All 84 hypotheses, at fair prices and with the marginEach mark is one pre-registered hypothesis. Horizontally, its e-value at fair prices on a logarithmic axis from 0.01 to 1,680; vertically, the same test with the market's margin charged, on a logarithmic axis from 0.001 to 100. Mark size grows with the number of bouts. Six marks clear 20 at fair prices; one also clears 20 with the margin, at 21.2, on the most favourable of five seeds. None comes near 1,680. Ten marks are under 0.01 at fair prices and stack left of the axis.20201,680, for all 84e with the margin1001010.10.010.0010.010.1110100201,680e at fair prices← 10 under 0.01

Long win streak 4+

Form and momentum, 308 bouts

e = 151 at fair pricese = 21.2 with the margin

Rank 1 of 84 at fair prices

Clears 20 at fair prices and with the margin, on the most favourable of five seeds: its five-seed median is 12.3. The bar for all 84 at once is 1,680.

Clears 20 at fair pricesand with the margin

157 bouts688 bouts

Point at a mark, or use the arrow keys.

Each mark is one pre-registered hypothesis. The horizontal axis is the e-value at fair prices, the vertical axis the same test with the market’s margin charged, both on a log scale. Mark size grows with the number of bouts in the segment. The rules mark e = 20, which rejects at the 5% level for a single pre-specified hypothesis. For a family of 84 the cutoff that is actually licensed is e-BH’s, e = 1,680 at one rejection, and nothing here comes near it. The single mark above the horizontal rule is also the most favourable of five walk-forward seeds: its five-seed median is 12.3, below the rule.

The standard correction did not apply

The 84 slices are overlapping subsets of one pool. Membership correlations across the 3,486 segment pairs run from −0.49 to +0.84, half of them negative, and 83 pairs share no bouts at all. Benjamini-Hochberg needs independence or PRDS; neither is established. On the 1,191-bout discovery window I actually searched, BH rejects four: one in the model’s favour and three against it. Benjamini-Yekutieli and e-BH, whose guarantees do hold here, reject none.

Rejections under BH, then under the corrections that hold here

Membership correlations across the 3,486 segment pairs

−0.49 +0.84 −10+1
Rejections among the 84 on the 1,191-bout discovery window
Benjamini-Hochbergneeds independence or PRDS; neither is established4
Benjamini-Yekutieliits guarantee holds here0
e-BHits guarantee holds here0

in the model’s favouragainst it

A segment written off too early

A segment the fixed-n analysis wrote off at p = 0.30 is still growing on the e-value scale: +0.0143 nats per bout against +0.0174 in the window that selected it, an e-value of 4.81 against a threshold of 20. It is not refuted. It is 100 bouts short at fair prices and 253 with the margin, which is seventeen months, or three and a half years.

The e-value so far, against the bar it still has time to reach
A segment written off too early: its e-value so far, and the bouts it still needsOn the e-value axis the segment stands at 4.81, growing by 0.0143 nats a bout, against a threshold of 20. At fair prices it is 100 bouts short, seventeen months; with the margin, 253 bouts, three and a half years.The e-value so far+0.0143 nats per bout, at fair prices110100201,680the bar, 20not refuted4.81At fair prices100 bouts: seventeen monthsWith the margin253 bouts: three and a half years050100150200250bouts short of 20
A segment written off too early: its e-value so far, and the bouts it still needsOn the e-value axis the segment stands at 4.81, growing by 0.0143 nats a bout, against a threshold of 20. At fair prices it is 100 bouts short, seventeen months; with the margin, 253 bouts, three and a half years.The e-value so far+0.0143 nats per bout, at fair prices11020100the bar, 20not refuted4.81At fair prices100 bouts: seventeen monthsWith the margin253 bouts: three and a half years0100200250bouts short of 20

Priced in full, a rule chosen after the fact falls short

A threshold rule chosen after the shape was visible, paid for by a mixture martingale over the 21 cuts it searched, clears the threshold at 31.28. Widen the mixture to the 21 cuts it could have searched in the opposite direction and it falls to 15.64. Add the other disagreement statistic it could have used and it falls to 10.43.

A rule chosen after the fact, as its search is paid for
What a rule chosen after the fact is worthThree mixture martingales on the e-value axis, each paying for more of the search. At fair prices: 31.28, 15.64, 10.43. With the market's margin: 8.47, 4.23, 2.82. Only the first, at fair prices, clears the threshold of 20.the bar, 20The search it ran31.288.47+ the direction it could have searched15.644.23+ the other disagreement statistic10.432.82110100201,680
What a rule chosen after the fact is worthThree mixture martingales on the e-value axis, each paying for more of the search. At fair prices: 31.28, 15.64, 10.43. With the market's margin: 8.47, 4.23, 2.82. Only the first, at fair prices, clears the threshold of 20.the bar, 20The search it ran31.288.47+ the direction it could have searched15.644.23+ the other disagreement statistic10.432.8211020100

at fair priceswith the market’s margin

With the market’s margin charged, the same three mixtures give 8.47, 4.23 and 2.82, and none of them clears the threshold.

Before any of it was used, the instrument was validated. Under a true null, with the prices and the tests held fixed and only the outcomes resampled, 40,000 replications give an expected e-value of 0.985 ± 0.018 per segment, with none of the 84 above its own alpha.

The headline result is negative: tested against the market’s price across the whole priced pool, the model gives e = 3.6 × 10−4 at fair prices and 3.1 × 10−8 with the margin.

Files

Working paper

Measure the instrument first

Detection floors for model selection and for LLM judges

Read the paperPDF, 126 KB

80% of the only improvement this pipeline’s winner model ever shipped, produced again by a change that does nothing: the same recipe, refit under another random seed.

Abstract

An evaluation reports that a candidate improved a metric. It almost never reports the smallest improvement it could have detected, and those are different claims. This paper measures the second one with a construction that needs no distributional assumption: the null lever, the same recipe refit under a different random seed. It changes nothing real, so everything it produces is instrument noise.

The one change that shipped is hard to tell from a rerun

On a 3,087-row walk-forward pool under five seeds, ten null comparisons manufacture effects up to 0.00207 nats. Re-seeding alone reproduces 80% of the only lever this leg ever shipped, and the pipeline had 56% power against its own shipped result.

Eleven candidates, one candidate scored three ways, and the lever that shipped

Everything measured on the main pool, against the detection floorChange in out-of-fold log-loss in nats, negative is better, from minus 0.006 to 0.006. The detection floor is plus or minus 0.00362. Ten re-seeds of an unchanged recipe produce effects up to 0.00207. The only change the winner model shipped is worth 0.0026, and the largest re-seed is 80% of it. Every mark lies inside the floor. The arrow keys step through the marks.detection floor, −0.00362+0.00362−0.006−0.004−0.00200.0020.0040.006Ten re-seeds of the same recipe, which change nothingEleven candidate levers from one studyOne lever, read three waysThe only change the winner model shippedlargest, 0.00207a re-seed, 80% of it−0.0026← betterworse →
Everything measured on the main pool, against the detection floorChange in out-of-fold log-loss in nats, negative is better, from minus 0.006 to 0.006. The detection floor is plus or minus 0.00362. Ten re-seeds of an unchanged recipe produce effects up to 0.00207. The only change the winner model shipped is worth 0.0026, and the largest re-seed is 80% of it. Every mark lies inside the floor. The arrow keys step through the marks.detection floor, ±0.00362−0.00400.004Ten re-seeds, which change nothingEleven candidate leversOne lever, read three waysThe only change ever shipped: −0.00260.00207a re-seed: 80% of it← betterworse →
The shaded band runs to the detection floor, 0.00362 nats: the smallest true effect this pipeline would detect with 80% power, one-sided at the 5% level. Every lever in the chart sits inside it. The shipped lever, at 0.0026, is larger than the largest effect re-seeding produced (0.00207) and just past the 0.00239 significance boundary, but still inside the band: the pipeline had 56% power against an effect that size.

More seeds help, up to a point

More seeds lower the floor from 0.00362 nats at one seed towards an asymptote at 0.00288. Fifteen seeds get within 2% of it and no budget gets past it, because 63% of the paired noise lives beyond any seed budget. On four degrees of freedom that share is bounded only to [17%, 83%], so it is reported as an estimate and not as a finding.

The detection floor, by the number of seeds

The detection floor, by the number of seedsFrom 0.00362 nats at one seed the floor falls towards an asymptote at 0.00288. It gets within 2% of it at 15 seeds and reaches 0.00292 at twenty.nats0.00300.00320.00340.0036asymptote, 0.002880.00362 nats at one seedfifteen seeds, within 2%15101520seeds
The detection floor, by the number of seedsFrom 0.00362 nats at one seed the floor falls towards an asymptote at 0.00288. It gets within 2% of it at 15 seeds and reaches 0.00292 at twenty.nats0.00300.00320.00340.0036asymptote, 0.002880.00362 nats at one seed15, within 2%15101520seeds

Each result needs its own floor

Recomputed on the 793-row pool where this project’s one large win was actually measured, the floor is 0.0119 rather than 0.00362. The win is 4× its own floor, not 13× another pool’s.

The same win, counted in each floor

The win, on the 793-row pool

Its own floor, 0.0119

4×

Another pool’s floor, 0.00362

13×

The same test works on an LLM classifier

On a deployed LLM classifier, two identical reruns disagree on 5.6% of items, and twenty of them move measured agreement by 2.6 points. The floor is 2.9 points.

Twenty identical reruns, against the floor

the floor, 2.9 points 2.6 01234

Change in measured agreement, in points.

File

Before the papers

Collatz conjecture

Published, Young Scientist, February 2025

Halve an even number; triple an odd one and add one. The conjecture says every starting number ends at 1, and nobody has proved it. My MYP personal project was an article on it, written in English in Grade 9 with my supervisor Kirill Kusakin: it works through the residues modulo 4 and modulo 3 that a smallest counterexample would have to fall in, and reviews other authors’ approaches.

It was published in Young Scientist (Юный учёный), No. 2 (87), pp. 96–99, and Letovo named it the best MYP personal project in Mathematics.

The article in the journalPDF, 135 KB

The path of a starting number down to 1, on a log scale1101001,00010,000111 steps9,232
The path of a starting number down to 1, on a log scale1101001,00010,000111 steps9,232
27 reaches 1 in 111 steps, climbing as high as 9,232 on the way. Each step halves an even number or triples an odd one and adds one; the vertical axis is logarithmic.

A sausage wrapped in newspaper

School project, 2024

Cut through a sausage wrapped in newspaper, unroll the paper, and the cut edge comes out as a wave. In Grade 9 I proved why it has to be a sine: its period is the sausage’s circumference, 2πr, and its height is set by the angle of the cut. Then I checked it on a real sausage.

The 2026 revision puts a number on that check. Traced from the original scan, the edge follows a sine to within 1.8% of its height, and its height is 0.233 of its period against a predicted 0.243.

A sausage wrapped in newspaperSchool project, revised in 2026. PDF, 1.4 MB

Fit the edge with
A sine misses the traced edge by 1.8% of its height. The paper’s edge, traced from the scan at its true proportions; underneath, the misfit, magnified three times.

The volume of a tea glass

School project, 2025

In Grade 9 I traced a Turkish tea glass with four parabolas, cut it into discs and let the discs get infinitely thin, summing them with the formulas for sums of powers. The glass came out at 216.9 cm³. Lifted out of a measuring beaker full of water, it took between 200 and 225 ml with it.

The 2026 translation corrects one copying error, which had put the total at 208.6. Both numbers sit inside the beaker’s window; the corrected one is the one the method gives.

The volume of a tea glassSchool project, translated in 2026. PDF, 983 KB

Their volumes add up to
213.3 cm³
As the discs get infinitely thin
216.9 cm³

Questions about the papers? Write to me.

roman@prigodskii.dev

Telegram, @prigodskiiGitHub, romanprigodskii