Text set in serif type with a dotted bar in the margin was drafted by an LLM (claude-opus-5) and sometimes reviewed by the author. The rest is the author's own. How to read this site
alpha-curriculum → schedule-opt → schedopt-v5
schedopt-v5
investigation
Intent
The previous round searched a free-form space of block curricula and ranked what it found by median crossing time. The campaign does not report the median. It reports the 90th percentile over seeds, which is the number that says whether a schedule works for nearly every seed or merely for a typical one, and a median-ranked search is blind to it: two candidates on the previous round's board share a median of 187.4 steps and differ by 87 in their 90th percentile.
This round therefore optimizes the quantity it reports. Each candidate gets its own dispersion parameter, shrunk toward the pooled value by how many of its seeds actually crossed, and two surrogates are fitted over the schedule space, one for location and one for dispersion, so the acquisition can ask how ragged a region is as well as how fast. The search itself is given the half the previous round lacked: trust regions that widen when they find something, shrink when they stop, and are replaced when they collapse, driven unattended across days rather than one round at a time.
The instrument is unchanged from the previous round, so every row of that
board carries forward as training data and the two are directly comparable. A
crossing is still confirmed by re-measuring frozen weights against fresh
minimal pairs rather than by waiting for a second waypoint. The design
is in the repository spec
2026-07-26-schedopt-v5-turbo-driver-design.md, and the campaign log,
including what the first session got wrong, is in V5.md.
Conclusions
The round set out to optimize the number the campaign reports and did not. Its own driver ranked candidates by a fitted lognormal 90th percentile, while every result the campaign states is a Kaplan-Meier one, and on the same seeds those two estimators disagree by enough to reorder the field. Once the confirmation is read, no schedule this round proposed is faster than the one it inherited, and the three that come closest were not the ones the search picked.
On one fresh cohort of 150 seeds shared by every arm, the previous round's single-step schedule reaches the band at 202.1 steps and this round's nominee at 216.2, a paired difference of 14.1 steps with an interval from 5.2 to 20.0 (the confirmation). Three v5 arms match the incumbent without separating from it, and none beats it. The pre-registered mechanism, a lower crossing-time dispersion paying for a later median, does not appear either: the median arrives 7.1 steps later as predicted, and the dispersion difference is −0.006 with an interval from −0.052 to 0.034.
Referenced by (1 direct)
Direct references:
Twenty-seven candidates on the board carry the full 150 seeds on the frozen instrument, so both estimators can be read on identical data. The fitted 90th percentile runs a mean 8.8 steps above the Kaplan-Meier one, with a spread from −0.3 to 20.7, and the size of the gap tracks the fitted dispersion at a correlation of 0.82: the fitted estimator extrapolates furthest past the data exactly where the upper tail is heaviest. The two rank the same twenty-seven candidates at a rank correlation of 0.19, which is close to no agreement at all.
The consequences are concrete. Thirteen of the twenty-seven beat the
incumbent on the fitted estimator and three beat it on the reported one, by
2.7, 1.6 and 0.2 steps, which is smaller than the 2.4 steps the
reported estimator itself moves between cohorts. The incumbent ranks
fourteenth of twenty-seven on the quantity the search optimized and fourth on
the quantity the campaign publishes. free-5134fafb, the candidate
with the best reported 90th percentile on the whole board at 203.9, ranks
nineteenth on the fitted one and was never sent to confirmation.
The fresh cohort settles which of the two carries information. Across the eleven arms taken to confirmation, held-out Kaplan-Meier reproduces search-cohort Kaplan-Meier to a mean of 2.4 steps and the search-cohort fitted value to 6.0, and the search ordering predicts the held-out ordering at a rank correlation of 0.86 when both sides are Kaplan-Meier against −0.34 when the search side is fitted. The search was working. It was aimed at a quantity that does not agree with the one the campaign answers to.
Referenced by (2 direct)
Of the twenty-six schedules measured at both 60 and 150 seeds, the
median crossing time does not move at all, while the reported 90th percentile
rises by a median of 3.0 steps and the fitted one by 9.9. The incumbent
is its own illustration: 198.9 at 60 seeds, 206.6 at 150, and 202.1 on a
cohort that took no part in selecting it. The effect is larger for candidates
promoted because they looked good shallow. free-46763f61 held
the board's nominal top rank through eighteen rounds of the first session on
20 seeds, where it read 200.0; at 150 it reads 210.4, behind the
incumbent it appeared to have beaten.
This matters more than a few steps of bias, because the ladder promotes on these readings. A candidate's fate is decided at 6 or 20 seeds, and at six the mean fitted dispersion across the board is 0.209 against 0.111 at 150, so promotion selects the luckiest shrinkage as readily as the fastest schedule.
Two searches built on different principles now agree on the answer. The previous round drew four hundred curricula at random and its winner holds α at 4.76 for 160 steps and then drops once to 0.81. This round proposed by surrogate over thirty free cells and the three arms that come closest hold α near 4.8 for 150 to 160 steps and then drop to a floor that jitters from cell to cell around 0.8. Written out as levels they look like twenty-five to twenty-nine stages; read as shapes they are the same two-level schedule with jitter on every cell. The one finalist that departs from it, the nominee, falls to zero at 170 steps, climbs back to 4.8 near 230 and descends a second time, and it is the slowest of the finalists on the fresh cohort.
The reading is that hold-high-then-drop-once is not one good schedule among many but the shape of the basin, and that the extra freedom of thirty cells buys nothing measurable over two. What is left to optimize inside it is where the drop falls, which is the one direction the trust regions cannot travel: moving a plateau cell from 4.6 to 1.0 is 0.72 in the unit encoding, against a proposal radius of 0.10 at its widest.
Three things this round leaves for the next one. The first is cheap and
obvious: re-rank the existing board on the reported estimator and confirm
what that says, starting with free-5134fafb, which costs one
confirmation run against a board that already exists. The second is the
ladder, which runs an independent successive-halving pass per round, so
schedules proposed in different rounds never compete and exactly one per
round reaches the full cohort; a candidate that loses its single promotion is
never reconsidered however good the board later says it is. The third is the
isotropic trust region described above. Fitting per-dimension length scales
was measured as a fix and rejected: on the real board it costs 196 times the
fitting time and cross-validates worse on both surrogates, so if the
anisotropy is worth having it has to come from a kernel that ties length
scales across neighbouring cells rather than from thirty free parameters.
Experiments
-
20260727_203158_schedopt_v5_confirm 2026-07-28
The v5 finalists, prior winners, incumbent, and flat controls are re-measured on one fresh paired seed cohort under the frozen crossing-quantile instrument.