Text set in serif type with a dotted bar in the margin was drafted by an LLM (claude-fable-5, claude-opus-4-8, claude-opus-5) and sometimes reviewed by the author. The rest is the author's own. How to read this site
alpha-curriculum → schedule-opt
schedule-opt
investigation
Intent
This investigation asks a single question inherited from the schedule line of
the alpha-curriculum work: what schedule gets a model to its
generalization peak fastest, and how sure can we be of the answer? The objective, fixed before any search ran, is the probability —
over random seeds — that the generalization score reaches a frozen threshold
within a frozen step budget. The campaign proceeds in gated phases:
calibration of the threshold and budget, a space-filling sweep of a
three-parameter schedule family, Bayesian optimization within it,
confirmation of the winners on held-out seeds never touched during the
search, and a nested test of whether a fourth shape parameter buys anything
real. Only held-out numbers are reported as results. The design and its
statistical reasoning live in the repository spec
2026-07-21-schedule-opt-campaign-design.md.
Conclusions
The campaign answered its question. The schedule that most reliably reaches the generalization band within a fixed budget is a high start that descends through the productive middle and touches the floor only near the end, and on 150 held-out seeds it clears the best fixed α by a wide margin: a descent from α 3.2 to 0.1 crosses on 0.94 of seeds against 0.67 for fixed α = 1.6, a paired difference significant past any doubt (held-out numbers).
The four phases each did one job. Calibration set the objective and flagged an apparent gap between the smooth family and the discrete burst schedule on it (calibration), later shown to be a confound. The space-filling design mapped a mostly empty box with a thin productive ridge and a first winner at 0.92, its ranking stable under resampling (design). The Bayesian search then climbed the ridge in a legible way: a first round of proposals that plunged too fast to the floor failed and taught the surface that the erosion zone is fatal, a correction to high-curvature late descents lifted it to 0.96, and a final round that raised the starting concentration reached an estimated 1.00 on the search pool. The held-out confirmation brought that back to earth by the expected winner's curse, to a true 0.91 to 0.94.
The winning shape connects to the wider curriculum work: build the number rule early at high α through collocational-bootstrapping, then descend. Here that picture is measured as the probability of crossing the band by a fixed budget rather than as final accuracy at the end of a long run, and the fixed-α baseline loses on it not because it is slow but because a third of its seeds never cross the band at all, where nearly every seed of the descent does.
The apparent gap between the smooth family and the burst schedule was settled in two stages that turned out to be two halves of one fact. A matched control (matched control) showed that at the frozen budget the burst and the best smooth descent cross at an identical rate, so the incumbent's edge had been a confound of horizon and seeds. Re-scoring those same trajectories at smaller budgets then inverted the reading: the tie was an artifact of the deadline, because the burst crosses roughly a hundred steps earlier and the smooth descent catches up only near the end. Whether the discrete structure matters depends entirely on whether the objective measures eventual arrival or haste.
The first objective saturated at its own ceiling, with the three held-out winners statistically inseparable, so the campaign re-froze a second operating point chosen from search and calibration data only, a higher threshold at a shorter budget where the incumbent sat mid-scale. The two operating points are never merged, and the re-freeze is reported as a re-freeze. Along the way a proposed longer dwell guard was rejected on measurement grounds: near the threshold the above-below state at consecutive waypoints is statistically independent, so a longer dwell compounds evaluation-timescale noise rather than testing persistence.
At the speed-measuring point the search told a clean story. The smooth family's ascending basin was a dead end. The fourth parameter, re-aimed by the deadline-artifact finding from an S-shaped smooth form to a tooth amplitude riding the ramp, paid immediately on the search pool (the amplitude round) and closed at a toothed late-diving descent (the search close). The held-out confirmation (Step F) then delivered the campaign's final, two-sided verdict: the teeth move the median crossing roughly 70 steps earlier than the smooth descent (the speed result), but the pre-registered crossing-probability bar for the fourth parameter is not met (the bar), and the optimized family exactly matches, without beating, the hand-built burst reference (the equivalence). A realization-horizon artifact in the search-side comparisons, caught because a winner's-curse gap came out negative, is documented with the result (the artifact) and is the concrete vindication of the rule that search-pool numbers are never results.
Read together, the two operating points sharpen the schedule story into one sentence with two clauses: a high-start descent that reaches the floor late is what makes nearly every seed cross at all, and discrete teeth through the productive middle are what make the crossing happen early. Reliability belongs to the envelope; speed belongs to the teeth; the burst the campaign started with already had both.
On the frozen objectives the campaign names one winner and declines to name a second. At the v1 operating point the winner is the smooth descent from α 3.2 to 0.1 with curvature γ 1.84 (cand3), crossing on 0.94 of held-out seeds against 0.67 for the baseline. At the v2 operating point the point-estimate leader is the modulated phi4star (ramp α 3.42 to 0.16, late curvature γ 3.65, tooth amplitude 0.69; selected in the Phase-4 search as r8_c3) at 0.867, but the top three arms — the two toothed schedules phi4star and the hand-built burst schedule burst_16, and the smooth descent phi3star at 0.793 — are pairwise inseparable on 150 seeds (McNemar p between 0.14 and 0.69), so on the crossing probability at the frozen deadline no single best schedule can be declared. What does separate the arms is timing, which is the same objective evaluated at earlier deadlines and was pre-registered as reported context, never as the selection criterion: at a 250-step budget the crossing probability reads 0.72 for phi4star and 0.61 for burst_16 against 0.01 for phi3star, and their median crossing steps are 221 and 218 against 289. A practitioner choosing for speed should take either of the two toothed arms, which the campaign cannot separate from each other; on the frozen objective itself it cannot even place them above the smooth descent with confidence.
| arm | crossing probability | Wilson 95 percent | median crossing step (seeds crossing) |
|---|---|---|---|
| phi4star | 0.867 | 0.803 to 0.912 | 221.3 (130 of 150) |
| burst_16 | 0.847 | 0.780 to 0.896 | 218.0 (127 of 150) |
| phi3star | 0.793 | 0.722 to 0.850 | 289.0 (119 of 150) |
| ascend3 | 0.460 | 0.382 to 0.540 | 266.3 (69 of 150) |
| fixed_16 | 0.367 | 0.294 to 0.446 | 285.6 (55 of 150) |
Referenced by (1 direct)
Direct references:
A third operating point replaced the deadline probability with the crossing time itself, scoring a schedule by the 90th percentile of when its seeds reach the band rather than by how many of them get there by a deadline. That objective has resolution the earlier ones ran out of, and it reordered the campaign's own winners. It also invited a search with no family: a candidate became K blocks of level and duration with K drawn rather than fixed, which is how the campaign's best schedule was found (schedopt_v4).
The round after it spent twenty-one rounds of surrogate-driven local search on the same space and improved nothing, for a reason worth carrying forward into any future round: its driver ranked candidates by a fitted lognormal quantile while every published number is a Kaplan-Meier one, and on identical seeds the two agree at a rank correlation of 0.19 (the estimator mismatch). A search is only as good as the agreement between what it ranks and what you report.
The fastest known schedule holds α at 4.76 for 160 steps and then drops once to 0.81 for the remaining 240 (free-5411cbfe). On 150 held-out seeds it reaches the band at a 90th percentile of 202.1 steps, with a median of 190.7 and one seed in 150 not crossing inside the budget. Measured on that one instrument, every schedule the campaign built by hand sits behind it: 230.0 for the toothed v2 winner, 285.0 for the smooth v2 descent, 331.4 for the v1 descent, and 337.9 for the best constant α at 1.8, with the constant at 1.6 never identifiable because 16% of its seeds do not cross at all (the held-out margins). Against the toothed arm the paired difference is 27.9 steps with an interval from 18.4 to 45.7, so the single step is faster than the teeth on this objective, though the teeth remain faster than the smooth descent they were built against.
Shaping α at all is worth about 136 steps against the best constant α, more than a third of the 400-step horizon, and that margin is the campaign's most robust number: it is the one comparison every round reproduces.
Referenced by (2 direct, 1 transitive)
Transitive (depth 1):
Referenced by (2 direct, 1 transitive)
Direct references:
Transitive (depth 1):
Sub-investigations
-
schedopt-v1 8 experiments
The v1 arm of the schedule-optimization campaign gathers every experiment scored against the original frozen objective: the probability, over random seeds, that unseen_mismatch accuracy reaches m_star = 92.5 within k = 400 steps, guarded by a two-waypoint dwell (
experiments/schedopt/config.json). Its members are the Phase-0 calibration that froze that objective, the Sobol design that first mapped the three-parameter schedule family, the four Bayesian-optimization rounds that climbed it, and the Phase-3 confirmation on held-out seeds. The objective reached saturation here: the winning descent crosses on 0.94 of held-out seeds and the top three candidates are statistically inseparable, which is what motivated re-freezing a second operating point. The two operating points are never merged, and no number recorded under v1 is comparable to a number under v2. -
schedopt-v2 6 experiments
The v2 arm of the schedule-optimization campaign gathers every experiment scored against the re-frozen objective: the probability, over random seeds, that unseen_mismatch accuracy reaches m_star = 95.5 within k = 350 steps, guarded by the same two-waypoint dwell (
experiments/schedopt/config_v2.json). Its members are the Step-D search round that re-searched the three-parameter schedule family at the corrected point, the Phase-4 rounds that added a tooth-amplitude parameter riding the ramp, and the Step-F confirmation on fresh held-out seeds. The operating point was re-frozen only after the Phase-3 confirmation had been seen, and it was chosen from search and calibration data alone so that the held-out comparison stayed honest. Nothing here confirms a v1 result: the re-freeze is always reported as a re-freeze, the two operating points are never merged, and no number recorded under v2 is comparable to a number under v1. -
schedopt-v3 0 experiments
The first two operating points of this campaign scored a schedule by a single number: the probability that a seed's generalization accuracy reaches a frozen threshold by a frozen deadline. That objective answered its question and then ran out of resolution — at the second operating point the three leading arms were pairwise inseparable on crossing probability (The re-frozen campaign: reliability and speed are separable virtues) while differing by roughly seventy steps in median crossing time. This investigation replaces the deadline Bernoulli with a frozen objective on the full distribution of the per-seed crossing time: minimize its 90th percentile under right-censoring at a horizon, so that an arm wins only if at least nine in ten seeds cross at all, and among such arms the one whose slowest decile crosses soonest. The change of objective is a fresh freeze; no number here is comparable to the v1 or v2 operating points, and the earlier winners enter only as reference arms with no incumbency. Four sub-investigations carry the phases: the tail diagnostics that decide whether fixed-concentration training stalls heavy-tailed, the restart baselines that classical theory demands if it does, the staged schedule search with its held-out confirmation, and the mechanism studies that turn a descriptive win into a causal account.
-
schedopt-v4 1 experiment
The first three rounds of this campaign each froze an objective, searched a parameterized family of curricula, and defended a winner. This one drops the family. A candidate is K blocks of α level and duration, with K drawn rather than fixed and the boundaries free integers, so the search space contains constant schedules, monotone descents and oscillations alike as special cases, and contains shapes none of the earlier families could express.
The reason to spend a round this way is that every family used so far was chosen after the fact to contain the arms that were already winning, which makes a search inside it evidence about the family rather than about the space. The node also carries the instrument change that makes the numbers here incomparable to the earlier rounds: a crossing is confirmed by re-measuring frozen weights against fresh minimal pairs rather than by waiting for a second waypoint.
-
schedopt-v5 1 experiment
The previous round searched a free-form space of block curricula and ranked what it found by median crossing time. The campaign does not report the median. It reports the 90th percentile over seeds, which is the number that says whether a schedule works for nearly every seed or merely for a typical one, and a median-ranked search is blind to it: two candidates on the previous round's board share a median of 187.4 steps and differ by 87 in their 90th percentile.
This round therefore optimizes the quantity it reports. Each candidate gets its own dispersion parameter, shrunk toward the pooled value by how many of its seeds actually crossed, and two surrogates are fitted over the schedule space, one for location and one for dispersion, so the acquisition can ask how ragged a region is as well as how fast. The search itself is given the half the previous round lacked: trust regions that widen when they find something, shrink when they stop, and are replaced when they collapse, driven unattended across days rather than one round at a time.
The instrument is unchanged from the previous round, so every row of that board carries forward as training data and the two are directly comparable. A crossing is still confirmed by re-measuring frozen weights against fresh minimal pairs rather than by waiting for a second waypoint. The design is in the repository spec
2026-07-26-schedopt-v5-turbo-driver-design.md, and the campaign log, including what the first session got wrong, is inV5.md.