Text set in serif type with a dotted bar in the margin was drafted by an LLM (claude-opus-4-8) and sometimes reviewed by the author. The rest is the author's own. How to read this site

schedopt_granularity

20260722_084517_schedopt_granularity · complete · published 2026-07-23 · seed 3000

part of investigation alpha-curriculum → schedule-opt → schedopt-v1

Intent

Intent: A matched control: does the burst shape raise the crossing probability, or was its calibration edge a confound? \@{schedopt-granularity-intent} A follow-up control for the schedule-opt campaign. In calibration the burst schedule incumbent crossed the band by the budget more often than any smooth probe, but it ran at a different segment granularity and horizon, 48 segments over 900 steps against the smooth arms' 16 over 400, so its edge confounded shape with those. This experiment removes the confound: the burst shape and the campaign's best smooth schedule are run at the same 16-segment granularity, the same 400-step horizon, and the same seeds, with a 48-segment burst added to check whether the teeth need fine granularity, and a fixed-α arm as an anchor. All are scored on the frozen objective, the probability of crossing 92.5 unseen_mismatch within 400 steps.

Hypothesis

Hypothesis: Matched, the burst edge should mostly vanish \@{schedopt-granularity-hypothesis} At matched granularity, horizon, and seeds the coarse burst and the smooth descent should cross at similar rates, meaning the calibration edge was largely the confound rather than the discrete structure. If the burst retains an edge it should appear only at fine granularity, the 48-segment arm beating the 16-segment one, which would say the teeth matter but need resolution to act.

Setup

model: GPT (nanoGPT-derived), weight-tied embeddings — 2 layers × 4 heads × 256 dim (~1,635,584 params) · block 50 · dropout 0.1 · vocab 167 tokens
training: AdamW (β 0.9/0.95, wd 0.1) · lr 0.0006 constant (no warmup, no decay) · batch 32 · 400 iterations · fresh init per replicate (seeds derived from base seed) · 25 seeds (base 3000)
training data: PCFG, zipfian noun–verb pairing (oneshot where noted) · 12,000 sentences per dataset (9,600 train — 1 epoch = 300 iters) · unseen holdout 10
code: commit abeca0504b65

arm	data regime	datasets	iters/dataset	epochs/dataset
burst_16	fresh-dataset-per-stage	16	25	0.02
burst_48	fresh-dataset-per-stage	48	[8, 9]	[0.01, 0.01]
fixed_16	fresh-dataset-per-stage	16	25	0.02
smooth_16	fresh-dataset-per-stage	16	25	0.02

Replicate this experiment

Get the code at this exact version:

git clone https://github.com/ClaireHobbs/imagining-syntax
cd imagining-syntax
git checkout abeca0504b65
pip install -e ".[dev]"

Download the config (experiment.yaml) and run:

imsyn exp run experiment.yaml

Checks

PASS children_complete PASS replicate_coverage PASS schedule_total_matches_final_waypoint PASS instrument_consistent PASS objective_computable

Results — objective: P(unseen_mismatch ≥ 92.5 by t = 400, dwell 2, smoothing 1)

arm	ŝ	wilson95	median crossing [CI]	crossed
burst_16	0.880	[0.700, 0.958]	212.0 [191.7, 215.9]	22/25
smooth_16	0.880	[0.700, 0.958]	304.8 [298.0, 314.2]	22/25
burst_48	0.760	[0.566, 0.885]	200.7 [196.5, 221.7]	19/25
fixed_16	0.640	[0.445, 0.798]	270.1 [248.2, 309.1]	16/25

Trajectory summary — unseen_mismatch

Evaluation data: seen_* probed at reference α = 1.4 · unseen_* = held-out pairings, uniform (α-independent) · 1000 pairs per condition

Each probe is a minimal pair: a grammatical sentence and its verb-number-flipped twin. The model scores correct when it assigns the grammatical version higher probability; accuracy = % correct over 1000 pairs per condition.
seen_match / seen_mismatch — noun–verb pairings that occur in training, sampled at fixed reference α = 1.4 (not at the schedule's own α, so arms stay comparable)
unseen_match / unseen_mismatch — held-out noun–verb pairings that never occur in training, sampled uniformly (α = 0), so their difficulty is identical across all arms and training αs
match vs mismatch — whether the prepositional objects agree in number with the subject; mismatch places attractor nouns between subject and verb (the AGREE-RECENT trap)

arm	end of training (t = 400)	flags
burst_16	91.1 ±17.6	⚠ seed_split: range 45.9-100.0 (3 low / 21 high of 25)
burst_48	86.4 ±24.2	⚠ seed_split: range 0.6-100.0 (1 low / 17 high of 25)
fixed_16	86.6 ±24.7	⚠ seed_split: range 8.8-100.0 (2 low / 20 high of 25)
smooth_16	86.4 ±20.8	⚠ seed_split: range 19.0-100.0 (1 low / 15 high of 25)

Conclusions

Conclusion: Matched, the burst buys nothing on the objective \@{schedopt-granularity-conclusion} The control resolves the confound. Run at the same segment granularity, the same 400-step horizon, and the same seeds, the burst schedule shape and the campaign's best smooth descent cross the band on an identical 0.88 of seeds, and the paired test finds no difference at all (McNemar p = 1.0, three seeds disagreeing each way). The apparent burst advantage in calibration, 1.00 against the smooth family's 0.94, was therefore the confound and not the teeth: it came from the incumbent's longer 900-step run and different seed set, both of which this experiment holds fixed. On the frozen objective the discrete drop-and-recover structure does nothing the smooth descent does not. The two schedules distribute their crossings differently across the 400 steps, but that is not the objective and has no bearing on it. The objective counts only whether a run has crossed by step 400, so a crossing at step 200 and one at step 399 are the same success, and both schedules bring 22 of 25 seeds across by the deadline. On the one quantity the campaign optimizes, they are equal. Result: Burst equals smooth on the objective at matched conditions \@{schedopt-granularity-tie} At 16 segments over 400 steps on the same 25 seeds, the burst shape and the smooth descent (α 3.2 to 0.1) each crossed 0.88 (22 of 25; Wilson 0.70 to 0.96), against 0.64 for fixed α = 1.6. Paired McNemar between burst and smooth: b = 3, c = 3, p = 1.0. Referenced by (2 direct, 4 transitive)Direct references:The incumbent's calibration edge was a confound, resolved to a tieWhat the campaign settled and what it left openTransitive (depth 1):Start concentrated, hold, then drop onceTransitive (depth 2):Build early, avoid the uniform tail, and land in the working rangeA matched control: does the burst shape raise the crossing probability, or was its calibration edge a confound?Transitive (depth 3):The intermediate peak is real but it is a slice of a moving target Finer teeth do not rescue the burst. Realized at 48 segments over the same 400 steps, eight or nine iterations per segment, the burst crossed on only 0.76 of seeds, below its own 16-segment version and below the smooth arm. Too few iterations per dataset leave each α too little time to act, so higher resolution costs more than it buys. The burst structure gains nothing from fine granularity on this objective. Referenced by (1 direct, 3 transitive)Direct references:Start concentrated, hold, then drop onceTransitive (depth 1):Build early, avoid the uniform tail, and land in the working rangeA matched control: does the burst shape raise the crossing probability, or was its calibration edge a confound?Transitive (depth 2):The intermediate peak is real but it is a slice of a moving target

Comparison figures

Children

Child	peak unseen_mismatch	Status
smooth_16	98.1	done
burst_16	94.5	done
burst_48	90.1	done
fixed_16	86.6	done

experiment.yaml