Text set in serif type with a dotted bar in the margin was drafted by an LLM (claude-fable-5, claude-opus-4-8) and sometimes reviewed by the author. The rest is the author's own. How to read this site

schedopt_bo_r5

20260722_124921_schedopt_bo_r5 · complete · published 2026-07-23 · seed 1000

part of investigation alpha-curriculum → schedule-opt → schedopt-v2

Intent

Intent: Round 5 of the Bayesian search, at the re-frozen operating point \@{schedopt-bo-r5-intent} Step D of the schedopt re-freeze continuation (Phase 2b), round 5: eight candidate schedules proposed by Thompson sampling from a Gaussian process re-fitted to every prior evaluation re-scored at the re-frozen operating point v2 (m_star=95.5, k=350, dwell 2) — the v1 objective saturated, and this round searches the speed-measuring point instead. Same search seed pool (common random numbers); adaptivity lives between rounds only; v1 and v2 numbers are never merged.

Hypothesis

Hypothesis: Each round improves on the incumbent, with shrinking returns \@{schedopt-bo-r5-hypothesis} At least one candidate improves on the current incumbent's crossing probability; the round-over-round gain shrinks as the surface is learned. Search-phase s-hat values are never reported as results.

Setup

model: GPT (nanoGPT-derived), weight-tied embeddings — 2 layers × 4 heads × 256 dim (~1,635,584 params) · block 50 · dropout 0.1 · vocab 167 tokens
training: AdamW (β 0.9/0.95, wd 0.1) · lr 0.0006 constant (no warmup, no decay) · batch 32 · 350 iterations · fresh init per replicate (seeds derived from base seed) · 25 seeds (base 1000)
training data: PCFG, zipfian noun–verb pairing (oneshot where noted) · 12,000 sentences per dataset (9,600 train — 1 epoch = 300 iters) · unseen holdout 10
code: commit 9b3a42cc16a5

data regime (all arms): fresh-dataset-per-stage — 14 dataset(s) × 25 iters each = 0.02 epochs per dataset

Replicate this experiment

Get the code at this exact version:

git clone https://github.com/ClaireHobbs/imagining-syntax
cd imagining-syntax
git checkout 9b3a42cc16a5
pip install -e ".[dev]"

Download the config (experiment.yaml) and run:

imsyn exp run experiment.yaml

Checks

PASS children_complete PASS replicate_coverage PASS schedule_total_matches_final_waypoint PASS instrument_consistent PASS objective_computable

Results — objective: P(unseen_mismatch ≥ 95.5 by t = 350, dwell 2, smoothing 1)

arm	ŝ	wilson95	median crossing [CI]	crossed
r5_c1	0.440	[0.267, 0.629]	251.4 [233.2, 269.6]	11/25
r5_c3	0.360	[0.202, 0.555]	268.2 [247.8, 315.3]	9/25
r5_c4	0.360	[0.202, 0.555]	250.6 [217.5, 294.5]	9/25
r5_c5	0.320	[0.172, 0.516]	258.6 [240.0, 316.2]	8/25
r5_c0	0.280	[0.143, 0.476]	225.0 [216.9, 267.5]	7/25
r5_c2	0.240	[0.115, 0.434]	241.6 [196.9, 271.8]	6/25
r5_c6	0.000	[0.000, 0.133]	—	0/25
r5_c7	0.000	[0.000, 0.133]	—	0/25

Trajectory summary — unseen_mismatch

Evaluation data: seen_* probed at reference α = 1.4 · unseen_* = held-out pairings, uniform (α-independent) · 1000 pairs per condition

Each probe is a minimal pair: a grammatical sentence and its verb-number-flipped twin. The model scores correct when it assigns the grammatical version higher probability; accuracy = % correct over 1000 pairs per condition.
seen_match / seen_mismatch — noun–verb pairings that occur in training, sampled at fixed reference α = 1.4 (not at the schedule's own α, so arms stay comparable)
unseen_match / unseen_mismatch — held-out noun–verb pairings that never occur in training, sampled uniformly (α = 0), so their difficulty is identical across all arms and training αs
match vs mismatch — whether the prepositional objects agree in number with the subject; mismatch places attractor nouns between subject and verb (the AGREE-RECENT trap)

arm	end of training (t = 350)	flags
r5_c0	86.6 ±6.2
r5_c1	93.4 ±6.1
r5_c2	86.6 ±5.9
r5_c3	89.2 ±10.6	⚠ seed_split: range 56.9-100.0 (2 low / 15 high of 25)
r5_c4	87.9 ±8.4
r5_c5	86.4 ±6.2
r5_c6	73.3 ±4.5
r5_c7	69.3 ±4.9

Conclusions

Conclusion: The ascending basin does not reach the incumbent, and the top-up stops \@{schedopt-bo-r5-conclusion} Round 5 is the first round at the re-frozen operating point v2 (m_star = 95.5, k = 350, dwell 2) and, by the pre-registered stop rule, the last. The refit surface had concentrated its posterior in the basin of ascending schedules, those that start near α 1.5 and rise late, and all eight candidates probed that basin or its edges. None reached the incumbent. Result: The ascending basin tops out below the descent \@{schedopt-bo-r5-ascending-ceiling} The best candidate, r5_c1 (α 1.62 rising to 2.21, γ 1.44), crossed on 0.44 of the pool seeds, Wilson interval 0.27 to 0.63, against 0.52 for the incumbent descent r3_c5 (α 3.21 falling to 0.08, γ 1.84). The six candidates inside the basin read 0.24 to 0.44. The two probes at its edges, a near-constant high arm (r5_c6, α 2.93 to 3.12) and a fast rise to α ≈ 4 (r5_c7), crossed on no seeds at all. The round sharpens the Step C reading rather than overturning the incumbent. The ascending basin is real and its members cross at useful rates, but its density in the re-scored surface partly reflected selection noise rather than a higher ceiling. With the best candidate 0.08 below the incumbent, the pre-registered stop rule (no improvement greater than 0.03 in the first round) ends the top-up after one round, and Step D carries forward r3_c5 together with r4_c1 (α 1.77 rising to 2.33, γ 1.60), which ties it at 0.52, as the two three-parameter schedules for the Phase-4 comparison. The incumbent sits inside the pre-registered 0.40 to 0.75 gate for Phase 4. These are search-phase values on the v2 objective, never merged with v1 numbers and not reportable results.

Comparison figures

Children

Child	peak unseen_mismatch	Status
r5_c0	90.0	done
r5_c1	93.4	done
r5_c2	89.8	done
r5_c3	89.2	done
r5_c4	87.9	done
r5_c5	88.4	done
r5_c6	73.9	done
r5_c7	69.3	done

experiment.yaml