Text set in serif type with a dotted bar in the margin was drafted by an LLM (claude-opus-4-8) and sometimes reviewed by the author. The rest is the author's own. How to read this site

schedopt_bo_r2

20260721_204316_schedopt_bo_r2 · complete · published 2026-07-23 · seed 1000

part of investigation alpha-curriculum → schedule-opt → schedopt-v1

Intent

Intent: Round 2 of the Bayesian search \@{schedopt-bo-r2-intent} Phase 2, round 2 of the schedopt campaign: eight candidate schedules proposed by Thompson sampling from a Gaussian process fitted to every prior evaluation, scored on the frozen objective (m_star=92.5, k=400) over the shared search seed pool. Adaptivity lives between rounds only; within this manifest the eight arms are fixed before any of them runs.

Hypothesis

Hypothesis: Each round improves on the incumbent, with shrinking returns \@{schedopt-bo-r2-hypothesis} At least one candidate improves on the current incumbent's crossing probability; the round-over-round gain shrinks as the surface is learned. Search-phase s-hat values are never reported as results.

Setup

model: GPT (nanoGPT-derived), weight-tied embeddings — 2 layers × 4 heads × 256 dim (~1,635,584 params) · block 50 · dropout 0.1 · vocab 167 tokens
training: AdamW (β 0.9/0.95, wd 0.1) · lr 0.0006 constant (no warmup, no decay) · batch 32 · 400 iterations · fresh init per replicate (seeds derived from base seed) · 25 seeds (base 1000)
training data: PCFG, zipfian noun–verb pairing (oneshot where noted) · 12,000 sentences per dataset (9,600 train — 1 epoch = 300 iters) · unseen holdout 10
code: commit 3d8011e561e5

data regime (all arms): fresh-dataset-per-stage — 16 dataset(s) × 25 iters each = 0.02 epochs per dataset

Replicate this experiment

Get the code at this exact version:

git clone https://github.com/ClaireHobbs/imagining-syntax
cd imagining-syntax
git checkout 3d8011e561e5
pip install -e ".[dev]"

Download the config (experiment.yaml) and run:

imsyn exp run experiment.yaml

Checks

PASS children_complete PASS replicate_coverage PASS schedule_total_matches_final_waypoint PASS instrument_consistent PASS objective_computable

Results — objective: P(unseen_mismatch ≥ 92.5 by t = 400, dwell 2, smoothing 1)

arm	ŝ	wilson95	median crossing [CI]	crossed
r2_c3	0.960	[0.805, 0.993]	316.4 [302.1, 329.9]	24/25
r2_c7	0.840	[0.653, 0.936]	320.5 [282.4, 345.0]	21/25
r2_c0	0.800	[0.609, 0.911]	357.7 [352.7, 365.3]	20/25
r2_c1	0.760	[0.566, 0.885]	247.6 [235.8, 254.0]	19/25
r2_c5	0.600	[0.407, 0.766]	310.0 [288.6, 323.7]	15/25
r2_c6	0.560	[0.371, 0.733]	266.3 [239.3, 300.0]	14/25
r2_c4	0.040	[0.007, 0.195]	238.8 [238.8, 238.8]	1/25
r2_c2	0.000	[0.000, 0.133]	—	0/25

Trajectory summary — unseen_mismatch

Evaluation data: seen_* probed at reference α = 1.4 · unseen_* = held-out pairings, uniform (α-independent) · 1000 pairs per condition

Each probe is a minimal pair: a grammatical sentence and its verb-number-flipped twin. The model scores correct when it assigns the grammatical version higher probability; accuracy = % correct over 1000 pairs per condition.
seen_match / seen_mismatch — noun–verb pairings that occur in training, sampled at fixed reference α = 1.4 (not at the schedule's own α, so arms stay comparable)
unseen_match / unseen_mismatch — held-out noun–verb pairings that never occur in training, sampled uniformly (α = 0), so their difficulty is identical across all arms and training αs
match vs mismatch — whether the prepositional objects agree in number with the subject; mismatch places attractor nouns between subject and verb (the AGREE-RECENT trap)

arm	end of training (t = 400)	flags
r2_c0	97.3 ±4.1
r2_c1	88.0 ±4.4
r2_c2	40.4 ±25.5	⚠ seed_split: range 0.1-96.4 (5 low / 1 high of 25)
r2_c3	95.8 ±8.1
r2_c4	45.0 ±19.7	⚠ seed_split: range 0.0-97.0 (2 low / 1 high of 25)
r2_c5	91.2 ±10.5
r2_c6	85.5 ±5.6
r2_c7	93.4 ±5.2

Conclusions

Conclusion: The late-descent correction lifts the incumbent to 0.96 \@{schedopt-bo-r2-conclusion} Round 2 improved, and it did so by correcting round 1's mistake. Where round 1 plunged to the floor early and failed, this round's winner descends from α 2.8 to 0.1 with high curvature, so it holds α high through the productive middle and only reaches the floor near the end, too briefly for erosion to bite within the budget. It crosses on 0.96 of seeds, a new search incumbent over the design's 0.92. The optimizer has learned the shape that matters: a late, not early, descent. Search-pool figure, not a reported result. Result: High-curvature late descent reaches 0.96 \@{schedopt-bo-r2-late-descent} The round's best, a descent from α 2.8 to 0.1 with curvature 2.56, crossed on 0.96 of 25 seeds, above the 0.92 design incumbent and the 0.80 best of round 1.

Comparison figures

Children

Child	peak unseen_mismatch	Status
r2_c0	97.3	done
r2_c1	91.2	done
r2_c2	50.1	done
r2_c3	99.0	done
r2_c4	50.6	done
r2_c5	91.2	done
r2_c6	89.8	done
r2_c7	93.4	done

experiment.yaml