Text set in serif type with a dotted bar in the margin was drafted by an LLM (claude-opus-4-8) and sometimes reviewed by the author. The rest is the author's own. How to read this site

schedopt_bo_r3

20260721_213056_schedopt_bo_r3 · complete · published 2026-07-23 · seed 1000

part of investigation alpha-curriculum → schedule-opt → schedopt-v1

Intent

Intent: Round 3 of the Bayesian search \@{schedopt-bo-r3-intent} Phase 2, round 3 of the schedopt campaign: eight candidate schedules proposed by Thompson sampling from a Gaussian process fitted to every prior evaluation, scored on the frozen objective (m_star=92.5, k=400) over the shared search seed pool. Adaptivity lives between rounds only; within this manifest the eight arms are fixed before any of them runs.

Hypothesis

Hypothesis: Each round improves on the incumbent, with shrinking returns \@{schedopt-bo-r3-hypothesis} At least one candidate improves on the current incumbent's crossing probability; the round-over-round gain shrinks as the surface is learned. Search-phase s-hat values are never reported as results.

Setup

model: GPT (nanoGPT-derived), weight-tied embeddings — 2 layers × 4 heads × 256 dim (~1,635,584 params) · block 50 · dropout 0.1 · vocab 167 tokens
training: AdamW (β 0.9/0.95, wd 0.1) · lr 0.0006 constant (no warmup, no decay) · batch 32 · 400 iterations · fresh init per replicate (seeds derived from base seed) · 25 seeds (base 1000)
training data: PCFG, zipfian noun–verb pairing (oneshot where noted) · 12,000 sentences per dataset (9,600 train — 1 epoch = 300 iters) · unseen holdout 10
code: commit fa157a107af5

data regime (all arms): fresh-dataset-per-stage — 16 dataset(s) × 25 iters each = 0.02 epochs per dataset

Replicate this experiment

Get the code at this exact version:

git clone https://github.com/ClaireHobbs/imagining-syntax
cd imagining-syntax
git checkout fa157a107af5
pip install -e ".[dev]"

Download the config (experiment.yaml) and run:

imsyn exp run experiment.yaml

Checks

PASS children_complete PASS replicate_coverage PASS schedule_total_matches_final_waypoint PASS instrument_consistent PASS objective_computable

Results — objective: P(unseen_mismatch ≥ 92.5 by t = 400, dwell 2, smoothing 1)

arm	ŝ	wilson95	median crossing [CI]	crossed
r3_c4	0.960	[0.805, 0.993]	347.1 [338.6, 349.0]	24/25
r3_c5	0.960	[0.805, 0.993]	309.6 [295.3, 320.4]	24/25
r3_c1	0.920	[0.750, 0.978]	360.8 [356.9, 364.3]	23/25
r3_c3	0.720	[0.524, 0.857]	294.7 [267.1, 322.2]	18/25
r3_c0	0.640	[0.445, 0.798]	253.7 [223.6, 283.9]	16/25
r3_c6	0.640	[0.445, 0.798]	252.8 [240.9, 285.3]	16/25
r3_c2	0.560	[0.371, 0.733]	367.1 [364.6, 370.3]	14/25
r3_c7	0.040	[0.007, 0.195]	372.2 [372.2, 372.2]	1/25

Trajectory summary — unseen_mismatch

Evaluation data: seen_* probed at reference α = 1.4 · unseen_* = held-out pairings, uniform (α-independent) · 1000 pairs per condition

Each probe is a minimal pair: a grammatical sentence and its verb-number-flipped twin. The model scores correct when it assigns the grammatical version higher probability; accuracy = % correct over 1000 pairs per condition.
seen_match / seen_mismatch — noun–verb pairings that occur in training, sampled at fixed reference α = 1.4 (not at the schedule's own α, so arms stay comparable)
unseen_match / unseen_mismatch — held-out noun–verb pairings that never occur in training, sampled uniformly (α = 0), so their difficulty is identical across all arms and training αs
match vs mismatch — whether the prepositional objects agree in number with the subject; mismatch places attractor nouns between subject and verb (the AGREE-RECENT trap)

arm	end of training (t = 400)	flags
r3_c0	85.6 ±5.9
r3_c1	96.5 ±9.2
r3_c2	97.8 ±4.7
r3_c3	90.0 ±7.0
r3_c4	94.5 ±8.6
r3_c5	91.5 ±15.8	⚠ seed_split: range 31.8-100.0 (1 low / 19 high of 25)
r3_c6	88.4 ±5.9
r3_c7	95.0 ±2.9

Conclusions

Conclusion: A plateau at 0.96, not a single lucky point \@{schedopt-bo-r3-conclusion} Round 3 did not improve the incumbent, and that is itself informative. Its two best candidates, descents from α 3.2 to 0.1 and 3.6 to 0.2, both crossed on 0.96 of seeds, joining round 2's winner at the same value. Three independent points of the same high-start, late-descent shape sitting at 0.96 is a robust plateau rather than one fortunate draw, which is what the selection bootstrap over the combined surface confirmed by spreading its winner-share across them. The smooth family's ceiling on this objective looks to be near 0.96. Search-pool figures, not reported results. Result: Three schedules converge at 0.96 \@{schedopt-bo-r3-plateau} Round 3's best two, descents from α 3.2 to 0.1 and 3.6 to 0.2, each crossed on 0.96 of 25 seeds, tying round 2's 2.8-to-0.1 winner; no candidate improved on the incumbent.

Comparison figures

Children

Child	peak unseen_mismatch	Status
r3_c0	90.6	done
r3_c1	96.9	done
r3_c2	97.8	done
r3_c3	90.7	done
r3_c4	97.7	done
r3_c5	98.4	done
r3_c6	91.6	done
r3_c7	95.0	done

experiment.yaml