Text set in serif type with a dotted bar in the margin was drafted by an LLM (claude-opus-4-8) and sometimes reviewed by the author. The rest is the author's own. How to read this site

schedopt_bo_r1

20260721_194647_schedopt_bo_r1 · complete · published 2026-07-23 · seed 1000

part of investigation alpha-curriculum → schedule-opt → schedopt-v1

Intent

Intent: Round 1 of the Bayesian search \@{schedopt-bo-r1-intent} Phase 2, round 1 of the schedopt campaign: eight candidate schedules proposed by Thompson sampling from a Gaussian process fitted to every prior evaluation, scored on the frozen objective (m_star=92.5, k=400) over the shared search seed pool. Adaptivity lives between rounds only; within this manifest the eight arms are fixed before any of them runs.

Hypothesis

Hypothesis: Each round improves on the incumbent, with shrinking returns \@{schedopt-bo-r1-hypothesis} At least one candidate improves on the current incumbent's crossing probability; the round-over-round gain shrinks as the surface is learned. Search-phase s-hat values are never reported as results.

Setup

model: GPT (nanoGPT-derived), weight-tied embeddings — 2 layers × 4 heads × 256 dim (~1,635,584 params) · block 50 · dropout 0.1 · vocab 167 tokens
training: AdamW (β 0.9/0.95, wd 0.1) · lr 0.0006 constant (no warmup, no decay) · batch 32 · 400 iterations · fresh init per replicate (seeds derived from base seed) · 25 seeds (base 1000)
training data: PCFG, zipfian noun–verb pairing (oneshot where noted) · 12,000 sentences per dataset (9,600 train — 1 epoch = 300 iters) · unseen holdout 10
code: commit d14005f641c2

data regime (all arms): fresh-dataset-per-stage — 16 dataset(s) × 25 iters each = 0.02 epochs per dataset

Replicate this experiment

Get the code at this exact version:

git clone https://github.com/ClaireHobbs/imagining-syntax
cd imagining-syntax
git checkout d14005f641c2
pip install -e ".[dev]"

Download the config (experiment.yaml) and run:

imsyn exp run experiment.yaml

Checks

PASS children_complete PASS replicate_coverage PASS schedule_total_matches_final_waypoint PASS instrument_consistent PASS objective_computable

Results — objective: P(unseen_mismatch ≥ 92.5 by t = 400, dwell 2, smoothing 1)

arm	ŝ	wilson95	median crossing [CI]	crossed
r1_c1	0.800	[0.609, 0.911]	289.1 [273.0, 348.0]	20/25
r1_c4	0.800	[0.609, 0.911]	289.1 [255.6, 318.1]	20/25
r1_c3	0.680	[0.484, 0.828]	273.8 [249.8, 293.2]	17/25
r1_c0	0.400	[0.234, 0.593]	344.8 [295.9, 349.7]	10/25
r1_c5	0.360	[0.202, 0.555]	244.8 [207.0, 285.0]	9/25
r1_c6	0.040	[0.007, 0.195]	198.4 [198.4, 198.4]	1/25
r1_c2	0.000	[0.000, 0.133]	—	0/25
r1_c7	0.000	[0.000, 0.133]	—	0/25

Trajectory summary — unseen_mismatch

Evaluation data: seen_* probed at reference α = 1.4 · unseen_* = held-out pairings, uniform (α-independent) · 1000 pairs per condition

Each probe is a minimal pair: a grammatical sentence and its verb-number-flipped twin. The model scores correct when it assigns the grammatical version higher probability; accuracy = % correct over 1000 pairs per condition.
seen_match / seen_mismatch — noun–verb pairings that occur in training, sampled at fixed reference α = 1.4 (not at the schedule's own α, so arms stay comparable)
unseen_match / unseen_mismatch — held-out noun–verb pairings that never occur in training, sampled uniformly (α = 0), so their difficulty is identical across all arms and training αs
match vs mismatch — whether the prepositional objects agree in number with the subject; mismatch places attractor nouns between subject and verb (the AGREE-RECENT trap)

arm	end of training (t = 400)	flags
r1_c0	89.1 ±12.5	⚠ seed_split: range 50.9-100.0 (2 low / 17 high of 25)
r1_c1	90.5 ±13.2	⚠ seed_split: range 51.0-100.0 (2 low / 20 high of 25)
r1_c2	24.4 ±23.6	⚠ seed_split: range 0.0-87.9 (10 low / 1 high of 25)
r1_c3	87.5 ±6.4
r1_c4	89.9 ±6.7
r1_c5	84.7 ±5.4
r1_c6	28.2 ±25.8	⚠ seed_split: range 0.0-78.4 (11 low / 1 high of 25)
r1_c7	70.3 ±4.3

Conclusions

Conclusion: No improvement, but a region ruled out \@{schedopt-bo-r1-conclusion} The first search round improved on nothing. Its best candidate reached a crossing probability of 0.80 against the design's incumbent at 0.92, and its value was the negative information: the proposals that descended fastest to a low floor, from α 3.4 to near 0.1 with low curvature, crossed on almost none of their seeds. Plunging into the near-uniform deep-uniform floor early leaves the rest of the run in the erosion zone, and the surface now carries that as a dead region. These are search-pool figures, not reported results. Result: Fast descent to a low floor fails \@{schedopt-bo-r1-fast-descent-dies} Best of round 1 was 0.80 (an ascent from α 1.6 to 3.7), below the 0.92 design incumbent. The two descents to a low floor with low curvature, 3.4 to 0.1 and 3.4 to 0.3, crossed on 0.00 and 0.04 of 25 seeds.

Comparison figures

Children

Child	peak unseen_mismatch	Status
r1_c0	89.1	done
r1_c1	90.5	done
r1_c2	49.9	done
r1_c3	90.4	done
r1_c4	91.3	done
r1_c5	88.8	done
r1_c6	49.9	done
r1_c7	70.8	done

experiment.yaml