imagining syntax experiment results

Experiments

ExperimentIntentStatusChildrenPublished
20260705_211010_depth_long Undertraining follow-up to depth (runs/20260705_133844_depth): continue the existing L=3 and L=4 per-iteration checkpoints from 1200 to 4800 training iterations (fresh-AdamW warm restart on each model's own dataset) and watch whether the rightward-shifted, seed-bimodal recency->agreement transitions recover the 2-layer behavior with a 4x budget. Covers the two transition bands densely (alpha 0.9-2.1) plus tail probes (2.5, 3.0). complete 2/2 2026-07-06
20260705_133844_depth How does depth (n_layer) affect subject-verb agreement generalization? Complements msize (which varied width at 2 layers): depth controls the number of sequential composition steps available, while width (256) and heads (4) are held fixed throughout. The n-layer 2 child is the msize n-embd 256 baseline re-run, giving an in-experiment anchor. complete 4/4 2026-07-06
20260705_105204_paper Full reproduction of the paper's central result (Figure 2): subject-verb agreement generalization across the Zipfian concentration sweep, against the oneshot (deterministic 1:1 pairing) limit. complete 2/2 2026-07-05
20260704_225157_msize How does embedding width (n_embd) affect subject-verb agreement generalization? One axis of model capacity only — depth (2 layers) and heads (4) are held fixed throughout. complete 6/6 2026-07-05
20260614_215604_vocab_holdout25 How does vocabulary size affect the Z-sweep agreement curve when the held-out (unseen) fraction of noun-verb pairs is fixed at 25%? Six Zipf sweeps at vocab sizes 20-160, each holding out a quarter of the pairs. complete 6/6 2026-07-05
20260509_153511_peak_umm_iter Locate each Z's true peak unseen_mismatch training iteration more precisely than the six coarse waypoints of paper_long_iter allow: take the coarse argmax per Z, then run a dense 9-point geometric bracket around it. complete 1/1 2026-07-05
20260506_205133_paper_long_iter How does subject-verb agreement generalization (unseen_mismatch) evolve over the course of training, across the full Z sweep? The paper reports accuracy at a fixed 1200-iteration budget; this run extends training to 10,000 iterations with checkpoints at six waypoints to see whether the fixed budget under- or over-shoots the agreement solution at each Z. complete 1/1 2026-07-05