paper

20260705_105204_paper · complete · published 2026-07-05 · seed 42

Intent

Full reproduction of the paper's central result (Figure 2): subject-verb agreement generalization across the Zipfian concentration sweep, against the oneshot (deterministic 1:1 pairing) limit.

Hypothesis

Reproduces the published finding: unseen_mismatch accuracy peaks at intermediate Z rather than rising monotonically with concentration, and the oneshot limit generalizes worse than the Zipfian optimum despite perfectly consistent pairings — variability in noun-verb pairing is what forces an abstract agreement rule.

Conclusions

Reproduced. images/figure_2.png is the composed two-panel figure.

• Zipfian panel: unseen_mismatch peaks at 93.1% at Z=1.2, falling off both toward the uniform end (low Z: near/below chance) and the concentrated end (Z=3.0: ~71%). Seen conditions saturate (seen_match 100%, seen_mismatch 99.6% at high Z); unseen_match reaches 100% at low Z.
• Oneshot panel: seen conditions are perfect (100/100) but unseen generalization collapses to 62.2% unseen_mismatch / 66.9% unseen_match — memorization of consistent pairings, not an abstract rule.
• Cross-experiment consistency: the 93.1% @ Z=1.2 peak equals the msize d=256 child's peak on the same grid — independent runs, same seeds policy, same result.
• Context from the backfilled companions: the peak's location in Z is partly a snapshot-timing effect of the 1200-iteration budget (see paper_long_iter/peak_umm_iter — higher Z peaks earlier then drifts), so Figure 2's shape should be read jointly with the peak-timing results.

Comparison figures

Children

experiment.yaml