imagining syntax

← faithful_v1

faithful_zipfian

done · peak unseen_mismatch 96.6

Configuration

n_layer
2
n_head
4
n_embd
256
base_seed
42

Figures

accuracy_vs_alpha.png
accuracy_vs_alpha.png

Results

seen_* generated at each grid point's own training α · unseen_* = held-out pairings, uniform (α-independent) · 1000 pairs per condition

αseen_matchseen_mismatchunseen_matchunseen_mismatch
0.0 100.0 ± 0.031.8 ± 27.7100.0 ± 0.030.7 ± 26.6
0.1 100.0 ± 0.013.8 ± 19.9100.0 ± 0.014.5 ± 20.1
0.2 100.0 ± 0.024.7 ± 19.6100.0 ± 0.024.0 ± 19.1
0.3 100.0 ± 0.011.9 ± 17.0100.0 ± 0.011.8 ± 16.9
0.4 100.0 ± 0.022.2 ± 20.5100.0 ± 0.023.3 ± 22.0
0.5 100.0 ± 0.024.1 ± 30.1100.0 ± 0.024.5 ± 30.6
0.6 100.0 ± 0.046.2 ± 37.6100.0 ± 0.046.9 ± 36.9
0.7 100.0 ± 0.032.8 ± 28.7100.0 ± 0.031.6 ± 27.9
0.8 100.0 ± 0.066.2 ± 30.3100.0 ± 0.065.0 ± 30.5
0.9 100.0 ± 0.080.5 ± 17.6100.0 ± 0.079.3 ± 18.5
1.0 100.0 ± 0.083.3 ± 21.9100.0 ± 0.082.0 ± 21.9
1.1 100.0 ± 0.089.4 ± 22.699.8 ± 0.488.1 ± 22.6
1.2 99.7 ± 0.299.0 ± 0.998.4 ± 0.996.6 ± 2.1
1.3 99.6 ± 0.398.7 ± 1.197.8 ± 1.593.2 ± 3.9
1.4 99.3 ± 0.798.7 ± 0.796.5 ± 2.092.5 ± 3.3
1.5 99.5 ± 0.398.5 ± 0.896.2 ± 1.690.1 ± 3.7
1.6 99.3 ± 0.498.4 ± 0.793.1 ± 1.388.7 ± 4.1
1.7 99.4 ± 0.398.5 ± 0.591.8 ± 2.084.0 ± 4.5
1.8 99.4 ± 0.299.0 ± 0.591.7 ± 1.787.7 ± 3.0
1.9 99.4 ± 0.399.2 ± 0.391.2 ± 2.787.0 ± 3.0
2.0 99.6 ± 0.299.1 ± 0.392.8 ± 1.885.7 ± 5.5
2.1 99.6 ± 0.399.2 ± 0.588.1 ± 3.282.5 ± 2.4
2.2 99.6 ± 0.299.0 ± 0.589.8 ± 2.983.7 ± 4.7
2.3 99.5 ± 0.299.3 ± 0.388.0 ± 2.483.6 ± 5.1
2.4 99.7 ± 0.299.5 ± 0.287.1 ± 3.880.1 ± 3.9
2.5 99.7 ± 0.399.4 ± 0.485.9 ± 4.480.5 ± 4.8
2.6 99.7 ± 0.299.4 ± 0.386.3 ± 2.580.6 ± 4.5
2.7 99.7 ± 0.199.5 ± 0.283.9 ± 3.478.3 ± 3.9
2.8 99.7 ± 0.299.5 ± 0.384.2 ± 3.077.2 ± 4.5
2.9 99.8 ± 0.299.6 ± 0.281.0 ± 2.876.0 ± 2.9
3.0 99.8 ± 0.299.7 ± 0.183.3 ± 2.579.5 ± 3.2

Downloads