imagining syntax experiment results

← vocab_holdout25

v060_h15

done · peak unseen_mismatch 89.5

Configuration

n_layer
2
n_head
4
n_embd
256
base_seed
42

Figures

accuracy_vs_alpha.png
accuracy_vs_alpha.png

Results

paramseen_matchseen_mismatchunseen_matchunseen_mismatch
0.0 100.0 ± 0.024.5 ± 27.7100.0 ± 0.025.0 ± 27.4
0.2 100.0 ± 0.024.6 ± 23.7100.0 ± 0.024.5 ± 23.5
0.4 100.0 ± 0.010.3 ± 15.2100.0 ± 0.09.9 ± 14.5
0.6 100.0 ± 0.023.8 ± 30.6100.0 ± 0.023.6 ± 30.5
0.8 100.0 ± 0.026.3 ± 21.7100.0 ± 0.025.9 ± 21.5
1.0 100.0 ± 0.027.2 ± 30.9100.0 ± 0.026.8 ± 30.8
1.2 100.0 ± 0.059.7 ± 29.4100.0 ± 0.060.1 ± 29.3
1.4 99.7 ± 0.389.8 ± 17.299.0 ± 1.287.0 ± 15.7
1.6 99.4 ± 0.595.4 ± 8.196.6 ± 2.889.5 ± 7.2
1.8 99.3 ± 0.498.1 ± 0.691.7 ± 3.487.0 ± 5.6
2.0 99.1 ± 0.598.4 ± 0.788.2 ± 2.283.0 ± 5.0
2.2 99.2 ± 0.498.2 ± 1.281.7 ± 4.276.6 ± 5.3
2.4 99.2 ± 0.398.4 ± 0.778.5 ± 2.870.5 ± 4.3
2.6 99.3 ± 0.598.9 ± 0.576.4 ± 3.271.6 ± 5.2
2.8 99.7 ± 0.299.1 ± 0.477.1 ± 3.269.4 ± 5.6
3.0 99.7 ± 0.299.1 ± 1.274.4 ± 2.867.4 ± 6.0

Downloads