imagining syntax experiment results

← vocab_holdout25

v120_h30

done · peak unseen_mismatch 85.5

Configuration

n_layer
2
n_head
4
n_embd
256
base_seed
42

Figures

accuracy_vs_alpha.png
accuracy_vs_alpha.png

Results

paramseen_matchseen_mismatchunseen_matchunseen_mismatch
0.0 80.3 ± 23.637.4 ± 27.780.7 ± 23.137.8 ± 27.9
0.2 75.6 ± 24.431.4 ± 19.075.2 ± 24.831.7 ± 20.1
0.4 85.2 ± 22.635.9 ± 20.484.9 ± 23.136.3 ± 20.7
0.6 85.1 ± 22.839.4 ± 26.285.3 ± 22.539.8 ± 26.1
0.8 90.1 ± 19.725.0 ± 18.189.8 ± 20.224.8 ± 17.3
1.0 85.0 ± 22.935.9 ± 22.585.5 ± 22.036.1 ± 22.6
1.2 85.5 ± 22.248.2 ± 20.985.1 ± 22.748.0 ± 20.4
1.4 88.6 ± 19.472.8 ± 27.888.1 ± 19.170.0 ± 27.1
1.6 93.4 ± 14.991.6 ± 14.288.8 ± 17.185.5 ± 15.9
1.8 97.3 ± 2.996.0 ± 3.486.0 ± 12.883.5 ± 11.6
2.0 96.8 ± 6.696.0 ± 7.683.5 ± 10.181.9 ± 9.4
2.2 97.8 ± 1.597.7 ± 1.280.3 ± 7.778.7 ± 5.8
2.4 96.6 ± 2.496.6 ± 1.764.5 ± 8.864.7 ± 8.3
2.6 96.4 ± 5.196.1 ± 5.168.0 ± 9.966.3 ± 9.1
2.8 98.2 ± 0.897.4 ± 2.065.2 ± 7.961.7 ± 8.1
3.0 97.9 ± 0.997.6 ± 1.463.4 ± 8.261.7 ± 7.4

Downloads