Skip to main content

When you don't train parameters of convolution in neural network

So, I tried very simple experiment, where parameters of convolution are initialized randomly and not trained at all. The results are better than what I would expect on MNIST dataset.

convolution,1,28,28,400,8,8,0.5,0
max,400,21,21,7,7
matrix,3600,130,0.5,-0.001
sigmoid,130
dropout,130,0.5
matrix,130,10,0.5,-0.001
softmax,10

0 at the end for convolution means 0 step size - do not train. Only parameters of other layers are trained. Trained with WideOpenThoughts.

Results (above 99% accuracy for MNIST):