Reproducing quantitative results

[thadikari]

Hi hjweide,
Thank you very much for the code. I am new to Theano and couldn't get my environment setup to run your code. However my question is, have you been able to reach an error rate of 1.90 (±0:10)% for MNIST 100 labels (as reported in original paper)? Even after 5000 epochs the lowest I get is somewhat around 5% (in my TensorFlow implementation).

[hjweide]

@thadikari I actually never tried to reproduce the quantitative results, I was more interested in reproducing the visualization of the digits sampled from the latent space of the autoencoder.

I'm afraid I'm not familiar with the MNIST-100 dataset. Could you provide a link for me please? Is it more difficult than standard MNIST? 5% on standard MNIST indeed sounds pretty high.

[thadikari]

@hjweide thanks for the reply!

I meant the standard MNIST with only 100 labeled images. Sorry about the confusion.
Specifically the error rate I quoted is 1:90 (±0:10) that is mentioned in the

• original Adversarial Autoencoders paper,
• Table 2,
• row 'Adversarial Autoencoders',
• column 'MNIST (100)'.

I couldn't make it lower beyond ~5.5% and was wondering if you had a better outcome.
Anyway thanks for the code and the reply.

[hjweide]

@thadikari Thanks for the clarification! I haven't read that paper in a while, but I remember now. I also had trouble choosing the model architecture based on just the paper, and was only able to get good results after I saw this discussion by the authors. If you write to the authors about reproducing the result, they may be able to help you out with the exact architecture they used for that experiment. I'm also curious about that now but I don't think I'll have time to work on this in the near future.

[thadikari]

Thanks for the quick reply! I will try to contact authors and will definitely let you know if there's any improvement.