reproduce a standard benchmark ?

[ghost]

Can this code reproduce a standard character level language model benchmark - or more usefully a word level language model benchmark? That would help to convince people it works.

I suggest you,

a) provide a link to some benchmark dataset, (the choice is yours).

b) show that your code can come close to a benchmark, (which should preferably be a published result).

If the code cannot come close to a benchmark, it's hard to consider it to be anything more than a toy?

Edit - Why not try the word level PR, on Penn Tree Bank - that would be reasonably convincing if it worked, compared to Wojciech Zaremba's code?

I suggest using the same model configurations to make it a like for like comparison.

[YafahEdelman]

Well, there's the blog post on this which has been pretty popular and seems convincing. Also, in terms of using the world level PR I made, it currently seems to use a huge amount of RAM (which I'm working on). A 1MB file takes a min of 4 GBs or so for me, which may make it difficult to run on the Penn Tree Bank. I'm working on fixing this now and once I do I'll definitely try it out on that sample.

[ghost]

@JacobEdelman, I'm not as easily convinced as you are?

I'd prefer using code which has been developed and documented in a peer reviewed paper compared to a blog?

Why should anyone use/prefer this code to Wojciech Zaremba's 2 layer LSTM model and code?

At the moment I'll stick to using/building on that code until someone can prove to me that this repo is better?