training_data_fraction is slow.

[sleepinyourhat]

If you're using a small fraction (10% or less), you spend most of your time hashing examples, GPU usage drops, and samples per second drops dramatically. The lazy solution for now is to use a cumbersome two different setups: training_data_fraction when training on >1% of the training data, and a custom data file and new task when training on less.

[iftenney]

Can we pre-generate the hashes during indexing for tasks where we want to do this?

[sleepinyourhat]

That might be the cleanest solution, yes.

[iftenney]

Another solution is sharded training files - @pappagari is working on this for Reddit I believe. For these experiments we could generate, say, 10 files and use only some of them.

[sleepinyourhat]

Relevant to @Jan21

[sleepinyourhat]

Shard-based solution sounds more efficient, but probably messier/less reproducible. May be wrong, though.

[zphang]

This is an automatically generated comment.

As we update jiant to v2.x, jiant v1.x has been migrated to https://github.com/nyu-mll/jiant-v1-legacy. As such, we are closing all issues relating to jiant v1.x in this repository.

If this issue is still affecting you in jiant v1.x, please follow up at nyu-mll/jiant-v1-legacy#180.

If this issue is still affecting you in jiant v2.x, reopen this issue or create a new one.