Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Readme unclear about training only extension vs training whole model #11

Open
rubmz opened this issue Oct 29, 2021 · 0 comments
Open

Readme unclear about training only extension vs training whole model #11

rubmz opened this issue Oct 29, 2021 · 0 comments

Comments

@rubmz
Copy link

rubmz commented Oct 29, 2021

Hi :-)

Really love the work you did here! And looking forward to give it a try myself, but I found two unclear documentation issues:

  1. Is training "extension only" makes a workable model, or I need to train a "whole" exBert model? It's confusing, because in the paper you explain that training a whole model is a very lengthy operation + It does not make sense that you would provide CLI for training exBert from scratch (we have bert for it), so what does the "only extension" vs "whole" exBert training means? I tried to decypher that by reading the paper and the readme... and I am still unsure which one I should go with.
  2. In the inputs to the CLI you specify path_to_state_dict_of_the_OFF_THE_SHELF_MODEL variable. Yet on the bert model I want to extend there is no such file: https://huggingface.co/onlplab/alephbert Is this input file mandatory? Is it expected that all BERT models on huggingface provide such state file?
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant