Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Problem while loading BERT pre-trained model for the NLI-Scorer #4

Open
xhc19930714 opened this issue Aug 9, 2019 · 14 comments
Open
Labels
help wanted Extra attention is needed

Comments

@xhc19930714
Copy link

sorry to bother you. I need your help. when I run the command(python MAML.py --cuda --model trs --batch_size 16 --use_sgd --lr 0.01 --meta_lr 0.0003 --meta_batch_size 16 --meta_optimizer adam --pretrain_emb --weight_sharing --emb_dim 300 --hidden_dim 300 --fix_dialnum_train --pointer_gen --save_path save/paml/)
I got the following result
image
so, can you give me some advice on fixing the problem? thx.

@xhc19930714
Copy link
Author

the last error was found in /PAML/utils/load_bert.py", line 325, in init

@andreamad8
Copy link
Member

Have you downloaded the BERT pre-trained model?
https://drive.google.com/file/d/1Qawz1pMcV0aGLVYzOgpHPgG5vLSKPOJ1/view?usp=sharing

@xhc19930714
Copy link
Author

Yes, I have already downloaded the BERT pre-trained model and followed your instructions to put the model in the /PAML/data/nli_model/

@xhc19930714
Copy link
Author

image
Did you mean this file?

@andreamad8
Copy link
Member

yes, I mean the pytorch_model.bin which yes should be inside the nli_model folder.

So, we never had this error, could you add print(model) before model.to(self.device) this shouldn't be None.

Maybe a version problem? we tested pytorch 4.* and 1.0

@andreamad8 andreamad8 changed the title need your help while I'm trying to reproducing your results. Problem while loading BERT pre-trained model for the NLI-Scorer Aug 9, 2019
@andreamad8 andreamad8 added the help wanted Extra attention is needed label Aug 9, 2019
@xhc19930714
Copy link
Author

I use conda create command to create an environment, then I run the pip install -r requirements.txt to match your dependencies, and I checked my torch version is 0.4.1.

@qiangqiang-he
Copy link

I need help.When I trained successfully.I want to fine-tuned.>>>>[python main_fine_tune.py --cuda --model trs --batch_size 16 --use_sgd --lr 0.01 --meta_lr 0.0003 --meta_batch_size 16 --meta_optimizer adam --pretrain_emb --weight_sharing --emb_dim 300 --hidden_dim 300 --pointer_gen --persona --save_path save/persona/${model} --save_path_dataset save/persona/ --test]
It generates error.IsADirectoryError:is a directory:'save/persona'.Even if I checked that 'save/persona' is existed.

@imrdong
Copy link

imrdong commented Aug 12, 2020

I need help.When I trained successfully.I want to fine-tuned.>>>>[python main_fine_tune.py --cuda --model trs --batch_size 16 --use_sgd --lr 0.01 --meta_lr 0.0003 --meta_batch_size 16 --meta_optimizer adam --pretrain_emb --weight_sharing --emb_dim 300 --hidden_dim 300 --pointer_gen --persona --save_path save/persona/${model} --save_path_dataset save/persona/ --test]
It generates error.IsADirectoryError:is a directory:'save/persona'.Even if I checked that 'save/persona' is existed.

Have you take the model with lowest PPL in validation set to finetune and test (replace ${model})

@xhc19930714
Copy link
Author

xhc19930714 commented Aug 12, 2020 via email

@imrdong
Copy link

imrdong commented Aug 13, 2020

Hi, I want to know how you trained the code? Did you train the code directly in your environment? I failed to train the code. When I run the code in my env, my GPU memory and utilization rate is very low. Did you modify the code and then run code in your env? At 2020-08-12 20:54:24, "mrdong" notifications@github.com wrote: I need help.When I trained successfully.I want to fine-tuned.>>>>[python main_fine_tune.py --cuda --model trs --batch_size 16 --use_sgd --lr 0.01 --meta_lr 0.0003 --meta_batch_size 16 --meta_optimizer adam --pretrain_emb --weight_sharing --emb_dim 300 --hidden_dim 300 --pointer_gen --persona --save_path save/persona/${model} --save_path_dataset save/persona/ --test] It generates error.IsADirectoryError:is a directory:'save/persona'.Even if I checked that 'save/persona' is existed. Have you take the model with lowest PPL in validation set to finetune and test (replace ${model}) — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or unsubscribe.

Hi, I trained the code in a virtual environment which match the dependencies in the requirements.txt, I do modify the code as there are some errors when I trained the code, but the errors I encountered are not the errors you mentioned, maybe you can try to run the code on another machine ranther than create a virtual environment on the same machine, and see what happens.

@xhc19930714
Copy link
Author

I have successfully trained all the PAML、 no_persona and persona models. Then I have fine-tuned the no-persona and persona models successfully. As for the paml model, when I have finished training the paml model , there is no model exist in the path ./save/paml like the other two. Should I have check the source code and fix them by myself, or maybe you have missed uploading some key code in the file MAML.py?

@leyi-123
Copy link

sorry to bother you. I need your help. when I run the command(python MAML.py --cuda --model trs --batch_size 16 --use_sgd --lr 0.01 --meta_lr 0.0003 --meta_batch_size 16 --meta_optimizer adam --pretrain_emb --weight_sharing --emb_dim 300 --hidden_dim 300 --fix_dialnum_train --pointer_gen --save_path save/paml/)
I got the following result
image
so, can you give me some advice on fixing the problem? thx.

Hi! I have the same problem as you. Can I ask how you solved it? Thanks!

@ghost
Copy link

ghost commented Dec 14, 2020

Hi, I want to konw what's the purpose of data_analysis.ipynb?Is it necessary for the code?Is there another way to get persona_map file

@SungWeiTseng
Copy link

Yes, I have already downloaded the BERT pre-trained model and followed your instructions to put the model in the /PAML/data/nli_model/

I'm sorry to bother you, the original link provided by the owner is no longer valid. If it's not too much trouble and you still have the model file, could you please share it with me?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
help wanted Extra attention is needed
Projects
None yet
Development

No branches or pull requests

6 participants