Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Could not reproduce #3

Open
Cansal7159 opened this issue May 24, 2024 · 0 comments
Open

Could not reproduce #3

Cansal7159 opened this issue May 24, 2024 · 0 comments

Comments

@Cansal7159
Copy link

I reproduce according to your tutor,but I meet a problem.

The problem:

root@autodl-container-7dda1180fa-8d089844:~/SuperGen# ./run_gen.sh QQP
Namespace(fp16=False, k=10, label='all', max_len=50, model_name_or_path='ctrl', model_type='ctrl', no_cuda=False, num_gen=25000, p=1.0, pretrain_corpus_dir='pretrain_corpus/openwebtext_questions.txt', print_res=True, repetition_penalty=None, repetition_reward=None, save_dir='temp_gen', seed=42, task='QQP', temperature='0')
05/24/2024 16:16:02 - WARNING - __main__ - device: cuda, n_gpu: 1, 16-bits training: False
Traceback (most recent call last):
  File "/root/miniconda3/lib/python3.7/site-packages/transformers/modeling_utils.py", line 1331, in from_pretrained
    state_dict = torch.load(resolved_archive_file, map_location="cpu")
  File "/root/miniconda3/lib/python3.7/site-packages/torch/serialization.py", line 608, in load
    return _legacy_load(opened_file, map_location, pickle_module, **pickle_load_args)
  File "/root/miniconda3/lib/python3.7/site-packages/torch/serialization.py", line 794, in _legacy_load
    deserialized_objects[key]._set_from_file(f, offset, f_should_read_directly)
RuntimeError: unexpected EOF, expected 18969398 more bytes. The file might be corrupted.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/root/miniconda3/lib/python3.7/site-packages/transformers/modeling_utils.py", line 1335, in from_pretrained
    if f.read().startswith("version"):
  File "/root/miniconda3/lib/python3.7/codecs.py", line 322, in decode
    (result, consumed) = self._buffer_decode(data, self.errors, final)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0x80 in position 0: invalid start byte

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "gen_train_data.py", line 561, in <module>
    main()
  File "gen_train_data.py", line 548, in main
    generator = SuperGenGenerator(args)
  File "gen_train_data.py", line 179, in __init__
    self.model = CTRLLMHeadModelWithRepReward.from_pretrained(args.model_name_or_path)
  File "/root/miniconda3/lib/python3.7/site-packages/transformers/modeling_utils.py", line 1345, in from_pretrained
    f"Unable to load weights from pytorch checkpoint file for '{pretrained_model_name_or_path}' "
OSError: Unable to load weights from pytorch checkpoint file for 'ctrl' at '/root/.cache/huggingface/transformers/039b344e7dfade42f761aeefe39d5468bd9a19e5bb487e79fef0194f52f5d564.1571f66b33d88ee286ae12b94e3e0229fd037b6fd9a43a193bf808edb3380858'If you tried to load a PyTorch model from a TF 2.0 checkpoint, please set from_tf=True. 
Traceback (most recent call last):
  File "src/gen_utils.py", line 140, in <module>
    main()
  File "src/gen_utils.py", line 132, in main
    gen_file_dict = read_files(args.read_dir, task)
  File "src/gen_utils.py", line 45, in read_files
    for f in filenames:
UnboundLocalError: local variable 'filenames' referenced before assignment
root@autodl-container-7dda1180fa-8d089844:~/SuperGen# ^C
root@autodl-container-7dda1180fa-8d089844:~/SuperGen# python gen_train_data.py --task $SST-2 --label $LABEL=all --save_dir $SAVE_DIR --num_gen $NUM_GEN
usage: gen_train_data.py [-h] [--pretrain_corpus_dir PRETRAIN_CORPUS_DIR] [--task TASK] [--label LABEL] [--model_type MODEL_TYPE] [--model_name_or_path MODEL_NAME_OR_PATH]
                         [--temperature TEMPERATURE] [--repetition_reward REPETITION_REWARD] [--repetition_penalty REPETITION_PENALTY] [--p P] [--k K] [--seed SEED]
                         [--no_cuda NO_CUDA] [--fp16 FP16] [--num_gen NUM_GEN] [--max_len MAX_LEN] [--save_dir SAVE_DIR] [--print_res]
gen_train_data.py: error: argument --save_dir: expected one argument
root@autodl-container-7dda1180fa-8d089844:~/SuperGen# python gen_train_data.py --task $SST-2 --label $LABEL=all --save_dir $SAVE_DIR=temp_gen --num_gen $NUM_GEN
usage: gen_train_data.py [-h] [--pretrain_corpus_dir PRETRAIN_CORPUS_DIR] [--task TASK] [--label LABEL] [--model_type MODEL_TYPE] [--model_name_or_path MODEL_NAME_OR_PATH]
                         [--temperature TEMPERATURE] [--repetition_reward REPETITION_REWARD] [--repetition_penalty REPETITION_PENALTY] [--p P] [--k K] [--seed SEED]
                         [--no_cuda NO_CUDA] [--fp16 FP16] [--num_gen NUM_GEN] [--max_len MAX_LEN] [--save_dir SAVE_DIR] [--print_res]
gen_train_data.py: error: argument --num_gen: expected one argument

How should I fix it?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant