Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Problem_info.txt not generated. #110

Open
jmschnei opened this issue Aug 4, 2022 · 2 comments
Open

Problem_info.txt not generated. #110

jmschnei opened this issue Aug 4, 2022 · 2 comments

Comments

@jmschnei
Copy link

jmschnei commented Aug 4, 2022

I am trying to generate an MTL model with two problems by copying the example from the Tutorial, but changing the task names and order ('seq_tag' first and 'cls' after). When I try to run the code (just like in the tutorial) to train the model I get the following error:

KeyError                                  Traceback (most recent call last)
[<ipython-input-15-d96ce789aae6>](https://localhost:8080/#) in <module>()
      5     problem_type_dict=problem_type_dict_s,
      6     processing_fn_dict=processing_fn_dict_s,
----> 7     continue_training=False
      8 )

3 frames
[/usr/local/lib/python3.7/dist-packages/m3tl/base_params.py](https://localhost:8080/#) in get_problem_info(self, problem, info_name)
    623                        "    1. path {1} exists. this is usually created when creating tfrecord for problem. \n"
    624                        "    2. {0} exists in 'problem_info' key of {2}. ".format(problem, problem_info_json_path, self.params_path))
--> 625             raise KeyError(err_str)
    626 
    627         if info_name is None:

KeyError: "cannot get t2s_ner_t2s_topic's problem info, seems you have not created tfrecord for problem. \nplease make sure either: \n    1. path tmp/t2s_ner_t2s_topic/problem_info.txt exists. this is usually created when creating tfrecord for problem. \n    2. t2s_ner_t2s_topic exists in 'problem_info' key of models/t2s_ner_t2s_topic_ckpt/params.json. "

I am running the code in a Colab (I do not know if this is important).

If I generate the problem_info.txt file manually, then the model is trained without any problem. Is it done that the problem_info.txt has to be created manually, am I supposed to do something that is not included in the tutorial or is something not properly working?

@JayYip
Copy link
Owner

JayYip commented Aug 5, 2022

It's supposed to be created automatically. Could you provide a run-able example?

My first guess is that the function name of pre-processing function is not the same as its problem name.

@jmschnei
Copy link
Author

jmschnei commented Aug 5, 2022

I think I was able to solve the problem. I was defining the problem, problem type and pre-processing functions properly, but it seems that I had to register the new. problem in the params. For that I used the following code:

params.register_problem(problem_name='t2s_ner_1', problem_type='seq_tag', processing_fn=t2s_ner_1)
params.register_problem(problem_name='t2s_re_1', problem_type='seq_tag', processing_fn=t2s_re_1)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants