IndexError occurs when generation

[ZJUhjx]

When I using your code to run a chinese dataset(the format is the same as wikibio dataset), some error occurs during generation part--"IndexError: index 574664924280828 is out of range for dimension 0 (of size 5)", and I find that it occurs because "maxprobs, top2k = torch.topk(wrd_dist.view(-1), 2 * K)" the top2k are all nan elements. I debug this for a long time but can not solve it, so how can I deal with it?

[ZJUhjx]

Traceback (most recent call last):
File "/home/h50001979/.pycharm_helpers/pydev/pydevd.py", line 1758, in
main()
File "/home/h50001979/.pycharm_helpers/pydev/pydevd.py", line 1752, in main
globals = debugger.run(setup['file'], None, None, is_module)
File "/home/h50001979/.pycharm_helpers/pydev/pydevd.py", line 1147, in run
pydev_imports.execfile(file, globals, locals) # execute the script
File "/home/h50001979/neural-template-gen/chsmm.py", line 1398, in
gen_from_src()
File "/home/h50001979/neural-template-gen/chsmm.py", line 1312, in gen_from_src
gen_from_srctbl(src_tbl, top_temps, coeffs, src_line=src_line)
File "/home/h50001979/neural-template-gen/chsmm.py", line 1212, in gen_from_srctbl
len_lps, row2tblent, row2feats)
File "/home/h50001979/neural-template-gen/chsmm.py", line 779, in gen_one_ar
args.beamsz, corpus)
File "/home/h50001979/neural-template-gen/chsmm.py", line 691, in temp_ar_bs
nulens[len(new_hyps)][0] = curr_lens[anc][0] + 1

[swiseman]

I you're getting nans, then presumably wrd_dist, which should contain log probabilities, contains nans, so I would recommend trying to figure out how that's happening. Is it possible that somehow you're getting negative numbers on line 507, which become nan after the log on line 513?

[ZJUhjx]

Yeah, thanks a lot! You are totally right, I'm getting negative numbers on line 507, which become nan after the log on line 513, but I don't know why this happens =.=. Is there any recommended solution?

[xiaozhi2015]

@ZJUhjx
How did you turn Chinese dataset to the format same as wikibio. And did you successfully apply this project to Chinese dataset?