ValueError: Imaginary component

[npapargyr]

Thanks for sharing such an amazing work!

While running the train_vq.py, after the warm-up iters I am getting the following error:

2023-04-14 22:31:45,883 INFO Training on t2m, motions are with 22 joints
Reading checkpoints/t2m/Comp_v6_KLD005/opt.txt
Loading Evaluation Model Wrapper (Epoch 28) Completed!!
100%|██████████| 23384/23384 [00:22<00:00, 1033.65it/s]
  0%|          | 0/1460 [00:00<?, ?it/s]Total number of motions 20942
100%|██████████| 1460/1460 [00:02<00:00, 497.87it/s]
Pointer Pointing at 0
2023-04-14 22:32:37,786 INFO Warmup. Iter 200 :  lr 0.00004 	 Commit. 0.29882 	 PPL. 82.50 	 Recons.  0.70914
2023-04-14 22:32:59,786 INFO Warmup. Iter 400 :  lr 0.00008 	 Commit. 1.13799 	 PPL. 128.26 	 Recons.  0.50615
2023-04-14 22:33:22,425 INFO Warmup. Iter 600 :  lr 0.00012 	 Commit. 2.25313 	 PPL. 217.44 	 Recons.  0.39540
2023-04-14 22:33:43,956 INFO Warmup. Iter 800 :  lr 0.00016 	 Commit. 3.23983 	 PPL. 270.35 	 Recons.  0.32911
Traceback (most recent call last):
  File "/media/npapargyr/C07AF5B07AF5A2F8/WD_2TB_Elements/t2m-gpt/train_vq.py", line 132, in <module>
    best_fid, best_iter, best_div, best_top1, best_top2, best_top3, best_matching, writer, logger = eval_trans.evaluation_vqvae(
  File "/home/npapargyr/anaconda3/envs/onnx/lib/python3.8/site-packages/torch/autograd/grad_mode.py", line 27, in decorate_context
    return func(*args, **kwargs)
  File "/media/npapargyr/C07AF5B07AF5A2F8/WD_2TB_Elements/t2m-gpt/utils/eval_trans.py", line 98, in evaluation_vqvae
    fid = calculate_frechet_distance(gt_mu, gt_cov, mu, cov)
  File "/media/npapargyr/C07AF5B07AF5A2F8/WD_2TB_Elements/t2m-gpt/utils/eval_trans.py", line 547, in calculate_frechet_distance
    raise ValueError('Imaginary component {}'.format(m))
ValueError: Imaginary component 2.1971031947393037e+113

Any ideas?

[katefive]

I have the same issue. Did you solve this problem?