nvrtc error

[Ha0Tang]

Dataset length: 60000
Dataset length: 256
Setting up Perceptual loss...
Downloading: "https://download.pytorch.org/models/alexnet-owt-4df8aa71.pth" to /cluster/home/tangha/.cache/t
orch/hub/checkpoints/alexnet-owt-4df8aa71.pth
100%|█████████████████████████████████████████████████████████████████████| 233M/233M [00:02<00:00, 106MB/s$
Loading model from: /cluster/work/mcvd-pytorch/models/weights/v0.1/alex.pth
...[net-lin [alex]] initialized
...Done

video_gen dataloader: 0%| | 0/1 [00:00<?, ?it/s]I
NFO - ncsn_runner.py - 2022-09-03 16:48:22,970 - (1) Video Pred
INFO - ncsn_runner.py - 2022-09-03 16:48:22,971 - PREDICTING 20 frames, using a 5 frame model conditioned on
5 frames, subsample=1000, preds_per_test=1

Generating video frames: 100%|███████████████████████████████████████████████| 4/4 [16:49<00:00, 252.40s/it]
INFO - ncsn_runner.py - 2022-09-03 17:05:21,209 - fvd1 True, fvd2 False, fvd3 False[16:49<00:00, 252.36s/it]

video_gen dataloader: 0%| | 0/1 [17:01<?, ?it/s]
ERROR - main.py - 2022-09-03 17:05:24,564 - Traceback (most recent call last):
File "main.py", line 404, in main
runner.train()
File "/cluster/work/mcvd-pytorch/runners/ncsn_runner.py", line 497, in train
vid_metrics = self.video_gen(scorenet=test_scorenet, ckpt=step, train=True)
File "/cluster/home/.local/lib/python3.8/site-packages/torch/autograd/grad_mode.py", line 27, in de
corate_context
return func(args, **kwargs)
File "/cluster/work/mcvd-pytorch/runners/ncsn_runner.py", line 1940, in video_gen
real_embeddings.append(get_fvd_feats(real_fvd, i3d=i3d, device=self.config.device))
File "/cluster/work/mcvd-pytorch/models/fvd/fvd.py", line 55, in get_fvd_feats
embeddings = get_feats(videos, i3d, device, bs)
File "/cluster/work/mcvd-pytorch/models/fvd/fvd.py", line 48, in get_feats
feats = np.vstack([feats, detector(torch.stack([preprocess_single(video) for video in videos[ibs:(i+1)*
bs]]).to(device), **detector_kwargs).detach().cpu().numpy()])
File "/cluster/home/.local/lib/python3.8/site-packages/torch/nn/modules/module.py", line 889, in _c
all_impl
result = self.forward(*input, **kwargs)
File "/cluster/home/.local/lib/python3.8/site-packages/torch/nn/parallel/data_parallel.py", line 16
5, in forward
return self.module(*inputs[0], **kwargs[0])
File "/cluster/home/.local/lib/python3.8/site-packages/torch/nn/modules/module.py", line 889, in _c
all_impl
result = self.forward(*input, **kwargs)
RuntimeError: nvrtc: error: failed to open libnvrtc-builtins.so.11.1.
Make sure that libnvrtc-builtins.so.11.1 is installed correctly.
nvrtc compilation failed:

#define NAN __int_as_float(0x7fffffff)
#define POS_INFINITY __int_as_float(0x7f800000)
#define NEG_INFINITY __int_as_float(0xff800000)

template
device T maximum(T a, T b) {
return isnan(a) ? a : (a > b ? a : b);
}