Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cron test error -- ./modules/transfer_mmar.ipynb #352

Closed
wyli opened this issue Sep 15, 2021 · 1 comment · Fixed by Project-MONAI/MONAI#2954 or Project-MONAI/MONAI#2968
Closed

Comments

@wyli
Copy link
Contributor

wyli commented Sep 15, 2021

Describe the bug
logs from https://github.com/Project-MONAI/MONAI/runs/3604008777?check_suite_focus=true

real	0m14.757s
user	0m18.861s
sys	0m12.270s
Running ./modules/transfer_mmar.ipynb
Checking PEP8 compliance...
Running notebook...
Before:
    "max_epochs = 50\n",
After:
    "max_epochs = 1\n",
Before:
    "val_interval = 2\n",
After:
    "val_interval = 1\n",

Executing:   0%|          | 0/30 [00:00<?, ?cell/s]
Executing:   3%|▎         | 1/30 [00:00<00:23,  1.24cell/s]
Executing:  10%|█         | 3/30 [00:04<00:39,  1.47s/cell]
Executing:  13%|█▎        | 4/30 [00:04<00:28,  1.08s/cell]
Executing:  20%|██        | 6/30 [00:06<00:24,  1.03s/cell]
Executing:  37%|███▋      | 11/30 [01:38<03:51, 12.20s/cell]
Executing:  57%|█████▋    | 17/30 [03:49<03:44, 17.24s/cell]
Executing:  70%|███████   | 21/30 [03:55<01:47, 11.96s/cell]
Executing:  87%|████████▋ | 26/30 [03:57<00:30,  7.65s/cell]
Executing:  87%|████████▋ | 26/30 [03:59<00:36,  9.20s/cell]
Traceback (most recent call last):
  File "/opt/conda/bin/papermill", line 8, in <module>
    sys.exit(papermill())
  File "/opt/conda/lib/python3.8/site-packages/click/core.py", line 829, in __call__
    return self.main(*args, **kwargs)
  File "/opt/conda/lib/python3.8/site-packages/click/core.py", line 782, in main
    rv = self.invoke(ctx)
  File "/opt/conda/lib/python3.8/site-packages/click/core.py", line 1066, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/opt/conda/lib/python3.8/site-packages/click/core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "/opt/conda/lib/python3.8/site-packages/click/decorators.py", line 21, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "/opt/conda/lib/python3.8/site-packages/papermill/cli.py", line 250, in papermill
    execute_notebook(
  File "/opt/conda/lib/python3.8/site-packages/papermill/execute.py", line 122, in execute_notebook
    raise_for_execution_errors(nb, output_path)
  File "/opt/conda/lib/python3.8/site-packages/papermill/execute.py", line 234, in raise_for_execution_errors
    raise error
papermill.exceptions.PapermillExecutionError: 
---------------------------------------------------------------------------
Exception encountered at "In [14]":
---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
/tmp/ipykernel_76540/512289039.py in <module>
     16     step = 0
     17     set_determinism(seed=42)
---> 18     for batch_data in train_loader:
     19         step += 1
     20         inputs, labels = (

/opt/conda/lib/python3.8/site-packages/torch/utils/data/dataloader.py in __next__(self)
    519             if self._sampler_iter is None:
    520                 self._reset()
--> 521             data = self._next_data()
    522             self._num_yielded += 1
    523             if self._dataset_kind == _DatasetKind.Iterable and \

/opt/conda/lib/python3.8/site-packages/torch/utils/data/dataloader.py in _next_data(self)
   1201             else:
   1202                 del self._task_info[idx]
-> 1203                 return self._process_data(data)
   1204 
   1205     def _try_put_index(self):

/opt/conda/lib/python3.8/site-packages/torch/utils/data/dataloader.py in _process_data(self, data)
   1227         self._try_put_index()
   1228         if isinstance(data, ExceptionWrapper):
-> 1229             data.reraise()
   1230         return data
   1231 

/opt/conda/lib/python3.8/site-packages/torch/_utils.py in reraise(self)
    429             # instantiate since we don't know how to
    430             raise RuntimeError(msg) from None
--> 431         raise exception
    432 
    433 

AttributeError: Caught AttributeError in DataLoader worker process 0.
Original Traceback (most recent call last):
  File "/opt/conda/lib/python3.8/site-packages/torch/utils/data/_utils/worker.py", line 287, in _worker_loop
    data = fetcher.fetch(index)
  File "/opt/conda/lib/python3.8/site-packages/torch/utils/data/_utils/fetch.py", line 52, in fetch
    return self.collate_fn(data)
  File "/opt/conda/lib/python3.8/site-packages/monai/data/utils.py", line 275, in list_data_collate
    return default_collate(data)
  File "/opt/conda/lib/python3.8/site-packages/torch/utils/data/_utils/collate.py", line 74, in default_collate
    return {key: default_collate([d[key] for d in batch]) for key in elem}
  File "/opt/conda/lib/python3.8/site-packages/torch/utils/data/_utils/collate.py", line 74, in <dictcomp>
    return {key: default_collate([d[key] for d in batch]) for key in elem}
  File "/opt/conda/lib/python3.8/site-packages/torch/utils/data/_utils/collate.py", line 84, in default_collate
    return [default_collate(samples) for samples in transposed]
  File "/opt/conda/lib/python3.8/site-packages/torch/utils/data/_utils/collate.py", line 84, in <listcomp>
    return [default_collate(samples) for samples in transposed]
  File "/opt/conda/lib/python3.8/site-packages/torch/utils/data/_utils/collate.py", line 74, in default_collate
    return {key: default_collate([d[key] for d in batch]) for key in elem}
  File "/opt/conda/lib/python3.8/site-packages/torch/utils/data/_utils/collate.py", line 74, in <dictcomp>
    return {key: default_collate([d[key] for d in batch]) for key in elem}
  File "/opt/conda/lib/python3.8/site-packages/torch/utils/data/_utils/collate.py", line 74, in default_collate
    return {key: default_collate([d[key] for d in batch]) for key in elem}
  File "/opt/conda/lib/python3.8/site-packages/torch/utils/data/_utils/collate.py", line 74, in <dictcomp>
    return {key: default_collate([d[key] for d in batch]) for key in elem}
  File "/opt/conda/lib/python3.8/site-packages/torch/utils/data/_utils/collate.py", line 53, in default_collate
    numel = sum(x.numel() for x in batch)
  File "/opt/conda/lib/python3.8/site-packages/torch/utils/data/_utils/collate.py", line 53, in <genexpr>
    numel = sum(x.numel() for x in batch)
AttributeError: 'numpy.ndarray' object has no attribute 'numel'
@wyli
Copy link
Contributor Author

wyli commented Sep 17, 2021

this is still an issue

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant