wholeslidedata

[4479917]

Dear Mart,

I attempted to run Docker, and it seems that the preinstalled version of wholeslidedata is 0.1.0. However, I encountered some issues as certain files in this version appear to be unavailable. Specifically, I received the following error messages: 'ModuleNotFoundError: No module named 'wholeslidedata.extensions'' and 'ModuleNotFoundError: No module named 'wholeslidedata.annotation.structures'.'

In response, I uninstalled wholeslidedata 0.1.0 and installed version 0.0.16. Yet, I encountered another issue with this version, resulting in the error message: 'AttributeError: module 'wholeslidedata.iterators' has no attribute 'PatchConfiguration'.'

Could you please inform me which version of wholeslidedata you used, and do you have any guidance on how to proceed from here?

Thank you for your assistance.

[martvanrijthoven]

Dear 4479917,

Thank you for your interest in HookNet-TLS. Could you please provide me with the complete error trace?
HookNet-TLS should work with the latests wholeslidedata version (main branch = 0.1.0).
If you could provide me with the complete error trace, I would be happy to help to debug the problem.

Sorry for the inconvenience.

Best wishes,
Mart

[4479917]

Thank you very much for your prompt response.

I did this "python3 -m hooknettls hooknettls.default.image_path=/home/4479917/example.tif hooknettls.default.mask_path=/home/4479917/example_mask.tif"

It returns:
/usr/local/lib/python3.8/dist-packages/wholeslidedata/image/wholeslideimage.py:78: UserWarning: spacing 0.5 outside margin (0.3%) for [50.0, 100.00135898590088, 200.00813007354736, 400.0380039215088, 800.0760078430176, 1600.1520156860352, 3201.694107055664, 6408.956146240234, 12840.243530273438, 25770.278930664062, 51903.521728515625], returning closest spacing: 50.0
warnings.warn(
Process CommanderForkProcess-3:
Traceback (most recent call last):
File "/usr/local/lib/python3.8/dist-packages/wholeslidedata/buffer/patchcommander.py", line 74, in create_message
return next(self._messages)
StopIteration

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/usr/lib/python3.8/multiprocessing/process.py", line 315, in _bootstrap
self.run()
File "/usr/local/lib/python3.8/dist-packages/concurrentbuffer/commander.py", line 71, in run
self._message(buffer_id)
File "/usr/local/lib/python3.8/dist-packages/concurrentbuffer/commander.py", line 74, in _message
message = self._commander.create_message(*args, **kwargs)
File "/usr/local/lib/python3.8/dist-packages/wholeslidedata/buffer/patchcommander.py", line 77, in create_message
return next(self._messages)
StopIteration
2024-04-12 19:15:46.125181: I tensorflow/core/platform/cpu_feature_guard.cc:193] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations: AVX2 FMA
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
2024-04-12 19:15:46.612501: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1532] Created device /job:localhost/replica:0/task:0/device:GPU:0 with 13797 MB memory: -> device: 0, name: Tesla T4, pci bus id: 0000:41:00.0, compute capability: 7.5
loading weights... /home/user/pathology-hooknet-tls/weights.h5
Create output folder
Creating lock file: /output/images/examplehooknettls.lock
Run inference
Init writers...
/usr/local/lib/python3.8/dist-packages/wholeslidedata/image/wholeslideimage.py:78: UserWarning: spacing 0.5 outside margin (0.3%) for [50.0, 100.00135898590088, 200.00813007354736, 400.0380039215088, 800.0760078430176, 1600.1520156860352, 3201.694107055664, 6408.956146240234, 12840.243530273438, 25770.278930664062, 51903.521728515625], returning closest spacing: 50.0
warnings.warn(
write: /home/user/tmp/example_hooknettls.tif
Creating: /home/user/tmp/example_hooknettls.tif....
Spacing: 50.0
Dimensions: (73703, 50706)
Tile_shape: (1024, 1024)
write: /home/user/tmp/example_hooknettls_heat1.tif
Creating: /home/user/tmp/example_hooknettls_heat1.tif....
Spacing: 50.0
Dimensions: (73703, 50706)
Tile_shape: (1024, 1024)
write: /home/user/tmp/example_hooknettls_heat2.tif
Creating: /home/user/tmp/example_hooknettls_heat2.tif....
Spacing: 50.0
Dimensions: (73703, 50706)
Tile_shape: (1024, 1024)
Applying...
0it [00:00, ?it/s]
/usr/local/lib/python3.8/dist-packages/numpy/core/fromnumeric.py:3432: RuntimeWarning: Mean of empty slice.
return _methods._mean(a, axis=axis, dtype=dtype,
/usr/local/lib/python3.8/dist-packages/numpy/core/_methods.py:190: RuntimeWarning: invalid value encountered in double_scalars
ret = ret.dtype.type(ret / rcount)
average batch time: nan
average prediction time: nan
Saving...
Segmentation fault (core dumped)

Not sure if it's an issue about wholeslidedata?

[martvanrijthoven]

It seems that your input image does not contain the correct spacings or wholeslidedata is reading the spacings wrong:
[50.0, 100.00135898590088, 200.00813007354736, 400.0380039215088, 800.0760078430176, 1600.1520156860352, 3201.694107055664, 6408.956146240234, 12840.243530273438, 25770.278930664062, 51903.521728515625]
The image should contain 0.5 and 2.0 values in this list.

Would it be possible to share the image with me? Then I can try to debug it.

Best wishes,
Mart

[4479917]

Dear Mart,

Thank you very much for your explanation.

For the first input, I used save_image_at_spacing.py from pathology-whole-slide-data to convert svs to tiff with spacing 0.5 and 2. It's file 424218_EAS43977_HE.tif. For the second input, it's file segmented_output.tif. Here in the link are the images:
https://drive.google.com/drive/folders/1wABmWVJOr3QGXLWa_EScqOpvkEjyRsd_?usp=sharing

It does not have spacing warnings anymore, but it stopped moving forward after running a while. Here is the traceback:
python3 -m hooknettls hooknettls.default.image_path=/home/4479917/424218_EAS43977_HE.tif hooknettls.default.mask_path=/home/4479917/424218_EAS43977_HE_mask.tif
2024-04-17 14:06:32.945552: I tensorflow/core/platform/cpu_feature_guard.cc:193] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations: AVX2 FMA
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
2024-04-17 14:06:33.331746: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1532] Created device /job:localhost/replica:0/task:0/device:GPU:0 with 13797 MB memory: -> device: 0, name: Tesla T4, pci bus id: 0000:41:00.0, compute capability: 7.5
loading weights... /home/user/pathology-hooknet-tls/weights.h5
Create output folder
Creating lock file: /output/images/424218_EAS43977_HEhooknettls.lock
Run inference
Init writers...
write: /home/user/tmp/424218_EAS43977_HE_hooknettls.tif
Creating: /home/user/tmp/424218_EAS43977_HE_hooknettls.tif....
Spacing: 0.5025998745463595
Dimensions: (41832, 40844)
Tile_shape: (1024, 1024)
write: /home/user/tmp/424218_EAS43977_HE_hooknettls_heat1.tif
Creating: /home/user/tmp/424218_EAS43977_HE_hooknettls_heat1.tif....
Spacing: 0.5025998745463595
Dimensions: (41832, 40844)
Tile_shape: (1024, 1024)
write: /home/user/tmp/424218_EAS43977_HE_hooknettls_heat2.tif
Creating: /home/user/tmp/424218_EAS43977_HE_hooknettls_heat2.tif....
Spacing: 0.5025998745463595
Dimensions: (41832, 40844)
Tile_shape: (1024, 1024)
Applying...
0%| | 0/953 [00:00<?, ?it/s]2024-04-17 14:06:35.098043: I tensorflow/stream_executor/cuda/cuda_dnn.cc:384] Loaded cuDNN version 8100
0%|▍ | 2/953 [00:06<43:05, 2.72s/it]

It stops here. Could you kindly provide some guidance on how to proceed? Thank you very much for your time and help!

[martvanrijthoven]

It might be that this is due to a shared memory issue. If you are using docker could you please try to add --shm-size=4G when starting the docker.

Please let me know if it still hangs or if you are not using docker.

[4479917]

Thanks a bunch of for your help on this Mart! I added --shm-size=4G, now it can proceed. But stopped at the step of writing json file.

Here is the traceback:
Applying...
0%| | 0/953 [00:00<?, ?it/s]2024-04-19 14:35:57.449930: I tensorflow/stream_executor/cuda/cuda_dnn.cc:384] Loaded cuDNN version 8100
100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 953/953 [05:07<00:00, 3.10it/s]
average batch time: 0.004079683488156616
average prediction time: 0.2824230131481876
Saving...
Total time was 13453
Total reading time was 0
Total base writing time was 3787
Total pyramid downsampling time was 0
Total pyramid writing time was 9666
Total time determining min/max was 953
Copying from: /home/user/tmp/424218_EAS43977_HE_hooknettls.tif
Copying to: /output/images/424218_EAS43977_HE_hooknettls.tif
Removing tmp file...
Copying done.
Total time was 16500
Total reading time was 0
Total base writing time was 4932
Total pyramid downsampling time was 0
Total pyramid writing time was 11568
Total time determining min/max was 953
Copying from: /home/user/tmp/424218_EAS43977_HE_hooknettls_heat1.tif
Copying to: /output/images/424218_EAS43977_HE_hooknettls_heat1.tif
Removing tmp file...
Copying done.
Total time was 16944
Total reading time was 0
Total base writing time was 5026
Total pyramid downsampling time was 2
Total pyramid writing time was 11918
Total time determining min/max was 953
Copying from: /home/user/tmp/424218_EAS43977_HE_hooknettls_heat2.tif
Copying to: /output/images/424218_EAS43977_HE_hooknettls_heat2.tif
Removing tmp file...
Copying done.
Stopping iterator
Releasing lock file /output/images/424218_EAS43977_HEhooknettls.lock
writing json file: /output/images/filtered/424218_EAS43977_HE_hooknettls_tls_filtered.json
Traceback (most recent call last):
File "/usr/lib/python3.8/runpy.py", line 194, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/usr/lib/python3.8/runpy.py", line 87, in _run_code
exec(code, run_globals)
File "/home/user/pathology-hooknet-tls/hooknettls/main.py", line 8, in
objects = build_config(config_reader.read()["default"])
File "/usr/local/lib/python3.8/dist-packages/dicfg/factory.py", line 124, in build_config
return _ObjectFactory(deepcopy(config)).build_config()
File "/usr/local/lib/python3.8/dist-packages/dicfg/factory.py", line 26, in build_config
return self._build(self._configuration)
File "/usr/lib/python3.8/functools.py", line 912, in _method
return method.get(obj, cls)(*args, **kwargs)
File "/usr/local/lib/python3.8/dist-packages/dicfg/factory.py", line 38, in _build_dict
config[key] = self._build_object(value)
File "/usr/local/lib/python3.8/dist-packages/dicfg/factory.py", line 67, in _build_object
return attribute(*args, **kwargs)
File "/home/user/pathology-hooknet-tls/hooknettls/postprocessing.py", line 235, in gc_filtering
filtered_tls_annotations = filter_gc_annotations(
File "/home/user/pathology-hooknet-tls/hooknettls/postprocessing.py", line 147, in filter_gc_annotations
if not in_tls(tls_tree, gc_annotation.geometry):
File "/home/user/pathology-hooknet-tls/hooknettls/postprocessing.py", line 131, in in_tls
tls_box = box(*tls_annotation.bounds)
AttributeError: 'numpy.int64' object has no attribute 'bounds'

I really appreciate your time and patience. Thanks a lot in advance.

[martvanrijthoven]

You are welcome :)

For your latest problem it seems that shapely2.x has updated the interface of strtree. I have updated the docker file to install shapely1.8.4, which should work with hooknet-tls code. I will soon try to update the code to make it compatible with shapely2.0, but if you now rebuild the docker with the updated dockerfile, it should not give this error anymore.

Sorry for the inconvenience. Please let me know if you still run into issues.

[4479917]

Thank you very much for your great help!

Best,
Ranran

[4479917]

Dear Mart, I really appreciate this great work you have done in TLS detection. Is it possible for you to share the script used for pretraining the weights? Your assistance in this matter would be greatly valuable. Thank you very much in advance!

[martvanrijthoven]

Dear Ranran,

For training i used the wholeslidedata package en hooknet model. You can find example code to train it here: https://github.com/DIAGNijmegen/pathology-hooknet/blob/master/notebooks/HookNetPracticalGuide.ipynb

Please let me know if you have any further questions.

Best wishes,
Mart

[4479917]

Dear Mart,

Thank you for your guidance. I tried that ipynb, the example image /tmp/TCGA-21-5784-01Z-00-DX1.tif from https://drive.google.com/uc?id=1NefnQu3e0l4WR7Xb809gpVORXrKpJb0q cannot be found, so I downloaded TCGA-21-5784-01Z-00-DX1.svs from TCGA directly and then converted it to tif with spacings 0.5 and 2, but the size of this tif seems different from the example data.

I am wondering if the data you used to pretrain HookNet-TLS corresponds to the datasets provided in this repository (the annotations are 1020 annotations_approx_05um xml files which you uploaded recently, and the images are their corresponding 1020 TIF files, which need to be downloaded from TCGA?) Additionally, could you advise on the appropriate processing steps for the .svs images obtained from TCGA? Should they be converted to .tif format with spacings of 0.5 and 2, or is there another approach to ensure that the tif can fit the annotations you provided?

Thank you very much!

Best,
Ranran

[martvanrijthoven]

Dear Ranran,

I have seen your email and responded, lets discuss this further in our meeting :).

Best wishes,
Mart