Problem with using directional method "PREDICTED" #19
Comments
|
Hi, thanks for getting in touch :-)
how did you train the crYOLO model? You need to train with filament annotations to get a filament model which can use "PREDICTED". The general model is only for single particles.
I didn't do a systematic comparison, but PREDICTED should be more flexible regarding the filament shape. CONVOLUTION expects that the filament has certain width.
Can you tell me what kind of error you saw? I just redid the installation as described in the tutorial and its working on my side. Best, |
|
Thanks for your response! Perhaps the napari is cause of all problems.
I still cannot install napari by conda. Here is the log.
I also tried download napari in dependent , it stopped in solving enviromnet. conda install -c conda-forge napari=0.4.17`
Collecting package metadata (current_repodata.json): / DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): conda.anaconda.org:443
DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): repo.anaconda.com:443
DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): conda.anaconda.org:443
DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): repo.anaconda.com:443
DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): repo.anaconda.com:443
DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): repo.anaconda.com:443
DEBUG:urllib3.connectionpool:https://repo.anaconda.com:443 "GET /pkgs/r/linux-64/current_repodata.json HTTP/1.1" 304 0 | DEBUG:urllib3.connectionpool:https://conda.anaconda.org:443 "GET /conda-forge/noarch/current_repodata.json HTTP/1.1" 200 None
DEBUG:urllib3.connectionpool:https://conda.anaconda.org:443 "GET /conda-forge/linux-64/current_repodata.json HTTP/1.1" 200 None - DEBUG:urllib3.connectionpool:https://repo.anaconda.com:443 "GET /pkgs/main/noarch/current_repodata.json HTTP/1.1" 304 0 - DEBUG:urllib3.connectionpool:https://repo.anaconda.com:443 "GET /pkgs/main/linux-64/current_repodata.json HTTP/1.1" 304 0
DEBUG:urllib3.connectionpool:https://repo.anaconda.com:443 "GET /pkgs/r/noarch/current_repodata.json HTTP/1.1" 304 0 done
Solving environment: unsuccessful initial attempt using frozen solve. Retrying with flexible solve.
Solving environment: unsuccessful attempt using repodata from current_repodata.json, retrying with next repodata source.
Collecting package metadata (repodata.json): | DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): conda.anaconda.org:443
DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): conda.anaconda.org:443
DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): repo.anaconda.com:443
DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): repo.anaconda.com:443
DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): repo.anaconda.com:443
DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): repo.anaconda.com:443 DEBUG:urllib3.connectionpool:https://conda.anaconda.org:443 "GET /conda-forge/noarch/repodata.json HTTP/1.1" 200 None DEBUG:urllib3.connectionpool:https://conda.anaconda.org:443 "GET /conda-forge/linux-64/repodata.json HTTP/1.1" 200 None DEBUG:urllib3.connectionpool:https://repo.anaconda.com:443 "GET /pkgs/main/linux-64/repodata.json HTTP/1.1" 304 0
DEBUG:urllib3.connectionpool:https://repo.anaconda.com:443 "GET /pkgs/r/linux-64/repodata.json HTTP/1.1" 304 0 | DEBUG:urllib3.connectionpool:https://repo.anaconda.com:443 "GET /pkgs/main/noarch/repodata.json HTTP/1.1" 304 0 / DEBUG:urllib3.connectionpool:https://repo.anaconda.com:443 "GET /pkgs/r/noarch/repodata.json HTTP/1.1" 304 0 done
Solving environment: \I'm not sure if all this is caused by network issues or the absence of this package on conda： conda install napari`
PackagesNotFoundError: The following packages are not available from current channels:
- napariFinally I try to download it by pip, thanksfully it works. （napari 0.4.18）
Is it because I deleted so many things that cryolo can't recognize the training data as filament? Maybe the napari installed from pip have some bug. I cannot load .cox data into CRYOLO when i start trainning. #####################################################
Important debugging information.
In case of any problems, please provide this information.
#####################################################
/home/em/anaconda3/envs/cryolo/bin/cryolo_gui.py train
-c config_cryolo.json
-w 5
-nc 16
--gpu_fraction 1.0
-e 10
-lft 2
--seed 10
#####################################################
###############################################
New version of crYOLO available
Local version: 1.8.4
Latest version: 1.9.6
More information here:
https://cryolo.readthedocs.io/en/latest/changes.html
###############################################
###############################################
The following training image sizes were detected:
4096 x 4096 ( N: 19 )
crYOLO will train in mode: SQUARE
###############################################
Reading old CBOX format file
2023-09-28 22:37:44.408501: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
WARNING:root:Limited tf.compat.v2.summary API due to missing TensorBoard installation.
Using TensorFlow backend.
2023-09-28 22:37:45.391510: I tensorflow/core/platform/profile_utils/cpu_utils.cc:94] CPU Frequency: 2000000000 Hz
2023-09-28 22:37:45.392759: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x55c37ef32190 initialized for platform Host (this does not guarantee that XLA will be used). Devices:
2023-09-28 22:37:45.392783: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): Host, Default Version
2023-09-28 22:37:45.396331: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcuda.so.1
2023-09-28 22:37:45.632035: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x55c37e424c40 initialized for platform CUDA (this does not guarantee that XLA will be used). Devices:
2023-09-28 22:37:45.632077: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): NVIDIA GeForce RTX 2080 Ti, Compute Capability 7.5
2023-09-28 22:37:45.633273: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1666] Found device 0 with properties:
name: NVIDIA GeForce RTX 2080 Ti major: 7 minor: 5 memoryClockRate(GHz): 1.545
pciBusID: 0000:1a:00.0
2023-09-28 22:37:45.633341: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
2023-09-28 22:37:45.645550: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublas.so.11
2023-09-28 22:37:45.674337: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcufft.so.10
2023-09-28 22:37:45.674665: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcurand.so.10
2023-09-28 22:37:45.675267: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusolver.so.11
2023-09-28 22:37:45.676324: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusparse.so.11
2023-09-28 22:37:45.676471: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudnn.so.8
2023-09-28 22:37:45.678536: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1794] Adding visible gpu devices: 0
2023-09-28 22:37:45.678600: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
2023-09-28 22:37:46.174525: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1206] Device interconnect StreamExecutor with strength 1 edge matrix:
2023-09-28 22:37:46.174567: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1212] 0
2023-09-28 22:37:46.174575: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1225] 0: N
2023-09-28 22:37:46.176901: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1351] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 6624 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:1a:00.0, compute capability: 7.5)
Traceback (most recent call last):
File "/home/em/anaconda3/envs/cryolo/lib/python3.8/site-packages/pandas/core/indexes/base.py", line 2895, in get_loc
return self._engine.get_loc(casted_key)
File "pandas/_libs/index.pyx", line 70, in pandas._libs.index.IndexEngine.get_loc
File "pandas/_libs/index.pyx", line 101, in pandas._libs.index.IndexEngine.get_loc
File "pandas/_libs/hashtable_class_helper.pxi", line 1675, in pandas._libs.hashtable.PyObjectHashTable.get_item
File "pandas/_libs/hashtable_class_helper.pxi", line 1683, in pandas._libs.hashtable.PyObjectHashTable.get_item
KeyError: '_CoordinateZ'
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/home/em/anaconda3/envs/cryolo/lib/python3.8/site-packages/cryolo/CoordsIO.py", line 230, in read_cbox_boxfile
z=starfile['cryolo']['_CoordinateZ'][i],
File "/home/em/anaconda3/envs/cryolo/lib/python3.8/site-packages/pandas/core/frame.py", line 2906, in __getitem__
indexer = self.columns.get_loc(key)
File "/home/em/anaconda3/envs/cryolo/lib/python3.8/site-packages/pandas/core/indexes/base.py", line 2897, in get_loc
raise KeyError(key) from err
KeyError: '_CoordinateZ'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/em/anaconda3/envs/cryolo/bin/cryolo_gui.py", line 8, in <module>
sys.exit(_main_())
File "/home/em/anaconda3/envs/cryolo/lib/python3.8/site-packages/cryolo/cryolo_main.py", line 455, in _main_
Gooey(
File "/home/em/anaconda3/envs/cryolo/lib/python3.8/site-packages/gooey/python_bindings/gooey_decorator.py", line 134, in <lambda>
return lambda *args, **kwargs: func(*args, **kwargs)
File "/home/em/anaconda3/envs/cryolo/lib/python3.8/site-packages/cryolo/cryolo_main.py", line 424, in main
train.main(args)
File "/home/em/anaconda3/envs/cryolo/lib/python3.8/site-packages/cryolo/train.py", line 516, in main
parse_dict = preprocess.parse_annotation(
File "/home/em/anaconda3/envs/cryolo/lib/python3.8/site-packages/cryolo/preprocessing.py", line 150, in parse_annotation
filaments = CoordsIO.read_cbox_boxfile(boxpath, int(cell_h))
File "/home/em/anaconda3/envs/cryolo/lib/python3.8/site-packages/cryolo/CoordsIO.py", line 265, in read_cbox_boxfile
return read_cbox_boxfile_old(path)
File "/home/em/anaconda3/envs/cryolo/lib/python3.8/site-packages/cryolo/CoordsIO.py", line 273, in read_cbox_boxfile_old
boxreader = np.atleast_2d(np.genfromtxt(path))
File "/home/em/anaconda3/envs/cryolo/lib/python3.8/site-packages/numpy/lib/npyio.py", line 2103, in genfromtxt
raise ValueError(errmsg)
ValueError: Some errors were detected !
Line #3 (got 2 columns instead of 1)
Line #13 (got 5 columns instead of 1)
Line #14 (got 5 columns instead of 1)
Line #15 (got 5 columns instead of 1)
Line #16 (got 5 columns instead of 1)
Line #17 (got 5 columns instead of 1)
Line #18 (got 5 columns instead of 1)
Line #19 (got 5 columns instead of 1)
Line #20 (got 5 columns instead of 1)
Line #21 (got 5 columns instead of 1)
Line #22 (got 5 columns instead of 1)
Line #23 (got 5 columns instead of 1)
Line #24 (got 5 columns instead of 1)
Line #25 (got 5 columns instead of 1)
Line #26 (got 5 columns instead of 1)
Line #27 (got 5 columns instead of 1)
Line #28 (got 5 columns instead of 1)
Line #38 (got 5 columns instead of 1)
Line #39 (got 5 columns instead of 1)
Line #40 (got 5 columns instead of 1)
Line #41 (got 5 columns instead of 1)
Line #42 (got 5 columns instead of 1)
Line #43 (got 5 columns instead of 1)
Line #44 (got 5 columns instead of 1)
Line #45 (got 5 columns instead of 1)
Line #46 (got 5 columns instead of 1)
Line #47 (got 5 columns instead of 1)
Line #48 (got 5 columns instead of 1)
Line #49 (got 5 columns instead of 1)
Line #50 (got 5 columns instead of 1)
Line #51 (got 5 columns instead of 1)
Line #52 (got 5 columns instead of 1)
Line #53 (got 5 columns instead of 1)
Line #54 (got 5 columns instead of 1)
Line #55 (got 5 columns instead of 1)
Line #56 (got 5 columns instead of 1)
Line #57 (got 5 columns instead of 1)
Line #58 (got 5 columns instead of 1)
Line #59 (got 5 columns instead of 1)
Line #60 (got 5 columns instead of 1)
Line #61 (got 5 columns instead of 1)
Line #62 (got 5 columns instead of 1)
Line #63 (got 5 columns instead of 1)
Line #64 (got 5 columns instead of 1)
Line #65 (got 5 columns instead of 1)
Line #66 (got 5 columns instead of 1)
Line #67 (got 5 columns instead of 1)
Line #68 (got 5 columns instead of 1)
Line #69 (got 5 columns instead of 1)
Line #70 (got 5 columns instead of 1)
Line #71 (got 5 columns instead of 1)
Line #72 (got 5 columns instead of 1)
Line #73 (got 5 columns instead of 1)
Line #74 (got 5 columns instead of 1)
Line #75 (got 5 columns instead of 1)
Line #76 (got 5 columns instead of 1)
Line #77 (got 5 columns instead of 1)
Line #78 (got 5 columns instead of 1)
Line #79 (got 5 columns instead of 1)
Line #80 (got 5 columns instead of 1)
Line #81 (got 5 columns instead of 1)
Line #82 (got 5 columns instead of 1)
Line #83 (got 5 columns instead of 1)
Line #84 (got 5 columns instead of 1)
Line #85 (got 5 columns instead of 1)
Line #86 (got 5 columns instead of 1)
Line #87 (got 5 columns instead of 1)
Line #88 (got 5 columns instead of 1)
Line #89 (got 5 columns instead of 1)
Line #90 (got 5 columns instead of 1)
Line #91 (got 5 columns instead of 1)
Line #92 (got 5 columns instead of 1)
Line #93 (got 5 columns instead of 1)
Line #94 (got 5 columns instead of 1)
Line #95 (got 5 columns instead of 1)
Line #96 (got 5 columns instead of 1)
Line #97 (got 5 columns instead of 1)
Line #98 (got 5 columns instead of 1)
Line #99 (got 5 columns instead of 1)
Line #100 (got 5 columns instead of 1)
Line #101 (got 5 columns instead of 1)
Line #102 (got 5 columns instead of 1)
Line #103 (got 5 columns instead of 1)
Line #104 (got 5 columns instead of 1)
Line #105 (got 5 columns instead of 1)
Line #106 (got 5 columns instead of 1)
Line #107 (got 5 columns instead of 1)
Line #108 (got 5 columns instead of 1)
Line #109 (got 5 columns instead of 1)
Line #110 (got 5 columns instead of 1)
Line #111 (got 5 columns instead of 1)
Line #112 (got 5 columns instead of 1)
Line #113 (got 5 columns instead of 1)
Line #114 (got 5 columns instead of 1)
Line #115 (got 5 columns instead of 1)
Line #116 (got 5 columns instead of 1)
Line #117 (got 5 columns instead of 1)
Line #118 (got 5 columns instead of 1)
Line #119 (got 5 columns instead of 1)
I check the .cox file： I changed the format, keeping only the X and Y coordinates of the box and the size of the box： With the new format, cryolo train can work. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Hello!
I use crYOLO for filament picking. I have chosen "Activate filament mode" and the crYOLO model. Although there were no errors when I was running predictions, the log indicates that I can't use the directional method "PREDICTED":
##########
The directional method "PREDICTED" can't be used as your model is not a filament model.You need to retrain your picking model. Fall back to old directional method "CONVOLUTION".
##########
How to use directional method "PREDICTED"? Also I want to konw weather PREDICTED is more precise than CONVOLUTION ?
By the way, tutorials document of Install napari and the boxmanager plugin is incorrect. I cannot install napari by conda while the command
pip install naparican work. :)The text was updated successfully, but these errors were encountered: