We can use Wechat if it is fine for you. My id: Dongshu392-TohokuU (fgdal) dongzengao@DONG:/mnt/d/AAA_DATA/Test_data/3DBIE-SolarPV-main/3DBIE-SolarPV-main/UDA-Seg$ bash train_with_sd.sh /home/dongzengao/anaconda3/envs/fgdal/lib/python3.8/site-packages/torch/distributed/launch.py:178: FutureWarning: The module torch.distributed.launch is deprecated and will be removed in future. Use torchrun. Note that --use_env is set by default in torchrun. If your script expects `--local_rank` argument to be set, please change it to read from `os.environ['LOCAL_RANK']` instead. See https://pytorch.org/docs/stable/distributed.html#launch-utility for further instructions warnings.warn( WARNING:torch.distributed.run: ***************************************** Setting OMP_NUM_THREADS environment variable for each process to be 1 in default, to avoid your system being overloaded, please further tune the variable for optimal performance in your application as needed. ***************************************** Traceback (most recent call last): File "train_src.py", line 13, in from core.datasets import build_dataset File "/mnt/d/AAA_DATA/Test_data/3DBIE-SolarPV-main/3DBIE-SolarPV-main/UDA-Seg/core/datasets/__init__.py", line 1, in from .build import build_dataset File "/mnt/d/AAA_DATA/Test_data/3DBIE-SolarPV-main/3DBIE-SolarPV-main/UDA-Seg/core/datasets/build.py", line 4, in from .dataset_path_catalog import DatasetCatalog File "/mnt/d/AAA_DATA/Test_data/3DBIE-SolarPV-main/3DBIE-SolarPV-main/UDA-Seg/core/datasets/dataset_path_catalog.py", line 3, in from .cityscapes_self_distill import cityscapesSelfDistillDataSet ModuleNotFoundError: No module named 'core.datasets.cityscapes_self_distill' Traceback (most recent call last): File "train_src.py", line 13, in from core.datasets import build_dataset File "/mnt/d/AAA_DATA/Test_data/3DBIE-SolarPV-main/3DBIE-SolarPV-main/UDA-Seg/core/datasets/__init__.py", line 1, in from .build import build_dataset File "/mnt/d/AAA_DATA/Test_data/3DBIE-SolarPV-main/3DBIE-SolarPV-main/UDA-Seg/core/datasets/build.py", line 4, in from .dataset_path_catalog import DatasetCatalog File "/mnt/d/AAA_DATA/Test_data/3DBIE-SolarPV-main/3DBIE-SolarPV-main/UDA-Seg/core/datasets/dataset_path_catalog.py", line 3, in from .cityscapes_self_distill import cityscapesSelfDistillDataSet ModuleNotFoundError: No module named 'core.datasets.cityscapes_self_distill' ERROR:torch.distributed.elastic.multiprocessing.api:failed (exitcode: 1) local_rank: 0 (pid: 1531) of binary: /home/dongzengao/anaconda3/envs/fgdal/bin/python Traceback (most recent call last): File "/home/dongzengao/anaconda3/envs/fgdal/lib/python3.8/runpy.py", line 192, in _run_module_as_main return _run_code(code, main_globals, None, File "/home/dongzengao/anaconda3/envs/fgdal/lib/python3.8/runpy.py", line 85, in _run_code exec(code, run_globals) File "/home/dongzengao/anaconda3/envs/fgdal/lib/python3.8/site-packages/torch/distributed/launch.py", line 193, in main() File "/home/dongzengao/anaconda3/envs/fgdal/lib/python3.8/site-packages/torch/distributed/launch.py", line 189, in main launch(args) File "/home/dongzengao/anaconda3/envs/fgdal/lib/python3.8/site-packages/torch/distributed/launch.py", line 174, in launch run(args) File "/home/dongzengao/anaconda3/envs/fgdal/lib/python3.8/site-packages/torch/distributed/run.py", line 752, in run elastic_launch( File "/home/dongzengao/anaconda3/envs/fgdal/lib/python3.8/site-packages/torch/distributed/launcher/api.py", line 131, in __call__ return launch_agent(self._config, self._entrypoint, list(args)) File "/home/dongzengao/anaconda3/envs/fgdal/lib/python3.8/site-packages/torch/distributed/launcher/api.py", line 245, in launch_agent raise ChildFailedError( torch.distributed.elastic.multiprocessing.errors.ChildFailedError: ============================================================ train_src.py FAILED ------------------------------------------------------------ Failures: [1]: time : 2024-04-21_23:29:58 host : DONG. rank : 1 (local_rank: 1) exitcode : 1 (pid: 1532) error_file: traceback : To enable traceback see: https://pytorch.org/docs/stable/elastic/errors.html ------------------------------------------------------------ Root Cause (first observed failure): [0]: time : 2024-04-21_23:29:58 host : DONG. rank : 0 (local_rank: 0) exitcode : 1 (pid: 1531) error_file: traceback : To enable traceback see: https://pytorch.org/docs/stable/elastic/errors.html ============================================================ /home/dongzengao/anaconda3/envs/fgdal/lib/python3.8/site-packages/torch/distributed/launch.py:178: FutureWarning: The module torch.distributed.launch is deprecated and will be removed in future. Use torchrun. Note that --use_env is set by default in torchrun. If your script expects `--local_rank` argument to be set, please change it to read from `os.environ['LOCAL_RANK']` instead. See https://pytorch.org/docs/stable/distributed.html#launch-utility for further instructions warnings.warn( WARNING:torch.distributed.run: ***************************************** Setting OMP_NUM_THREADS environment variable for each process to be 1 in default, to avoid your system being overloaded, please further tune the variable for optimal performance in your application as needed. ***************************************** Traceback (most recent call last): File "train_SEmsf_dnt_adv_BD+FD_FreezeBackbone.py", line 15, in Traceback (most recent call last): File "train_SEmsf_dnt_adv_BD+FD_FreezeBackbone.py", line 15, in from core.datasets import build_dataset File "/mnt/d/AAA_DATA/Test_data/3DBIE-SolarPV-main/3DBIE-SolarPV-main/UDA-Seg/core/datasets/__init__.py", line 1, in from core.datasets import build_dataset File "/mnt/d/AAA_DATA/Test_data/3DBIE-SolarPV-main/3DBIE-SolarPV-main/UDA-Seg/core/datasets/__init__.py", line 1, in from .build import build_dataset File "/mnt/d/AAA_DATA/Test_data/3DBIE-SolarPV-main/3DBIE-SolarPV-main/UDA-Seg/core/datasets/build.py", line 4, in from .build import build_dataset File "/mnt/d/AAA_DATA/Test_data/3DBIE-SolarPV-main/3DBIE-SolarPV-main/UDA-Seg/core/datasets/build.py", line 4, in from .dataset_path_catalog import DatasetCatalog File "/mnt/d/AAA_DATA/Test_data/3DBIE-SolarPV-main/3DBIE-SolarPV-main/UDA-Seg/core/datasets/dataset_path_catalog.py", line 3, in from .dataset_path_catalog import DatasetCatalog File "/mnt/d/AAA_DATA/Test_data/3DBIE-SolarPV-main/3DBIE-SolarPV-main/UDA-Seg/core/datasets/dataset_path_catalog.py", line 3, in from .cityscapes_self_distill import cityscapesSelfDistillDataSet ModuleNotFoundError: No module named 'core.datasets.cityscapes_self_distill' from .cityscapes_self_distill import cityscapesSelfDistillDataSet ModuleNotFoundError: No module named 'core.datasets.cityscapes_self_distill' ERROR:torch.distributed.elastic.multiprocessing.api:failed (exitcode: 1) local_rank: 0 (pid: 1578) of binary: /home/dongzengao/anaconda3/envs/fgdal/bin/python Traceback (most recent call last): File "/home/dongzengao/anaconda3/envs/fgdal/lib/python3.8/runpy.py", line 192, in _run_module_as_main return _run_code(code, main_globals, None, File "/home/dongzengao/anaconda3/envs/fgdal/lib/python3.8/runpy.py", line 85, in _run_code exec(code, run_globals) File "/home/dongzengao/anaconda3/envs/fgdal/lib/python3.8/site-packages/torch/distributed/launch.py", line 193, in main() File "/home/dongzengao/anaconda3/envs/fgdal/lib/python3.8/site-packages/torch/distributed/launch.py", line 189, in main launch(args) File "/home/dongzengao/anaconda3/envs/fgdal/lib/python3.8/site-packages/torch/distributed/launch.py", line 174, in launch run(args) File "/home/dongzengao/anaconda3/envs/fgdal/lib/python3.8/site-packages/torch/distributed/run.py", line 752, in run elastic_launch( File "/home/dongzengao/anaconda3/envs/fgdal/lib/python3.8/site-packages/torch/distributed/launcher/api.py", line 131, in __call__ return launch_agent(self._config, self._entrypoint, list(args)) File "/home/dongzengao/anaconda3/envs/fgdal/lib/python3.8/site-packages/torch/distributed/launcher/api.py", line 245, in launch_agent raise ChildFailedError( torch.distributed.elastic.multiprocessing.errors.ChildFailedError: ============================================================ train_SEmsf_dnt_adv_BD+FD_FreezeBackbone.py FAILED ------------------------------------------------------------ Failures: [1]: time : 2024-04-21_23:30:04 host : DONG. rank : 1 (local_rank: 1) exitcode : 1 (pid: 1579) error_file: traceback : To enable traceback see: https://pytorch.org/docs/stable/elastic/errors.html ------------------------------------------------------------ Root Cause (first observed failure): [0]: time : 2024-04-21_23:30:04 host : DONG. rank : 0 (local_rank: 0) exitcode : 1 (pid: 1578) error_file: traceback : To enable traceback see: https://pytorch.org/docs/stable/elastic/errors.html ============================================================ Optionally, required to modify the network in the code to conduct self distill Traceback (most recent call last): File "test.py", line 16, in from core.datasets import build_dataset File "/mnt/d/AAA_DATA/Test_data/3DBIE-SolarPV-main/3DBIE-SolarPV-main/UDA-Seg/core/datasets/__init__.py", line 1, in from .build import build_dataset File "/mnt/d/AAA_DATA/Test_data/3DBIE-SolarPV-main/3DBIE-SolarPV-main/UDA-Seg/core/datasets/build.py", line 4, in from .dataset_path_catalog import DatasetCatalog File "/mnt/d/AAA_DATA/Test_data/3DBIE-SolarPV-main/3DBIE-SolarPV-main/UDA-Seg/core/datasets/dataset_path_catalog.py", line 3, in from .cityscapes_self_distill import cityscapesSelfDistillDataSet ModuleNotFoundError: No module named 'core.datasets.cityscapes_self_distill' /home/dongzengao/anaconda3/envs/fgdal/lib/python3.8/site-packages/torch/distributed/launch.py:178: FutureWarning: The module torch.distributed.launch is deprecated and will be removed in future. Use torchrun. Note that --use_env is set by default in torchrun. If your script expects `--local_rank` argument to be set, please change it to read from `os.environ['LOCAL_RANK']` instead. See https://pytorch.org/docs/stable/distributed.html#launch-utility for further instructions warnings.warn( WARNING:torch.distributed.run: ***************************************** Setting OMP_NUM_THREADS environment variable for each process to be 1 in default, to avoid your system being overloaded, please further tune the variable for optimal performance in your application as needed. ***************************************** Traceback (most recent call last): File "train_self_distill.py", line 14, in from core.datasets import build_dataset File "/mnt/d/AAA_DATA/Test_data/3DBIE-SolarPV-main/3DBIE-SolarPV-main/UDA-Seg/core/datasets/__init__.py", line 1, in from .build import build_dataset File "/mnt/d/AAA_DATA/Test_data/3DBIE-SolarPV-main/3DBIE-SolarPV-main/UDA-Seg/core/datasets/build.py", line 4, in Traceback (most recent call last): File "train_self_distill.py", line 14, in from core.datasets import build_dataset File "/mnt/d/AAA_DATA/Test_data/3DBIE-SolarPV-main/3DBIE-SolarPV-main/UDA-Seg/core/datasets/__init__.py", line 1, in from .dataset_path_catalog import DatasetCatalog File "/mnt/d/AAA_DATA/Test_data/3DBIE-SolarPV-main/3DBIE-SolarPV-main/UDA-Seg/core/datasets/dataset_path_catalog.py", line 3, in from .cityscapes_self_distill import cityscapesSelfDistillDataSet ModuleNotFoundError: No module named 'core.datasets.cityscapes_self_distill' from .build import build_dataset File "/mnt/d/AAA_DATA/Test_data/3DBIE-SolarPV-main/3DBIE-SolarPV-main/UDA-Seg/core/datasets/build.py", line 4, in from .dataset_path_catalog import DatasetCatalog File "/mnt/d/AAA_DATA/Test_data/3DBIE-SolarPV-main/3DBIE-SolarPV-main/UDA-Seg/core/datasets/dataset_path_catalog.py", line 3, in from .cityscapes_self_distill import cityscapesSelfDistillDataSet ModuleNotFoundError: No module named 'core.datasets.cityscapes_self_distill' ERROR:torch.distributed.elastic.multiprocessing.api:failed (exitcode: 1) local_rank: 0 (pid: 1644) of binary: /home/dongzengao/anaconda3/envs/fgdal/bin/python Traceback (most recent call last): File "/home/dongzengao/anaconda3/envs/fgdal/lib/python3.8/runpy.py", line 192, in _run_module_as_main return _run_code(code, main_globals, None, File "/home/dongzengao/anaconda3/envs/fgdal/lib/python3.8/runpy.py", line 85, in _run_code exec(code, run_globals) File "/home/dongzengao/anaconda3/envs/fgdal/lib/python3.8/site-packages/torch/distributed/launch.py", line 193, in main() File "/home/dongzengao/anaconda3/envs/fgdal/lib/python3.8/site-packages/torch/distributed/launch.py", line 189, in main launch(args) File "/home/dongzengao/anaconda3/envs/fgdal/lib/python3.8/site-packages/torch/distributed/launch.py", line 174, in launch run(args) File "/home/dongzengao/anaconda3/envs/fgdal/lib/python3.8/site-packages/torch/distributed/run.py", line 752, in run elastic_launch( File "/home/dongzengao/anaconda3/envs/fgdal/lib/python3.8/site-packages/torch/distributed/launcher/api.py", line 131, in __call__ return launch_agent(self._config, self._entrypoint, list(args)) File "/home/dongzengao/anaconda3/envs/fgdal/lib/python3.8/site-packages/torch/distributed/launcher/api.py", line 245, in launch_agent raise ChildFailedError( torch.distributed.elastic.multiprocessing.errors.ChildFailedError: ============================================================ train_self_distill.py FAILED ------------------------------------------------------------ Failures: [1]: time : 2024-04-21_23:30:12 host : DONG. rank : 1 (local_rank: 1) exitcode : 1 (pid: 1645) error_file: traceback : To enable traceback see: https://pytorch.org/docs/stable/elastic/errors.html ------------------------------------------------------------ Root Cause (first observed failure): [0]: time : 2024-04-21_23:30:12 host : DONG. rank : 0 (local_rank: 0) exitcode : 1 (pid: 1644) error_file: traceback : To enable traceback see: https://pytorch.org/docs/stable/elastic/errors.html ============================================================