icu-benchmarks train raises pkg_resources.DistributionNotFound on osx-arm64

[mlondschien]

Stacktrace

(yaib) ~/code/YAIB [development] $ icu-benchmarks train -d data -n mimic -t Regression -tn KidneyFunction -m ElasticNet
Traceback (most recent call last):
  File "/Users/mlondschien/mambaforge/envs/yaib/bin/icu-benchmarks", line 33, in <module>
    sys.exit(load_entry_point('yaib', 'console_scripts', 'icu-benchmarks')())
  File "/Users/mlondschien/mambaforge/envs/yaib/bin/icu-benchmarks", line 25, in importlib_load_entry_point
    return next(matches).load()
  File "/Users/mlondschien/mambaforge/envs/yaib/lib/python3.10/importlib/metadata/__init__.py", line 171, in load
    module = import_module(match.group('module'))
  File "/Users/mlondschien/mambaforge/envs/yaib/lib/python3.10/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 688, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 883, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "/Users/mlondschien/code/YAIB/icu_benchmarks/run.py", line 13, in <module>
    from icu_benchmarks.tuning.hyperparameters import choose_and_bind_hyperparameters
  File "/Users/mlondschien/code/YAIB/icu_benchmarks/tuning/hyperparameters.py", line 10, in <module>
    from icu_benchmarks.models.utils import JsonResultLoggingEncoder, log_table_row, Align
  File "/Users/mlondschien/code/YAIB/icu_benchmarks/models/utils.py", line 12, in <module>
    from pytorch_lightning.loggers.logger import Logger
  File "/Users/mlondschien/mambaforge/envs/yaib/lib/python3.10/site-packages/pytorch_lightning/__init__.py", line 6, in <module>
    from lightning_utilities import module_available
  File "/Users/mlondschien/mambaforge/envs/yaib/lib/python3.10/site-packages/lightning_utilities/__init__.py", line 6, in <module>
    from lightning_utilities.core.apply_func import apply_to_collection
  File "/Users/mlondschien/mambaforge/envs/yaib/lib/python3.10/site-packages/lightning_utilities/core/__init__.py", line 4, in <module>
    from lightning_utilities.core.imports import compare_version, module_available
  File "/Users/mlondschien/mambaforge/envs/yaib/lib/python3.10/site-packages/lightning_utilities/core/imports.py", line 13, in <module>
    import pkg_resources
  File "/Users/mlondschien/mambaforge/envs/yaib/lib/python3.10/site-packages/pkg_resources/__init__.py", line 3328, in <module>
    def _initialize_master_working_set():
  File "/Users/mlondschien/mambaforge/envs/yaib/lib/python3.10/site-packages/pkg_resources/__init__.py", line 3302, in _call_aside
    f(*args, **kwargs)
  File "/Users/mlondschien/mambaforge/envs/yaib/lib/python3.10/site-packages/pkg_resources/__init__.py", line 3340, in _initialize_master_working_set
    working_set = WorkingSet._build_master()
  File "/Users/mlondschien/mambaforge/envs/yaib/lib/python3.10/site-packages/pkg_resources/__init__.py", line 631, in _build_master
    ws.require(__requires__)
  File "/Users/mlondschien/mambaforge/envs/yaib/lib/python3.10/site-packages/pkg_resources/__init__.py", line 968, in require
    needed = self.resolve(parse_requirements(requirements))
  File "/Users/mlondschien/mambaforge/envs/yaib/lib/python3.10/site-packages/pkg_resources/__init__.py", line 829, in resolve
    dist = self._resolve_dist(
  File "/Users/mlondschien/mambaforge/envs/yaib/lib/python3.10/site-packages/pkg_resources/__init__.py", line 870, in _resolve_dist
    raise DistributionNotFound(req, requirers)
pkg_resources.DistributionNotFound: The 'pytorch-cuda==11.8' distribution was not found and is required by yaib

The issue lies here, where installation requirements are parsed from the environment.yml (not environment_mps.yml), which includes pytorch-cuda, that is unsatisfiable.

This could be solved by detecting whether the code is run on an x86/osx-arm64 machine and then parsing the corresponding environment file. Alternatively, one can also just comment out the offending line. IMO the latter is the much cleaner option. What is the benefit of enforcing exact requirements?

[rvandewater]

Addressed in #126. This was an alternative way of installing via pip. Will see to add this as an optional parameter in the future.

[mlondschien]

Thank you :)