[Core] Ray.init() returiningTypeError: 'NoneType' object is not iterable

[harryjordancma]

What happened + What you expected to happen

Simply trying to run ray.init() throws error

Error message:

2023-11-30 18:14:16,304 ERROR services.py:1329 -- Failed to start the dashboard , return code 1
2023-11-30 18:14:16,305 ERROR services.py:1354 -- Error should be written to 'dashboard.log' or 'dashboard.err'. We are printing the last 20 lines for you. See '[https://docs.ray.io/en/master/ray-observability/ray-logging.html#logging-directory-structure'](https://docs.ray.io/en/master/ray-observability/ray-logging.html#logging-directory-structure%27) to find where the log file is.
2023-11-30 18:14:16,305 ERROR services.py:1398 -- 
The last 20 lines of /tmp/ray/session_2023-11-30_18-14-14_035354_3799/logs/dashboard.log (it contains the error message from the dashboard): 
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ubuntu/miniconda/envs/py311/lib/python3.11/site-packages/ray/dashboard/utils.py", line 134, in get_all_modules
    raise e
  File "/home/ubuntu/miniconda/envs/py311/lib/python3.11/site-packages/ray/dashboard/utils.py", line 121, in get_all_modules
    importlib.import_module(name)
  File "/home/ubuntu/miniconda/envs/py311/lib/python3.11/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<frozen importlib._bootstrap>", line 1204, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1176, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1147, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 690, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 940, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "/home/ubuntu/miniconda/envs/py311/lib/python3.11/site-packages/ray/dashboard/modules/test/test_agent.py", line 6, in <module>
    import ray.dashboard.modules.test.test_utils as test_utils
  File "/home/ubuntu/miniconda/envs/py311/lib/python3.11/site-packages/ray/dashboard/modules/test/test_utils.py", line 3, in <module>
    import async_timeout
ModuleNotFoundError: No module named 'async_timeout'
Traceback (most recent call last):
  File "/notebooks/Untitled.py", line 31, in <module>
    ray.init()
  File "/home/ubuntu/miniconda/envs/py311/lib/python3.11/site-packages/ray/_private/client_mode_hook.py", line 103, in wrapper
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/home/ubuntu/miniconda/envs/py311/lib/python3.11/site-packages/ray/_private/worker.py", line 1567, in init
    _global_node = ray._private.node.Node(
                   ^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ubuntu/miniconda/envs/py311/lib/python3.11/site-packages/ray/_private/node.py", line 310, in __init__
    self.start_ray_processes()
  File "/home/ubuntu/miniconda/envs/py311/lib/python3.11/site-packages/ray/_private/node.py", line 1357, in start_ray_processes
    resource_spec = self.get_resource_spec()
                    ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ubuntu/miniconda/envs/py311/lib/python3.11/site-packages/ray/_private/node.py", line 549, in get_resource_spec
    ).resolve(is_head=self.head, node_ip_address=self.node_ip_address)
      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ubuntu/miniconda/envs/py311/lib/python3.11/site-packages/ray/_private/resource_spec.py", line 200, in resolve
    accelerator_manager.get_current_node_num_accelerators()
  File "/home/ubuntu/miniconda/envs/py311/lib/python3.11/site-packages/ray/_private/accelerators/neuron.py", line 74, in get_current_node_num_accelerators
    for neuron_device in neuron_devices:
TypeError: 'NoneType' object is not iterable

Versions / Dependencies

Working in AWS EC2 on an Ubuntu22 AMI (not sure of details - can look them up if useful).

Using mamba to create a fresh environment, and pulling in ray 2.8.0.

name: py311
channels: 
  - conda-forge
  - nodefaults
dependencies:
  - python=3.11
  - ipykernel
  - pip
  - ray-default

Reproduction script

Having built env, as above (or otherwise).

# %%
import ray
 
# %%
ray.init()

Issue Severity

Medium: It is a significant difficulty but I can work around it.

[jtlz2]

What is your workaround?

Why don't you just pip install async_timeout?

[jonathan-anyscale]

Possible duplicate with #41267