Got ModuleNotFoundError: No module named 'clvm.CLVMObject'

[ibamong]

After I installed and try simple cmd. I got this error. any idea : D

[roschler]

I am getting this error when I run py.tests:

========================================= test session starts ==========================================
platform linux -- Python 3.7.5, pytest-6.2.4, py-1.10.0, pluggy-0.13.1
rootdir: /home/user/GitHub/chia/clvm_tools
collected 125 items / 6 errors / 119 selected

================================================ ERRORS ================================================
_________________________________ ERROR collecting tests/clvmc_test.py _________________________________
ImportError while importing test module '/home/user/GitHub/chia/clvm_tools/tests/clvmc_test.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
/usr/lib/python3.7/importlib/__init__.py:127: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
tests/clvmc_test.py:10: in <module>
    from clvm_tools import clvmc
clvm_tools/clvmc.py:9: in <module>
    from ir import reader
ir/reader.py:6: in <module>
    from clvm.CLVMObject import CLVMObject
E   ModuleNotFoundError: No module named 'clvm.CLVMObject'
_________________________________ ERROR collecting tests/curry_test.py _________________________________
ImportError while importing test module '/home/user/GitHub/chia/clvm_tools/tests/curry_test.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
/usr/lib/python3.7/importlib/__init__.py:127: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
tests/curry_test.py:3: in <module>
    from clvm_tools.binutils import assemble, disassemble
clvm_tools/binutils.py:6: in <module>
    from ir.reader import read_ir
ir/reader.py:6: in <module>
    from clvm.CLVMObject import CLVMObject
E   ModuleNotFoundError: No module named 'clvm.CLVMObject'
_______________________________ ERROR collecting tests/ir_reader_test.py _______________________________
ImportError while importing test module '/home/user/GitHub/chia/clvm_tools/tests/ir_reader_test.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
/usr/lib/python3.7/importlib/__init__.py:127: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
tests/ir_reader_test.py:1: in <module>
    from ir import reader
ir/reader.py:6: in <module>
    from clvm.CLVMObject import CLVMObject
E   ModuleNotFoundError: No module named 'clvm.CLVMObject'
_______________________________ ERROR collecting tests/ir_writer_test.py _______________________________
ImportError while importing test module '/home/user/GitHub/chia/clvm_tools/tests/ir_writer_test.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
/usr/lib/python3.7/importlib/__init__.py:127: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
tests/ir_writer_test.py:1: in <module>
    from ir import reader, writer
ir/reader.py:6: in <module>
    from clvm.CLVMObject import CLVMObject
E   ModuleNotFoundError: No module named 'clvm.CLVMObject'
_____________________________ ERROR collecting tests/pattern_match_test.py _____________________________
ImportError while importing test module '/home/user/GitHub/chia/clvm_tools/tests/pattern_match_test.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
/usr/lib/python3.7/importlib/__init__.py:127: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
tests/pattern_match_test.py:1: in <module>
    from clvm_tools.binutils import assemble
clvm_tools/binutils.py:6: in <module>
    from ir.reader import read_ir
ir/reader.py:6: in <module>
    from clvm.CLVMObject import CLVMObject
E   ModuleNotFoundError: No module named 'clvm.CLVMObject'
________________________________ ERROR collecting tests/reader_test.py _________________________________
ImportError while importing test module '/home/user/GitHub/chia/clvm_tools/tests/reader_test.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
/usr/lib/python3.7/importlib/__init__.py:127: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
tests/reader_test.py:1: in <module>
    from ir.reader import read_ir
ir/reader.py:6: in <module>
    from clvm.CLVMObject import CLVMObject
E   ModuleNotFoundError: No module named 'clvm.CLVMObject'
=========================================== warnings summary ===========================================
venv/lib/python3.7/site-packages/pkg_resources/_vendor/pyparsing.py:943
  /home/user/GitHub/chia/clvm_tools/venv/lib/python3.7/site-packages/pkg_resources/_vendor/pyparsing.py:943: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated since Python 3.3,and in 3.9 it will stop working
    collections.MutableMapping.register(ParseResults)

venv/lib/python3.7/site-packages/pkg_resources/_vendor/pyparsing.py:3226
  /home/user/GitHub/chia/clvm_tools/venv/lib/python3.7/site-packages/pkg_resources/_vendor/pyparsing.py:3226: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated since Python 3.3,and in 3.9 it will stop working
    elif isinstance( exprs, collections.Iterable ):

-- Docs: https://docs.pytest.org/en/stable/warnings.html
======================================= short test summary info ========================================
ERROR tests/clvmc_test.py
ERROR tests/curry_test.py
ERROR tests/ir_reader_test.py
ERROR tests/ir_writer_test.py
ERROR tests/pattern_match_test.py
ERROR tests/reader_test.py
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! Interrupted: 6 errors during collection !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
==================================== 2 warnings, 6 errors in 0.62s =====================================

[arvidn]

you also need to install clvm into your venv, and possibly also clvm_rs

[arvidn]

all these dependencies are installed into a venv by the setup.sh script in the chia-blockchain repo

[eFishCent]

Can someone who is running into this issue please post their exact OS, python version, and python related packages they are attempting this so I can try to reproduce the problem internally to troubleshoot?

[trosos]

Thank you, @eFishCent, for looking into this.
The issue can be reproduced in the isolated environment provided by debootstrap:

Sandboxed reproduction of the issue

$ uname -srvmo
Linux 5.10.0-10-amd64 #1 SMP Debian 5.10.84-1 (2021-12-08) x86_64 GNU/Linux
$ sudo -i
# mkdir /tmp/chroot
# debootstrap --variant=minbase --include=python3-venv,git buster /tmp/chroot
# chroot /tmp/chroot
# git clone https://github.com/Chia-Network/clvm_tools.git /clvm_tools
# cd /clvm_tools
# git reset --hard 10014eb79376c8ce314cef39b0f5a5280f34f73c
# python3 -m venv venv
# . ./venv/bin/activate
(venv) # pip install -e .
Obtaining file:///clvm_tools
  Installing build dependencies ... done
Collecting clvm>=0.9.2 (from clvm-tools==0.4.4)
  Downloading https://files.pythonhosted.org/packages/7a/9e/999f3a4a58dcdc18fd2003f41cc9d2402fde1bad8823bcb3b8e2eaeccf88/clvm-0.41-py3-none-any.whl
Installing collected packages: clvm, clvm-tools
  Running setup.py develop for clvm-tools
Successfully installed clvm-0.41 clvm-tools
(venv) # brun
Traceback (most recent call last):
  File "/clvm_tools/venv/bin/brun", line 33, in <module>
    sys.exit(load_entry_point('clvm-tools', 'console_scripts', 'brun')())
  File "/clvm_tools/venv/lib/python3.7/site-packages/pkg_resources/__init__.py", line 489, in load_entry_point
    return get_distribution(dist).load_entry_point(group, name)
  File "/clvm_tools/venv/lib/python3.7/site-packages/pkg_resources/__init__.py", line 2793, in load_entry_point
    return ep.load()
  File "/clvm_tools/venv/lib/python3.7/site-packages/pkg_resources/__init__.py", line 2411, in load
    return self.resolve()
  File "/clvm_tools/venv/lib/python3.7/site-packages/pkg_resources/__init__.py", line 2417, in resolve
    module = __import__(self.module_name, fromlist=['__name__'], level=0)
  File "/clvm_tools/clvm_tools/cmds.py", line 14, in <module>
    from ir import reader
  File "/clvm_tools/ir/reader.py", line 6, in <module>
    from clvm.CLVMObject import CLVMObject
ModuleNotFoundError: No module named 'clvm.CLVMObject'

Workaround

As a workaround, it helps to hand-build clvm first (cmake>=3.14.0 required for this):

$ sudo -i
# mkdir /tmp/chroot
# debootstrap --variant=buildd --include=python3-venv,git,cmake,python3-dev,software-properties-common buster /tmp/chroot
# chroot /tmp/chroot
# add-apt-repository --update 'deb http://deb.debian.org/debian buster-backports main'
# apt -t=buster-backports install cmake cmake-data
# git clone https://github.com/Chia-Network/clvm.git /clvm
# git -C /clvm reset --hard fa1ba1034def7e5e57d2b13be78ccc0941e599ff
# git clone https://github.com/Chia-Network/clvm_tools.git /clvm_tools
# git -C /clvm_tools reset --hard 10014eb79376c8ce314cef39b0f5a5280f34f73c
# python3 -m venv venv
# . ./venv/bin/activate
(venv) # cd /clvm
(venv) # pip install -e '.[dev]'
Obtaining file:///clvm
  Installing build dependencies ... done
Collecting blspy>=0.9 (from clvm==0.9.8.dev9)
  Downloading https://files.pythonhosted.org/packages/5f/ec/10a8625f4b22ee7376fc2b199a40f372b26faa8408bc431d81a091cf32b2/blspy-1.0.9.tar.gz (161kB)
    100% |................................| 163kB 2.2MB/s 
  Installing build dependencies ... done
Collecting clvm_tools>=0.4.2 (from clvm==0.9.8.dev9)
  Downloading https://files.pythonhosted.org/packages/ac/a9/0249ef97d1e84658559b35317595c0e6164989770659bd62b2c3063f3ab0/clvm_tools-0.4.4-py3-none-any.whl
Collecting pytest (from clvm==0.9.8.dev9)
  Downloading https://files.pythonhosted.org/packages/d2/ac/556e4410326ce77eeb1d1ec35a3e3ec847fb3e5cb30673729d2eeeffc970/pytest-7.1.1-py3-none-any.whl (297kB)
    100% |................................| 307kB 1.9MB/s 
Collecting wheel (from blspy>=0.9->clvm==0.9.8.dev9)
  Using cached https://files.pythonhosted.org/packages/27/d6/003e593296a85fd6ed616ed962795b2f87709c3eee2bca4f6d0fe55c6d00/wheel-0.37.1-py2.py3-none-any.whl
Collecting packaging (from pytest->clvm==0.9.8.dev9)
  Using cached https://files.pythonhosted.org/packages/05/8e/8de486cbd03baba4deef4142bd643a3e7bbe954a784dc1bb17142572d127/packaging-21.3-py3-none-any.whl
Collecting attrs>=19.2.0 (from pytest->clvm==0.9.8.dev9)
  Downloading https://files.pythonhosted.org/packages/be/be/7abce643bfdf8ca01c48afa2ddf8308c2308b0c3b239a44e57d020afa0ef/attrs-21.4.0-py2.py3-none-any.whl (60kB)
    100% |................................| 61kB 2.6MB/s 
Collecting tomli>=1.0.0 (from pytest->clvm==0.9.8.dev9)
  Using cached https://files.pythonhosted.org/packages/97/75/10a9ebee3fd790d20926a90a2547f0bf78f371b2f13aa822c759680ca7b9/tomli-2.0.1-py3-none-any.whl
Collecting importlib-metadata>=0.12; python_version < "3.8" (from pytest->clvm==0.9.8.dev9)
  Downloading https://files.pythonhosted.org/packages/92/f2/c48787ca7d1e20daa185e1b6b2d4e16acd2fb5e0320bc50ffc89b91fa4d7/importlib_metadata-4.11.3-py3-none-any.whl
Collecting py>=1.8.2 (from pytest->clvm==0.9.8.dev9)
  Downloading https://files.pythonhosted.org/packages/f6/f0/10642828a8dfb741e5f3fbaac830550a518a775c7fff6f04a007259b0548/py-1.11.0-py2.py3-none-any.whl (98kB)
    100% |................................| 102kB 4.1MB/s 
Collecting iniconfig (from pytest->clvm==0.9.8.dev9)
  Downloading https://files.pythonhosted.org/packages/9b/dd/b3c12c6d707058fa947864b67f0c4e0c39ef8610988d7baea9578f3c48f3/iniconfig-1.1.1-py2.py3-none-any.whl
Collecting pluggy<2.0,>=0.12 (from pytest->clvm==0.9.8.dev9)
  Downloading https://files.pythonhosted.org/packages/9e/01/f38e2ff29715251cf25532b9082a1589ab7e4f571ced434f98d0139336dc/pluggy-1.0.0-py2.py3-none-any.whl
Collecting pyparsing!=3.0.5,>=2.0.2 (from packaging->pytest->clvm==0.9.8.dev9)
  Using cached https://files.pythonhosted.org/packages/80/c1/23fd82ad3121656b585351aba6c19761926bb0db2ebed9e4ff09a43a3fcc/pyparsing-3.0.7-py3-none-any.whl
Collecting zipp>=0.5 (from importlib-metadata>=0.12; python_version < "3.8"->pytest->clvm==0.9.8.dev9)
  Downloading https://files.pythonhosted.org/packages/52/c5/df7953fe6065185af5956265e3b16f13c2826c2b1ba23d43154f3af453bc/zipp-3.7.0-py3-none-any.whl
Collecting typing-extensions>=3.6.4; python_version < "3.8" (from importlib-metadata>=0.12; python_version < "3.8"->pytest->clvm==0.9.8.dev9)
  Downloading https://files.pythonhosted.org/packages/45/6b/44f7f8f1e110027cf88956b59f2fad776cca7e1704396d043f89effd3a0e/typing_extensions-4.1.1-py3-none-any.whl
Building wheels for collected packages: blspy
  Running setup.py bdist_wheel for blspy ... done
  Stored in directory: /root/.cache/pip/wheels/7b/ef/06/e8500f84c3e1e5a3d98fbdb0f43923d743ed63e4ab91074c93
Successfully built blspy
Installing collected packages: wheel, blspy, clvm-tools, pyparsing, packaging, attrs, tomli, zipp, typing-extensions, importlib-metadata, py, iniconfig, pluggy, pytest, clvm
  Running setup.py develop for clvm
Successfully installed attrs-21.4.0 blspy-1.0.9 clvm clvm-tools-0.4.4 importlib-metadata-4.11.3 iniconfig-1.1.1 packaging-21.3 pluggy-1.0.0 py-1.11.0 pyparsing-3.0.7 pytest-7.1.1 tomli-2.0.1 typing-extensions-4.1.1 wheel-0.37.1 zipp-3.7.0
(venv) # cd /clvm_tools
(venv) # pip install -e .
Obtaining file:///clvm_tools
  Installing build dependencies ... done
Requirement already satisfied: clvm>=0.9.2 in /clvm (from clvm-tools==0.4.4) (0.9.8.dev9)
Requirement already satisfied: blspy>=0.9 in /venv/lib/python3.7/site-packages (from clvm>=0.9.2->clvm-tools==0.4.4) (1.0.9)
Requirement already satisfied: wheel in /venv/lib/python3.7/site-packages (from blspy>=0.9->clvm>=0.9.2->clvm-tools==0.4.4) (0.37.1)
Installing collected packages: clvm-tools
  Found existing installation: clvm-tools 0.4.4
    Uninstalling clvm-tools-0.4.4:
      Successfully uninstalled clvm-tools-0.4.4
  Running setup.py develop for clvm-tools
Successfully installed clvm-tools
(venv) # brun
usage: brun [-h] [--strict] [--mempool] [-x] [-s STAGE] [-v] [-t] [-c]
            [--time] [-m MAX_COST] [-d] [--quiet] [-y SYMBOL_TABLE] [-n]
            [--backend BACKEND] [-i INCLUDE]
            path_or_code [env]
brun: error: the following arguments are required: path_or_code

Debian bullseye package versions work

Please also note that with newer packages (Debian bullseye), the documented method works:

$ uname -srvmo
Linux 5.10.0-10-amd64 #1 SMP Debian 5.10.84-1 (2021-12-08) x86_64 GNU/Linux
$ sudo -i
# mkdir /tmp/chroot
# debootstrap --variant=minbase --include=python3-venv,git bullseye /tmp/chroot
# chroot /tmp/chroot
# git clone https://github.com/Chia-Network/clvm_tools.git /clvm_tools
# cd /clvm_tools
# git reset --hard 10014eb79376c8ce314cef39b0f5a5280f34f73c
# python3 -m venv venv
# . ./venv/bin/activate
(venv) # pip install -e .
Obtaining file:///clvm_tools
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
    Preparing wheel metadata ... done
Collecting clvm>=0.9.2
  Downloading clvm-0.9.7-py3-none-any.whl (19 kB)
Collecting blspy>=0.9
  Downloading blspy-1.0.9-cp39-cp39-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (822 kB)
     |................................| 822 kB 1.0 MB/s 
Collecting wheel
  Using cached wheel-0.37.1-py2.py3-none-any.whl (35 kB)
Installing collected packages: wheel, blspy, clvm, clvm-tools
  Running setup.py develop for clvm-tools
Successfully installed blspy-1.0.9 clvm-0.9.7 clvm-tools wheel-0.37.1
(venv) # brun
usage: brun [-h] [--strict] [--mempool] [-x] [-s STAGE] [-v] [-t] [-c]
            [--time] [-m MAX_COST] [-d] [--quiet] [-y SYMBOL_TABLE] [-n]
            [--backend BACKEND] [-i INCLUDE]
            path_or_code [env]
brun: error: the following arguments are required: path_or_code

Speculation

From the above, it seems that the problem is in the version of the clvm wheel that gets downloaded:

1. With packages from Debian buster (older), following version is downloaded (which doesn't work):
   • clvm-0.41-py3-none-any.whl
2. With packages from Debian bullseye (newer), following version is downloaded (which works):
   • clvm-0.9.7-py3-none-any.whl (older?!)