x.py: XZ support is required to download LLVM with python3

[willus10245]

$ ./x.py check
Updating only changed submodules
Submodules updated in 0.06 seconds
error: XZ support is required to download LLVM
help: consider disabling `download-ci-llvm` or using python3
Build completed unsuccessfully in 0:00:00

I also get this error when running python3 x.py check

I think this may have something to do with the fact that I use asdf-vm to manage my python version

$ which python
/Users/<user>/.asdf/shims/python

$ which python3
/Users/<user>/.asdf/shims/python3

[jyn514]

@willus10245 what is the output of python3 --version?

[willus10245]

@jyn514 Python 3.8.0

[cuviper]

I think it is possible to build Python without lzma if you don't have those devel files, but I'm not sure how to check if that's what happened except that the module is missing from your build.

[jyn514]

Hmm, lzma support was added in 3.3 (according to https://docs.python.org/3/library/tarfile.html), so that shouldn't be the issue. I tried this with 3.8 and can't reproduce:

$ ~/Downloads/python-3.8.10-embed-amd64/python.exe x.py check                                          Updating only changed submodules
Submodules updated in 0.39 seconds
downloading https://ci-artifacts.rust-lang.org/rustc-builds/36a4d14c7edba21bba14df00b9e6e4a111dfc6f2/rust-dev-nightly-x86_64-pc-windows-msvc.tar.xz
extracting C:\Users\Jeremy Nelson\src\rust\build\cache\llvm-36a4d14c7edba21bba14df00b9e6e4a111dfc6f2-False\rust-dev-nightly-x86_64-pc-windows-msvc.tar.xz
    Finished dev [unoptimized + debuginfo] target(s) in 0.68s

Are you using a fork of python or something? If you put support_xz in its own file and run it without the error handling, what errors do you get?

rust/src/bootstrap/bootstrap.py

Lines 19 to 22 in 3e827cc

	with tempfile.NamedTemporaryFile(delete=False) as temp_file:
	temp_path = temp_file.name
	with tarfile.open(temp_path, "w:xz"):
	pass

[cuviper]

Could try a command like: python3 -c "import lzma; print(dir(lzma))"

[willus10245]

I wonder it asdf-python is doing something weird on the install. When running the command @cuviper suggested, I got

$ python3 -c "import lzma; print(dir(lzma))"

Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/Users/scottwiggins/.asdf/installs/python/3.8.0/lib/python3.8/lzma.py", line 27, in <module>
    from _lzma import *
ModuleNotFoundError: No module named '_lzma'

[jyn514]

Ok. This doesn't sound like something we can fix; the error message is correct, you can either disable download-ci-llvm or switch to a python that has XZ support.

[willus10245]

Looks like asdf-python uses pyenv/python-build under the hood. Passing the flags from this comment to asdf install python <version> fixed the issue.

CFLAGS="-I$(brew --prefix xz)/include" LDFLAGS="-L$(brew --prefix xz)/lib" asdf install python 3.8.0