SABnzbd ERROR: Failed building wheel for cryptography #4104

syncrysys · 2021-02-12T15:40:53Z

ADMIN EDIT

Do the following to pull pre-compiled Python module wheels from the piwheels.org repository:

echo -e '[global]\nextra-index-url=https://www.piwheels.org/simple/' > /etc/pip.conf

Or add extra-index-url=https://www.piwheels.org/simple/ to the [global] section if that config file exists already (it's not present by default).

Getting this error when trying to install SABnzbd on a Rpi 4B newest DietPi version,

RPi 4 Model B (armv7l) | IP: 192.168.178.43 | Use up/down buttons to scroll text ──────────────┤ DietPi-Software ├──────────────────────────────────────────────────┐
                                            │ Collecting notify2                                                                                                     
                                            │   Downloading notify2-0.3.1-py2.py3-none-any.whl (8.0 kB)                                                              
                                            │ Collecting more-itertools>=2.6                                                                                         
                                            │   Downloading more_itertools-8.7.0-py3-none-any.whl (48 kB)                                                            
                                            │ Collecting six>=1.11.0                                                                                                 
                                            │   Downloading six-1.15.0-py2.py3-none-any.whl (10 kB)                                                                  
                                            │ Collecting jaraco.functools                                                                                            
                                            │   Downloading jaraco.functools-3.2.0-py3-none-any.whl (6.7 kB)                                                         
                                            │ Collecting sgmllib3k                                                                                                   
                                            │   Downloading sgmllib3k-1.0.0.tar.gz (5.8 kB)                                                                          
                                            │ Collecting jaraco.collections                                                                                          
                                            │   Downloading jaraco.collections-3.2.0-py3-none-any.whl (9.7 kB)                                                       
                                            │ Collecting zc.lockfile                                                                                                 
                                            │   Downloading zc.lockfile-2.0-py2.py3-none-any.whl (9.7 kB)                                                            
                                            │ Collecting tempora>=1.8                                                                                                
                                            │   Downloading tempora-4.0.1-py3-none-any.whl (14 kB)                                                                   
                                            │ Collecting pytz                                                                                                        
                                            │   Downloading pytz-2021.1-py2.py3-none-any.whl (510 kB)                                                                
                                            │ Collecting cffi>=1.12                                                                                                  
                                            │   Using cached cffi-1.14.5-cp37-cp37m-linux_armv7l.whl                                                                 
                                            │ Collecting pycparser                                                                                                   
                                            │   Using cached pycparser-2.20-py2.py3-none-any.whl (112 kB)                                                            
                                            │ Collecting jaraco.text                                                                                                 
                                            │   Downloading jaraco.text-3.5.0-py3-none-any.whl (8.1 kB)                                                              
                                            │ Collecting jaraco.classes                                                                                              
                                            │   Downloading jaraco.classes-3.2.0-py3-none-any.whl (5.6 kB)                                                           
                                            │ Requirement already satisfied: setuptools in /usr/local/lib/python3.7/dist-packages (from zc.lockfile->cherrypy->-r    
                                            │ requirements.txt (line 8)) (53.0.0)                                                                                    
                                            │ Building wheels for collected packages: cheetah3, sabyenc3, configobj, cryptography, sgmllib3k                         
                                            │   Building wheel for cheetah3 (setup.py): started                                                                      
                                            │   Building wheel for cheetah3 (setup.py): finished with status 'done'                                                  
                                            │   Created wheel for cheetah3: filename=Cheetah3-3.2.6-cp37-cp37m-linux_armv7l.whl size=199720                          
                                            │ sha256=22b1149dba8cdca93c0c8fe2b4ffaeb2ac25fde4fc01f2991e84780f3c47c13a                                                
                                            │   Stored in directory: /root/.cache/pip/wheels/52/28/39/2925ea0945f5002635a052a3e59fc794f412332ef5e23b53a1             
                                            │   Building wheel for sabyenc3 (setup.py): started                                                                      
                                            │   Building wheel for sabyenc3 (setup.py): finished with status 'done'                                                  
                                            │   Created wheel for sabyenc3: filename=sabyenc3-4.0.2-cp37-cp37m-linux_armv7l.whl size=27466                           
                                            │ sha256=eca0933a6ea523d2916bcaf51ee58066e2462ab16a3787d9a5cfa8c46a966c0c                                                
                                            │   Stored in directory: /root/.cache/pip/wheels/0e/91/28/fe66d02c53f79141e7c7fa4862d68357001a38c7eaf24aa6aa             
                                            │   Building wheel for configobj (setup.py): started                                                                     
                                            │   Building wheel for configobj (setup.py): finished with status 'done'                                                 
                                            │   Created wheel for configobj: filename=configobj-5.0.6-py3-none-any.whl size=34547                                    
                                            │ sha256=3cea559ac50ec48d0ea1c703e9bec1f4a50d1c7b10c4b7ae0dfbbd0115227588                                                
                                            │   Stored in directory: /root/.cache/pip/wheels/0d/c4/19/13d74440f2a571841db6b6e0a273694327498884dafb9cf978             
                                            │   Building wheel for cryptography (PEP 517): started                                                                   
                                            │   Building wheel for cryptography (PEP 517): finished with status 'error'                                              
                                            │   ERROR: Command errored out with exit status 1:                                                                       
                                            │    command: /usr/bin/python3 /usr/local/lib/python3.7/dist-packages/pip/_vendor/pep517/_in_process.py build_wheel      
                                            │                                                                                                                        
                                            │                        Retry          : Re-run the last command that failed                 ↑                          
                                            │                        DietPi-Config  : Edit network, APT/NTP mirror settings etc           ▮                          
                                            │                                                                                                                        
                                            │                                                                                                                        
                                            │                                  <Ok>                                      <Exit>                                      
                                            │                                                                                                                      │ 
                                            └──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┘

The text was updated successfully, but these errors were encountered:

Joulinar · 2021-02-12T21:08:25Z

Hi,

many thanks for your message. Looks like cryptography require rust to be build. At least that's the error I could replicate on my test system.

  error: Can not find Rust compiler
  ----------------------------------------
  ERROR: Failed building wheel for cryptography
Failed to build cryptography

Let me check how to fix this

MichaIng · 2021-02-12T21:39:47Z

Indeed, this is quite now: pyca/cryptography#5410
I guess after piwheels.org implementation this is only required for ARMv8.

We could use a temporary Rust tool chain like for Bitwarden_RS: https://github.com/MichaIng/DietPi/blob/dev/dietpi/dietpi-software#L5764-L5772

Nasty since cryptography is used by a large amount of Python modules. Short-term workaround could be to use stick with an older version, but earlier or later software will depend on the recent versions, i.e. the Rust tool chain, when no pre-compiled wheel is available 🤔.

Joulinar · 2021-02-12T21:42:20Z

@MichaIng
it's a ARMv7 32bit system. I could verify it on my RPi3B+ as well, even that I had piwheels.org activated

Yes might we need same procedure like Bitwarden. Unfortunately cargo version shipped via apt install is not sufficient

error: Rust 1.41.1 does not match extension requirement >=1.45.0

EDIT 1
using the same procedure as Bitwarden will give following error
error: no override and no default toolchain set

EDIT 2
we are not the only one missing cryptography package on piwheels.org piwheels/packages#188

EDIT 3
OK I got it working if I use G_EXEC ./rustup-init.sh -y instead of G_EXEC ./rustup-init.sh -y --default-toolchain none

Question is if we would need to create an own software install for rust if it is used more often now?

MichaIng · 2021-02-12T22:05:39Z

Dammit true, since v3.4 the builds failed: https://www.piwheels.org/project/cryptography/

New trend that projects move partly to Rust, but nasty that the quite large Rust tool chain is then required, and rustup is by default installed into the users home dir, not system-wide.

Question is if we would need to create an own software install for rust if it is used more often now?

Reasonable anyway, and find a good way to install it so that all users can use it, inkl. cargo to run sources directly like a runtime environment. I hope it can work like we do with Go for OpenBazaar currently: Install to /mnt/dietpi_userdata/rust and then load the tool chain binaries via /etc/bashrc.d/rust.sh into every users PATH.

Joulinar · 2021-02-12T22:09:05Z

probably stupid idea but what about pulling cargo from bullseye ??

MichaIng · 2021-02-12T22:13:26Z

This will just be outdated as well earlier or later and I'd try to avoid mixing Debian versions wherever possible, as this caused us a lot of problems in the past. So I'd first try to use rustup as mentioned.

As long as it's for the cryptography module only, we could stick with a temporary install as for Bitwarden_RS. It could be installed together with Python 3 pip automatically, on ARM, as potentially it is required for every pip install, then automatically removed when dietpi-software exits. But we should offer a permanent install as well.

MichaIng · 2021-02-12T22:19:37Z

@syncrysys
As a real solution will take a bit longer, for you the following should work as workaround:

sudo rm -R /root/.cache/pip
sudo pip3 install --force-reinstall 'cryptography==3.3.2'

The rerun the SABnzbd install.

Joulinar · 2021-02-12T22:44:17Z

hmm not sure if this is working @MichaIng

I installed cryptography-3.3.2 but installation is still trying to setup cryptography-3.4.4, which is failing

Requirement already satisfied: cryptography in /usr/local/lib/python3.7/dist-packages (from -r requirements.txt (line 3)) (3.3.2)
Collecting cryptography
  Downloading cryptography-3.4.4.tar.gz (545 kB)

AH I guess because of the -U flag used > pip3 install -Ur requirements.txt

MichaIng · 2021-02-13T10:25:16Z

Yes that's it, so:

sed -i 's/pip3 install -Ur requirements.txt/pip3 install -r requirements.txt/' /boot/dietpi/dietpi-software
rm -R /root/.cache/pip
pip3 install --force-reinstall 'cryptography==3.3.2'

syncrysys · 2021-02-14T01:21:01Z

Thanks for the replies, the commands are working.
Is there any ETA for a full fix?

MichaIng · 2021-02-14T10:29:35Z

I just found that it is not an issue on ARMv8 since PyPI ships aarch64 wheels: https://pypi.org/project/cryptography/#files
And the recent ARMv6+ARMv7 builds on piwheels for Buster succeeded as well, just yesterday: https://www.piwheels.org/project/cryptography/

On Stretch it doesn't matter since it's not supported anymore anyway due to Python >=3.6 dependency.

I.e. solved already. I'll do a quick test.

Implementing the Rust toolchain as install options remains a need anyway, but it looks like in case of Python modules the need for builds fades, when PyPI starts to ship aarch64 wheels on a regular basis 👍.

Joulinar · 2021-02-14T10:43:13Z

But this require piwheels to be activated, which is not the case until 6.35 😉

MichaIng · 2021-02-14T10:46:33Z

To add piwheels:

echo -e '[global]\nextra-index-url=https://www.piwheels.org/simple/' > /etc/pip.conf

Or add extra-index-url=https://www.piwheels.org/simple/ to the [global] section if that config exists already (it's not present by default).

[ INFO ] DietPi-Software | pip3 install -Ur requirements.txt, please wait...
Looking in indexes: https://pypi.org/simple, https://www.piwheels.org/simple/
Ignoring feedparser: markers 'python_version == "3.5"' don't match your environment
Ignoring pywin32: markers 'sys_platform == "win32"' don't match your environment
Ignoring pyobjc: markers 'sys_platform == "darwin"' don't match your environment
Collecting sabyenc3>=4.0.0
  Downloading https://www.piwheels.org/simple/sabyenc3/sabyenc3-4.0.2-cp37-cp37m-linux_armv7l.whl (27 kB)
Collecting cheetah3>=3.0.0
  Downloading https://www.piwheels.org/simple/cheetah3/Cheetah3-3.2.6-cp37-cp37m-linux_armv7l.whl (199 kB)
Requirement already satisfied: cryptography in /usr/local/lib/python3.7/dist-packages (from -r requirements.txt (line 3)) (3.3.1)
WARNING: Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ProtocolError('Connection aborted.', RemoteDisconnected('Remote end closed connection without response'))': /simple/cryptography/
Collecting cryptography
  Downloading https://www.piwheels.org/simple/cryptography/cryptography-3.4.5-cp37-cp37m-linux_armv7l.whl (1.5 MB)
Requirement already satisfied: feedparser>=6.0.0 in /usr/local/lib/python3.7/dist-packages (from -r requirements.txt (line 5)) (6.0.2)
Collecting configobj
  Downloading https://www.piwheels.org/simple/configobj/configobj-5.0.6-py3-none-any.whl (36 kB)
Collecting cheroot<8.4.3
  Downloading cheroot-8.4.1-py2.py3-none-any.whl (91 kB)
Collecting cherrypy
  Downloading https://www.piwheels.org/simple/cherrypy/CherryPy-18.6.0-py2.py3-none-any.whl (419 kB)
Collecting portend
  Downloading https://www.piwheels.org/simple/portend/portend-2.7.1-py3-none-any.whl (5.3 kB)
Requirement already satisfied: chardet in /usr/local/lib/python3.7/dist-packages (from -r requirements.txt (line 10)) (4.0.0)
Collecting notify2
  Downloading https://www.piwheels.org/simple/notify2/notify2-0.3.1-py2.py3-none-any.whl (8.0 kB)
Requirement already satisfied: six>=1.11.0 in /usr/local/lib/python3.7/dist-packages (from cheroot<8.4.3->-r requirements.txt (line 7)) (1.15.0)
Collecting more-itertools>=2.6
  Downloading https://www.piwheels.org/simple/more-itertools/more_itertools-8.7.0-py3-none-any.whl (48 kB)
Collecting jaraco.functools
  Downloading https://www.piwheels.org/simple/jaraco-functools/jaraco.functools-3.2.0-py3-none-any.whl (6.7 kB)
Requirement already satisfied: sgmllib3k in /usr/local/lib/python3.7/dist-packages (from feedparser>=6.0.0->-r requirements.txt (line 5)) (1.0.0)
Collecting jaraco.collections
  Downloading https://www.piwheels.org/simple/jaraco-collections/jaraco.collections-3.2.0-py3-none-any.whl (9.7 kB)
Collecting zc.lockfile
  Downloading https://www.piwheels.org/simple/zc-lockfile/zc.lockfile-2.0-py2.py3-none-any.whl (9.7 kB)
Collecting tempora>=1.8
  Downloading https://www.piwheels.org/simple/tempora/tempora-4.0.1-py3-none-any.whl (14 kB)
Requirement already satisfied: pytz in /usr/local/lib/python3.7/dist-packages (from tempora>=1.8->portend->-r requirements.txt (line 9)) (2020.5)
Requirement already satisfied: cffi>=1.12 in /usr/local/lib/python3.7/dist-packages (from cryptography->-r requirements.txt (line 3)) (1.14.4)
Requirement already satisfied: pycparser in /usr/local/lib/python3.7/dist-packages (from cffi>=1.12->cryptography->-r requirements.txt (line 3)) (2.20)
Collecting jaraco.text
  Downloading https://www.piwheels.org/simple/jaraco-text/jaraco.text-3.5.0-py3-none-any.whl (8.1 kB)
Collecting jaraco.classes
  Downloading https://www.piwheels.org/simple/jaraco-classes/jaraco.classes-3.2.0-py3-none-any.whl (5.6 kB)
Requirement already satisfied: setuptools in /usr/local/lib/python3.7/dist-packages (from zc.lockfile->cherrypy->-r requirements.txt (line 8)) (51.0.0)
Installing collected packages: more-itertools, jaraco.functools, tempora, jaraco.text, jaraco.classes, zc.lockfile, portend, jaraco.collections, cheroot, sabyenc3, notify2, cryptography, configobj, cherrypy, cheetah3
  Attempting uninstall: cryptography
    Found existing installation: cryptography 3.3.1
    Uninstalling cryptography-3.3.1:
      Successfully uninstalled cryptography-3.3.1
Successfully installed cheetah3-3.2.6 cheroot-8.4.1 cherrypy-18.6.0 configobj-5.0.6 cryptography-3.4.5 jaraco.classes-3.2.0 jaraco.collections-3.2.0 jaraco.functools-3.2.0 jaraco.text-3.5.0 more-itertools-8.7.0 notify2-0.3.1 portend-2.7.1 sabyenc3-4.0.2 tempora-4.0.1 zc.lockfile-2.0
[  OK  ] DietPi-Software | pip3 install -Ur requirements.txt

MichaIng · 2021-02-14T20:06:09Z

Sad fact about a system-wide Rust: rust-lang/rustup#2383
Looks like it needs to be a root user or dedicated "rust" user instance, when binaries are compiled, and a separate Rust instance for each service that runs a Rust program via cargo from sources.

There are Debian packages which install system-wide rustc and cargo binaries, but it seems this kind of sharing binaries is not intended, nor safe by design...

Joulinar · 2021-02-14T20:12:45Z

yesterday I tried to install cargo via apt, but the version shipped was not sufficient to build cryptography
This is as well stated on cryptography web site https://cryptography.io/en/latest/installation.html#debian-ubuntu

MichaIng · 2021-02-14T20:34:18Z

It works now since cryptography has just lowered the minimum cargo version. But what I wanted to say is that even if we implement it a way like the Debian package does, it won't work safely like it is with other compilers and runtime systems like Java or gcc. It's meant to be a per-user environment.

Joulinar · 2021-02-14T21:22:30Z

It works now since cryptography has just lowered the minimum cargo version

quite some movement on cryptography atm

MichaIng · 2021-02-14T22:55:50Z

I mark this as closed. Rust is an own topic for a future release.

ravenclaw900 added the Investigating 🤔 label Feb 12, 2021

MichaIng added ARMv8 Bug 🐞 labels Feb 12, 2021

MichaIng added this to the v6.35 milestone Feb 12, 2021

MichaIng added ARMbaby and removed ARMv8 labels Feb 12, 2021

Joulinar added the Workaround available 🆗 Workaround is available/has been implemented, but a definite solution should be found when possible. label Feb 14, 2021

MichaIng added ARMv6 ARMv7 and removed ARMbaby labels Feb 14, 2021

MichaIng added Solution available 🥂 Definite solution has been done and removed Workaround available 🆗 Workaround is available/has been implemented, but a definite solution should be found when possible. Investigating 🤔 labels Feb 14, 2021

MichaIng closed this as completed Feb 14, 2021

Joulinar mentioned this issue Oct 7, 2021

Cant install HomeAssistant with DietPi-Software on RPi 2 Model B (armv7l) #4372

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

SABnzbd ERROR: Failed building wheel for cryptography #4104

SABnzbd ERROR: Failed building wheel for cryptography #4104

syncrysys commented Feb 12, 2021 •

edited by MichaIng

Loading

Joulinar commented Feb 12, 2021

MichaIng commented Feb 12, 2021

Joulinar commented Feb 12, 2021 •

edited

Loading

MichaIng commented Feb 12, 2021 •

edited

Loading

Joulinar commented Feb 12, 2021

MichaIng commented Feb 12, 2021 •

edited

Loading

MichaIng commented Feb 12, 2021

Joulinar commented Feb 12, 2021 •

edited

Loading

MichaIng commented Feb 13, 2021

syncrysys commented Feb 14, 2021

MichaIng commented Feb 14, 2021 •

edited

Loading

Joulinar commented Feb 14, 2021

MichaIng commented Feb 14, 2021

MichaIng commented Feb 14, 2021

Joulinar commented Feb 14, 2021

MichaIng commented Feb 14, 2021

Joulinar commented Feb 14, 2021 •

edited

Loading

MichaIng commented Feb 14, 2021

SABnzbd ERROR: Failed building wheel for cryptography #4104

SABnzbd ERROR: Failed building wheel for cryptography #4104

Comments

syncrysys commented Feb 12, 2021 • edited by MichaIng Loading

ADMIN EDIT

Joulinar commented Feb 12, 2021

MichaIng commented Feb 12, 2021

Joulinar commented Feb 12, 2021 • edited Loading

MichaIng commented Feb 12, 2021 • edited Loading

Joulinar commented Feb 12, 2021

MichaIng commented Feb 12, 2021 • edited Loading

MichaIng commented Feb 12, 2021

Joulinar commented Feb 12, 2021 • edited Loading

MichaIng commented Feb 13, 2021

syncrysys commented Feb 14, 2021

MichaIng commented Feb 14, 2021 • edited Loading

Joulinar commented Feb 14, 2021

MichaIng commented Feb 14, 2021

MichaIng commented Feb 14, 2021

Joulinar commented Feb 14, 2021

MichaIng commented Feb 14, 2021

Joulinar commented Feb 14, 2021 • edited Loading

MichaIng commented Feb 14, 2021

syncrysys commented Feb 12, 2021 •

edited by MichaIng

Loading

Joulinar commented Feb 12, 2021 •

edited

Loading

MichaIng commented Feb 12, 2021 •

edited

Loading

MichaIng commented Feb 12, 2021 •

edited

Loading

Joulinar commented Feb 12, 2021 •

edited

Loading

MichaIng commented Feb 14, 2021 •

edited

Loading

Joulinar commented Feb 14, 2021 •

edited

Loading