Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

META: Python 3.10 / 3.11 package status #5043

Closed
9 of 10 tasks
th0ma7 opened this issue Jan 3, 2022 · 39 comments · Fixed by #5216
Closed
9 of 10 tasks

META: Python 3.10 / 3.11 package status #5043

th0ma7 opened this issue Jan 3, 2022 · 39 comments · Fixed by #5216
Labels

Comments

@th0ma7
Copy link
Contributor

th0ma7 commented Jan 3, 2022

Python 3.10 package status

High-level python PR

Packages formally using python 3.x

PACKAGE Python 3.10 migration Published Python 3.11 migration Published 3.11 COMMENT
bazarr ✔️ ✔️ Follow-up update #5139 (@smaarn)
beets ✔️ ✔️ #4946
borgbackup ✔️ ✔️ #5048 (co-author @sunlei)
Fixed: borgbackup/borg#6105
domoticz Version online uses python38
duplicity ✔️ ✔️ #5054
fishnet ❌️ ❌️ @mayweed Rewritten in Rust (removal #5216)
flexget ✔️ ✔️ #5176
homeassistant ✔️ ✔️ #5579, #5096, ARMv5 support pending: numpy/numpy#20664
Looks like working based on testing fbd9819
Confirmed, fixed #5155
mercurial ✔️ ✔️ #5045
octoprint ✔️ ✔️ #5217
plexpy-custom ✔️ Ready for migration but BUG with the package install, to remove?
rdiff-backup ✔️ ✔️ #5447
rutorrent ✔️ ✔️ Migration to DSM7 #5455, #5669, #5617
sabnzbd ✔️ ✔️ ✔️ ✔️ #5109 (@Safihre)
#5137
#5527
sickchill ✔️ ✔️ #5431 @miigotu
salt-master ✔️ ✔️ #5017 co-author: @arnebjarne
salt-minion ✔️ ✔️ #5017 co-author: @arnebjarne, #5145
tvheadend ✔️ ✔️
znc ✔️ ✔️ #4951, #5091

Packages formally using python 2.x (#4524)

PACKAGE python 3.10 migration DSM7 migration Python 3.10 Published Python 3.11 migration Published 3.11 COMMENT
deluge ✔️ ✔️ ✔️ #5728 #4153, #5202, #5398
ffsync ✔️ ✔️ #5942
haproxy
salt-master ✔️ ✔️ ✔️ #5017 co-author: @arnebjarne

Framework clean-up

apps dependent packages Removed COMMENT
python3 (cross, spk, native) none ✔️ #5316
python38 (cross, spk, native) none TODO to remove once all python3.x packages published
cross/asyncstdlib none ✔️ #5175
cross/borgbackup borgbackup ✔️ #5048
cross/cryptography duplicity n/a Mandatory to build abi3 wheels
cross/cryptography-legacy python2 TODO to remove once all python3.x packages published
cross/deluge deluge ✔️ #5398
cross/dtlssocket homeassistant N/A Unable to migrate to pip
cross/duplicity duplicity ✔️ #5054
cross/gevent ffsync
cross/greenlet ffsync, python38
cross/immutables python3, python38 TODO to remove once all python3.x packages published
cross/libtorrent deluge ✔️ #5398
cross/lxml python2, python3, python38
cross/m2crypto python2
cross/mercurial mercurial ✔️ #5045
cross/msgpack-python python2, python3, python38
cross/numpy homeassistant ✔️ #5096, ARMv5 support pending: numpy/numpy#20664
Fixed #5155
cross/octoprint octoprint ✔️ #5217
cross/pillow none ✔️ #5175
cross/pillow_py2 python2
cross/pocketsphinx none ✔️ #5175
cross/poetry none ✔️ #5508
cross/pyalsa python2
cross/pyaudio python2
cross/pycryptodome none ✔️ #5175
cross/pycryptodomex none ✔️ #5175
cross/pycurl python2, python3, python38
cross/PyNaCl none ✔️ #5175
cross/pyyaml python2, python3, python38
cross/pyzmq python2
cross/rencode deluge ✔️ #5398
cross/ruamel.yaml python3, python38 TODO to remove once all python3.x packages published
cross/sphinxbase cross/pocketsphinx ✔️ #5175
cross/stockfish-niklasf spk/fishnet ✔️ @mayweed Rewritten in Rust (removal #5216)
cross/ujson homeassistant ✔️ #5096, Solved: ultrajson/ultrajson#496
cross/umemcache none ✔️ #5175
cross/uwsgi python2
cross/webrtcvad none ✔️ #5175
@th0ma7 th0ma7 changed the title Python 3.10 package status META: Python 3.10 package status Jan 3, 2022
@th0ma7 th0ma7 added the dsm 7 label Jan 3, 2022
@hgy59 hgy59 pinned this issue Jan 4, 2022
@hgy59
Copy link
Contributor

hgy59 commented Jan 13, 2022

@th0ma7 octoprint cannot install the plugins due to permission restrictions for the pip module/install folder.
any suggestions?

2022-01-13 21:37:20,165 - octoprint.server - INFO - Listening on http://0.0.0.0:8088 and http://[::]:8088
2022-01-13 21:37:20,169 - octoprint.plugins.discovery - INFO - Registered OctoPrint instance on ds-arm for SSDP
2022-01-13 21:37:21,012 - octoprint.plugins.pluginmanager - INFO - Loaded plugin repository data from https://plugins.octoprint.org/plugins.json
2022-01-13 21:37:22,577 - octoprint.plugins.announcements - INFO - Loaded channel _important from https://octoprint.org/feeds/important.xml in 0.41s
2022-01-13 21:37:23,919 - octoprint.plugins.announcements - INFO - Loaded channel _releases from https://octoprint.org/feeds/releases.xml in 0.31s
2022-01-13 21:37:23,938 - octoprint.util.pip - INFO - Using "/volume1/@appstore/octoprint/env/bin/python3 -m pip" as command to invoke pip
2022-01-13 21:37:23,954 - octoprint.util.pip - INFO - pip installs to /volume1/@appstore/octoprint/env/lib/python3.10/site-packages (writable -> no), --user flag needed -> no, virtual env -> yes
2022-01-13 21:37:23,955 - octoprint.util.pip - INFO - ==> pip ok -> NO!
2022-01-13 21:37:23,955 - octoprint.util.pip - ERROR - Cannot use this pip install, can't write to the install dir and also can't use --user for installing. Check your setup and the permissions on /volume1/@appstore/octoprint/env/lib/python3.10/site-packages.
2022-01-13 21:37:23,989 - octoprint.plugin - ERROR - Error while calling plugin softwareupdate
Traceback (most recent call last):
  File "/volume1/@appstore/octoprint/share/OctoPrint/src/octoprint/plugin/__init__.py", line 271, in call_plugin
    result = getattr(plugin, method)(*args, **kwargs)
  File "/volume1/@appstore/octoprint/share/OctoPrint/src/octoprint/util/__init__.py", line 1737, in wrapper
    return f(*args, **kwargs)
  File "/volume1/@appstore/octoprint/share/OctoPrint/src/octoprint/plugins/softwareupdate/__init__.py", line 224, in on_after_startup
    self._check_environment()
  File "/volume1/@appstore/octoprint/share/OctoPrint/src/octoprint/plugins/softwareupdate/__init__.py", line 362, in _check_environment
    and get_comparable_version(versions["pip"])
  File "/volume1/@appstore/octoprint/share/OctoPrint/src/octoprint/util/version.py", line 170, in get_comparable_version
    version_string = normalize_version(version_string)
  File "/volume1/@appstore/octoprint/share/OctoPrint/src/octoprint/util/version.py", line 211, in normalize_version
    if "-" in version:
TypeError: argument of type 'NoneType' is not iterable

Edit:

  • DSM 6, aarch64-6.1
  • octoprint UI is running successfully

@th0ma7
Copy link
Contributor Author

th0ma7 commented Jan 14, 2022

That's odd, never faced that issue. And good thing it isn't yet released, leaving time to settle on this.

@th0ma7
Copy link
Contributor Author

th0ma7 commented Jan 14, 2022

Just a thought, I now wonder if the virtualenv didn't appeared to be installed by root user during the installation phase? Although I recall last time I tested homeassistant that many many other wheels where being installed after the fact with no problem. Perhaps something changed somehow...

I'll try to check this out over the week-end.

@th0ma7
Copy link
Contributor Author

th0ma7 commented Jan 18, 2022

octoprint cannot install the plugins due to permission restrictions for the pip module/install folder. any suggestions?

I wasn't able to reproduce... I was able to successfully install Bed-Visualizer from the Install new Plugins... option. Although I noticed that a particular version of setuptools was YANKED on pypi, perhaps your install ended-up using it?

Installing plugin "Bed Level Visualizer" from https://github.com/jneilliii/OctoPrint-BedLevelVisualizer/archive/master.zip...
/volume1/@appstore/octoprint/env/bin/python3 -m pip --disable-pip-version-check install file:///tmp/tmpgfzttzb2/OctoPrint-BedLevelVisualizer-master.zip --no-cache-dir
Processing /tmp/tmpgfzttzb2/OctoPrint-BedLevelVisualizer-master.zip
Preparing metadata (setup.py): started
Preparing metadata (setup.py): finished with status 'done'
Requirement already satisfied: OctoPrint in /volume1/@appstore/octoprint/env/lib/python3.10/site-packages/OctoPrint-1.7.2-py3.10.egg (from Bed-Visualizer==1.1.1) (1.7.2)
Requirement already satisfied: OctoPrint-FileCheck>=2021.2.23 in /volume1/@appstore/octoprint/env/lib/python3.10/site-packages (from OctoPrint->Bed-Visualizer==1.1.1) (2021.2.23)
Requirement already satisfied: OctoPrint-FirmwareCheck>=2021.10.11 in /volume1/@appstore/octoprint/env/lib/python3.10/site-packages (from OctoPrint->Bed-Visualizer==1.1.1) (2021.10.11)
Requirement already satisfied: OctoPrint-PiSupport>=2021.10.28 in /volume1/@appstore/octoprint/env/lib/python3.10/site-packages (from OctoPrint->Bed-Visualizer==1.1.1) (2021.10.28)
Requirement already satisfied: markupsafe<2.0,>=1.1 in /volume1/@appstore/octoprint/env/lib/python3.10/site-packages (from OctoPrint->Bed-Visualizer==1.1.1) (1.1.1)
Requirement already satisfied: markdown<3.2,>=3.1 in /volume1/@appstore/octoprint/env/lib/python3.10/site-packages (from OctoPrint->Bed-Visualizer==1.1.1) (3.1.1)
Requirement already satisfied: regex!=2018.11.6 in /volume1/@appstore/octoprint/env/lib/python3.10/site-packages (from OctoPrint->Bed-Visualizer==1.1.1) (2021.10.23)
Requirement already satisfied: wrapt<1.13,>=1.12.1 in /volume1/@appstore/octoprint/env/lib/python3.10/site-packages (from OctoPrint->Bed-Visualizer==1.1.1) (1.12.1)
Requirement already satisfied: flask<2,>=1.1.4 in /volume1/@appstore/octoprint/env/lib/python3.10/site-packages (from OctoPrint->Bed-Visualizer==1.1.1) (1.1.4)
Requirement already satisfied: Jinja2<3,>=2.11.3 in /volume1/@appstore/octoprint/env/lib/python3.10/site-packages (from OctoPrint->Bed-Visualizer==1.1.1) (2.11.3)
Requirement already satisfied: Flask-Login<0.6,>=0.5 in /volume1/@appstore/octoprint/env/lib/python3.10/site-packages (from OctoPrint->Bed-Visualizer==1.1.1) (0.5.0)
Requirement already satisfied: Flask-Babel<2,>=1.0 in /volume1/@appstore/octoprint/env/lib/python3.10/site-packages (from OctoPrint->Bed-Visualizer==1.1.1) (1.0.0)
Requirement already satisfied: Flask-Assets<3,>=2.0 in /volume1/@appstore/octoprint/env/lib/python3.10/site-packages (from OctoPrint->Bed-Visualizer==1.1.1) (2.0)
Requirement already satisfied: werkzeug<2,>=1.0.1 in /volume1/@appstore/octoprint/env/lib/python3.10/site-packages (from OctoPrint->Bed-Visualizer==1.1.1) (1.0.1)
Requirement already satisfied: itsdangerous<2,>=1.1.0 in /volume1/@appstore/octoprint/env/lib/python3.10/site-packages (from OctoPrint->Bed-Visualizer==1.1.1) (1.1.0)
Requirement already satisfied: cachelib<0.2,>=0.1 in /volume1/@appstore/octoprint/env/lib/python3.10/site-packages (from OctoPrint->Bed-Visualizer==1.1.1) (0.1.1)
Requirement already satisfied: PyYAML<6,>=5.4.1 in /volume1/@appstore/octoprint/env/lib/python3.10/site-packages (from OctoPrint->Bed-Visualizer==1.1.1) (5.4.1)
Requirement already satisfied: pyserial<4,>=3.4 in /volume1/@appstore/octoprint/env/lib/python3.10/site-packages (from OctoPrint->Bed-Visualizer==1.1.1) (3.5)
Requirement already satisfied: netaddr<0.9,>=0.8 in /volume1/@appstore/octoprint/env/lib/python3.10/site-packages (from OctoPrint->Bed-Visualizer==1.1.1) (0.8.0)
Requirement already satisfied: watchdog==0.10.4 in /volume1/@appstore/octoprint/env/lib/python3.10/site-packages (from OctoPrint->Bed-Visualizer==1.1.1) (0.10.4)
Requirement already satisfied: sarge==0.1.6 in /volume1/@appstore/octoprint/env/lib/python3.10/site-packages (from OctoPrint->Bed-Visualizer==1.1.1) (0.1.6)
Requirement already satisfied: netifaces<1,>=0.11 in /volume1/@appstore/octoprint/env/lib/python3.10/site-packages (from OctoPrint->Bed-Visualizer==1.1.1) (0.11.0)
Requirement already satisfied: pylru<2,>=1.2 in /volume1/@appstore/octoprint/env/lib/python3.10/site-packages (from OctoPrint->Bed-Visualizer==1.1.1) (1.2.0)
Requirement already satisfied: pkginfo<2,>=1.7.1 in /volume1/@appstore/octoprint/env/lib/python3.10/site-packages (from OctoPrint->Bed-Visualizer==1.1.1) (1.7.1)
Requirement already satisfied: requests<3,>=2.26.0 in /volume1/@appstore/octoprint/env/lib/python3.10/site-packages (from OctoPrint->Bed-Visualizer==1.1.1) (2.26.0)
Requirement already satisfied: semantic_version<3,>=2.8.5 in /volume1/@appstore/octoprint/env/lib/python3.10/site-packages (from OctoPrint->Bed-Visualizer==1.1.1) (2.8.5)
Requirement already satisfied: psutil<6,>=5.8 in /volume1/@appstore/octoprint/env/lib/python3.10/site-packages (from OctoPrint->Bed-Visualizer==1.1.1) (5.8.0)
Requirement already satisfied: Click<8,>=7.1.2 in /volume1/@appstore/octoprint/env/lib/python3.10/site-packages (from OctoPrint->Bed-Visualizer==1.1.1) (7.1.2)
Requirement already satisfied: future<1,>=0.18.2 in /volume1/@appstore/octoprint/env/lib/python3.10/site-packages (from OctoPrint->Bed-Visualizer==1.1.1) (0.18.2)
Requirement already satisfied: websocket-client<1,>=0.59 in /volume1/@appstore/octoprint/env/lib/python3.10/site-packages (from OctoPrint->Bed-Visualizer==1.1.1) (0.59.0)
Requirement already satisfied: emoji<2,>=1.4.2 in /volume1/@appstore/octoprint/env/lib/python3.10/site-packages (from OctoPrint->Bed-Visualizer==1.1.1) (1.6.1)
Requirement already satisfied: sentry-sdk<2,>=1.3.1 in /volume1/@appstore/octoprint/env/lib/python3.10/site-packages (from OctoPrint->Bed-Visualizer==1.1.1) (1.4.3)
Requirement already satisfied: filetype<2,>=1.0.7 in /volume1/@appstore/octoprint/env/lib/python3.10/site-packages (from OctoPrint->Bed-Visualizer==1.1.1) (1.0.8)
Requirement already satisfied: zipstream-new<1.2,>=1.1.8 in /volume1/@appstore/octoprint/env/lib/python3.10/site-packages (from OctoPrint->Bed-Visualizer==1.1.1) (1.1.8)
Requirement already satisfied: blinker<2,>=1.4 in /volume1/@appstore/octoprint/env/lib/python3.10/site-packages (from OctoPrint->Bed-Visualizer==1.1.1) (1.4)
Requirement already satisfied: feedparser<7,>=6.0.8 in /volume1/@appstore/octoprint/env/lib/python3.10/site-packages (from OctoPrint->Bed-Visualizer==1.1.1) (6.0.8)
Requirement already satisfied: tornado<7,>=6 in /volume1/@appstore/octoprint/env/lib/python3.10/site-packages (from OctoPrint->Bed-Visualizer==1.1.1) (6.1)
Requirement already satisfied: zeroconf<0.34,>=0.33 in /volume1/@appstore/octoprint/env/lib/python3.10/site-packages (from OctoPrint->Bed-Visualizer==1.1.1) (0.33.4)
Requirement already satisfied: immutabledict<3,>=2.1 in /volume1/@appstore/octoprint/env/lib/python3.10/site-packages (from OctoPrint->Bed-Visualizer==1.1.1) (2.2.1)
Requirement already satisfied: pathvalidate<3,>=2.4.1 in /volume1/@appstore/octoprint/env/lib/python3.10/site-packages (from OctoPrint->Bed-Visualizer==1.1.1) (2.5.0)
Requirement already satisfied: colorlog<6,>=5.0.1 in /volume1/@appstore/octoprint/env/lib/python3.10/site-packages (from OctoPrint->Bed-Visualizer==1.1.1) (5.0.1)
Requirement already satisfied: unidecode in /volume1/@appstore/octoprint/env/lib/python3.10/site-packages (from OctoPrint->Bed-Visualizer==1.1.1) (1.3.2)
Requirement already satisfied: pathtools>=0.1.1 in /volume1/@appstore/octoprint/env/lib/python3.10/site-packages (from watchdog==0.10.4->OctoPrint->Bed-Visualizer==1.1.1) (0.1.2)
Requirement already satisfied: sgmllib3k in /volume1/@appstore/octoprint/env/lib/python3.10/site-packages (from feedparser<7,>=6.0.8->OctoPrint->Bed-Visualizer==1.1.1) (1.0.0)
Requirement already satisfied: webassets>=2.0 in /volume1/@appstore/octoprint/env/lib/python3.10/site-packages (from Flask-Assets<3,>=2.0->OctoPrint->Bed-Visualizer==1.1.1) (2.0)
Requirement already satisfied: Babel>=2.3 in /volume1/@appstore/octoprint/env/lib/python3.10/site-packages (from Flask-Babel<2,>=1.0->OctoPrint->Bed-Visualizer==1.1.1) (2.9.1)
Requirement already satisfied: pytz in /volume1/@appstore/octoprint/env/lib/python3.10/site-packages (from Flask-Babel<2,>=1.0->OctoPrint->Bed-Visualizer==1.1.1) (2021.3)
Requirement already satisfied: setuptools>=36 in /volume1/@appstore/octoprint/env/lib/python3.10/site-packages (from markdown<3.2,>=3.1->OctoPrint->Bed-Visualizer==1.1.1) (60.5.0)
Requirement already satisfied: charset-normalizer~=2.0.0 in /volume1/@appstore/octoprint/env/lib/python3.10/site-packages (from requests<3,>=2.26.0->OctoPrint->Bed-Visualizer==1.1.1) (2.0.10)
Requirement already satisfied: urllib3<1.27,>=1.21.1 in /volume1/@appstore/octoprint/env/lib/python3.10/site-packages (from requests<3,>=2.26.0->OctoPrint->Bed-Visualizer==1.1.1) (1.26.8)
Requirement already satisfied: idna<4,>=2.5 in /volume1/@appstore/octoprint/env/lib/python3.10/site-packages (from requests<3,>=2.26.0->OctoPrint->Bed-Visualizer==1.1.1) (3.3)
Requirement already satisfied: certifi>=2017.4.17 in /volume1/@appstore/octoprint/env/lib/python3.10/site-packages (from requests<3,>=2.26.0->OctoPrint->Bed-Visualizer==1.1.1) (2021.10.8)
Requirement already satisfied: six in /volume1/@appstore/octoprint/env/lib/python3.10/site-packages (from websocket-client<1,>=0.59->OctoPrint->Bed-Visualizer==1.1.1) (1.16.0)
Requirement already satisfied: ifaddr>=0.1.7 in /volume1/@appstore/octoprint/env/lib/python3.10/site-packages (from zeroconf<0.34,>=0.33->OctoPrint->Bed-Visualizer==1.1.1) (0.1.7)
Building wheels for collected packages: Bed-Visualizer
Building wheel for Bed-Visualizer (setup.py): started
Building wheel for Bed-Visualizer (setup.py): finished with status 'done'
Created wheel for Bed-Visualizer: filename=Bed_Visualizer-1.1.1-py3-none-any.whl size=4182731 sha256=78edd206f54028eeb9218469443099aa8caaf5a231226448d2cf28619aeb095f
Stored in directory: /tmp/pip-ephem-wheel-cache-bul75zdr/wheels/b5/d6/23/c582f689507a9b7cf7420184f0fe2c5b595600fc122bac8917
Successfully built Bed-Visualizer
Installing collected packages: Bed-Visualizer
Successfully installed Bed-Visualizer-1.1.1
Done!

Would you mind uninstalling it / reinstalling it and see if the issue is still there? On my end I'll give it another shot using a fully clean tree on master to check if I can reproduce.

arch: kvmx64-7.0

EDIT: I'll also give it a shot on another arch such as armv7 / armv5 that I have available at home

EDIT2: Found the same log file as you referred, and all is good there as well (re-built from maser). Also still able to install an additional plugin:

2022-01-17 21:27:32,944 - octoprint.util.pip - INFO - Using "/volume1/@appstore/octoprint/env/bin/python3 -m pip" as command to invoke pip
2022-01-17 21:27:34,086 - octoprint.util.pip - INFO - Version of pip is 21.3.1
2022-01-17 21:27:34,088 - octoprint.util.pip - INFO - pip installs to /volume1/@appstore/octoprint/env/lib/python3.10/site-packages (writable -> yes), --user flag needed -> no, virtual env -> yes
2022-01-17 21:27:34,088 - octoprint.util.pip - INFO - ==> pip ok -> yes
2022-01-17 21:27:34,143 - octoprint.plugin.core - INFO - Initialized 17 plugin implementation(s)
2022-01-17 21:27:34,232 - octoprint.plugin.core - INFO - 17 plugin(s) registered with the system:

...
2022-01-17 21:31:35,501 - octoprint.plugins.pluginmanager - INFO - The plugin was installed successfully: Bed Visualizer, version 1.1.1
2022-01-17 21:31:35,893 - octoprint.plugins.tracking - INFO - Sent tracking event install_plugin, payload: {'plugin': 'bedlevelvisualizer', 'plugin_version': '1.1.1'}

EDIT 3: same on armv7, no issues:

2022-01-17 21:55:10,110 - octoprint.server.util.watchdog - INFO - ... initial scan done.
2022-01-17 21:55:11,037 - octoprint.plugins.discovery - INFO - Registered 'OctoPrint instance on DS115j-armv7._http._tcp.local.' for _http._tcp.local.
2022-01-17 21:55:12,074 - octoprint.plugins.discovery - INFO - Registered 'OctoPrint instance on DS115j-armv7._octoprint._tcp.local.' for _octoprint._tcp.local.
2022-01-17 21:55:12,086 - octoprint.plugins.discovery - INFO - Registered OctoPrint instance on DS115j-armv7 for SSDP
2022-01-17 21:55:12,106 - octoprint.server - INFO - Listening on http://0.0.0.0:8088 and http://[::]:8088
2022-01-17 21:55:13,905 - octoprint.plugins.pluginmanager - INFO - Loaded plugin repository data from https://plugins.octoprint.org/plugins.json
2022-01-17 21:55:22,503 - octoprint.plugins.announcements - INFO - Loaded channel _important from https://octoprint.org/feeds/important.xml in 1.5s
2022-01-17 21:55:29,709 - octoprint.plugins.announcements - INFO - Loaded channel _releases from https://octoprint.org/feeds/releases.xml in 2.4s
2022-01-17 21:55:33,037 - octoprint.util.pip - INFO - Using "/volume1/@appstore/octoprint/env/bin/python3 -m pip" as command to invoke pip
2022-01-17 21:55:33,082 - octoprint.util.pip - INFO - pip installs to /volume1/@appstore/octoprint/env/lib/python3.10/site-packages (writable -> yes), --user flag needed -> no, virtual env -> yes
2022-01-17 21:55:33,083 - octoprint.util.pip - INFO - ==> pip ok -> yes
2022-01-17 21:55:33,179 - octoprint.plugins.softwareupdate - INFO - Minimum free storage across all update relevant locations is 120.1MB. That is considered insufficient for updating.
2022-01-17 21:55:33,181 - octoprint.plugins.softwareupdate - INFO - Fetching check overlays from https://plugins.octoprint.org/update_check_overlay.json
2022-01-17 21:55:36,966 - octoprint.plugins.announcements - INFO - Loaded channel _blog from https://octoprint.org/feeds/octoblog.xml in 2.0s
2022-01-17 21:55:42,437 - octoprint.plugins.announcements - INFO - Loaded channel _plugins from https://plugins.octoprint.org/feed.xml in 1.4s
2022-01-17 21:55:47,198 - octoprint.plugins.announcements - INFO - Loaded channel _octopi from https://octoprint.org/feeds/octopi.xml in 1.2s
2022-01-17 21:55:48,419 - octoprint.plugins.pluginmanager - INFO - Loaded plugin notices data from https://plugins.octoprint.org/notices.json
2022-01-17 21:55:51,217 - octoprint.util.pip - INFO - Using "/volume1/@appstore/octoprint/env/bin/python3 -m pip" as command to invoke pip
2022-01-17 21:55:51,225 - octoprint.util.pip - INFO - pip installs to /volume1/@appstore/octoprint/env/lib/python3.10/site-packages (writable -> yes), --user flag needed -> no, virtual env -> yes
2022-01-17 21:55:51,226 - octoprint.util.pip - INFO - ==> pip ok -> yes
2022-01-17 21:55:51,354 - octoprint.plugins.softwareupdate - INFO - Saved version cache to disk

@th0ma7
Copy link
Contributor Author

th0ma7 commented Mar 2, 2022

@miigotu the sickchill package is in theory ready for publishing. Was there any update planned that could benefit from being part of the python310 update?

@hgy59 on a similar note, I haven't published homeassistant just yet. Are you planning for an update anything soon?

@miigotu
Copy link
Contributor

miigotu commented Mar 2, 2022

@miigotu the sickchill package is in theory ready for publishing. Was there any update planned that could benefit from being part of the python310 update?

@hgy59 on a similar note, I haven't published homeassistant just yet. Are you planning for an update anything soon?

Sickchill should work fine from 3.6.2 to python 3.11.x, and possibly newer as long as all of the dependencies that require c-extenstion/rust-extensions are built.

@hgy59
Copy link
Contributor

hgy59 commented Mar 2, 2022

@hgy59 on a similar note, I haven't published homeassistant just yet. Are you planning for an update anything soon?

As I mentioned earlier, any homeassistant update needs cryptography ... that needs setuptools_rust to build the wheel.
(I am currently on vacation and have no access to my spk dev env).

@th0ma7
Copy link
Contributor Author

th0ma7 commented Mar 2, 2022

Thnx for your feedbacks to both of you. So it looks like rust is mandatory for any next steps...

@miigotu
Copy link
Contributor

miigotu commented Mar 2, 2022

Thnx for your feedbacks to both of you. So it looks like rust is mandatory for any next steps...

Yes, rust is going to become the defacto replacement for all c extensions over the next few years, and cryptography is a requirement of any application that makes web requests unless a pure python replacement comes along some day (crosses fingers). Cryptography could be done entirely in python with a small performance hit.

@th0ma7
Copy link
Contributor Author

th0ma7 commented Mar 5, 2022

rust is on my todo list but considering I've been sick over the last moth this will have to wait a little longer until I get better (or someone else tackle this over). In the meantime, we can stick with cryptography version 3.3.2 ...

@th0ma7
Copy link
Contributor Author

th0ma7 commented Mar 26, 2022

@mayweed is there any interest in upgrading fishnet?
Current version uses an older python3 and is 1.18.1 while new version is at 2.5.1.
EDIT: Note that I can help in getting there but I have no clue how to use this software.

@mayweed
Copy link
Contributor

mayweed commented Mar 28, 2022

No, fishnet was rewritten in Rust actually. So it's a whole New package now...

@Safihre
Copy link
Contributor

Safihre commented Mar 28, 2022

@mayweed So for now we can just abandon it until someone updates it? I don't think we do rust yet..

@miigotu
Copy link
Contributor

miigotu commented Mar 30, 2022

No, fishnet was rewritten in Rust actually. So it's a whole New package now...
@mayweed So for now we can just abandon it until someone updates it? I don't think we do rust yet..

I wonder if utilizing docker for pure rust builds is a good option for synocommunity?

FROM rust:1.59.0 AS builder
WORKDIR /fishnet
RUN mkdir /source
RUN git clone git@bitbucket.org:Pumalo/docker-conf.git /fichnet && (git submodule update --init --recursive || true) && cargo build --release -vv

FROM scratch as export-stage
RUN mkdir -p /output/$TARGETPLATFORM
COPY --from=builder /fishnet/fishnet /output/$TARGETPLATFORM/
name: Build all necessary architectures for fishnet

jobs:
  build:
    runs-on: ubuntu-latest
    steps:
    - name: Set up QEMU
      uses: docker/setup-qemu-action@v1
    - name: Set up Docker Buildx
      uses: docker/setup-buildx-action@v1
    - name: Cache Docker layers
      uses: actions/cache@v2
      with:
        path: /tmp/.buildx-cache
        key: ${{ runner.os }}-buildx-${{ github.sha }}
        restore-keys: |
          ${{ runner.os }}-buildx-
    - name: Build docker image
      uses: docker/build-push-action@v2
      with:
        context: .
        platforms: linux/amd64,linux/arm64, <add all arch here>
        push: false
        output: type=local,dest=output_dir
        tags: fishnet-build
        cache-from: type=local,src=/tmp/.buildx-cache
        cache-to: type=local,dest=/tmp/.buildx-cache-new

You'd have all of the binaries for each platform in output_dir/$TARGETPLATFORM/fishnet, and then you add that to your spk however you want. I'm not sure how difficult it would be to offload some of the work to github actions in this way but it might be beneficial.

Maybe a third party repository to automatically build these binaries and add them as release artifacts, thatr synocommunnity can wget in the build process.

@miigotu
Copy link
Contributor

miigotu commented Mar 30, 2022

Working on a POC just to see if I can build rust binaries that way: https://github.com/miigotu/fishnet-binaries

@hgy59
Copy link
Contributor

hgy59 commented Mar 30, 2022

Working on a POC just to see if I can build rust binaries that way: https://github.com/miigotu/fishnet-binaries

We already have rust support in spksrc. The following (cli) tools are built with rust:

  • cross/bat
  • cross/dua
  • cross/dutree
  • cross/exa
  • cross/ripgrep

@miigotu
Copy link
Contributor

miigotu commented Mar 31, 2022

Working on a POC just to see if I can build rust binaries that way: https://github.com/miigotu/fishnet-binaries

We already have rust support in spksrc. The following (cli) tools are built with rust:

  • cross/bat
  • cross/dua
  • cross/dutree
  • cross/exa
  • cross/ripgrep

I know that, but I still want to see if I can do this thing that way lol

@hgy59
Copy link
Contributor

hgy59 commented May 30, 2022

As I mentioned earlier, any homeassistant update needs cryptography ... that needs setuptools_rust to build the wheel.

@th0ma7 any progress on building cryptography (cross) wheel with rust? (homeassistant package needs an update 😄).

@th0ma7
Copy link
Contributor Author

th0ma7 commented May 30, 2022

any progress on building cryptography (cross) wheel with rust? (homeassistant package needs an update 😄).

Nope, not yet. Nothing started neither.

@Safihre
Copy link
Contributor

Safihre commented Jun 18, 2022

Updated that python3 package was removed. Should we also remove/disable it on https://synocommunity.com/ ?

@hgy59
Copy link
Contributor

hgy59 commented Jun 18, 2022

Updated that python3 package was removed. Should we also remove/disable it on https://synocommunity.com/ ?

There are still packages that depend on python3 (i.e. 3.7) in the repo, that are not updated to Python 3.10/3.8 yet

  • plexpy-custom
  • rdiff-backup
  • rutorrent (?)

@th0ma7
Copy link
Contributor Author

th0ma7 commented Jun 18, 2022

There are still packages that depend on python3 (i.e. 3.7) in the repo, that are not updated to Python 3.10/3.8 yet

  • plexpy-custom
  • rdiff-backup
  • rutorrent (?)

@hgy59 good catch, while they where all migrated to python310 but not published yet. And as for python38, only one left is domoticz.

@hogi1200
Copy link

hogi1200 commented Jul 5, 2022

any progress on building cryptography (cross) wheel with rust? (homeassistant package needs an update 😄).

Nope, not yet. Nothing started neither.

@th0ma7
may i ask if you already have a plan, when you will add rust support to enable cross compilation of newer cryptography versions?
seems like everybody here thinks you are the right guy for this task. 🙏 😉

my target is to be able to build a wheel of cryptography 36.0.2, so that I can upgrade my homeassistant core installation using that wheel.
but i am still a noob to spksrc cross-compilation toolchain, so i kind of depend on a working framework.

@th0ma7
Copy link
Contributor Author

th0ma7 commented Jul 5, 2022

Sadly my cycles are really limited at the moment and will remain for a few weeks. Realistically this might slip until end of summer early fall unless i stumble on an easy solution before then.

@hogi1200
Copy link

hogi1200 commented Jul 6, 2022

@th0ma7 thanks for the quick response.
Your contributions are still highly appreciated 👍

@hgy59
Copy link
Contributor

hgy59 commented Aug 12, 2022

@th0ma7 did you ever consider gateone package? (#4524 (comment))

@th0ma7
Copy link
Contributor Author

th0ma7 commented Aug 13, 2022

@th0ma7 did you ever consider gateone package? (#4524 (comment))

Nope, and sadly no clue what taht is (on vacation with limited internet)

@miigotu
Copy link
Contributor

miigotu commented Aug 13, 2022

@th0ma7 did you ever consider gateone package? (#4524 (comment))

Nope, and sadly no clue what taht is (on vacation with limited internet)

"Gate One is an HTML5-powered terminal emulator and SSH client"

https://github.com/liftoff/GateOne

@th0ma7
Copy link
Contributor Author

th0ma7 commented Oct 13, 2022

@cytec I tested the plexpy-custom package which makes use github.com/Tautulli/Tautulli.git
Is there an interest at keeping this package alive and maintained? Thnx in advance.

@th0ma7
Copy link
Contributor Author

th0ma7 commented Oct 13, 2022

any progress on building cryptography (cross) wheel with rust? (homeassistant package needs an update smile).

Nope, not yet. Nothing started neither.

@th0ma7 may i ask if you already have a plan, when you will add rust support to enable cross compilation of newer cryptography versions? seems like everybody here thinks you are the right guy for this task. pray wink

my target is to be able to build a wheel of cryptography 36.0.2, so that I can upgrade my homeassistant core installation using that wheel. but i am still a noob to spksrc cross-compilation toolchain, so i kind of depend on a working framework.

@hogi1200 rust wheels is now available on master, enjoy :)

@th0ma7 th0ma7 mentioned this issue Nov 3, 2022
9 tasks
@Casuallynoted
Copy link

Any information on Mylar? Currently gives Python errors, I saw this Issue linked to an issue about Mylar but don't see it in the table in the original post.

@hgy59
Copy link
Contributor

hgy59 commented Nov 13, 2022

@Casuallynoted you find the request for Mylar (3) in #4232

@Safihre Safihre changed the title META: Python 3.10 package status META: Python 3.10 / 3.11 package status Dec 12, 2022
@th0ma7
Copy link
Contributor Author

th0ma7 commented Feb 27, 2023

@SynoCommunity/developers is there still interest in domoticz? It's the only py38 package remaining.

@miigotu
Copy link
Contributor

miigotu commented Feb 28, 2023

@SynoCommunity/developers is there still interest in domoticz? It's the only py38 package remaining.

I had thought this was a Russian software, but I just learned it is from India. Honestly, there are much more advanced and widely used smart home operating systems and standalone softwares such as home assistant and zigbee2mqtt.

@th0ma7
Copy link
Contributor Author

th0ma7 commented May 7, 2023

@SynoCommunity/developers I've noticed that there is only domoticz that isn't yet migrated over to a newer version of python leaving python38 still in the code. Anyone interested in updating it or should it simply get removed entirely?

EDIT: Re-reding my previous post, if no one is against it I'll remove domoticz along with python38 entirely.

@Safihre
Copy link
Contributor

Safihre commented May 7, 2023

You have my vote. Nobody bothered to fix it in years..

@hgy59
Copy link
Contributor

hgy59 commented May 8, 2023

There is still a PR for domoticz (#4730)
Since domoticz package does not depend on Python 3.8 package, we can remove Python 3.8 packages from the repo (after double checking that none of the published packages depend on python 3.8)
But we should keep python38 in native and cross folders (until #4730 is updated to at least python 3.10)

@th0ma7
Copy link
Contributor Author

th0ma7 commented May 9, 2023

There is still a PR for domoticz (#4730)

That PR is 2 years old... you really intent to pursue that? Anyway, I'll revert my commit from #5724 for now.

EDIT: @hgy59 I ended-up updating python38 up to latest version until we get rid of it entirely. Let me know if I can help at completing domoticz migration towards 3.10-3.11 asap to clean-up older python2 and python38 along with the many cross/*

@th0ma7
Copy link
Contributor Author

th0ma7 commented Oct 1, 2024

Closing as since all packages got migrated to 3.11.

@th0ma7 th0ma7 closed this as completed Oct 1, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging a pull request may close this issue.

7 participants