Chore: Upgrade to python 3.10 #1688
Conversation
✅ Deploy Preview for specter-desktop-docs ready!
To edit notification comments on pull requests, go to your Netlify site settings. |
|
I'd prefer to keep all the versions stable as we migrate to 3.10, otherwise we might fight too many battles at the same time. So if you want to upgrade other stuff as well, let's try to do that on a separate PR? |
|
I see 2 possible options:
Which one did you have in mind? |
|
I'd separate these two issues. Why does an upgrade to python 3.10 involves an upgrade of ubuntu? hashlib is not bringing its own version of openssl so this is a problem of the underlying OS, right? This has been discussed on hwi: bitcoin-core/HWI#305 At the end, this must be solved in embit and i'll suggest to create an issue there and discuss there. |
Ok. I will do the upgrade to python 3.10 in an older ubuntu version and reduce dependency version changes to a minimum.
Ok, will you create an issue in embit? |
Well, i thought, you started it, you go for it :-) ?! I hope there will be a more consistent and confirmed way to go forward. Maybe @stepansnigirev can judge what's the problem with Simplexum/python-bitcointx#65. Seems like Marco Falke is ok with it: bitcoin/bitcoin#23710 (comment) |
|
When building a docker image following https://github.com/cryptoadvance/specter-desktop/blob/master/docker/python-bitcoind/Readme.md , I get Am I doing something wrong? @k9ert : If you have time, it would be great if you could help with the docker images.
|
|
released new version of embit with a pure-python implementation of ripemd160 taken from bitcoin/test_framework |
I'm not sure why this is happening. However, we should remove it anyway. Qt4 is for the Bitcoin-UI and we don't care about that. Just remove that one. Probably if you migrate to the newest version of bitcoind, you need I guess only one of them need it. probably (assuming you use the newest version). Btw, you can also contact me on telegram and/or signal any time. |
|
Thanks. I will try that (looks good so far). I am not clear on which docker images are used for which purpose (I assume some are used for the github automated checks?).
|
- Added libzmq to build process, to enable future use cases such as cryptoadvance#778 - added --without-gui to remove configure warning that gui will not be built.
|
Good question. I would start in this order:
I think the best approach is that you push to your own registry and replace the references. When stuff works, i can rebuild and push to the appropiate And thank you very much for your work. |
|
Thanks, that explanation is very helpful.
When I upgrade to jammy, I also have to increase the embit version to 0.4.13. |
- pushed python-bitcoind image
- updated docker references
|
I am stuck.... somehow cirrus ci doesn't pull the docker images any more from gitlab registry I guess I have to use a public gitlab project... Trying that next |
|
In 6823a2e I reference the docker image 14-wine-mono-08.22, see electron-userland/electron-builder#6922 (comment) |
|
So:
We might still want more testing but i guess we can proceed to the last step, replacing the Docker-images with official ones. As far as i can see, here is the list of image-changes: cypress-base-ubuntu-jammy (new)
cypress-python --> cypress-python-jammy
cirrus-focal --> cirrus-jammy
python-bitcoind
electron-builder
|
|
|
|
|
||
| # The block below is created with pip-compile in python 3.8, such that requirements.txt is backwards compatible with python 3.8 | ||
| # This package is not needed for python 3.10 | ||
| backports.zoneinfo==0.2.1 ; python_version < '3.10' \ |
There was a problem hiding this comment.
@k9ert : Unfortunately pip-compile does not support these environment markers, and this block has to be placed here manually after doing pip-compile (in python 3.10). Do you know a better way?
There was a problem hiding this comment.
I'm not sure i understand the issue.
requirements.txt is independent on the python version. I see the issue that dependencies might differ between python-versions but have no idea how this could potentially be addressed.
There was a problem hiding this comment.
The issue is a difference of pip-compile in different python versions:
- Delete
requirements.txt pip-compile --generate-hashes requirements.inin python 3.10backports.zoneinfowill be missing inrequirements.txtand will have to be manually copied in again.
or:
- Delete
requirements.txt pip-compile --generate-hashes requirements.inin python 3.8backports.zoneinfowill be inrequirements.txt.
|
I've just created yet another test-release |
docker/electron-builder/README.md
Outdated
| @@ -17,6 +17,9 @@ docker run --rm -ti \ | |||
| build the image like: | |||
There was a problem hiding this comment.
@k9ert : This command
docker run --rm -ti \
--env-file <(env | grep -iE 'DEBUG|NODE_|ELECTRON_|YARN_|NPM_|CI|CIRCLE|TRAVIS_TAG|TRAVIS|TRAVIS_REPO_|TRAVIS_BUILD_|TRAVIS_BRANCH|TRAVIS_PULL_REQUEST_|APPVEYOR_|CSC_|GH_|GITHUB_|BT_|AWS_|STRIP|BUILD_') \
--env ELECTRON_CACHE="/root/.cache/electron" \
--env ELECTRON_BUILDER_CACHE="/root/.cache/electron-builder" \
-v ${PWD}:/project \
-v ${PWD##*/}-node-modules:/project/node_modules \
-v ~/.cache/electron:/root/.cache/electron \
-v ~/.cache/electron-builder:/root/.cache/electron-builder \
registry.gitlab.com/relativisticelectron/specter-desktop/electron-builder:latest
returns an error for me
docker: open /proc/self/fd/11: no such file or directory.
See 'docker run --help'.
So I have no way of testing the electron builder inside docker...
There was a problem hiding this comment.
I was not able to reproduce your issue with:
- Docker version 20.10.14, build a224086
- Ubuntu 20.04.4 LTS
developing/fixing those pipelines is always a horrible pain in the ass. Ever been and i can't imagine that will ever change much. Sometimes it's worth, sometimes better (e.g. cirrus has a way to run the pipeline locally).
I usually do a mixture but often enough let the pipeline run for the 20th time rather than trying to reproduce locally (while doing something else in the meantime).
I've created a new tag and pushed it this time to my fork in order to not pollute the main project. You could potentially do that as well: https://github.com/cryptoadvance/specter-desktop/blob/master/docs/continuous-integration.md#cicd-dev-env-setup
|
There is a major roadblock with the |
|
ok, after upgrading docker (now 20.10.18) on the buildagent and three reboots, it seem to somehow be more stable now. I will now create yet another one but for the main repo: soon here: |
|
No, testing the plugins which are shipped directly is not covered by any tests. We don't even have a good test-concept for the plugins. But we might even have more severe issues here at hand. When i try to run Specter-1.13.0-pre5.AppImage on my Ubuntu 20.04.4 LTS, i get: Not good. I guess i have to dive into the the compatibility details of glibc again. |
I can reproduce your error in Virtualbox with Ubuntu 20.04. The error suggests that libpython3.10 needs at least glibc 2.35. Ubuntu 20.04 has however My initial idea for a path forward would be (also suggested here):
|
|
Fully agree to all your points. Effectively, i came to the same conclusion almost exactly 2 years ago but forgot about it 🤦 . |
|
Ok, rolled back the electron-builder and created yet another test-release. soon here: I've also smoketested the extensions with python3.10 in a pip-installation, works! I guess after testing the mentioned pre6, we might be ready to merge, no?! |
Yes.
|
|
Thank you very much appreciated 🙇♂️ |
From #1686
TODO:
Fix hashlib/openssl error Specter Python 3.10 compatibility; Ubuntu 22.04 has python 3.10 #1686 (comment) Possible Reason: RIPEMD160 does not work openssl/openssl#16994Create embit issue RIPEMD160 missing in Openssl3 diybitcoinhardware/embit#28 Solved in embit 0.4.13Fixed black error via Incompatible with click 8.1.0 (ImportError: cannot import name '_unicodefun' from 'click') psf/black#2964 (comment) see PR here: BugFix: Update black version to fix error #1708sudo docker build -t testspecter .andsudo docker build -t testspecter .of https://github.com/cryptoadvance/specter-desktop/blob/master/Dockerfile with modified python versions 3.7, 3.8, 3.9, 3.10 (3.6 was incompatible leading to Fix: Change specter compatibility to ">=3.7,<3.10" due to dependency #1707 )relativisticelectronwithcryptoadvance, check/rename docker image tag , build and push docker images to cryptoadvanceelectron-builder:latest, Was everything built successfully?