micropython: update to 1.23.0 and fix compile on external mbedtls3 #24664
Conversation
|
This PR also subsumes #23795 (tag @nzmichaelh) |
|
build failure |
|
@jefferyto Should the remaining patch in micropython-lib be removed, and |
gstrauss
force-pushed
the
micropython-mbedtls-3.x
branch
from
181513c
to
77dc4a9
Compare
Thanks! |
|
@jefferyto please review. Note: I removed the See micropython/micropython-lib@23df50d for upstream commit with commit message "unix-ffi: Remove "unix_ffi" argument from require(). And describe how to use |
|
@gstrauss it did compile but when I tried requesting by mpython it didn't have any CA certificates loaded edit: no CA cert is intended default statue of micropython, but because mbedtls3.x removed ssl mode option on client mbedtls refuse to connect |
|
micropython/micropython-lib#838 |
|
@orangepizza that default in the new python-only ssl code is an awful default for security. Thank you for highlighting it. Do you happen to know if this is a change in behavior from micropython-lib v1.22.0? Attempting to modify this PR to use micropython-lib v1.22.0 reveals an issue I did not immediately know how to solve, which is a missing "re" python module for unix-ffi, so the build fails. You're welcome to copy or resubmit a different PR if you come up with a different solution. I'm just trying to help a little bit with the openwrt mbedtls 3.x migration. |
|
It always had awful default but mbedtls 3 killed cert_none on tls 1.3 so now we have non working default: while it's documented to not support multiple certs in cacert file mbedtls context itself supports, so I'm not sure if it actually not working on full unix (this have microcontrollers that wouldnt) but looking at micropython mbedtls hook I see no reason it'd break buy multiple certs in pem, it calls mbedtls_x509_crt_parse directly on buffer |
|
@Ansuel I will take a look but might have some questions. I just updated micropython/micropython#15547 to try to get micropython submodule lib/mbedtls point to mbedtls 3.6.2. Hopefully it will be part of micropython 1.24.0. |
|
@Ansuel I renamed |
gstrauss
force-pushed
the
micropython-mbedtls-3.x
branch
2 times, most recently
from
cb1be35
to
b092247
Compare
|
@jefferyto please review. Note: I removed the All tests pass except for (temporary?) provisioning error for Test Build / Test i386_pentium-mmx (pull_request) See micropython/micropython-lib@23df50d for upstream commit with commit message "unix-ffi: Remove "unix_ffi" argument from require(). And describe how to use |
|
FYI: the non-working micropython default (with mbedtls 3.x) has been reported upstream by @orangepizza |
Running |
|
Oh, unfortunate. Please rebase. |
I am not well-versed in micropython enough to comment whether or not this approach is recommended (or not) by micropython developers, or if there are alternative approaches which might be used instead. I'll remove my commit which removed
I am a C programmer and have written foreign function interfaces (FFI) for various scripting languages. However, I am not an expert in micropython and so the limit of my confidence is running the automated tests. If they have good coverage, then that is great. If they do not, then I politely request the help of others to test and validate. |
gstrauss
force-pushed
the
micropython-mbedtls-3.x
branch
2 times, most recently
from
07d0052
to
5c72154
Compare
|
I removed the patches to micropython. This PR now contains only changes to micropython-lib. @Ansuel one of us should make a separate PR for your patch to quell compiler uninitialized variable warnings, and whether or not to add my patch which removes the copy of cdefs.h that you added and replaces with a small number of bog-standard macros. In this PR, powerpc CI tests are failing to download packages. Other platforms are failing with More work is needed for unix-ffi integration. @jefferyto I think micropython-lib with mbedtls has been broken in 23.05 for a while, so have you considered removing it instead of fixing it? |
|
@gstrauss: The main micropython update to 1.23.0 has been merged: Important : A 1.24.0 has been released few days ago: |
|
@gstrauss please try replacing 001-build-unix-ffi.patch with: --- a/tools/build.py
+++ b/tools/build.py
@@ -289,7 +289,7 @@ def _update_index_package_metadata(index
index_package_json["path"] = package_path
-def build(output_path, hash_prefix_len, mpy_cross_path):
+def build(output_path, unix_ffi, hash_prefix_len, mpy_cross_path):
import manifestfile
import mpy_cross
@@ -315,7 +315,10 @@ def build(output_path, hash_prefix_len,
# For now, don't process unix-ffi. In the future this can be extended to
# allow a way to request unix-ffi packages via mip.
- lib_dirs = ["micropython", "python-stdlib", "python-ecosys"]
+ lib_dirs = ["unix-ffi"] if unix_ffi else ["micropython", "python-stdlib", "python-ecosys"]
+
+ if unix_ffi:
+ manifestfile.BASE_LIBRARY_NAMES = ("unix-ffi",) + manifestfile.BASE_LIBRARY_NAMES
mpy_version, _mpy_sub_version = mpy_cross.mpy_version(mpy_cross=mpy_cross_path)
mpy_version = str(mpy_version)
@@ -446,6 +449,7 @@ def main():
cmd_parser = argparse.ArgumentParser(description="Compile micropython-lib for serving to mip.")
cmd_parser.add_argument("--output", required=True, help="output directory")
+ cmd_parser.add_argument("--unix-ffi", action="store_true", help="process unix-ffi packages")
cmd_parser.add_argument("--hash-prefix", default=8, type=int, help="hash prefix length")
cmd_parser.add_argument("--mpy-cross", default=None, help="optional path to mpy-cross binary")
cmd_parser.add_argument("--micropython", default=None, help="path to micropython repo")
@@ -455,7 +459,7 @@ def main():
sys.path.append(os.path.join(args.micropython, "tools")) # for manifestfile
sys.path.append(os.path.join(args.micropython, "mpy-cross")) # for mpy_cross
- build(args.output, hash_prefix_len=max(4, args.hash_prefix), mpy_cross_path=args.mpy_cross)
+ build(args.output, args.unix_ffi, hash_prefix_len=max(4, args.hash_prefix), mpy_cross_path=args.mpy_cross)
if __name__ == "__main__": |
* update micropython-lib to hash for 1.23.0 * remove micropython-lib unix-uffi patches Co-authored-by: Jeffery To <jeffery.to@gmail.com> Signed-off-by: Glenn Strauss <gstrauss@gluelogic.com>
gstrauss
force-pushed
the
micropython-mbedtls-3.x
branch
from
5c72154
to
b9b0145
Compare
|
@jefferyto Thank you. Your suggestion did the trick. I added you as co-author on the commit. Only test that fails is on powerpc_464fp platform with temporary error: |
Maintainer: @jefferyto Jeffery To jeffery.to@gmail.com
Compile tested: mips_24kc_musl
Description:
See micropython/micropython-lib@23df50d for upstream commit with commit message "unix-ffi: Remove "unix_ffi" argument from require(). And describe how to use
add_library()instead."This PR is a subsequent attempt at #24102, though has not yet been run-tested. (tag @orangepizza)