ffsync: Update syncstorage-rs to v0.17.3 #6187

mreid-tt · 2024-08-02T21:27:06Z

Description

This PR includes the following:

Update syncstorage-rs to v0.17.3
Update diesel to v2.1.6
Remove patch for pyo3 (no longer needed)
Update identity sync URI in documentation

Fixes #

Checklist

Build rule all-supported completed successfully
New installation of package completed successfully
Package upgrade completed successfully (Manually install the package again)
Package functionality was tested
Any needed documentation is updated/created

Type of change

Package update

mreid-tt · 2024-08-03T04:05:10Z

hey @hgy59, I'm not very familiar with rust packages and was wondering if you could assist with troubleshooting this update. Thus far the builds seem to be failing with this message:

error: failed to run custom build command for `mysqlclient-sys v0.4.0`

Caused by:
  process didn't exit successfully: `/github/workspace/spk/ffsync/work-x64-6.2.4/diesel-2.2.2/target/release/build/mysqlclient-sys-4d1b6445f4b60699/build-script-build` (exit status: 101)
  --- stdout

After doing some research it seems to be similar to this issue: diesel-rs/diesel#4056 but the solutions presented don't seem to resolve the issue with the build.

hgy59 · 2024-08-03T07:13:21Z

@mreid-tt https://github.com/sgrif/mysqlclient-sys does not provide an official release 0.4.0, latest is still 0.3.0. Probably it is too early to use 0.4.0.

hgy59 · 2024-08-03T07:50:47Z

@mreid-tt

we have

# we must define the mysql db socket, since the rust binaries (mysqlclient-sys)
# do not read settings from bin/mysql_conf
export "MYSQL_DB_SOCKET=/run/mysqld/mysqld10.sock"

There are probably other settings that must be defined in the environment.

unfortunately I can't see the real build error...

mreid-tt · 2024-08-03T11:17:03Z

unfortunately I can't see the real build error...

Understood. In the meantime I've tried an alternate approach suggested in diesel-rs/diesel#4056. Rather than using the latest diesel v2.2.2, I used v2.1.6. This allowed the builds for aarch64 and x64 to succeed. For the other architectures, we seem to have other build errors:

$ make arch-evansport-7.1 -C ./spk/python311
  note: This error originates from a subprocess, and is likely not a problem with pip.
error: subprocess-exited-with-error

× pip subprocess to install build dependencies did not run successfully.
│ exit code: 1
╰─> See above for output.

Further up in the log file I see:

0:01:05 load avg: 1.22 [25/44] test_json
Failed to import test module: test.test_json.test_tool
Traceback (most recent call last):
  File "/github/workspace/spk/python311/work-evansport-7.1/Python-3.11.5/Lib/unittest/loader.py", line 407, in _find_test_path
    module = self._get_module_from_name(name)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/github/workspace/spk/python311/work-evansport-7.1/Python-3.11.5/Lib/unittest/loader.py", line 350, in _get_module_from_name
    __import__(name)
  File "/github/workspace/spk/python311/work-evansport-7.1/Python-3.11.5/Lib/test/test_json/test_tool.py", line 10, in <module>
    from test.support.script_helper import assert_python_ok
  File "/github/workspace/spk/python311/work-evansport-7.1/Python-3.11.5/Lib/test/support/script_helper.py", line 11, in <module>
    import zipfile
  File "/github/workspace/spk/python311/work-evansport-7.1/Python-3.11.5/Lib/zipfile.py", line 6, in <module>
    import binascii
ImportError: libz.so.1: cannot open shared object file: No such file or directory

0:01:05 load avg: 1.22 [26/44] test_long -- test_json failed (uncaught exception)

Perhaps you can help with the above error?

hgy59 · 2024-08-03T12:49:32Z

@mreid-tt @th0ma7 The first errror when building python is:

  ModuleNotFoundError: No module named 'setuptools.command.bdist_wheel'

probably something in the python wheel creation has changed (update of pip, setuptools, ...?)

Found similar error here:
Dao-AILab/flash-attention#1014

hgy59 · 2024-08-03T13:25:03Z

The build of python311 fails for several archs

for arch-armv7-7.1 the following error occurred while creating the cross environment:
(as commented above with ModuleNotFoundError: No module named 'setuptools.command.bdist_wheel')

2024-08-03T10:57:21.6373962Z . /github/workspace/spk/python311/work-armv7-7.1/crossenv/bin/activate && cd /github/workspace/spk/python311/work-armv7-7.1/Python-3.11.5 && env LD_LIBRARY_PATH=/github/workspace/spk/python311/work-armv7-7.1/Python-3.11.5 PYO3_CROSS_LIB_DIR=/github/workspace/spk/python311/work-armv7-7.1/install/var/packages/python311/target/lib/ PYO3_CROSS_INCLUDE_DIR=/github/workspace/spk/python311/work-armv7-7.1/install/var/packages/python311/target/include/ OPENSSL_LIB_DIR=/github/workspace/spk/python311/work-armv7-7.1/install/var/packages/python311/target/lib/ OPENSSL_INCLUDE_DIR=/github/workspace/spk/python311/work-armv7-7.1/install/var/packages/python311/target/include/ PKG_CONFIG_LIBDIR=/github/workspace/spk/python311/work-armv7-7.1/install//var/packages/python311/target/lib/pkgconfig WORK_DIR=/github/workspace/spk/python311/work-armv7-7.1 INSTALL_PREFIX=/var/packages/python311/target TC_WORK_DIR=/github/workspace/toolchain/syno-armv7-7.1/work TC=syno-armv7-7.1 SYSROOT="/github/workspace/toolchain/syno-armv7-7.1/work/arm-unknown-linux-gnueabi/arm-unknown-linux-gnueabi/sysroot" LD="/github/workspace/toolchain/syno-armv7-7.1/work/arm-unknown-linux-gnueabi/bin/arm-unknown-linux-gnueabi-ld" LDSHARED="/github/workspace/toolchain/syno-armv7-7.1/work/arm-unknown-linux-gnueabi/bin/arm-unknown-linux-gnueabi-gcc -shared" CPP="/github/workspace/toolchain/syno-armv7-7.1/work/arm-unknown-linux-gnueabi/bin/arm-unknown-linux-gnueabi-cpp" NM="/github/workspace/toolchain/syno-armv7-7.1/work/arm-unknown-linux-gnueabi/bin/arm-unknown-linux-gnueabi-nm" CC="/github/workspace/toolchain/syno-armv7-7.1/work/arm-unknown-linux-gnueabi/bin/arm-unknown-linux-gnueabi-gcc" AS="/github/workspace/toolchain/syno-armv7-7.1/work/arm-unknown-linux-gnueabi/bin/arm-unknown-linux-gnueabi-as" RANLIB="/github/workspace/toolchain/syno-armv7-7.1/work/arm-unknown-linux-gnueabi/bin/arm-unknown-linux-gnueabi-ranlib" CXX="/github/workspace/toolchain/syno-armv7-7.1/work/arm-unknown-linux-gnueabi/bin/arm-unknown-linux-gnueabi-g++" AR="/github/workspace/toolchain/syno-armv7-7.1/work/arm-unknown-linux-gnueabi/bin/arm-unknown-linux-gnueabi-ar" STRIP="/github/workspace/toolchain/syno-armv7-7.1/work/arm-unknown-linux-gnueabi/bin/arm-unknown-linux-gnueabi-strip" OBJDUMP="/github/workspace/toolchain/syno-armv7-7.1/work/arm-unknown-linux-gnueabi/bin/arm-unknown-linux-gnueabi-objdump" OBJCOPY="/github/workspace/toolchain/syno-armv7-7.1/work/arm-unknown-linux-gnueabi/bin/arm-unknown-linux-gnueabi-objcopy" READELF="/github/workspace/toolchain/syno-armv7-7.1/work/arm-unknown-linux-gnueabi/bin/arm-unknown-linux-gnueabi-readelf" CFLAGS="-I/github/workspace/toolchain/syno-armv7-7.1/work/arm-unknown-linux-gnueabi/arm-unknown-linux-gnueabi/sysroot/usr/include -D__ARM_PCS_VFP=1 -I/github/workspace/spk/python311/work-armv7-7.1/install/var/packages/python311/target/include -D_LARGEFILE64_SOURCE -D_FILE_OFFSET_BITS=64 -L /github/workspace/spk/python311/work-armv7-7.1/install/var/packages/python311/target/lib -I /github/workspace/spk/python311/work-armv7-7.1/install/var/packages/python311/target/include" CPPFLAGS="-I/github/workspace/toolchain/syno-armv7-7.1/work/arm-unknown-linux-gnueabi/arm-unknown-linux-gnueabi/sysroot/usr/include -D__ARM_PCS_VFP=1 -I/github/workspace/spk/python311/work-armv7-7.1/install/var/packages/python311/target/include " CXXFLAGS="-I/github/workspace/toolchain/syno-armv7-7.1/work/arm-unknown-linux-gnueabi/arm-unknown-linux-gnueabi/sysroot/usr/include -D__ARM_PCS_VFP=1 -I/github/workspace/spk/python311/work-armv7-7.1/install/var/packages/python311/target/include " LDFLAGS="-L/github/workspace/toolchain/syno-armv7-7.1/work/arm-unknown-linux-gnueabi/arm-unknown-linux-gnueabi/sysroot/lib -D__ARM_PCS_VFP=1 -L/github/workspace/spk/python311/work-armv7-7.1/install/var/packages/python311/target/lib -Wl,--rpath-link,/github/workspace/spk/python311/work-armv7-7.1/install/var/packages/python311/target/lib -Wl,--rpath,/var/packages/python311/target/lib " CARGO_HOME="/github/workspace/distrib/cargo" RUSTUP_HOME="/github/workspace/distrib/rustup" RUSTUP_TOOLCHAIN="stable" CARGO_BUILD_TARGET="armv7-unknown-linux-gnueabihf" CARGO_TARGET_ARMV7_UNKNOWN_LINUX_GNUEABIHF_AR="/github/workspace/toolchain/syno-armv7-7.1/work/arm-unknown-linux-gnueabi/bin/arm-unknown-linux-gnueabi-ar" CARGO_TARGET_ARMV7_UNKNOWN_LINUX_GNUEABIHF_LINKER="/github/workspace/toolchain/syno-armv7-7.1/work/arm-unknown-linux-gnueabi/bin/arm-unknown-linux-gnueabi-gcc" CARGO_TARGET_ARMV7_UNKNOWN_LINUX_GNUEABIHF_RUSTFLAGS=" " TC_GCC=$(eval $(echo /github/workspace/spk/python311/work-armv7-7.1/../../../toolchain/syno-armv7-7.1/work/arm-unknown-linux-gnueabi/bin/arm-unknown-linux-gnueabi-gcc -dumpversion) 2>/dev/null || true) TC_GLIBC=2.26 TC_KERNEL=3.10.108 pip --disable-pip-version-check install cffi==1.15.1 cryptography==41.0.3 Cython==0.29.36 flit==3.9.0 poetry==1.6.1 scikit-build==0.17.6 setuptools-rust==1.7.0 setuptools-scm==7.1.0 numpy==1.25.2 maturin==1.2.3
2024-08-03T10:57:21.9509546Z WARNING: The directory '/github/home/.cache/pip' or its parent directory is not owned or is not writable by the current user. The cache has been disabled. Check the permissions and owner of that directory. If executing pip with sudo, you should use sudo's -H flag.
2024-08-03T10:57:22.1798532Z Collecting cffi==1.15.1
2024-08-03T10:57:22.2082662Z   Downloading cffi-1.15.1.tar.gz (508 kB)
2024-08-03T10:57:22.2307425Z      ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 508.5/508.5 kB 25.6 MB/s eta 0:00:00
2024-08-03T10:57:22.2957392Z   Preparing metadata (setup.py): started
2024-08-03T10:57:22.6790275Z   Preparing metadata (setup.py): finished with status 'done'
2024-08-03T10:57:22.8466590Z Collecting cryptography==41.0.3
2024-08-03T10:57:22.8561370Z   Downloading cryptography-41.0.3.tar.gz (630 kB)
2024-08-03T10:57:22.8624619Z      ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 630.1/630.1 kB 160.7 MB/s eta 0:00:00
2024-08-03T10:57:22.9775575Z   Installing build dependencies: started
2024-08-03T10:57:27.3930666Z   Installing build dependencies: finished with status 'error'
2024-08-03T10:57:27.4025718Z   error: subprocess-exited-with-error
2024-08-03T10:57:27.4026782Z   
2024-08-03T10:57:27.4027833Z   × pip subprocess to install build dependencies did not run successfully.
2024-08-03T10:57:27.4029020Z   │ exit code: 1
2024-08-03T10:57:27.4029657Z   ╰─> [101 lines of output]
2024-08-03T10:57:27.4032418Z       WARNING: The directory '/github/home/.cache/pip' or its parent directory is not owned or is not writable by the current user. The cache has been disabled. Check the permissions and owner of that directory. If executing pip with sudo, you should use sudo's -H flag.
2024-08-03T10:57:27.4033881Z       Collecting setuptools>=61.0.0
2024-08-03T10:57:27.4035302Z         Obtaining dependency information for setuptools>=61.0.0 from https://files.pythonhosted.org/packages/e1/58/e0ef3b9974a04ce9cde2a7a33881ddcb2d68450803745804545cdd8d258f/setuptools-72.1.0-py3-none-any.whl.metadata
2024-08-03T10:57:27.4036727Z         Downloading setuptools-72.1.0-py3-none-any.whl.metadata (6.6 kB)
2024-08-03T10:57:27.4037215Z       Collecting wheel
2024-08-03T10:57:27.4038593Z         Obtaining dependency information for wheel from https://files.pythonhosted.org/packages/7d/cd/d7460c9a869b16c3dd4e1e403cce337df165368c71d6af229a74699622ce/wheel-0.43.0-py3-none-any.whl.metadata
2024-08-03T10:57:27.4040891Z         Downloading wheel-0.43.0-py3-none-any.whl.metadata (2.2 kB)
2024-08-03T10:57:27.4041662Z       Collecting cffi>=1.12
2024-08-03T10:57:27.4042341Z         Downloading cffi-1.16.0.tar.gz (512 kB)
2024-08-03T10:57:27.4043341Z            ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 512.9/512.9 kB 26.2 MB/s eta 0:00:00
2024-08-03T10:57:27.4044179Z         Installing build dependencies: started
2024-08-03T10:57:27.4045095Z         Installing build dependencies: finished with status 'done'
2024-08-03T10:57:27.4045940Z         Getting requirements to build wheel: started
2024-08-03T10:57:27.4046916Z         Getting requirements to build wheel: finished with status 'done'
2024-08-03T10:57:27.4047775Z         Installing backend dependencies: started
2024-08-03T10:57:27.4048685Z         Installing backend dependencies: finished with status 'done'
2024-08-03T10:57:27.4049736Z         Preparing metadata (pyproject.toml): started
2024-08-03T10:57:27.4050728Z         Preparing metadata (pyproject.toml): finished with status 'error'
2024-08-03T10:57:27.4051783Z         error: subprocess-exited-with-error
2024-08-03T10:57:27.4052373Z       
2024-08-03T10:57:27.4053060Z         × Preparing metadata (pyproject.toml) did not run successfully.
2024-08-03T10:57:27.4053879Z         │ exit code: 1
2024-08-03T10:57:27.4054396Z         ╰─> [65 lines of output]
2024-08-03T10:57:27.4056835Z             /github/workspace/spk/python311/work-armv7-7.1/crossenv/build/lib/python3.11/site-packages/setuptools/config/setupcfg.py:508: SetuptoolsDeprecationWarning: The license_file parameter is deprecated, use license_files instead.
2024-08-03T10:57:27.4059055Z               warnings.warn(msg, warning_class)
2024-08-03T10:57:27.4059662Z             running dist_info
2024-08-03T10:57:27.4060421Z             creating /tmp/pip-modern-metadata-ftxs_63w/cffi.egg-info
2024-08-03T10:57:27.4061497Z             writing /tmp/pip-modern-metadata-ftxs_63w/cffi.egg-info/PKG-INFO
2024-08-03T10:57:27.4063040Z             writing dependency_links to /tmp/pip-modern-metadata-ftxs_63w/cffi.egg-info/dependency_links.txt
2024-08-03T10:57:27.4064627Z             writing entry points to /tmp/pip-modern-metadata-ftxs_63w/cffi.egg-info/entry_points.txt
2024-08-03T10:57:27.4066389Z             writing requirements to /tmp/pip-modern-metadata-ftxs_63w/cffi.egg-info/requires.txt
2024-08-03T10:57:27.4068016Z             writing top-level names to /tmp/pip-modern-metadata-ftxs_63w/cffi.egg-info/top_level.txt
2024-08-03T10:57:27.4069501Z             writing manifest file '/tmp/pip-modern-metadata-ftxs_63w/cffi.egg-info/SOURCES.txt'
2024-08-03T10:57:27.4070912Z             reading manifest file '/tmp/pip-modern-metadata-ftxs_63w/cffi.egg-info/SOURCES.txt'
2024-08-03T10:57:27.4072002Z             reading manifest template 'MANIFEST.in'
2024-08-03T10:57:27.4072727Z             adding license file 'LICENSE'
2024-08-03T10:57:27.4073791Z             writing manifest file '/tmp/pip-modern-metadata-ftxs_63w/cffi.egg-info/SOURCES.txt'
2024-08-03T10:57:27.4075075Z             creating '/tmp/pip-modern-metadata-ftxs_63w/cffi-1.16.0.dist-info'
2024-08-03T10:57:27.4075937Z             Traceback (most recent call last):
2024-08-03T10:57:27.4077947Z               File "/github/workspace/spk/python311/work-armv7-7.1/crossenv/build/lib/python3.11/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 353, in <module>
2024-08-03T10:57:27.4079649Z                 main()
2024-08-03T10:57:27.4081446Z               File "/github/workspace/spk/python311/work-armv7-7.1/crossenv/build/lib/python3.11/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 335, in main
2024-08-03T10:57:27.4083360Z                 json_out['return_val'] = hook(**hook_input['kwargs'])
2024-08-03T10:57:27.4084114Z                                          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-08-03T10:57:27.4086328Z               File "/github/workspace/spk/python311/work-armv7-7.1/crossenv/build/lib/python3.11/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 149, in prepare_metadata_for_build_wheel
2024-08-03T10:57:27.4088344Z                 return hook(metadata_directory, config_settings)
2024-08-03T10:57:27.4089066Z                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-08-03T10:57:27.4091035Z               File "/github/workspace/spk/python311/work-armv7-7.1/crossenv/build/lib/python3.11/site-packages/setuptools/build_meta.py", line 188, in prepare_metadata_for_build_wheel
2024-08-03T10:57:27.4092716Z                 self.run_setup()
2024-08-03T10:57:27.4094358Z               File "/github/workspace/spk/python311/work-armv7-7.1/crossenv/build/lib/python3.11/site-packages/setuptools/build_meta.py", line 174, in run_setup
2024-08-03T10:57:27.4095865Z                 exec(code, locals())
2024-08-03T10:57:27.4096464Z               File "<string>", line 183, in <module>
2024-08-03T10:57:27.4098253Z               File "/github/workspace/spk/python311/work-armv7-7.1/crossenv/build/lib/python3.11/site-packages/setuptools/__init__.py", line 87, in setup
2024-08-03T10:57:27.4099854Z                 return distutils.core.setup(**attrs)
2024-08-03T10:57:27.4100513Z                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-08-03T10:57:27.4102246Z               File "/github/workspace/spk/python311/work-armv7-7.1/crossenv/build/lib/python3.11/site-packages/setuptools/_distutils/core.py", line 185, in setup
2024-08-03T10:57:27.4103764Z                 return run_commands(dist)
2024-08-03T10:57:27.4104342Z                        ^^^^^^^^^^^^^^^^^^
2024-08-03T10:57:27.4106297Z               File "/github/workspace/spk/python311/work-armv7-7.1/crossenv/build/lib/python3.11/site-packages/setuptools/_distutils/core.py", line 201, in run_commands
2024-08-03T10:57:27.4107449Z                 dist.run_commands()
2024-08-03T10:57:27.4108504Z               File "/github/workspace/spk/python311/work-armv7-7.1/crossenv/build/lib/python3.11/site-packages/setuptools/_distutils/dist.py", line 973, in run_commands
2024-08-03T10:57:27.4109549Z                 self.run_command(cmd)
2024-08-03T10:57:27.4110505Z               File "/github/workspace/spk/python311/work-armv7-7.1/crossenv/build/lib/python3.11/site-packages/setuptools/dist.py", line 1217, in run_command
2024-08-03T10:57:27.4111386Z                 super().run_command(command)
2024-08-03T10:57:27.4112483Z               File "/github/workspace/spk/python311/work-armv7-7.1/crossenv/build/lib/python3.11/site-packages/setuptools/_distutils/dist.py", line 992, in run_command
2024-08-03T10:57:27.4113348Z                 cmd_obj.run()
2024-08-03T10:57:27.4114278Z               File "/github/workspace/spk/python311/work-armv7-7.1/crossenv/build/lib/python3.11/site-packages/setuptools/command/dist_info.py", line 42, in run
2024-08-03T10:57:27.4115277Z                 bdist_wheel = self.get_finalized_command('bdist_wheel')
2024-08-03T10:57:27.4115749Z                               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-08-03T10:57:27.4116822Z               File "/github/workspace/spk/python311/work-armv7-7.1/crossenv/build/lib/python3.11/site-packages/setuptools/_distutils/cmd.py", line 305, in get_finalized_command
2024-08-03T10:57:27.4117861Z                 cmd_obj = self.distribution.get_command_obj(command, create)
2024-08-03T10:57:27.4118354Z                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-08-03T10:57:27.4119419Z               File "/github/workspace/spk/python311/work-armv7-7.1/crossenv/build/lib/python3.11/site-packages/setuptools/_distutils/dist.py", line 864, in get_command_obj
2024-08-03T10:57:27.4120366Z                 klass = self.get_command_class(command)
2024-08-03T10:57:27.4120764Z                         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-08-03T10:57:27.4121749Z               File "/github/workspace/spk/python311/work-armv7-7.1/crossenv/build/lib/python3.11/site-packages/setuptools/dist.py", line 954, in get_command_class
2024-08-03T10:57:27.4122680Z                 self.cmdclass[command] = cmdclass = ep.load()
2024-08-03T10:57:27.4123104Z                                                     ^^^^^^^^^
2024-08-03T10:57:27.4124053Z               File "/github/workspace/native/python311/work-native/install/usr/local/lib/python3.11/importlib/metadata/__init__.py", line 202, in load
2024-08-03T10:57:27.4124959Z                 module = import_module(match.group('module'))
2024-08-03T10:57:27.4125371Z                          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-08-03T10:57:27.4126299Z               File "/github/workspace/native/python311/work-native/install/usr/local/lib/python3.11/importlib/__init__.py", line 126, in import_module
2024-08-03T10:57:27.4127213Z                 return _bootstrap._gcd_import(name[level:], package, level)
2024-08-03T10:57:27.4127694Z                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-08-03T10:57:27.4128274Z               File "<frozen importlib._bootstrap>", line 1204, in _gcd_import
2024-08-03T10:57:27.4128849Z               File "<frozen importlib._bootstrap>", line 1176, in _find_and_load
2024-08-03T10:57:27.4129541Z               File "<frozen importlib._bootstrap>", line 1140, in _find_and_load_unlocked
2024-08-03T10:57:27.4130262Z             ModuleNotFoundError: No module named 'setuptools.command.bdist_wheel'
2024-08-03T10:57:27.4130749Z             [end of output]
2024-08-03T10:57:27.4131013Z       
2024-08-03T10:57:27.4131466Z         note: This error originates from a subprocess, and is likely not a problem with pip.
2024-08-03T10:57:27.4132075Z       error: metadata-generation-failed
2024-08-03T10:57:27.4132398Z       
2024-08-03T10:57:27.4132761Z       × Encountered error while generating package metadata.
2024-08-03T10:57:27.4133268Z       ╰─> See above for output.
2024-08-03T10:57:27.4133556Z       
2024-08-03T10:57:27.4133907Z       note: This is an issue with the package mentioned above, not pip.
2024-08-03T10:57:27.4134382Z       hint: See above for details.
2024-08-03T10:57:27.4134674Z       
2024-08-03T10:57:27.4135116Z       [notice] A new release of pip is available: 23.2.1 -> 24.2
2024-08-03T10:57:27.4135653Z       [notice] To update, run: pip install --upgrade pip
2024-08-03T10:57:27.4136037Z       [end of output]
2024-08-03T10:57:27.4136271Z   
2024-08-03T10:57:27.4136686Z   note: This error originates from a subprocess, and is likely not a problem with pip.
2024-08-03T10:57:27.4137274Z error: subprocess-exited-with-error
2024-08-03T10:57:27.4137512Z 
2024-08-03T10:57:27.4137889Z × pip subprocess to install build dependencies did not run successfully.
2024-08-03T10:57:27.4138394Z │ exit code: 1
2024-08-03T10:57:27.4138662Z ╰─> See above for output.
2024-08-03T10:57:27.4138853Z 
2024-08-03T10:57:27.4139175Z note: This error originates from a subprocess, and is likely not a problem with pip.
2024-08-03T10:57:27.4969015Z make[3]: *** [Makefile:184: python311_post_install] Error 1
2024-08-03T10:57:27.4970267Z make[3]: Leaving directory '/github/workspace/cross/python311'
2024-08-03T10:57:27.4972939Z make[2]: *** [../../mk/spksrc.depend.mk:54: depend_target] Error 2
2024-08-03T10:57:27.4974081Z make[2]: Leaving directory '/github/workspace/spk/python311'
2024-08-03T10:57:27.4994969Z make[1]: *** [../../mk/spksrc.supported.mk:71: build-arch-armv7-7.1] Error 1
2024-08-03T10:57:27.4996134Z make[1]: Leaving directory '/github/workspace/spk/python311'
2024-08-03T10:57:27.5000143Z rm -fr work-*/.copy_done \
2024-08-03T10:57:27.5000804Z        work-*/.depend_done \
2024-08-03T10:57:27.5001403Z        work-*/.icon_done \
2024-08-03T10:57:27.5001982Z        work-*/.strip_done \
2024-08-03T10:57:27.5002532Z        work-*/conf \
2024-08-03T10:57:27.5003030Z        work-*/scripts \
2024-08-03T10:57:27.5003534Z        work-*/staging \
2024-08-03T10:57:27.5004028Z        work-*/tc_vars.mk \
2024-08-03T10:57:27.5004549Z        work-*/tc_vars.cmake \
2024-08-03T10:57:27.5005098Z        work-*/tc_vars.meson \
2024-08-03T10:57:27.5005665Z        work-*/package.tgz \
2024-08-03T10:57:27.5006214Z        work-*/INFO \
2024-08-03T10:57:27.5006675Z        work-*/PLIST \
2024-08-03T10:57:27.5007192Z        work-*/PACKAGE_ICON* \
2024-08-03T10:57:27.5007780Z        work-*/WIZARD_UIFILES
2024-08-03T10:57:28.0313155Z rm -fr work-armv7-7.1/db-6.2.32
2024-08-03T10:57:28.8495580Z rm -fr work-armv7-7.1/bzip2-1.0.8
2024-08-03T10:57:28.9568986Z rm -fr work-armv7-7.1/gdbm-1.23
2024-08-03T10:57:29.0184261Z rm -fr work-armv7-7.1/expat-2.6.2
2024-08-03T10:57:29.0912773Z rm -fr work-armv7-7.1/libffi-3.4.6
2024-08-03T10:57:29.1772054Z rm -fr work-armv7-7.1/util-linux-2.38.1
2024-08-03T10:57:29.3615039Z rm -fr work-armv7-7.1/ncurses-6.4
2024-08-03T10:57:29.4768346Z rm -fr work-armv7-7.1/openssl-3.1.6
2024-08-03T10:57:29.8191603Z rm -fr work-armv7-7.1/Python-3.11.5
2024-08-03T10:57:30.1040834Z rm -fr work-armv7-7.1/readline-8.2
2024-08-03T10:57:30.2407573Z rm -fr work-armv7-7.1/sqlite-autoconf-3450200
2024-08-03T10:57:30.3080783Z rm -fr work-armv7-7.1/xz-5.4.5
2024-08-03T10:57:30.4101812Z rm -fr work-armv7-7.1/zlib-1.3.1
2024-08-03T10:57:30.4950452Z make: Leaving directory '/github/workspace/spk/python311'

My local build of python311 does not fail for arch-armv7-7.1.
So I guess it depends either on the github runner image (Ubuntu 22.04) or the version of ghcr.io/synocommunity/spksrc:latest.

my successful build used a 12 months old ghcr.io/synocommunity/spksrc with sha 65b3e95ec808

hgy59 · 2024-08-03T13:31:37Z

Further up in the log file I see:

you can ignore errors in python tests. Most of the tests fail for cross compiled targets. The tests are executed to optimize the python code. So optimization for cross compiled versions is limitted.

Includes update to diesel 2.2.2

This reverts commit 814696a.

mreid-tt · 2024-09-03T12:24:19Z

@hgy59, it seems that the latest changes to the repo fixed the build issues. A re-basing allowed my PR to build without issue.

This reverts commit 88e157d.

hgy59 · 2024-09-04T16:12:04Z

it seems that the latest changes to the repo fixed the build issues. A re-basing allowed my PR to build without issue.

if related to
https://github.com/SynoCommunity/spksrc/pull/6218/files#diff-a29335ae733917c15ea391777fbc8888f1365d19c70a854c32e7aa82f46435f5

it was at least worth the effort...

mreid-tt · 2024-09-05T02:35:56Z

@hgy59 I think I need some help with the requirements files. Comparing the two releases (mozilla-services/syncstorage-rs@0.14.3...0.17.3), and looking at the requirements files I see the following changes:

requirements.txt

- cryptography==3.4.8
+ cryptography==42.0.8

tools/tokenserver/requirements.txt

- pyramid==2.0.1
+ pyramid==2.0.2
+ datadog
+ backoff

I'm not sure how to incorporate these changes into the src files (or even if I need to since it builds and runs successfully). Let me know your thoughts.

EDIT: I looked back at your recommendations for this in #5939 (comment) but as I don't have a local dev environment (no x86 arch), I don't think your script to generate the requirements will work for me.

EDIT: I've looked back at additional recommendations from @th0ma7 in #5939 (comment) and I've made a go at updating the requirements. Hopefully it will build okay.

EDIT: The build seems to have been successful and I've tested an upgrade install which works well. I updated the named dependency (testfixtures) but I'm not sure about updating the further dependencies which were at the end of the requirements files.

hgy59 · 2024-09-05T20:39:15Z

There must be an error somewhere
the build log tells:

===>  Install config file
===>  Patch binaries built with rust to find shared libraries
===>  Creating package icons for ffsync
===>  Generating service scripts for ffsync
===>  Creating /spksrc/spk/ffsync/work-x64-7.1/conf/resource
===>  Creating /spksrc/spk/ffsync/work-x64-7.1/conf/privilege
===>  (privilege) run-as: package
===>  (privilege) DSM >= 7 /spksrc/spk/ffsync/work-x64-7.1/conf/privilege
/bin/sh: 1: Syntax error: Unterminated quoted string
(cd /spksrc/spk/ffsync/work-x64-7.1/staging && find . -mindepth 1 -maxdepth 1 -not -empty | tar cpzf /spksrc/spk/ffsync/work-x64-7.1/package.tgz --owner=root --group=root --files-from=/dev/stdin)
===>  Creating INFO file for ffsync

…requirements

mreid-tt · 2024-09-05T23:20:52Z

There must be an error somewhere

Hmm, looking at the logs there are a number of warnings about statements falling through and deprecations but I couldn't see anything specific in the ffsync section of the build. The python311 build on the other hand had quite a few like these:

ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
pip-tools 7.4.1 requires build>=1.0.0, but you have build 0.10.0 which is incompatible.

This happens during the installation of multiple versions of setuptools from what I could see and happened after the uninstall of a newer version of build. I don't know if this is contributing to the error in ffsync.

mreid-tt · 2024-09-05T23:48:46Z

/bin/sh: 1: Syntax error: Unterminated quoted string

It's not clear to me what is running at the time this error is thrown. The privilege file gets created successfully with the command immediately before. Additionally, the package.tgz file is correctly included in the package with the command immediately after. Maybe a debugging build can help illuminate the issue.

Outside of that the package installs correctly and I've configured Firefox clients to point to the server that is installed and the SyncStorage service works perfectly. I really can't detect any impact from this error.

th0ma7 · 2024-09-06T00:06:33Z

There must be an error somewhere

Hmm, looking at the logs there are a number of warnings about statements falling through and deprecations but I couldn't see anything specific in the ffsync section of the build. The python311 build on the other hand had quite a few like these:
ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
pip-tools 7.4.1 requires build>=1.0.0, but you have build 0.10.0 which is incompatible.
This happens during the installation of multiple versions of setuptools from what I could see and happened after the uninstall of a newer version of build. I don't know if this is contributing to the error in ffsync.

Don't worry about this one in particular as it is when i am forcing a downgrade of pip to install an older numpy in the crossenv. I then re-upgrade it to latest with no issue

mreid-tt · 2024-09-06T19:18:52Z

@hgy59 I've resolved the build error as the log now looks like this:

  ===>  Install config file
  ===>  Patch binaries built with rust to find shared libraries
  ===>  Creating package icons for ffsync
  ===>  Generating service scripts for ffsync
  ===>  Creating /github/workspace/spk/ffsync/work-x64-7.1/conf/resource
  ===>  Creating /github/workspace/spk/ffsync/work-x64-7.1/conf/privilege
  ===>  (privilege) run-as: package
  ===>  (privilege) DSM >= 7 /github/workspace/spk/ffsync/work-x64-7.1/conf/privilege
  (cd /github/workspace/spk/ffsync/work-x64-7.1/staging && find . -mindepth 1 -maxdepth 1 -not -empty | tar cpzf /github/workspace/spk/ffsync/work-x64-7.1/package.tgz --owner=root --group=root --files-from=/dev/stdin)
  ===>  Creating INFO file for ffsync

When I looked more into the build sequence the last command before the error was in this section:

spksrc/mk/spksrc.service.mk

Line 340 in 2dc9159

    
           # Generate privilege file for service user (prefixed to avoid collision with busybox account)

The next sections after this dealt with "service configuration for admin port" as well as "DSM UI configuration (app/config)". Part of this dealt with the service description. I guessed that for the syntax error shown it must have something to do with quotes. I noted that the description of the package had three single quotes in the description:

spksrc/spk/ffsync/Makefile

Line 17 in 2dc9159

    
           DESCRIPTION = An implementation of the Mozilla Sync-1.5 Server protocol used by the sync service in Firefox 29 and later. You can use Mozilla Sync Server to exchange browser data \(bookmarks, history, open tabs, passwords, add-ons, and the like\) between 'clients' in a manner that respects a user's security and privacy. The service runs on port 8132.

I then did a search in the repo for other packages with single quotes and noted a package that had the single quotes escaped and recently built without error:

spksrc/spk/chromaprint/Makefile

Line 11 in 2dc9159

    
           DESCRIPTION = Chromaprint is the core component of the AcoustID project. It\'s a client-side library that implements a custom algorithm for extracting fingerprints from any audio source.

So I then simply escaped all the single quotes and the build completed without error. One interesting finding was that the INFO file which is generated in the package has the same description whether or not the single quotes are escaped or not so it may have something to do with a function that is using this information. Perhaps this line may be the culprit:

spksrc/mk/spksrc.service.mk

Line 460 in 2dc9159

SERVICE_DESC=$(shell echo ${DESCRIPTION} | sed -e 's/\\//g' -e 's/"/\\"/g')

Either way, it seems safer to escape single quotes.

hgy59 · 2024-09-06T20:23:31Z

Either way, it seems safer to escape single quotes.

I have now updated the description and examples of the DESCRIPTION variable in the Makefile variables wiki page

hgy59

LGTM

ready to merge and publish (if @th0ma7 has no objections regarding python)

th0ma7

Lgtm, and have'nt found any changes to the pyhton311 neither, have i missed anything @hgy59 ?

mreid-tt changed the title ~~ffsync: Update to syncstorage-rs v0.17.1~~ ffsync: Update syncstorage-rs to v0.17.1 Aug 2, 2024

mreid-tt self-assigned this Aug 2, 2024

mreid-tt added the build/rust Requires rust build system label Aug 2, 2024

mreid-tt force-pushed the ffsync-update branch from b98cfc2 to 814696a Compare August 3, 2024 00:59

mreid-tt added the status/help-wanted label Aug 3, 2024

mreid-tt added 4 commits September 3, 2024 05:52

Update to v0.17.1

f376b40

Includes update to diesel 2.2.2

Patch for mysqlclient-sys

b87b8c5

Revert "Patch for mysqlclient-sys"

ae0e1f8

This reverts commit 814696a.

Revert to diesel-2.1.6

385abfb

mreid-tt force-pushed the ffsync-update branch from 13612a0 to 385abfb Compare September 3, 2024 09:52

mreid-tt removed the status/help-wanted label Sep 3, 2024

Bump to v0.17.2

515743a

mreid-tt changed the title ~~ffsync: Update syncstorage-rs to v0.17.1~~ ffsync: Update syncstorage-rs to v0.17.2 Sep 3, 2024

mreid-tt added 3 commits September 3, 2024 08:51

Attempt bump to diesel-2.2.3

88e157d

Revert "Attempt bump to diesel-2.2.3"

027cb41

This reverts commit 88e157d.

Bump to v0.17.3

ffe131f

mreid-tt changed the title ~~ffsync: Update syncstorage-rs to v0.17.2~~ ffsync: Update syncstorage-rs to v0.17.3 Sep 4, 2024

Update install wizard

a6a4249

mreid-tt added the status/ready-to-merge label Sep 4, 2024

mreid-tt requested a review from hgy59 September 4, 2024 12:31

mreid-tt removed the status/ready-to-merge label Sep 5, 2024

mreid-tt added the status/needs-review label Sep 5, 2024

mreid-tt added 2 commits September 5, 2024 08:02

Attempt to update requirements

a1e1178

Use latest version of testfixtures

c3b4ee5

ffsync: update requirements to the latest versions found by original …

9b363a2

…requirements

Attempt to fix quotes in Makefile

081af60

hgy59 approved these changes Sep 6, 2024

View reviewed changes

th0ma7 approved these changes Sep 6, 2024

View reviewed changes

hgy59 merged commit 2af6c9c into SynoCommunity:master Sep 6, 2024
17 checks passed

mreid-tt deleted the ffsync-update branch September 6, 2024 21:47

mreid-tt added status/published Published and activated (may take up to 48h until visible in DSM package manager) and removed status/needs-review labels Sep 6, 2024

This was referenced Sep 12, 2024

ffsync: Update syncstorage-rs to v0.17.4 #6228

Open

bazarr: update 1.4.2 => 1.4.5 and migrate to ffmpeg 7 #6164

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

ffsync: Update syncstorage-rs to v0.17.3 #6187

ffsync: Update syncstorage-rs to v0.17.3 #6187

mreid-tt commented Aug 2, 2024 •

edited

Loading

mreid-tt commented Aug 3, 2024

hgy59 commented Aug 3, 2024

hgy59 commented Aug 3, 2024

mreid-tt commented Aug 3, 2024 •

edited

Loading

hgy59 commented Aug 3, 2024

hgy59 commented Aug 3, 2024

hgy59 commented Aug 3, 2024

mreid-tt commented Sep 3, 2024

hgy59 commented Sep 4, 2024

mreid-tt commented Sep 5, 2024 •

edited

Loading

hgy59 commented Sep 5, 2024

mreid-tt commented Sep 5, 2024

mreid-tt commented Sep 5, 2024

th0ma7 commented Sep 6, 2024

mreid-tt commented Sep 6, 2024

hgy59 commented Sep 6, 2024

hgy59 left a comment

th0ma7 left a comment

ffsync: Update syncstorage-rs to v0.17.3 #6187

ffsync: Update syncstorage-rs to v0.17.3 #6187

Conversation

mreid-tt commented Aug 2, 2024 • edited Loading

Description

Checklist

Type of change

mreid-tt commented Aug 3, 2024

hgy59 commented Aug 3, 2024

hgy59 commented Aug 3, 2024

mreid-tt commented Aug 3, 2024 • edited Loading

hgy59 commented Aug 3, 2024

hgy59 commented Aug 3, 2024

hgy59 commented Aug 3, 2024

mreid-tt commented Sep 3, 2024

hgy59 commented Sep 4, 2024

mreid-tt commented Sep 5, 2024 • edited Loading

hgy59 commented Sep 5, 2024

mreid-tt commented Sep 5, 2024

mreid-tt commented Sep 5, 2024

th0ma7 commented Sep 6, 2024

mreid-tt commented Sep 6, 2024

hgy59 commented Sep 6, 2024

hgy59 left a comment

Choose a reason for hiding this comment

th0ma7 left a comment

Choose a reason for hiding this comment

mreid-tt commented Aug 2, 2024 •

edited

Loading

mreid-tt commented Aug 3, 2024 •

edited

Loading

mreid-tt commented Sep 5, 2024 •

edited

Loading