Triggering dependabot updates for docker images building on standard many/musllinux containers

[MusicalNinjaDad]

Hi all,

Does anyone have any success getting dependabot to spot when e.g. quay.io/pypa/manylinux2014_x86_64 changes?

Background:
I am pre-building containers with rust installed to reduce the pipeline build time for pyo3 extension libraries: e.g. for manylinux-x86
Particularly if building for architectures which run under emulation in pipeline this saves hours in the overall build time...

Sadly dependabot doesn't automatically recognise the yyyy-mm-dd-sha tag format, the docs claim it will understand a simple yyyy-mm-dd tag. This means I need to either manually think to rebuild my containers (delays), or run a scheduled job to build them anyway (wasteful of resources)

Has anyone found a way around this problem?

Thanks!

[henryiii]

Have you tried https://github.com/PyO3/maturin-action? While it does tie you to GHA, it has great cross-compilation support, and is faster than cibuildwheel since it can build all the rust parts then build the Python specific parts for each wheel, due to it's integration with maturin. If you have a rust project, it's worth a try.

[MusicalNinjaDad]

I decided to use setuptools-rust rather than maturin for a few reasons:

• My target setup is likely too complex for maturin, mixing native python with multiple rust modules and publishing some of the rust work as independent crates. The maturin & setuptools-rust docs suggest the later for more complex cases
• I prefer to extend than replace and I know and like setuptools ... so why learn a new build system
• cibuildwheel already skips recompiling for each python version if you target a reduced abi; if targeting the full interfaces I'd guess a compile for each version makes sense ... So I don't expect there would be any performance difference between compiling all the rust and then building the wheels vs doing it together with cibw
• the reception at pyo3 was rather unwelcoming overall - compared to an amazing experience here! Which underlines for me the choice to stick with pypa tooling as much as possible

It's a good tip to look at how maturin do things to create their base containers, though, I hadn't thought of that ...

[mayeut]

The suggestion by Henry makes sense for Rust.

However it seems there's still something to be done at pypa/manylinux level to improve Dependabot support in any case so I will add a new tag format which shall help with this.

cibuildwheel uses a custom script to update image tags: https://github.com/pypa/cibuildwheel/blob/main/bin/update_docker.py

[mayeut]

You should now be able to use the new tag format: 2024.06.08.3 is the latest tag. It seems that referencing this discussion from #1626 did not add a link here.

[MusicalNinjaDad]

Thanks!

I've updated my Dockerfile to pin 2024.06.08.2 and run a manual dependabot update - it didn't recommend a PR to update to 2024.06.08.3...
My guess is that it may work with the next date update ... I'll keep an eye on it
https://github.com/dependabot/dependabot-core/blob/main/docker/README.md doesn't say anything about accepting suffixes on dates, which is a bit odd, as I'd imagine you're not the only ones who may need to push more than once a day and use date tags.

Should I let you know whether it works byy posting a comment in this discussion, or better over on pyap/manylinux ?

[mayeut]

Should I let you know whether it works byy posting a comment in this discussion, or better over on pyap/manylinux ?

This discussion is now in pyap/manylinux.

Dependabot uses SemVer so it might limit the version to 3 components (contrary to PEP440).

[mayeut]

I slightly modified the format, should be better now with a build number. (it works as a pep440 version too as it considers the build number as a post release segment).

[MusicalNinjaDad]

Thanks - that's working wonderfully!