docker-compose should be compatible with latest python libraries it uses

[ChenQi1989]

Currently docker-compose has maximum version requirement for several python libraries such jsonschema, requests, PyYAML, etc.
Is it possible for docker-compose to be compatible with the latest versions of these libraries? Why do docker-compose have such requirement?

[rumpl]

We had problems in the past with libraries changing things in a backwards incompatible way. Our solution is to be very conservative in the version we allow for compose. We update those when new versions are out and we have tested compose with then. What library in particular are you talking about? We could test compose with the latest.

[ChenQi1989]

Thanks for the info.

Generally I'm talking about all libraries specified in setup.py. They all have maximum version requirements. But in my working environment, the following 3 library requirements are causing problem for me.
PyYAML
requests
jsonschema

In particular, when old python requests module is required, it in turn has maximum version requirement for its dependencies, e.g. urllib3 and idna.

P.S.
I'm working for the Yocto project which usually will upgrade python modules to the latest versions.

[ChenQi1989]

Hi rumpl,

I can help test docker-compose with latest libraries. Is there some test suite or criteria?

[ndeloof]

Our test suite don't cover all potential use case of a widely adopted tool, so we are very conservative with version updates, as long as they are not required for docker-compose to achieve it's goals.

I'm working for the Yocto project which usually will upgrade python modules to the latest versions.

I'm not sure I understand why you ask for this in this "Linux distribution" context. Don't you bundle use our binary releases?

[ChenQi1989]

@ndeloof
Actually, we build docker-compose from source. Yocto is a project to build packages from source codes to generate a Linux image. It usually uses the latest stable versions of the packages.
What do you mean by 'bundle use our binary releases'? Thanks in advance.

[ndeloof]

Our official releases are distributed on https://github.com/docker/compose/releases
By "building from source" do you mean you also run pyinstaller or just use pip installation on your distribution?

Anyway, we don't want to get automatic version bump by using a non-constrained version range. We use to bump dependencies on a regular basis. Maybe we should enable dependabot so - at least - we get suggestion on potential upgrades as they get released upstream ?

[ChenQi1989]

@ndeloof
By "building from source", we actually do python3 setup.py build' and python3 setup.py install'. (The actually commands are a little different.)

I'd like to double check with you, using non-constrained version range is not an option, from docker-compose's point view. Right?

[ndeloof]

I confirm non-constrained version range is not an option. We want a reproducible build. And our primary distribution process is by binary release (where we get full control on the python runtime in use), pure python is just an alternate installation for unsupported platforms (typically: arm, see #6831)

[ndeloof]

We have enabled dependabot on compose repo so we will get notified on dependency updates by a PR and can act accordingly.