Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Versioning change (for API update) #146

Closed
masklinn opened this issue Nov 7, 2022 · 2 comments
Closed

Versioning change (for API update) #146

masklinn opened this issue Nov 7, 2022 · 2 comments
Milestone

Comments

@masklinn
Copy link
Contributor

masklinn commented Nov 7, 2022

Aside from actually designing the API, that's probably the biggest issue with #93 (#116): how to update the version?

Per #75 it's clearly important to have a good versioning scheme, and to reflect the version of ua-parser/uap-core since that's, well, the core.

But PEP 440 doesn't really provide a multi-versioning scheme:

  • local versions (+) are specifically for locally patched versions (in other ecosystems it's used for multi-versioning)
  • epochs (!) are for completely switching up the versioning scheme, in order to ensure ordering remains consistent when switching between e.g. m.m.m and date-based versioning schemes

The two contenders then seem to be:

  • extending / appending the version numbers as the "version segments" are illimited, e.g. 1.0.0.0.16 would be uap-python 1.0.0 using uap-core 0.16, this would allow users to select data versions within API versions, not the reverse but that doesn't seem very useful
  • splitting the package in two independently versioned libraries, one which would only expose regexes.py and the other the API and the matching machinery proper

The latter would increase the complexity (and moving parts), though it would selecting things more finely. However it's not clear that this division would be correct: if we start e.g. accelerating the matcher via a native implementation, the data from a ua_core.py package might not be used at all anymore.

On Calver

I want to explicitly reject calver as it was mentioned in #75: I don't think it's a good fit, calver is not useful for uap-python itself as the releases are irregular and mostly contingent on uap-core, but at the same time would not allow cleanly updating the API (#93), the library component makes the information of a semver version at least somewhat useful (though how hard semver principles are respected can be complicated).

uap-core could independently adopt calver, which would be a concern for the 0.x releases of uap-python, but probably not otherwise problematic (technically calver works fine in PEP 440, the only issue is switching from calver back to semver, as the version number has to regress, that's what epochs are for). It's not clear how good it would be though, as -core releases are also somewhat erratic, and it's not entirely clear what value date information would provide. The question would also come of whether calver releases would be as needed (though at most once a month) or at hard-set interval (à la ubuntu), possibly delaying data releases for users.

@masklinn masklinn added this to the 1.0 milestone Feb 6, 2024
@masklinn
Copy link
Contributor Author

masklinn commented Nov 4, 2024

So one advantage of option 2 is the versioning could be "floaty" so it updates on its own kinda, and furthermore it would be possible to release periodic alphas / prereleases of uap-core, as the official releases are cut pretty infrequently (see #211 for an exemple of motivation), though I'd have to look at how that works exactly e.g. is it possible for a consumer of the package to get onto the pre-release stream without "normal" consumers being affected? So e.g. we could have a monthly (?) release of a precompiled uap-core called 0.19.devX, where X would either be a numerical increment or a YYYYMM date (so e.g. 0.19.dev202411).

It could also be easier for debian to do the thing.

And finally the legacy package could be updated to make use of the split sub-package as well.

The drawback is I'd have to understand how to split the package into two somehow? This might also complexify tests, as they use regexes.yaml in both raw and precompiled forms.

Also the packages would be co-dependent: ua-parser depends on the precompiled expressions, but the precompiled expressions depend on ua-parser (to know what types to initialise).

@masklinn
Copy link
Contributor Author

masklinn commented Nov 8, 2024

Whelp, kinda hitting a few issues on the splitting:

  • because the dependency being extracted is entirely virtual (the package itself is created at runtime, then it's compiled into a wheel) a preprocessing step is needed to run the test suite, but tox does not support preprocessing steps (would require a tox plugin)
  • the test data and regexes.yaml are paired and updated in lockstep, so testing with precompiled data and an uap-core submodule does not guarantee they're in lockstep, on the other hand we want to test precompiled data (and have to for the legacy tests) otherwise there's no guarantee they work...

masklinn added a commit to masklinn/uap-python that referenced this issue Nov 18, 2024
The goal of this is the ability to generate wheels for precompiled
instances of uap-core, at whatever version we want.

The goal here is ultimately to move all the precompiled stuff outside
of uap-python itself and in a separate wheel, in order to generate
wheels for precompiled instances of uap-core at whatever version we want

1. It resolves ua-parser#146 by splitting the versioning of the API and that of
   the (pre-compiled) data, this is an issue for 1.0 as that detaches
   uap-python's versioning from uap-core's.
2. It allows users to update the API and the precompiled dataset
   separately, something they would otherwise need to do via yaml.
3. It fixes ua-parser#221 by allowing the regular release of "preview"
   precompiled regexes from uap-core snapshots e.g. we could release
   0.19.dev202412 at the start of december with whatever uap-core
   merged between the previous prerelease and then. This should not be
   picked up by pip by default, but would allow users to access those
   prerelases via `pip install --pre`.
4. If done well enough, it might allow users to build bespoke
   precompiled datasets so they don't have to pick between custom
   rules and precompiled (not sure there's any demand for this but it
   seems like it might be useful).
5. If it works well enough it might actually be possible to have 0.x
   use the legacy codegen package meaning it should not need to be
   updated anymore.

This is implemented via hatch build hooks (which seem seem simpler
than doing it via setuptools in the end).

Adding `regexes.yaml` to the sdist via artifacts is a bit strange but
necessary in order to generate a complete sdist which a wheel can be
built from (even though the release script will likely only push the
wheel).
masklinn added a commit to masklinn/uap-python that referenced this issue Nov 23, 2024
The goal of this is the ability to generate wheels for precompiled
instances of uap-core, at whatever version we want.

1. It resolves ua-parser#146 by splitting the versioning of the API and that of
   the (pre-compiled) data, this is an issue for 1.0 as that detaches
   uap-python's versioning from uap-core's.
2. It allows users to update the API and the precompiled dataset
   separately, something they would otherwise need to do via yaml.
3. It fixes ua-parser#221 by allowing the regular release of "preview"
   precompiled regexes from uap-core snapshots e.g. we could release
   0.19.dev202412 at the start of december with whatever uap-core
   merged between the previous prerelease and then. This should not be
   picked up by pip by default, but would allow users to access those
   prerelases via `pip install --pre`.
4. If done well enough, it might allow users to build bespoke
   precompiled datasets so they don't have to pick between custom
   rules and precompiled (not sure there's any demand for this but it
   seems like it might be useful).
5. If it works well enough it might actually be possible to have 0.x
   use the legacy codegen package meaning it should not need to be
   updated anymore.

This is implemented via hatch build hooks (which seem seem simpler
than doing it via setuptools in the end).

Adding `regexes.yaml` to the sdist via artifacts is a bit strange but
necessary in order to generate a complete sdist which a wheel can be
built from (even though the release script will likely only push the
wheel).
masklinn added a commit to masklinn/uap-python that referenced this issue Nov 23, 2024
The goal of this is the ability to generate wheels for precompiled
instances of uap-core, at whatever version we want.

1. It resolves ua-parser#146 by splitting the versioning of the API and that of
   the (pre-compiled) data, this is an issue for 1.0 as that detaches
   uap-python's versioning from uap-core's.
2. It allows users to update the API and the precompiled dataset
   separately, something they would otherwise need to do via yaml.
3. It fixes ua-parser#221 by allowing the regular release of "preview"
   precompiled regexes from uap-core snapshots e.g. we could release
   0.19.dev202412 at the start of december with whatever uap-core
   merged between the previous prerelease and then. This should not be
   picked up by pip by default, but would allow users to access those
   prerelases via `pip install --pre`.
4. If done well enough, it might allow users to build bespoke
   precompiled datasets so they don't have to pick between custom
   rules and precompiled (not sure there's any demand for this but it
   seems like it might be useful).
5. If it works well enough it might actually be possible to have 0.x
   use the legacy codegen package meaning it should not need to be
   updated anymore.

This is implemented via hatch build hooks (which seem seem simpler
than doing it via setuptools in the end).

Adding `regexes.yaml` to the sdist via artifacts is a bit strange but
necessary in order to generate a complete sdist which a wheel can be
built from (even though the release script will likely only push the
wheel).
masklinn added a commit to masklinn/uap-python that referenced this issue Nov 23, 2024
The goal of this is the ability to generate wheels for precompiled
instances of uap-core, at whatever version we want.

1. It resolves ua-parser#146 by splitting the versioning of the API and that of
   the (pre-compiled) data, this is an issue for 1.0 as that detaches
   uap-python's versioning from uap-core's.
2. It allows users to update the API and the precompiled dataset
   separately, something they would otherwise need to do via yaml.
3. It fixes ua-parser#221 by allowing the regular release of "preview"
   precompiled regexes from uap-core snapshots e.g. we could release
   0.19.dev202412 at the start of december with whatever uap-core
   merged between the previous prerelease and then. This should not be
   picked up by pip by default, but would allow users to access those
   prerelases via `pip install --pre`.
4. If done well enough, it might allow users to build bespoke
   precompiled datasets so they don't have to pick between custom
   rules and precompiled (not sure there's any demand for this but it
   seems like it might be useful).
5. If it works well enough it might actually be possible to have 0.x
   use the legacy codegen package meaning it should not need to be
   updated anymore.

This is implemented via hatch build hooks (which seem seem simpler
than doing it via setuptools in the end).

Adding `regexes.yaml` to the sdist via artifacts is a bit strange but
necessary in order to generate a complete sdist which a wheel can be
built from (even though the release script will likely only push the
wheel).
masklinn added a commit to masklinn/uap-python that referenced this issue Nov 23, 2024
The goal of this is the ability to generate wheels for precompiled
instances of uap-core, at whatever version we want.

1. It resolves ua-parser#146 by splitting the versioning of the API and that of
   the (pre-compiled) data, this is an issue for 1.0 as that detaches
   uap-python's versioning from uap-core's.
2. It allows users to update the API and the precompiled dataset
   separately, something they would otherwise need to do via yaml.
3. It fixes ua-parser#221 by allowing the regular release of "preview"
   precompiled regexes from uap-core snapshots e.g. we could release
   0.19.dev202412 at the start of december with whatever uap-core
   merged between the previous prerelease and then. This should not be
   picked up by pip by default, but would allow users to access those
   prerelases via `pip install --pre`.
4. If done well enough, it might allow users to build bespoke
   precompiled datasets so they don't have to pick between custom
   rules and precompiled (not sure there's any demand for this but it
   seems like it might be useful).
5. If it works well enough it might actually be possible to have 0.x
   use the legacy codegen package meaning it should not need to be
   updated anymore.

This is implemented via hatch build hooks (which seem seem simpler
than doing it via setuptools in the end).

Adding `regexes.yaml` to the sdist via artifacts is a bit strange but
necessary in order to generate a complete sdist which a wheel can be
built from (even though the release script will likely only push the
wheel).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant