-
-
Notifications
You must be signed in to change notification settings - Fork 12
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Use SOVERSION as migration marker? #67
Comments
We can technically use an It would be like this:
But maybe having |
That would be an obvious fix, but it'd mean we'd have to change the pinning and all feedstocks from |
Do we? Couldn't we instead have
|
Thanks for the input! TBH, I just didn't see an obvious solution, but I'd be thrilled if we find a way for that to work! So thinking through your suggestion a bit: obviously we'd have to change the current run-export, which would do So then I had been thinking that we'd then have to set Is that along the lines of what you had in mind? |
You can do the same as what |
Not sure if I can model it after that of course, and naming can be adapted/bikeshedded. Though how is it different from the If I understand the parallel correctly (which isn't 1:1 because gfortran is a compiler not a lib), you're suggesting that Assuming I haven't misunderstood something, I kinda prefer the range-idea from @xhochy so far. |
The |
Has there ever been a substantial issue with re2-migrations? From what I've seen they've all been quite smooth. What I mean is: do we need the ability to install different versions? I guess the upside would be to avoid the migrator and just bump I don't have a strong preference on this (well, except that I don't like |
No, they were all super-smooth. I'd rather use the same approach as |
To me,
Fine by me. I'll prepare that in #68. |
I see two options how to package everything but the libs: The gfortran way:
But then I don't know if the headers can never change between soversions...? 🤔 The alternative would be:
Any preferences or advice? |
Actually, gfortran still contains an unversioned I decided to model this after zlib, which really only contains the versioned libs on unix, resp the DLL without the import library on windows. |
Why would you want a co-installation? Isn't this about having a migration marker? |
☝️ I'm trying to follow along here... If we have the sover in the output name, obviously coinstallation becomes (much more easily) possible. IIUC, this would obviate the need for migrations at all, because we could just bump the pinning and let it percolate (as artifacts built against different re2 soversions would remain co-installable). If we have one migration per year (average sover bump in re2), that's already very nice - perhaps no migrations is even better? I was trying to follow the inputs here (and the best practice example from zlib), but I don't feel strongly about the setup. I saw an opportunity to cut down on the number of migrants and opened this issue, that's pretty much it. |
co-installability is not easy to enforce. Following situation is not desirable
|
Makes sense, thanks for the explanation. Do we want to explicitly suppress co-installability, or just ensure we migrate for new soversions? If the former, how do we suppress it, given the sover in the output name? Something like a run-constraint on Regarding which files should go into the lib-package and which into the metapackage, please let me know in #68 (currently following zlib model, as I said; aside from the naming of the output obviously). |
Again, please have a look at how |
I've looked at https://github.com/conda-forge/ctng-compilers-feedstock/blob/main/recipe/meta.yaml and it's not obvious. Perhaps you mean
together with
and
? Then
and
? If so, this is effectively what's implemented in #68 - please review there. If not, please explain what should be done. |
You have to look at https://github.com/conda-forge/ctng-compiler-activation-feedstock/blob/main/recipe/meta.yaml too because that's where |
OK, so that adds:
on top of what I described above. It's the same run-export as for I'm wondering if we can find a way to interact more constructively here? Either I've understood the situation (and already implemented it in #68), or I'm not getting what you want to point me to. Could you please either review #68, or explain what should be done? |
It would have been easier to tell you exactly what to do, but I didn't do that because then I'd have to explain the next time another issue comes up as well. I"m just trying to nudge you towards the solution. Hope that you don't find it condescending or anything like that.
Above
|
That presupposes that I'd be unable to learn from the explanation, which is somewhat condescending, actually. But water under the bridge, thanks for the answer. I've mentioned before that there's no 1:1 correspondence between the compiler setup (which needs an activation package) and the situation here, which is what's making it hard for me to see what you're trying to nudge me towards. So far the discussion in this thread referred only to outputs If - for example - you want me to add an unversioned Trying to build on your hint, Concretely, say I add a run-export on
So this doesn't avoid co-installation. Looking again at https://github.com/conda-forge/ctng-compiler-activation-feedstock/blob/main/recipe/meta.yaml, the run-export actually has a pin, i.e. |
Not true
It does, if you follow mamba create -n test libgfortran-ng libgfortran4 libgfortran5 |
Not needed. You can have |
OK cool. So this would translate to
How does this allow us to reduce the number of rebuilds we need for |
To recap, we want both:
Looking again at @xhochy's suggestion, this looks like a substantially more straight-forward way to achieve this:
That way, it's trivially impossible to co-install |
I don't understand this question. First of all, do you agree with the following statements?
There is. |
Not true either. Since downstream packages will be depending on only |
Yes, I agree with them, though it's not exactly obvious to me from the various run-exports and constraints, for one because it's spread across 4 outputs in two feedstocks, with no cross-references or explanations, and two because it's just not trivial to run a brain-based resolver on these constraints.
I think what tripped me up so badly was how the constraint of The penny may have finally dropped though that
That'd be pretty easy to solve by letting For me the overall situation is that there's one proposed solution that I find really hard to understand1, and there's one solution that takes far less mental gymnastics (for me) to verify as doing what it's supposed to. Footnotes
|
What solution are you talking about? All the solutions you have mentioned so far do allow co-installibility. I'm suggesting the option
It's a very simple recipe. |
You can also do - name: re2
build:
run_exports:
- {{ pin_subpackage("libre2-" ~ soversion) }}
- libre2
requirements:
host:
- {{ pin_subpackage("libre2-" ~ soversion, exact=True) }}
- {{ pin_subpackage("libre2", exact=True) }}
run:
- {{ pin_subpackage("libre2-" ~ soversion, exact=True) }}
- {{ pin_subpackage("libre2", exact=True) }}
- name: libre2
requirements:
host:
- {{ pin_subpackage("libre2-" ~ soversion, exact=True) }}
run:
- {{ pin_subpackage("libre2-" ~ soversion, exact=True) }}
run_constrained:
- re2 {{ version }}.*
- libre2 {{ version }}.*
- name: libre2-{{ soversion }}
requirements:
run_constrained:
- re2 {{ version }}.*
- libre2 {{ version }}.* if you want to separate the library from the headers, pkg-config files etc. |
The one where I said "That'd be pretty easy to solve by [adding a run-dep on
I claim that this is functionally equivalent to your proposal, which I'm including below for comparison (with the annotations I'd want to add for mere mortals like my future self).
I'm pretty confident I can prove that equivalence in terms of constraints if you don't believe me. To me the former is clearly easier to parse/resolve mentally, because it doesn't need as much indirection. That said, let's take a step back. We (@xhochy and I) started down the libgfortran road with the apparent misconception
As you explained we want to avoid co-installability. So the question arises why we want to version the outputs at all, rather than just have
If you don't think this is a better solution, please explain why (without rhetorical questions). |
|
OK, both of these make sense to me, thanks. OTOH "taking the experience of other package management software" is a bit fuzzy, so having an argument that justifies the extra complexity in the context of our setup is still a reasonable ask IMO. In any case, I'm fine with "paying" for the libre2-X setup (it's probably be good to document that approach w.r.t. library patterns in conda-forge). I've updated the implemented the implementation in #68 to use the run-export on |
If you could be specific on a few that you like that would help point our readings. I like RPM |
To this point: it would be important to know if distro's have designed their Looking at ubuntu & debian, there's only one lib per OS-version (with the exception of the testing-ground of debian-sid), so I couldn't immediately test what happens when trying to install both. fedora and arch don't make use of the soversion it seems. So I looked at the libgfortran situation, where both 4 & 5 exist for ubuntu 20.04. Spinning up a container and installing both succeeds: Installation log for
|
I'm reviving #68 now, which follows the gfortran pattern (versioned output-names; not co-installable). If anyone still has concerns, please comment on the PR. |
First version bump to take advantage of this new infrastructure: conda-forge/conda-forge-pinning-feedstock#5555 |
re2 provides a SOVERSION since google/re2@13ebb37, and it's only been updated twice since then (roughly once a year). Since it's explicitly marked as "ABI version", I presume we could use this as a reference for migrating re2, which should avoid a whole bunch of rebuilds.
However, we cannot supplant the existing version scheme of re2 (at the very least because the current SOVER 11.0.0 will always be less than the already published 2023.03.02), so it's not obvious to me what we could do without introducing another output (which however doesn't sound very appealing in terms of usage and having to rename it in all recipes)
Thoughts? @conda-forge/re2
CC @conda-forge/core
The text was updated successfully, but these errors were encountered: