Revert PR with disputed dependencies

[roed314]

#36964 was inappropriately merged, since two dependencies were still disputed. This PR attempts to revert the merge. It was created by

git revert -m 1 6ecb1d8532bc9b59ec03a1c7f3b1ce86c6b43cdb

📝 Checklist

• The title is concise and informative.
• The description explains in detail what this PR is about.
• I have linked a relevant issue or discussion.
• I have created tests covering the changes.
• I have updated the documentation and checked the documentation preview.

⌛ Dependencies

None, though it will also revert #36676 and #36951.

[mkoeppe]

This PR should not be merged because it is obviously not an improvement.

[mkoeppe]

There was absolutely nothing inappropriate in Volker merging the PR that he merged. He's the release manager.

[roed314]

Perhaps the people who were against #36676 and #36951 would disagree; I'd like to hear from them. I'm creating this in support of the process we have in place for resolving disputed PRs, which got mangled this time (we're still working out how it needs to coordinate with dependencies).

I'm open to other suggestions for how to proceed, from the people who were objecting to the code in #36676 and #36951 that got accidentally merged in #36964. In the meantime, I'm going to sleep.

[mkoeppe]

Easy suggestion: How about doing nothing rather than creating additional damage?

[saraedum]

This PR should not be merged because it is obviously not an improvement.

I don't think this PR is about the question of improvement or not. It's entirely a procedural question. When people voted on #36964, they did not vote on the dependencies of it. After all, the missing dependencies did not have the sufficient majorities to be set to positive review yet. Also, the diff could not even be viewed on GitHub, so I am sure most people voted on the description and discussion of the PR, not on the actual change set ("The diff you're trying to view is too large. We only load the first 3000 changed files.")

There was absolutely nothing inappropriate in Volker merging the PR that he merged. He's the release manager.

I don't think it was ever claimed that @vbraun did anything wrong here. It was just a procedural misunderstanding. He simply missed the message about not merging this yet (and nobody is blaming him for that.) The (disputed) PR mechanism needs a bit of fine-tuning to take dependencies into account. But that should probably be discussed on sage-devel.

Easy suggestion: How about doing nothing rather than creating additional damage?

I believe that not merging this is harmful for the community because it ignores the votes on those dependencies. I volunteer to create PRs to restore the state of the disputed dependencies and #36964 itself.

[github-actions]

Documentation preview for this PR (built with commit f16eb4b; changes) is ready! 🎉
This preview will update shortly after each push to this PR.

[tornaria]

I verified this PR by running git revert -m 1 6ecb1d8532bc9b59ec03a1c7f3b1ce86c6b43cdb on a clean checkout of 10.4.beta3 and comparing the tree with the one in this PR.

Edit: not that I mistrust @roed314 in any way. But trust is not really transitive, a review is supposed to be independent of the author, and it makes no sense for anybody to read the diff.

[mkoeppe]

I don't think this PR is about the question of improvement or not.

That's exactly the question, and the only question that is part of our project's review procedures.

[mkoeppe]

This cannot be merged. Enough is enough.

[roed314]

I'm setting this back to positive review based on

• In favor: @roed314 @tornaria
• Against: @mkoeppe

[mkoeppe]

Unbelievable.

[jhpalmieri]

@mkoeppe you made no objection to Julian's comment "@vbraun please do not merge this one yet. It depends on two disputed PRs that have not received positive review yet." #36964 should have not been given a positive review because of its dependencies. This corrects that problem.

[mkoeppe]

That's not how it works. Our review procedures do not have special rules for PRs that "only revert a PR".

[jhpalmieri]

+1 from me.

[mkoeppe]

This PR has not even been reviewed. It cannot even be voted on.

[roed314]

@tornaria checked that it did a very simple task: revert another PR that was accidentally merged. That's a review. This is just reverting state back to what develop was yesterday.

[mkoeppe]

We don't merge PRs that are not improvements. Our review criteria are clear regarding this.

[jplab]

Hello,

So, let me see if I can summarize the situation correctly:

• It seems to me that Add # sage_setup: distribution directives to all files, remove remaining # coding: utf-8 #36964 had the required votes to be set as positive review
• Add # sage_setup: distribution directives to all files, remove remaining # coding: utf-8 #36964 got merged.
• Add # sage_setup: distribution directives to all files, remove remaining # coding: utf-8 #36964 depends on two disputed PRs
• This PR reverts Add # sage_setup: distribution directives to all files, remove remaining # coding: utf-8 #36964
• The Release Manager suggested to create such a PR to revert the changes of Add # sage_setup: distribution directives to all files, remove remaining # coding: utf-8 #36964

Before any further discussion occurs, I believe it would be a good thing that the points above shall be double-checked. Then the following things shall be clarified (my question might sound naive, but I would like clear answers):

• Can or can't a PR be set as positive review although it has disputed PRs as dependencies (I want a clarification on this comment )
• Where can I find the two dependencies that are disputed? In the PR description of Add # sage_setup: distribution directives to all files, remove remaining # coding: utf-8 #36964, I see three dependencies and they are all merged.

Before that, I do not see how a clear decision can be taken of the present PR.

[dimpase]

+1 to merge this

[mkoeppe]

@roed314 For the record, I consider it a spoiled ballot because of (1) the misleading title of the poll (https://groups.google.com/g/sage-devel/c/glI1YgArcXc/m/U0B_YF8hBAAJ); (2) the adversarial haste to revert these positively reviewed PRs; (3) the continued denial that violations of our reviewing code put these PRs into "disputed" status; (4) the failure to repair the damage inflicted by sage-devel posts that may have intimidated voters; just deleting them is insufficient.

[mkoeppe]

Merging this has also backed out #36951, which has the necessary votes as pointed out in #37796 (comment)
Who's going to fix it?

[tornaria]

Merging this has also backed out #36951, which has the necessary votes as pointed out in #37796 (comment) Who's going to fix it?

In #36951 (comment) you did not count my -1.

Please see: #36951 (comment)

In any case, if you can rebase that on current develop, we can have a look. I agree on moving to pyproject.toml features, but let's do it right. I'm sure we can figure out a way to respect the meaning of build-system requires (should be only setuptools) and have a static pyproject.toml, while getting the build-wheel requirements right.

[mkoeppe]

let's do it right. I'm sure we can figure out a way to respect the meaning of build-system requires (should be only setuptools) and have a static pyproject.toml

Once more, Gonzalo, setuptools does not support this. What you keep declaring as the "correct meaning" of PEP 517/518, even after I explained it to you, is simply not a thing in actual Python packaging practice.

[tornaria]

Here's the supported and recommended way to customize the build dependencies:
https://setuptools.pypa.io/en/latest/build_meta.html#dynamic-build-dependencies-and-other-build-meta-tweaks

[mkoeppe]

Of course, by declaring and implementing one's own build backend instead of setuptools.build_meta, anything can be implemented.

[tornaria]

Of course, by declaring and implementing one's own build backend instead of setuptools.build_meta, anything can be implemented.

It's just writing a function that returns a list of dependencies. This is not implementing a build backend, and the code is all spelled out in the link above (except for the [...] part where we would place the build requirements).

But moreover, since this would be a python function, it would also allow us to create the build requirements dynamically, which seems like a big advantage.

Currently the build requirements are determined when one runs the ./bootstrap shell script which does the following afaict:

1. use the shell script sage-get-system-packages to get the install requirements of packages from build/pkgs/$PKG_BASE/*requirements.txt.
2. write this information into m4/sage_spkg_versions.m4 and m4/sage_spkg_versions_toml.m4.
3. run m4 to substitute this information in pyproject.toml.m4 producing pyproject.toml.

But in fact, step 1 could also be done in python in get_requires_for_build_wheel(), as long as the information on sagemath-standard dependencies can be included in the sdist (this doesn't mean the information has to be duplicated in all modularized distributions: it can be a single file that gets included in every sdist, similar to how VERSION.txt is included in every sdist).

[mkoeppe]

it would also allow us to create the build requirements dynamically, which seems like a big advantage.

Also an obvious big disadvantage, namely that the dependency is no longer visible in the metadata file. It's running counter to the trend to move this kind of stuff from setup.py (= available at runtime of the build) to a static file.

[mkoeppe]

step 1 could also be done in python in get_requires_for_build_wheel(), as long as the information on sagemath-standard dependencies can be included in the sdist

Sure it could. Hard to see how that would possibly be an improvement though. The current design keeps the reference to the centralized version metadata strictly in the bootstrapping step of our monorepository, and after this we have clean source trees that do not make a reference to such things.

[tornaria]

it would also allow us to create the build requirements dynamically, which seems like a big advantage.

Also an obvious big disadvantage, namely that the dependency is no longer visible in the metadata file. It's running counter to the trend to move this kind of stuff from setup.py (= available at runtime of the build) to a static file.

That's a good point, since we would want the code to be written only once (in sage-setup) but each package have its own set of dependencies. Maybe this information can be stored in pyproject.toml (in a [sage-setup] section or something) and write get_requires_for_build_wheel() in the shim-backend code so that it gets the metadata from the file. What do you think?

[mkoeppe]

I think that reducing the dependencies for the "build_sdist" step is a micro-optimization that is not worth spending time on (or I would probably already have done it).

[saraedum]

Merging this has also backed out #36951, which has the necessary votes as pointed out in #37796 (comment) Who's going to fix it?

There are branches 36951, 36964, 36676 on my fork that cherry-pick the changes from the original PRs. I am happy to create PRs with these changes if you want me to but since the original PRs were yours, it's maybe easier if you create the PRs starting from these branches.

[mkoeppe]

Please go ahead, Julian.

[saraedum]

@mkoeppe I created #37900 to reproduce #36676. #36676 is still open though I think it was in essence merged and reverted. Probably #36676 should be closed now but I leave that to you.

[saraedum]

There are now #37900, #37901, #37902 which revert this revert.

[mkoeppe]

Thanks @saraedum.