remove myself from the repository

[ericLemanissier]

Conan-center-index has very regrettably become less and less welcoming to contributors in the passed months. Among other things:

• pull requests are being locked, in the absence of code of conduct violation ([rapidjson] fix for gcc14 compile #23662 protobuf: add v4.25.3, v5.26.0 #21622 [config] Re-order Mac M1 configuration #23594)
• useful tools (1 as of today) are being removed, without any sort of warning to the people relying on them 0ff926d (prepare for more tools removal: Maintenance: remove outdated Conan 2 linter #23757)
• pain points reported repeatedly are not being addressed ([service] Why PRs review is so slow? #17921 [service] Pull Request Request: bump ubuntu images to bionic #13472)

[conan-center-bot]

Conan v1 pipeline ❌

Sorry, the build is only launched for Access Request users. You can request access writing in this issue.

Conan v2 pipeline ❌

Note: Conan v2 builds are now mandatory. Please read our discussion about it.

The v2 pipeline failed. Please, review the errors and note this is required for pull requests to be merged. In case this recipe is still not ported to Conan 2.x, please, ping @conan-io/barbarians on the PR and we will help you.

See details:

Sorry, the build is only launched for Access Request users. You can request access writing in this issue.

[jcar87]

Hi @ericLemanissier , we respect your decision and allow me to extend the thanks from the team for your long # contributions to the project.

Since its inception, Conan Center has grown significantly. These days the remote receives request at a rate of 180 million per month, and this is also reflected in the pull request traffic. This puts a strain on our resources as maintainers, both at a human level and an infrastructure level.
This has been such that we have reorganised the way the team works, making sure that everybody in the wider team has the right knowledge to review pull requests, make a decision, and also since we’re all involved in the development of the Conan client, that we “close the loop” and identify improvements in the client that Conan Center can also benefit from.

less and less welcoming

When it comes to the contribution process, this is feedback that we have been receiving, continually, for the past two years - not only in the repository itself, not only in private, but also in person in conferences. There has been very consistent feedback that the contribution process to Conan center and the automations around it are noisy and confusing, and we continually notice users confused about who are the maintainers of the repository. Currently, PRs get messages from automations from up to 5 different bots, some of which are not maintained by us. Some are pull requests comments, some are GitHub actions that make code annotations, in some the information is duplicated (I suspect we have two automations that track recipes that are “ready to review”). Similar feedback was provided to us by new members of the team that have joined in the past year: that it is not clear what is mandatory, and who amongst the reviewers are decision makers.

This led us to review the contribution process of similar repositories - and indeed we agree as a team that the contribution process is not welcoming to new and casual contributors.
On top of this, we are also contending with the aging infrastructure behind Conan Center - the pipeline has outgrown the demand, and it is currently too slow (or rather, slower than what the compute capabilities allow), and difficult to maintain.

0ff926d

This tends to contribute to the noise and in the vast majority of cases, and in most cases the mentioned users never interact. There is also no process to identify which users are or are not actively contributing, and this has caused misleading CI alerts in the past (failed workflows send emails to the contributor, IIRC). This has confused users, since it alerts, in the PR, users that are not affiliated with the project, sending a rather confusing message about ownership. The team has also felt “less empowered” to make decisions - despite the fact that they are the maintainers of the repository. I personally think it's important the team feels empowered to make decisions, even if they are the wrong ones. Mistakes can happen, but the team will learn from them to provide a better experience to our users. If widely use recipes present problems, we will be the ones that are contacted, and we will be the ones to respond (under pressure). We have not seen examples of other repositories operating this way. We are very happy for contributors to keep an eye out for recipes they are interested in, I suspect the automation can be repurposed and run on a separate GitHub repository. What we ask is that it does not leave comments in the pull request. If the notified users want to comment on the PR themselves, they are welcome to.

https://github.com/conan-io/conan-center-index/pull/23757

The “v2” linter is an unfortunate piece of code. Our colleague that maintained it has since left the team, it recently this has been posting flat out wrong diagnostics on pull requests - confusing our contributors, and hindering our job as reviewers. When the linter was introduced, I remember explicitly that this was done with the understanding that it was limited in time (While CI was not checking recipes for v2), and that everything was shown as warnings that were optional (because v2 was not mandatory back then). It has since transpired that despite having an expiration date, a more generic python linter was also added as part of the functionality. For what it is worth, we have been monitoring the failures - and the true positive rates of issues that are otherwise not caught by the proper CI run - is actually quite low. So in the short term we have determined that it is a relatively low risk to remove, until we repurpose the useful bits in the new redesigned pipeline. I am grateful for your PR to remove the v2 aspect of it and keep the generic linter, but we don’t currently have the resources to maintain this piece of code - this also means reviewing any contributions to the .github folder.

pain points reported repeatedly are not being addressed ([service] Why PRs review is so slow? #17921

I wholeheartedly sympathize with users and contributors who find that their pull requests and reported issues are not being resolved quickly enough. After all, they are using their own time and efforts to contribute to the project. On the one hand, as projects grow larger in user base, so do the number of requests, reaching a point where maintainers are unable to resolve them as quickly as they come. This has made us analyze how we work and how we can improve this.
We have already applied some measures to mitigate the need for too many PRs in the first place. For example:

• We have been extending the use of version ranges in more recipes where feasible, this has a triple effect: reduce version conflicts experienced by users, reduce the need to open PRs to fix said version conflicts, and, to a lesser extent, reduce the change of new PRs experiencing “missing binaries” for the configurations we support.
• We have disabled automation of “indiscriminate” version bumps. They are still performed, but we try to ensure that the guiding factor is reducing version conflicts experienced by our users, while updating to new versions when it is reasonable to do so. We have an internal CI job checking the repository every night for version conflicts for a “sample” project of common dependencies, so that we can get alerted and fix this before our users experience this. And have a couple proof of concepts to automate this to the point where the repo should always be as close to self-consistent as it can feasibly be.

On the other hand, the volume is such that we need to prioritize - and we believe it makes sense to align ourselves with the demands of the users of Conan Center, which is a much larger group than both maintainers and community contributors. Oftentimes receive PRs with unclear motivations, or where it is unclear if it benefits users (consumers of Conan Center packages via the remote, or the repository). Please also keep in mind that some users are enterprise users, running their own infrastructure, operating with their own forks of Conan Center - that reach out to us directly. So we have a broader visibility to help us understand the pain points of our users.
We have a triple task: to continue adapting our infrastructure and resources to cope with an ever-growing user base and the associated demand, to find solutions to reduce the amount of pull requests that need to be created in the first place - and do these two things while we continue reviewing PRs and ensuring that they are good to be merged: we are part of the software supply chain - and users place a level of trust on us. Our user base and the support matrix are large enough that any missteps can cause widespread issues - whereas in the past this would not have been the case with a smaller user base.
This last point is also why we have held off on some proposed PRs until there is more confidence and more validation, or why some PRs are being closed.

pull requests are being locked, in the absence of code of conduct violation ([rapidjson] fix for gcc14 compile #23662 protobuf: add v4.25.3, v5.26.0 #21622 [config] Re-order Mac M1 configuration #23594)

This is also part of our efforts to focus on what matters. For #23662, the user already had a locally served fork of the recipe, as well as publicly available versions in Conan Center that had the fix they needed. There was nothing else to discuss - and locking discussions is a useful mechanism to avoid unfruitful discussions. For #23662, the discussion around the test package has happened in 3 different pull requests if I’m not mistaken - we are asking that if there are actual issues, that these are reported and discussed in a separate issue. Worth mentioning that we have added more testing to the protobuf test package as a result of these discussions. We do have plans to reinstate or have the ability to include more complex tests, which will be rolled out in the coming months For #23594 - we had a look at what was going on under the hood and determined that the proposed changes had no effect. It’s an internal discussion as far as I’m concerned, and I apologize that the PR description led you (or anyone) to believe that it would improve the issue.

As for the point of adapting our infrastructure and resources, this is what we have been working on in the past few months - the following is the changelog for our new CI features that are ready to be rolled out in the next quarter, and most of the bullet points are already working in the current implementation:

• New contributors are no longer required to request access in a separate PR before their PR can run on CI. Conan maintainers will be immediately notified via a dedicated channel, and can start a CI run after inspecting the changeset. There’s no need for additional wait or requests of any kind.
• Improved CI build times
  • bottlenecks have been identified and mitigated in order to provide a faster service.
  • Better use of available compute resources
  • We expect the majority of PRs to report results in 30 minutes or less when there is agent availability, and in less than 4 hours for all cases in busier periods (or for larger projects, such as Qt). Figures might change as we perform wider testing.
• Improved reliability - the causes for many sporadic failures have been identified and addressed - some examples:
  Significantly reduced instances of “git clone” against conan-center-index (at a large scale, intermittent github failures were noticed)
  • Fixed issue where the pipeline would erroneously report that the PR is touching multiple recipes
  • Fixed issue where the pipeline would fail if too much time had passed between the first recipe checks and the first package build, due to agent availability.
  • Increased Jenkins reliability due to queue management (see below)
• CI runs are no longer re-started when the pull request description is updated.
• New unified queue for CI runs:
  • Unified, always-on: any request submitted while the system is under maintenance will preserve its place in the build queue, and once the service is resumed, will be built in order of arrival
  • Queue shaping: the rate at which jobs are started can be slowed down based on how busy the CI system is
  • Priority queue: maintainers can prioritize certain pull requests during busy periods - for example, to address issues that affect many users, or to shorten the pull request review feedback loop, with the goal of merging a PR they are actively reviewing.
• Ability to process pull requests that modify multiple recipes. These can be opened by anyone, but will require approval by maintainers before running on CI. A single PR could result in building several hundreds (even thousands!) of binary packages, in the order of hundreds of hours, so this will be allowed very sparingly.
• Improved ability to add support for new platforms. The team can now easily generate all binaries for a new configuration before making it a requirement on CI.
  • Soon after the rollout, we will be adding support for newer versions of gcc, clang and apple-clang.
• Centralized pull request checks progress and result reporting:
  • Build results are reported as github checks
  • Results are published and updated as the build progresses, and not only when it finishes. Build logs can be immediately inspected for a specific package_id as soon as CI builds it.
  • All checks are reported as github checks (no longer in PR comments)
  • It will be clear to contributors and reviewers which checks are optional and which are a blocker for the PR.
  • At a later date: Reformatted the way in which results from hooks are presented - making it easier for contributors and reviewers to address changes
• As the repository and the user base grows, so does the amount of PRs that the Conan team needs to review - and we are focused in shortening the CI feedback loop, granting the following abilities to Conan team maintainers:
  • Maintainers can be directly notified (via a dedicated channel) once CI results are available for a PR they have just reviewed, or that they have assigned. The aim is for them to go back to it as soon as possible, within the same day.
  • When troubleshooting an issue that is only affecting a single platform, ability to trigger a manual build isolated to the platform at question. Once fixed, the default set of platforms will be built. This ensures quicker results, and better use of resources.
• Fixes to the test_package workflow:
  • Correct testing when the tested reference is a tool_requires
  • Complex test packages that require multiple package IDs that are built as part of the same run, the test stage can be deferred until all packages are generated (e.g. protobuf)

Our focus is to make sure that the widest possible user base benefits from what we offer in Conan Center, and that pull requests are prioritized and reviewed as quickly as possible, where resources allow. Additionally, we want to make sure the PRs we prioritize are aligned with that goal, and that contributors have a better and more friendly interaction with all the automations in the CI system.

Thank you for your continued contributions, and the Conan Center community will be here should you decide to come back.

[Croydon]

You have been an incredible contributor to Bincrafters and CCI, and in general with all of your recipes, tooling and ideas for the Conan ecosystem. I hope that this is a "see you later" instead of a goodbye @ericLemanissier ❤️

---

I also feel somewhat alienated by some decisions in the last months. I think that part of this situation is the shift from having discussions on GitHub in the open, where everyone can contribute to finding a solution, towards more and more team internal meetings where only the finalized decisions are getting presented (the team decided that...). I understand that this happens to some degree naturally as the size of paid employees grow, but maybe CCI has to decide how much community involvement is wanted vs. "we make the rules and you can only contribute recipes following those rules".

[paulharris]

I've been away from the conan world for the last few months so I feel like I've missed a lot of the recent struggles. I'm not a full time tooling guy, so I drift back and forth between different development modes.

I've personally burned a lot of my time tooling up with conan v1 and upgrading to conan v2, but I kept going with the belief that It Would All Work Out In The End.

I would've burned a lot more time and gotten completely stuck without @ericLemanissier's help and efforts along the way. JFrog has effectively had a valuable unpaid employee in Eric for a long time now, and conan will be worse off without him.

I see his comments and I subscribe to track PRs and issues, I branch and merge the patches that are still pending (for long periods of time). I don't send a lot of messages of support and "this affects me too" because I didn't want to spam everyone, but perhaps I should've to make it known what problems were blockers for more people than just one guy.

The scale is increasing, and we are going to need more people with the skills and experience in fixing the difficult recipe problems. Eric (and other high-volume contributors) are like a canary in a coal mine. If he is happy, chances are many other package users and maintainers are happy (but we won't hear from them as there are no problems).
We can't make them happy immediately, the problems are too complex, but perhaps conan can aim some resources at specifically working with difficult recipes and high-volume contributors. If you reduce the friction, you could unlock a lot more value out of freely provided resources like Eric.

[paulharris]

PRs get messages from automations from up to 5 different bots

Well I found some of those bots handy.
I especially like the I detected other pull requests that are modifying xxxx bot.
The organisation of pending PRs is difficult in github (or anywhere), I've often discovered other PRs (both historical and current) through those links and it has helped me understand the recipe and the other pending problems.

The fact that the bot was written should tell the team that we need something like this for people like me.
Perhaps there are bots that need to be changed, but please don't throw the baby out with the bathwater.

[memsharded]

Hi @Croydon @paulharris and all,

I could discuss a lot of things in this thread, but I’ll try to be short. The recent controversial items have been:

• Deactivate automatic version bumps. We had tons of feedback and evidence of how these were hurting the ConanCenter users. When you have to connect every 2 weeks in zoom calls with upset Fortune 100 companies teams because ConanCenter broke their builds that were nicely working for the Nth time in the last 2 months, you realize that something is not right. We finally had to take the decision not to keep merging them almost automatically. It has been a few weeks and so far it has proved quite effective. Remember, we have not “disallowed” version bumps - we are just asking that contributors help us prioritize them by expressing their motivation. The team is now trying to avoid them by moving to version ranges, checking for potential widespread conflicts, and by not building them automatically CI has better resources to handle PRs with a higher priority.
• Simplified test-packages. Package recipes were not affected, the package build was still identical, but we had to remove some of the complexity of the “test_package” to allow the CI pipeline to unblock some challenging cases. It meant a bit less coverage, but no significant risk.
• Removed bots: We are modernizing the CI pipeline for Conan 2 builds. The “this recipe has been modified by these other PRs” is already implemented in our new pipeline. We are ready to deploy this bit of functionality as early as next week. We will also update the PR template to point users to see a list of PRs broken down by recipe they touch - this may even avoid unnecessary PRs in the first place! Note that for some of the automations that may be useful to some contributors, we have received very consistent feedback from tons of potential contributors that were overwhelmed and so confused by what happened when they opened a PR that immediately abandoned their efforts. Modernizing the CI pipeline and using recommended practices such as using Github checks instead of PR comments is a current large effort.

Everytime we have had to take one of these decisions, even when they were discussed in public, we have got just really strong push back. This also diminishes our work as maintainers. There have been instances where we have been told that we should just only approve and merge PRs if they are green, neglecting not only the expertise and way broader perspective of the team, but also the responsibility that the maintainers have with the wider community of Conan users.

To summarize, I’d like to reiterate my acknowledgment, say a very big thank you to @ericLemanissier for the amazing contributions that you have made. I fully understand that you are not feeling aligned with the direction that ConanCenter needs to move, and you do not plan to contribute any further. Even if you don’t believe me, we have done our best, a humongous effort to try to serve you as well as possible, so while I am happy how we have joined efforts this time, I am really sad to see and listen to the hate and resentment that I appreciate in these comments and actions, I think it is not deserved, and I had hoped that we could part ways in a more friendly manner. I really wish you all the best.

[paulharris]

I see your work @memsharded, and commend your efforts. Like I said, I've been away for a while and I can see there are a lot of positive improvements in that time (conan, the command itself, is working a lot better for me today than it did before).

On the 3 points you made:

• Yes I can understand wanting to avoid automerging. It is a frustration of mine that something changes and the build breaks when it was working not long ago, for whatever reasons.
• Simpler test-packages are fine, but keep in mind that someone went through the effort of building that test because something broke, and it would be good to test that it keeps working in the future. Perhaps we can have more complex "integration tests" that recipe authors can write. If those can be run offline, or on one separate CI machine, that would probably help catch a bunch more problems, even if those problems are detected after the merge (but hopefully before users trip over them). This would be off the critical path of recipe maintenance, but serve as a way for consumers to test things without being the test subjects themselves.
  ** For example, big heavy recipes like QT and the future VTK have a lot of options. Some of the more specialised options shouldn't be enabled by default, ie for CCI, and some options can only be enabled exclusive to other options. But there are people who use those options, and it would be comforting to know there is a test somewhere that is building and checking in a little more depth that those options are built and checked. It is the off-the-beaten-track problems that will make conan more reliable and less fragile.
• I'm happy to see new bots that will do an even better job than the old bots.

[ericLemanissier]

Thank you all for the kind words and reactions. It's very much appreciated 🙏 . And thanks to the team for taking the time to write so detailed answers.

I am really sad to see and listen to the hate and resentment that I appreciate in these comments and actions

@memsharded There is absolutely no hate in me. I'll be explicit about the other side of the coin, hopefully sounding less childish and ungrateful than in my first message: Conan is a great project, and I know you (the team) are all putting in a tremendous amount of effort to make it live and improve it, both on conan and conan-center-index side. For Conan, new features like conan graph explain, runners, local recipes index repositories etc. are huge improvements which are very nice to use. Also for conan, I did not contribute much, but each time I did it has been a breath of fresh air, with very fast feedback. It's a polar opposite to CCI. My understanding is that the very long/never-ending time for reviews and the very long time needed to evolve configurations (#13472) are caused by a lack of human resources, and I know(from private slack conversations) you are all working very hard to do a major overhaul of the CI. I already wrote it, and it felt hilarious/disconnected from reality to some of you, but I'll reiterate because I still believe it: from where I stand the CI has been doing its job well for many months. it has already come a long way since the dark days of unexpected failures, and no CI results being posted (and it's thanks to your hard work).

There have been instances where we have been told that we should just only approve and merge PRs if they are green, neglecting not only the expertise and way broader perspective of the team, but also the responsibility that the maintainers have with the wider community of Conan users.

To be clear, I do not agree with all what was said by the op on issue 17921 : CCI needs more reviews, not less. They are absolutely fundamental for the quality of recipes. I reacted to the suppression of automatic merge of bumps because it means the reviewers have even less time to review other pull requests.

pull requests are being locked, in the absence of code of conduct violation ([rapidjson] fix for gcc14 compile #23662 protobuf: add v4.25.3, v5.26.0 #21622 [config] Re-order Mac M1 configuration #23594)

This is also part of our efforts to focus on what matters. For #23662, the user already had a locally served fork of the recipe, as well as publicly available versions in Conan Center that had the fix they needed. There was nothing else to discuss - and locking discussions is a useful mechanism to avoid unfruitful discussions

@jcar87 This is a very strong and clear signal you are sending: the team can and will (and did) prevent technical discussions, without warning, even in the absence of spam, troll, disrespect, off-topic etc. I may be alone on this, but this is sufficient to drive me away from contributing for long. Code of conduct goes both ways.

By the way, to everybody: feel free to reuse code from my closed PRs or archived repositories. If it's written (and if it's useful), it might as well be used.

[Ahajha]

I'll also chime in, I haven't been super heavily involved as some of the other people here (thank you everyone who contributes!), but I've seen enough to form a few opinions:

• Regarding version bumps, I see both sides of the argument. Not auto-merging these is kinda silly from one perspective, but I understand not wanting to break things. I definitely agree with the move to use more version ranges - TBH I think ranges should be the default in most cases, but I understand the need for caution (see xz_utils).
• I'm all for reviews, but as has been frequently stated, the review times tend to be a problem. Me saying this doesn't magically fix the issue, but just consider me a +1 vote to the others who have said this. I have noticed recently at least a slow but steady decline recently of open PRs, as well as more involvement from the team, both of which I'm taking to mean that this is in progress, as hopefully this just means you're working on getting to a state where you can review things more quickly in the future.
• Closing and locking PRs in my eyes is not a good move, especially the recent ones where the discussion was civil. It very much has felt like the team is sort of "taking over" - making decisions internally even when the community clearly disagrees. I understand this is literally your job, but when that community puts in so much work I feel like their voices deserve more merit. It's also one thing to disagree with the community, another to stop the discussion entirely.

Please note as with others, this comes from me wanting to see CCI thrive - I enjoy working with Conan and the people who work on and use it, but these are certainly things that should be addressed.

[planetmarshall]

I have to echo the comments on long delays in PR reviews having an impact on the engagement of contributors. #22997 has now been green for months with no further progress. I can maintain recipes for my own use in my own repositories but I'm becoming increasingly disillusioned with making further contributions to CCI which is a shame.

[valgur]

Ditto for #24577 and related PRs that only get any attention only begrudgingly and semi-accidentally, it seems, despite the proposed solutions being polished and tested across tens of packages in my private fork (valgur/conan-center-index) for several months by now.

Comments like

We don't have any issues reported in Conan for openmp support, and in Conan Center we've never had requests or issues reported by users of the libraries, all openmp related discussions seem to be motivated by recipe maintainers. I should remind that our focus should always be the users - and that our efforts and priorities are guided by demand from the users of recipes. We don't want to run the risk of diverting resources for something that may not actually be used.

that imply that the few active CCI contributors somehow operate in a frivolous vacuum and don't actually count as real users (which is definitely not the case, I'm just not going to go into detail about the non-public uses of my contributions unnecessarily) is borderline insulting and really really does not help.

I definitely understand that there's a finite number of hours available for the maintainers and there's a need to heavily prioritize the work. A honest timeframe for features that you don't consider urgent or revenue-adjacent would be sufficient and more helpful.

I'm feeling quite close to revoking my CLA altogether, but I'm willing to give it another month or three.

[jwillikers]

I really appreciate the contributions of all of the community contributors, who work really hard to make Conan better. Like, @valgur, I've done work to improve / add packages for my company's use case. Sometimes that work has simply stagnated over time while other times has been shutdown with no real recourse other than perhaps opening an issue that seems unlikely to go anywhere at all. It's been pretty disheartening to see some of the best community contributors stop contributing due to these similar kinds of frustrations.