[MAINTAINERS] [Action Required] Your help reviewing changes and issues in opensearch-project/OpenSearch

[reta]

Is your feature request related to a problem? Please describe

The OpenSearch core development pace has accelerated significantly, thanks to large, and growing, number of contributors from half a dozen organizations. Unfortunately, out of 30+ listed maintainers, only a bit over a half have had any engagement in the past few months in both reviewing and merging pull requests or contributing code or content to the project. We understand that everyone has jobs and responsibilities outside of the project, but kindly ask the maintainers to step up by engaging with the project.

Describe the solution you'd like

1. Take on reviewing, and merging, at least 1 pull request and 1 issue. If you don't know which ones to start with, pick the oldest open PR or issue, and help the author drive it to closure.
2. If you prefer to be tagged on pull requests or issues related to specific areas of interest, please let others know.
3. Briefly review all unread issue and pull request notifications to keep up-to-date with the project.

Related component

Other

Describe alternatives you've considered

No response

Additional context

The risk of burnouts is real

[stephen-crawford]

Thanks for creating this @reta.

I would encourage everyone to set their preferences similar to @peternied (https://github.com/opensearch-project/OpenSearch/pull/11600/files). It seems like this would help people prioritize and also make sure they got the notifications they wanted.

[dblock]

I am doing (1) and (2) 👍

[andrross]

Thanks @reta, big +1 to this.

out of 30+ listed maintainers, only a bit over a half have had any engagement in the past few months

I know it's not exactly the point of this issue, but I would also encourage maintainers who are not able to contribute to change their status to "emeritus" so that our maintainer list accurately reflects who is active.

[peternied]

[Triage - attendees 1 2 3 4 5 6 7 8]
@reta Thanks for bringing up this issue, we are curious how we would go about closing this issue, did you have thoughts?

[dblock]

I propose we change the status of maintainers that haven't contributed in X time to emeritus to begin with to get the real picture, and close this issue when we have?

[reta]

I propose we change the status of maintainers that haven't contributed in X time to emeritus to begin with to get the real picture, and close this issue when we have?

👍 , it is also stated in the RESPONSIBILITIES,md, I think we could set a bar as high as 6 months, wdyt?

Maintainer status never expires. If a maintainer becomes inactive for a time (usually several months), or a maintainer can confirm that they are no longer involved with the project for any reason, the members of the admin team may make a pull request to move them to the "Emeritus" section of the MAINTAINERS.md, remove them from CODEOWNERS, and upon merging the pull request, revoke the maintainer level access. Any past maintainer can be reinstated via another pull request, and have their permissions restored by the members of the admin team at any time upon request.

[Pallavi-AWS]

+1 on moving maintainer status to emeritus if they have not had any meaningful contribution in 4-6 month window.

[dblock]

I am fine with any timeframe 4-12 months.

[peternied]

Here is a list for interactions [1] in the past 6 months - who wants to double check the numbers and make the changes?

Maintainer ID	Interactions in Last 6 Months	Suggested Status
abbashus	1	emeritus
anasalkouz	39	maintainer
andrross	385	maintainer
reta	832	maintainer
Bukhtawar	115	maintainer
CEHENKLE	8	maintainer
dbwiddis	26	maintainer
dblock	289	maintainer
gbbafna	112	maintainer
setiah	2	emeritus
kotwanikunal	409	maintainer
mch2	140	maintainer
msfroh	216	maintainer
nknize	29	maintainer
owaiskazi19	32	maintainer
peternied	475	maintainer
adnapibar	4	emeritus
Rishikesh1159	31	maintainer
ryanbogan	3	emeritus
sachinpkale	126	maintainer
saratvemulapalli	28	maintainer
shwetathareja	126	maintainer
sohami	104	maintainer
dreamer-89	46	maintainer
tlfeng	15	maintainer
VachaShah	40	maintainer

• Python script to pull from GitHub APIs [1] https://gist.github.com/peternied/c8ac7fb4552cc36890969ca4b33e2cb0

[peternied]

who wants to double check the numbers and make the changes?

@andrross or @dblock would you be willing to take this on?

[msfroh]

who wants to double check the numbers and make the changes?

I was curious about some of the numbers (folks who AFAIK stopped contributing after they left Amazon), so I clicked through.

It looks like it's counting issues and PRs that:

1. have had any updates (by anyone) in the last 6 months and
2. have ever received comments by the given people (at any time).

It does not guarantee that the activity by a given maintainer occurred in the last 6 months. Every "true" contribution in the past 6 months by a maintainer would get counted (since it would satisfy both conditions), so the above numbers are an upper bound.

[dblock]

I think @msfroh is right, this looks a bit suspicious. I think a contribution from a maintainer has to be something like at least 1 PR they approved or requested changes on.

@peternied If you want to turn your script into something more repeatable I opened opensearch-project/project-tools#82. There's existing code that will parse MAINTAINERS.md and makes GitHub API requests easily for you.

[peternied]

I've created a PR [1] to update the maintainers based on activity for those with no activity on the project.

• [1] Update maintainers based on activity since Jan 2024 #13204

[peternied]

out of 30+ listed maintainers, only a bit over a half have had any engagement in the past few months

@reta I wanted to take a moment to circle back on to this comment with a slight tweak - It doesn't feel like there are many maintainers in the project. What do you think of this position?

Dubious methodology aside, the maintainer interaction table [1] says the majority of maintainers are active. We have only a handful of maintainers that are in the less than 10 interaction count. I think this is the project is large enough to allow maintainers to be siloed in certain areas. I do not think the issue is maintainers aren't doing what is expected - but that maintainers are not collaborating together.

This issue has 5 out of 22 maintainers commenting on it. I think we are missing some important voices - I'm going to reach out offline to pull more folks in.

• [1] [MAINTAINERS] [Action Required] Your help reviewing changes and issues in opensearch-project/OpenSearch #12970 (comment)

[CEHENKLE]

I'm primarily doing "stuff" in .github, with my eyes always on core. I'll occasionally do things like this: #12148 but you really don't want me doing code reviews :) . Do you want to restrict maintainers to just folks who are doing CRs/PRs in code? I'm honestly not offended either way, and happy to move myself to emeritus if folks would prefer that. :)

I think the biggest thing that stands out to me is that @reta is more than double the next active person, and that's not sustainable :( I'd like to see some folks on this list like @saratvemulapalli, @Bukhtawar, @shwetathareja and @VachaShah get time allocated in their schedule specifically to work on putting up CRs to help pull some load off of @reta . What do you think @Pallavi-AWS ?

[owaiskazi19]

As the project encompasses a wide scope and maintainers possess expertise in specific modules, the task of reviewing every PR becomes quite challenging.

Few options we can have here to get more involvement from maintainers:

1. I liked the approach @peternied followed here for code ownership of specific packages. We can do something similar for every maintainer.
2. When considering contributions from maintainers, it's important to acknowledge that they're often juggling multiple projects. Rather than expecting them to solely submit new PRs, we can encourage their involvement through activities like actively reviewing issues or PRs. Their continued engagement until issues are resolved or PRs are merged is greatly appreciated.
3. While we already have a common channel for maintainers in the core on slack, we could enhance specificity by creating separate channels for different areas, such as search, indexing, plugins or documentation. Maintainers could check these channels once a day to stay updated on related PRs or updates. Additionally, automated bots could post PRs in these channels based on labels.
4. Recognize and appreciate the efforts of maintainers through badges or credits on community forums (similar to stack overflow).

[VachaShah]

With the increasing pace of the project, it has been difficult (for me) to stay up-to-date with all that is going on but I am going to commit to doing (1) and find my way through (2) and (3). Thank you @reta for opening this issue!

[sohami]

It doesn't feel like there are many maintainers in the project. What do you think of this position?

I think more maintainers will definitely help and as suggested above with some policy around active contributions in last X months. IMO setting a limit on just count could be tricky as contributions varies in terms of time/effort.

[reta]

@reta I wanted to take a moment to circle back on to this comment with a slight tweak - It doesn't feel like there are many maintainers in the project. What do you think of this position?

Thanks @peternied , I don't have any proof to be fair, but it feels like in quantitative measures ("by numbers"), we have enough maintainers (more would definitely help but this is another subject).

This issue has 5 out of 22 maintainers commenting on it. I think we are missing some important voices - I'm going to reach out offline to pull more folks in.

Some areas (like remote storage fe), are having a good number of maintainers watching the pull requests and issues out, this is definitely very positive (nonetheless I think having even more eyes would help there). But for most areas lack maintainers attention by large margin.

[kotwanikunal]

Reading through all the comments and thinking through it, I think the problem is not having enough active maintainers.

I am all for changing the maintainer status based on their contributions, but that necessarily does not solve for getting more activity from other maintainers, and might even lead to the existing maintainers being further overloaded with reviews and other responsibilities.

I think we can break down the solution as follows -

• Have more maintainers
• Have more contributions from maintainers

On 2, a suggestion which might be too simplistic and optimistic - but would tagging and assigning maintainers to issues and PRs at random help here?
Let's say I raise a PR - if it has been pending on maintainers for "x" amount of time, a bot starts assigning PRs to maintainers to help move things along (maybe even as soon as the PR is created)
And this can tie in with @peternied's contribution of personalizing areas that the maintainers are interested in, so you can apply your expertise and knowledge to the best :)

Just thinking out loud here, but if folks have more suggestions on how to improve activity, this would potentially solve the problem.

[Bukhtawar]

I'm primarily doing "stuff" in .github, with my eyes always on core. I'll occasionally do things like this: #12148 but you really don't want me doing code reviews :) . Do you want to restrict maintainers to just folks who are doing CRs/PRs in code? I'm honestly not offended either way, and happy to move myself to emeritus if folks would prefer that. :)

I think the biggest thing that stands out to me is that @reta is more than double the next active person, and that's not sustainable :( I'd like to see some folks on this list like @saratvemulapalli, @Bukhtawar, @shwetathareja and @VachaShah get time allocated in their schedule specifically to work on putting up CRs to help pull some load off of @reta . What do you think @Pallavi-AWS ?

While I understand and fully acknowledge the importance of these contributions to tackle operational load, we also tend to overlook, that a maintainer is sometimes also a force multiplier where delivering significant features that shape the overall road map for OpenSearch, requires mentoring newer contributors and building the next set of maintainers. I am not sure if we've noticed but a significant number of RFCs come from newer contributors and some maintainers including myself tend to just hit the like button.

I seriously feel that simply counting PR reviews or issues might not be the perfect measure to gauge a maintainer's contribution, what also matters is the impact one's contribution has had on the project. It might undermine not just a maintainer's contribution but contributors who can be good maintainers going forward whose nomination would otherwise be shot down.

The point is, we need to honour and respect contributions in its true sense, quality vs quantity.

Happy to discuss more. Thoughts @rramachand21 @backslasht @shwetathareja

[jainankitk]

While I understand and fully acknowledge the importance of these contributions to tackle operational load, we also tend to overlook, that a maintainer is sometimes also a force multiplier where delivering significant features that shape the overall road map for OpenSearch, requires mentoring newer contributors and building the next set of maintainers. I am not sure if we've noticed but a significant number of RFCs come from newer contributors and some maintainers including myself tend to just hit the like button.

+1 to @Bukhtawar's point. Although not a maintainer myself, I have been mentoring and encouraging newer contributors like @sgup432, @bowenlan-amzn, @kaushalmahi12, @niyatiagg, even outside of OpenSearch (eg - apache/lucene#12297 (comment)). I am assuming that is one of the more important responsibility of being a maintainer, instead of worrying about their own PR/commit metric.

I seriously feel that simply counting PR reviews or issues might not be the perfect measure to gauge a maintainer's contribution, what also matters is the impact one's contribution has had on the project. It might undermine not just a maintainer's contribution but contributors who can be good maintainers going forward whose nomination would otherwise be shot down.
The point is, we need to honour and respect contributions in its true sense, quality vs quantity.

1 contribution can be 30 min or few weeks task. The people being chosen for emeritus solely on the basis of number of contributions does not look correct to me. Also, I am not sure how removing the not contributing maintainers (and spending time writing scripts for that) is helping with the original problem of burnout. Is there a limitation on the number of maintainers or this is for encouraging existing maintainers into contributing? Speaking for core search, very few maintainers have the right expertise to thoroughly review the PRs

I would encourage everyone to set their preferences similar to @peternied (https://github.com/opensearch-project/OpenSearch/pull/11600/files). It seems like this would help people prioritize and also make sure they got the notifications they wanted.

This does make some sense, but given we have tags for most issues/PRs, notification should be tag based IMO.

[shwetathareja]

+1 to @Bukhtawar and @jainankitk's point that we should look at maintainer's role holistically that they are contributing meaningfully and not just tie with few aspects or create a leader board of some sort. I don't think we all need to check all boxes but rather divide it among ourselves to be more effective as a group.

This issue was created with a positive note on how to enable maintainers to contribute more. Most of the maintainers have expertise in one or few areas. Either, we can add our preference similar to @peternied or encourage maintainers to become more proactive on checking notifications. It would be good to identify areas where we lack maintainer's attention and address them.

[Bukhtawar]

We already have component level triages/meetups, assuming the current set of component that has been identified is regularly looking at the backlog, triaging issues and maintaining operational hygiene. If there is a need to identify more components in a way that splits up responsibilities more uniformly lets do that rather than a leaderboard as @shwetathareja also called out. We are growing and will continue to add more maintainers who would eventually divide and conquer.
The one thing that will help is mapping maintainers based on their preference to these components and see which area(s) do we lack more. As maintainers we should always be open to jumping components in order to help the project grow. @peternied do we have a list of issues that don't have a defined component or not getting enough maintainer traction?

[gbbafna]

I would also like to highlight the fact that we haven't able to add more maintainers. Since June 2023, we haven't added any maintainer. Apart from more contribution from existing maintainers, we need to think as a team how we can go about increasing the same or is there any impediment we need to remove to solve this. I do see lot of meaningful contributions (RFC, PR and reviews all around) as well as mentoring as well from many folks. Also, to add to @Bukhtawar 's point, we will need more help (more maintainers or help from existing ones) on components where we lack experts .

[Bukhtawar]

I would also like to highlight the fact that we haven't able to add more maintainers. Since June 2023, we haven't added any maintainer. Apart from more contribution from existing maintainers, we need to think as a team how we can go about increasing the same or is there any impediment we need to remove to solve this. I do see lot of meaningful contributions (RFC, PR and reviews all around) as well as mentoring as well from many folks. Also, to add to @Bukhtawar 's point, we will need more help (more maintainers or help from existing ones) on components where we lack experts .

I see opportunities where we can streamline the nomination process(I find it to be too open-ended that could undermine great contributions), define guidances on what "meaningful" and "impactful" contribution should look like(maybe quantify impact, making it more data driven), recognise contributors for their unique skills(rather that trying to check all boxes), re-define the veto process to ensure we respect everyones decision and avoiding proximity, recency biases that could unintentionally creep in. I am looking at opening a PR to refine the nomination process and provide clear guidances on how to calibrate contributions. Would love to hear feedback on the same

[rramachand21]

Agree @Bukhtawar - a standard criteria for becoming a maintainer would definitely help

[reta]

Do you want to restrict maintainers to just folks who are doing CRs/PRs in code? I'm honestly not offended either way, and happy to move myself to emeritus if folks would prefer that. :)

Thanks @CEHENKLE , by no means there was / is intent to restrict that, just siding with other folks - maintainer's role is multifold and every single contribution matters. Please continue helping with these areas, those as important as everything else, thank you.

[reta]

Folks, I think the discussions on this issue went off the subject (Your help reviewing changes and issues in opensearch-project/OpenSearch), which is very luckily the indicator of the complexity of the problem space.

The only and single goal of this targeted call for action was (and still is) to ask what could be done (if anything) so existing maintainers could be more engaged with the project, specifically with respect of reviewing changes (pull requests) and issues.

To finish up on the positive note, I think this issue has gotten quite an attention from the folks during last week, and hopefully we could keep the momentum and direct the engagements towards project operational needs.

[Bukhtawar]

Folks, I think the discussions on this issue went off the subject (Your help reviewing changes and issues in opensearch-project/OpenSearch), which is very luckily the indicator of the complexity of the problem space.

The only and single goal of this targeted call for action was (and still is) to ask what could be done (if anything) so existing maintainers could be more engaged with the project, specifically with respect of reviewing changes (pull requests) and issues.

To finish up on the positive note, I think this issue has gotten quite an attention from the folks during last week, and hopefully we could keep the momentum and direct the engagements towards project operational needs.

Thanks @reta. I think this discussion pops up in multiple forums so it's best we take actionable AIs from this else we might soon end up in a similar discussion elsewhere. The uber point is how can we better divide up the work and I think we had some good suggestions coming up as well which is worth capturing or following up on

[dblock]

The only and single goal of this targeted call for action was (and still is) to ask what could be done (if anything) so existing maintainers could be more engaged with the project, specifically with respect of reviewing changes (pull requests) and issues.

I can think of two things.

1. A maintainer that writes code is more engaged than one that does not. We can encourage maintainers to think backwards from the core project when they work in their respective businesses. I try to do it in many forums at Amazon, always asking "what code will this project contribute to OpenSearch core", or "is this implementation easier/faster/cheaper to do in the open source data plane vs. our proprietary control plane"?

2. A maintainer that reads every PR/issue/comment will inevitably engage on some. I think every maintainer here that wants to but doesn't have time to engage should go work with their manager at their paid job to explicitly say that being a maintainer of OpenSearch core is critical for their business, must become part of their job description, and requires a % of time allocated to that in explicit ways, and prioritized above other things that their colleagues can do.

PS: I should plug the fact that I am going to give a talk on "becoming a maintainer" at OpenSearchCon EU in May in Berlin and plan to source ideas from this thread!

[Bukhtawar]

A maintainer that writes code is more engaged than one that does not

I think it's hard to generalise this, for instance a maintainer can spend 20% time coding, 80% reviewing and mentoring other contributors. This strategy has generally worked better for atleast storage areas where we have been able to scale better by adding more contributors/maintainers and lining others for nominations. We were able to deliver significant features in the last year and have set a decent roadmap ahead. Thats why I feel maintainers acting as force-multipliers are critical from a scaling perspective. A single maintainer writing code most of the time can only do so much.

A maintainer that reads every PR/issue/comment will inevitably engage on some

There is absolutely no need to read every issue and spreading too thin. This is precisely why we have the components assigned and specific triaging meet-ups set up. Maintainers should rather focus on one or more core areas as needed to whilst ensuring all the operational demands are met. This helps foster a deep sense of ownership
The decision to commit to X% bandwidth should be done based on a careful analysis of the component areas, again for instance storage doesn't need any such commitments. And I would love to see how the work is distributed across components so that we can divide areas better and shuffle maintainer

[dblock]

There is absolutely no need to read every issue and spreading too thin.

@Bukhtawar I think you'll recognize that if you don't read every issue/PR, you can't possibly feel the pain across the board :) I review/comment/merge on so many contributions every day that it feels like a full time job, precisely because not enough people do that. If I didn't, a ton would fall through the cracks. I think we absolutely need both depth and breadth maintainers, and @reta's sentiment above is very much about this I believe.

[owaiskazi19]

And I would love to see how the work is distributed across components so that we can divide areas better

We could organize distinct teams of maintainers for various core functionalities. Tagging specific teams, such as opensearch-maintainers/indexing, to handle PRs or issues pertinent to their domain, or when a corresponding label is applied, could serve as a viable solution.

[peternied]

The weekly triage meeting [1] is a great way to increase breath for an once a week hour-long commitment. Anyone (+future-maintainers) let me know if you'd like to be on the calendar invite.

• [1] https://github.com/opensearch-project/OpenSearch/blob/main/TRIAGING.md

[rramachand21]

In addition to the above, the component triage meetings are also great way to increase breadth and depth - Any future maintainers and existing maintainers, please do join these as well - for example, next Storage Triage meeting is happening on 18th April - https://www.meetup.com/opensearch/events/299461631/

[Bukhtawar]

So In the light of component and general triage meetups do we really require maintainers to read up all issues and PR?? @dblock do you think we can give it a try?

[dblock]

So In the light of component and general triage meetups do we really require maintainers to read up all issues and PR?? @dblock do you think we can give it a try?

We don't today. All I am saying is that @reta and I do it, and I would say a solid 50% of issues and PRs would stay unattended, or be significantly delayed (many days without a response) if we didn't. @reta LMK if you feel the same, I'd be happy to be proven otherwise!

[reta]

@reta LMK if you feel the same, I'd be happy to be proven otherwise!

👍 totally with you here, thanks @dblock !

[anasalkouz]

I think we should focus on increasing number of contributions (e.g. reviews, comments on issues ...etc) rather than increasing number of maintainers. Penalizing non-active maintainers and moving them out of the list won't help solving the issue. In addition, you don't need to be a maintainer to make contributions into the project.

I propose a shift in our approach. Let's implement a system that acknowledges and incentivizes contributions from all members of our community. One suggestion is to compile a list of top contributors on a monthly or quarterly basis, akin to a "Contributor of the Month" accolade. This initiative would not only spotlight the efforts of dedicated individuals but also foster a sense of appreciation within our community. Celebrating these achievements collectively can serve as a powerful motivator for continued engagement and growth of contributions.

[saratvemulapalli]

Thanks @reta for this. I haven't been active lately but I'd more involved and interested in (2) in all things plugins.
Please tag me on PRs/Issues surrounding plugins.

[dbwiddis]

The only and single goal of this targeted call for action was (and still is) to ask what could be done (if anything) so existing maintainers could be more engaged with the project, specifically with respect of reviewing changes (pull requests) and issues.

There is simply too much noise and not enough use of tags to help bring attention to what needs attention.

• There are 1,539 open issues. Sorting features are useless: newest/most commented/recently updated often have engagement, oldest/least commented/least recently updated often are abandoned.
• A snapshot of the first page (25 issue) includes 7 flaky tests (135 of the 1539 issues have this tag) with no clear prioritization or indication of how much impact they have (one-off? failing every other test?)
• We have a single "untriaged" label but since there's a weekly meeting to look at those, there seems to be a reluctance to touch that tag and/or wait until it's been categorized before using the component labels to better organize/search.
• We have no means to prioritize requests. Everything goes in the same bucket. There's no way for individual issue submitters to call more attention to it without individually tagging people. For example, I submitted issue [BUG] The RestController always consumes content when handling real HTTP requests #13011 which I know exactly how to fix and will be happy to submit a PR, all I want is input from maintainers on picking option A or B. It's been 2 weeks. I guess I can just pick my preference and go with it, but there's not really any established guidance for how long to wait for feedback before assuming lack thereof is okay, or maybe it's not?

What would help me get more engaged:

1. Automate triaging of Flaky Test issues

• the issue could be regularly (weekly? monthly?) updated or re-created using GitHub API to find the open issues and correlates them to the number of failures in the last N jenkins runs similar to this but linked to actual open issue numbers.
• include some sort of automation that executes a test using the "REPRODUCE WITH" output of the gradle test. If the failure is always reproducible, add a specific label like "reproducible" as these are most easily debugged/fixed

2. Have a posted categorization of issue labels that we can assume someone else is tracking more closely and can personally deprioritize.

• For example, if I see "Indexing:Replication" or "Storage:Remote" those seem well-maintained.
• Which labels aren't? What labels can I exclude in an Issue Tracker bookmark that will save me time finding those overlooked diamonds in the rough?

3. Add a label that issue submitters can use to revive attention to their issues that scroll off the front page if they're simply waiting on a "go for it!" response to their query, like my example. Basically like the @here user in a Slack channel.