BUG: team plan pr comment shows pro pr format

[codecovdesign]

Problem to solve

When testing / auditing the team plan, we identified the PR comment and it showed the full pro version:

additionally, the check appears:

example: https://github.com/drazisil-org/mco-server/pull/6
related org/repo: https://app.codecov.io/plan/gh/drazisil-org

Solution

Show intended PR comment: patch only w/ out details and no project check

[codecovdesign]

@adrian-codecov just a note, @drazisil-codecov set this org to team plan in the DB when we were testing. Mentioning in case there is some other loose end in how we configed for testing for why it showed the pro plan version 🤔

[adrian-codecov]

@codecovdesign, have you seen a customer in the wild that's properly team plan showcase this behavior? Just want to make sure it's not what you're saying, a manual DB check rather than a widespread problem, so just want to confirm before diving

[codecovdesign]

@adrian-codecov I'll go double check the customer threads for screenshots; but in this case it was only the DB change. Is there a way we can recreate it from the start stripe purchase, then PR - this is recreate exact path?

EDIT: i found this one: codecov/feedback#54 (comment) but the issue is there we were experimenting at that time and it was unrelated to the team plan. Will keep thinking if there is another screenshot from somewhere 🤔

[codecovdesign]

from bug sync @drazisil-codecov @adrian-codecov :

• what happens when project is added to codecov.yaml + potentially changing default yaml to omit project
• @drazisil-codecov will help troubleshoot

[drazisil-codecov]

@aj-codecov and @codecovdesign Can you link me to where the changes were implemented? I can't any plans checks in notify at all.

[adrian-codecov]

Gentle ping to see how I proceed w/ this ticket, or if it was a false positive while we were experimenting, @drazisil-codecov @codecovdesign (leaving AJ out since he's out)

[drazisil-codecov]

I don't see any logic that changes the pr comment based on plan. Could you confirm the comment is correct if on team plan?

…

On Mon, Mar 11, 2024, 3:51 PM Adrian ***@***.***> wrote: Gentle ping to see how I proceed w/ this ticket, or if it was a false positive while we were experimenting, @drazisil-codecov <https://github.com/drazisil-codecov> @codecovdesign <https://github.com/codecovdesign> (leaving AJ out since he's out) — Reply to this email directly, view it on GitHub <#1303 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AQ7YB52CD4XPIBPPQ5TWSUTYXYKOZAVCNFSM6AAAAABD6QJMMGVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSOBZGMYTENBXGY> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[codecovdesign]

@adrian-codecov we should proceed with investigation, it doesn't appear to be a false positive, but it would be ideal to confirm either way. per our bug sync , @drazisil-codecov mentioned she'd look for where we are changing the file and sounds like couldn't find.

[adrian-codecov]

I found this, https://github.com/codecov/worker/blob/355325bb19f15d7bcfcfc023c59a6cf0c9fe8730/services/notification/notifiers/mixins/message/__init__.py#L75-L76, which determines the type of comment we select. If layout_name == "newfiles" or layout_name == "condensed_files", then we return the comment that does patch_only metrics, https://github.com/codecov/worker/blob/355325bb19f15d7bcfcfc023c59a6cf0c9fe8730/services/notification/notifiers/mixins/message/sections.py#L446-L447. It didn't seem that this fn took plan type into account, do you know if we implemented this for the team plan back in the day? Trying to determine if this is a bug or if this is something that we never did

[codecovdesign]

@adrian-codecov the intention was that it was to be implemented; I'm guessing it's a bug more than a missed issue, only since we did change it prior to team rollout, but that was unrelated to team and more experimental.

[adrian-codecov]

Gotchaa, ok that makes more sense, wanted to clarify the scope of the ticket, thanks!

[codecovdesign]

sync with @adrian-codecov

• (pro / team) the latest intended team plan PR comment (recent update per joey and ongoing epic)
• @adrian-codecov appears team plan comment is missing, looks like it'll need to be implemented from scratch
• this issue has some relationship with: Discovery: Make indirect changes and project changes check opt-in #1058, in terms of changing and/or gating yaml strategy based on intended use or plan

[adrian-codecov]

@codecovdesign this change is in, lmk if you see this misbehaving!

[drazisil-codecov]

👋 A customer reported that 4931b3c9835633bc88510b64a5a82c3385eb6021 had this behavior on 29 April, @adrian-codecov

[adrian-codecov]

Sweet! Checked and this is a users-teamm customer so all tracks, thanks for confirming @drazisil-codecov

[drazisil-codecov]

Sweet! Checked and this is a users-teamm customer so all tracks, thanks for confirming @drazisil-codecov

Sorry, I meant they got a project status check. This is a case where it was explicitly set in the codeov.yml, but it's my understanding it should not on team plan, right?

[adrian-codecov]

Project status check != PR comment, see this ticket for fixing project status #1660, that should be addressed there. This issue was to fix what we displayed on the PR comment if they were/weren't team plan