Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement publish notification emails #9341

Merged
merged 3 commits into from
Aug 28, 2024

Conversation

Turbo87
Copy link
Member

@Turbo87 Turbo87 commented Aug 26, 2024

I started out rebasing #2705, then updated it to the current code structure, and moved all of the email sending to a background job.

In short: when a new crate version is published, a corresponding send_publish_notifications background job containing the version_id will be queued. that background job will query the relevant publish information (crate name, version, publish time, and publisher name) from the database, will get the email addresses of all current owners of the crate, and then send publish notifications to each of these owners. If sending all notification fails we assume a network issue and retry the job. If only a subset fails we assume something else is wrong and won't retry.

Most `created_at` rows in the database default to `now()`, which can cause `insta` snapshots to be unstable when the content changes between runs due to the nature of time itself. This change adjusts the `emails_snapshot()` fn to replace the ISO8601 dates with a placeholder, which should cause the snapshots to become stable.
@Turbo87 Turbo87 added C-enhancement ✨ Category: Adding new behavior or a change to the way an existing feature works A-backend ⚙️ labels Aug 26, 2024
@Turbo87 Turbo87 force-pushed the publish-notifications branch from baf687d to 4c00c3c Compare August 26, 2024 16:13
@Turbo87 Turbo87 requested a review from a team August 26, 2024 16:17
Copy link
Contributor

@LawnGnome LawnGnome left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No concerns on the code, but one question about whether teams are being accounted for here.

I predict the next feature request we get will be the option to turn this off. 😆

// Find names and email addresses of all crate owners
let recipients = crate_owners::table
.filter(crate_owners::deleted.eq(false))
.filter(crate_owners::owner_kind.eq(OwnerKind::User))
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we also be looking up team owners?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we should, I'm just not sure that we can... :D

our publishing code currently performs a "get all owner teams, iterate through teams and query GitHub API to see if publisher is member of the team" operation, but that does not list the members of these teams. in other words: we'd need new GitHub API calls since we don't "cache" the team memberships on our side. if/when we have non-GitHub teams implemented this will probably become a lot easier.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ugh, I forgot that we don't cache that. Sorry, some Sourcegraph slipped through there.

Not blocking for this PR, but I'm wondering if it's worth the effort to figure that1 out sooner rather than later — given that part of the motivation here (at least from what I see) is supply chain security, not sending notifications for crates that are entirely owned by teams (such as the AWS crate ecosystem) feels problematic.

I'll give this some thought.

Footnotes

  1. For clarity, I'm more thinking just some level of caching here; not so much a full blown decoupled-from-GitHub team design.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm wondering if it's worth the effort to figure that out sooner rather than later

it then becomes a problem of cache invalidation. I think I'd personally rather see a proper teams concept in crates.io with a feature to optionally keep the team memberships in sync with some GitHub team. Pietro mentioned a while ago that this sync might be possible by having people install a GitHub app (?) in their orgs, but I haven't looked at any of this in detail yet.

@Turbo87
Copy link
Member Author

Turbo87 commented Aug 26, 2024

I predict the next feature request we get will be the option to turn this off. 😆

sure, but when you look at other package registries, they also don't seem to allow you to turn this off. if you really wanted to ignore these notifications it should be easy to implement a corresponding filter in your email server/client.

@LawnGnome
Copy link
Contributor

sure, but when you look at other package registries, they also don't seem to allow you to turn this off. if you really wanted to ignore these notifications it should be easy to implement a corresponding filter in your email server/client.

Oh, I don't actually think we should provide the option. I just expect someone to ask. 😅

@Turbo87 Turbo87 merged commit 69d19c4 into rust-lang:main Aug 28, 2024
10 checks passed
@Turbo87 Turbo87 deleted the publish-notifications branch August 28, 2024 09:54
Copy link

codecov bot commented Aug 28, 2024

Codecov Report

Attention: Patch coverage is 97.19626% with 3 lines in your changes missing coverage. Please review.

Project coverage is 89.21%. Comparing base (ee48d63) to head (4c00c3c).
Report is 14 commits behind head on main.

Files Patch % Lines
src/worker/jobs/send_publish_notifications.rs 96.70% 3 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #9341      +/-   ##
==========================================
+ Coverage   89.19%   89.21%   +0.01%     
==========================================
  Files         282      283       +1     
  Lines       28964    29071     +107     
==========================================
+ Hits        25834    25935     +101     
- Misses       3130     3136       +6     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Turbo87 added a commit to Turbo87/crates.io that referenced this pull request Aug 29, 2024
These were caused by a conflict between rust-lang#9342 and rust-lang#9341. Should've rebased first... 🙈
@Turbo87 Turbo87 mentioned this pull request Aug 29, 2024
Turbo87 added a commit that referenced this pull request Aug 29, 2024
These were caused by a conflict between #9342 and #9341. Should've rebased first... 🙈
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-backend ⚙️ C-enhancement ✨ Category: Adding new behavior or a change to the way an existing feature works
Projects
Archived in project
Development

Successfully merging this pull request may close these issues.

2 participants