Skip to content

Conversation

@beautifulentropy
Copy link
Member

@beautifulentropy beautifulentropy commented Aug 29, 2025

Add a periodic background job to the SFE with the following responsibilities:

  • Search for Zendesk tickets that have a status of "open" and an reviewStatus of "approved"
  • Process even numbered, odd numbered, or all Zendesk tickets depending on JSON configuration; this allows us to process tickets simultaneously in both DCs without the two SFEs having to coordinate locks
  • Re-validate fields from Zendesk tickets, perform any transformations as necessary (e.g. Account URI to ID)
  • If any ticket fields fail validation transition to Zendesk ticket to "pending" and leave a private comment concerning the issue
  • Write each approved and valid override request to the overrides table, in the enabled state
  • If any requested override is already administratively disabled, notify SFE via a private comment and transition the ticket to "pending"
  • Finally, close the ticket with a public comment that contains the relevant information

Closes #8166

@beautifulentropy beautifulentropy force-pushed the sfe-import-overrides branch 2 times, most recently from 0a40f0d to d149fd6 Compare August 29, 2025 21:05
@beautifulentropy beautifulentropy changed the title sfe: Periodically import approved rate limit overrides from Zendesk sfe: Periodically import approved rate limit override tickets Aug 29, 2025
@beautifulentropy beautifulentropy marked this pull request as ready for review August 29, 2025 21:06
@beautifulentropy beautifulentropy requested a review from a team as a code owner August 29, 2025 21:06
@beautifulentropy beautifulentropy requested review from aarongable, jprenken and jsha and removed request for jsha August 29, 2025 21:06
jprenken
jprenken previously approved these changes Aug 30, 2025
// - "even": process only tickets with even IDs
// - "odd": process only tickets with odd IDs
// If unspecified or empty, defaults to "all".
Mode string `validate:"omitempty,required_with=Interval,oneof=all even odd"`
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: Implementing this as two "M of N" settings (with the same underlying modulo bucketing logic) is pretty simple and would allow a lot more flexibility. But this all/even/odd mode is already implemented and will work fine for us, so NBD.

Copy link
Contributor

@jsha jsha left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Generally looks great! A few small tweaks, and one proposed refactoring for processTicket.

"Once the error has been corrected, change the status back to \"solved\" to retry.\n",
cause,
)
err := im.zendesk.UpdateTicketStatus(ticketID, "pending", privateBody, false)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As a possible followup: I'd love to split UpdateTicketStatus into UpdateTicketPublic and UpdateTicketPrivate, since boolean fields can make it easy to make copy-paste mistakes. Definitely not required as part of this PR.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like that idea. I had to write tests to make sure that we got the intended result and it should keyed me in to the idea that these two should be a little more separate.

@github-actions
Copy link
Contributor

github-actions bot commented Sep 3, 2025

@beautifulentropy, this PR appears to contain configuration and/or SQL schema changes. Please ensure that a corresponding deployment ticket has been filed with the new values.

@beautifulentropy beautifulentropy merged commit 3c3c365 into main Sep 4, 2025
13 checks passed
@beautifulentropy beautifulentropy deleted the sfe-import-overrides branch September 4, 2025 22:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

sfe: Add background job that regularly imports approved rate limit overrides from Zendesk

4 participants