Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(notification): add postgres queue #136

Merged
merged 1 commit into from
Oct 21, 2022
Merged

feat(notification): add postgres queue #136

merged 1 commit into from
Oct 21, 2022

Conversation

mabdh
Copy link
Member

@mabdh mabdh commented Oct 13, 2022

Changes

  • Add postgres queue
  • Add dlq concept
    • with postgres, we don't have a dedicated dlq table for now
      • enqueuing to dlq means updating the status to failed
      • dequeueing dlq means republishing the failed messages that has not expired
    • failing message could be marked as retryable or non-retryable, if client got 429 rate limit or 500, it is retriable. If there is wrong config or client got 4xx, it is non-retriable.
  • Add new cli
    • workers (workers mean another instance that run beside the server)
      • siren worker start notification_handler
      • siren worker start notification_dlq_handler
    • jobs (a task that runs once)
      • siren job run cleanup_queue - cleaning up messages that are published or stale in pending that already > specific threshold (default 7 days)
  • Introduce a MaskableString
    • When logging secret/personal credentials, string will be masked with asterix (*)
  • Add Pre and Post hook in receiver notification flow

@coveralls
Copy link

coveralls commented Oct 13, 2022

Pull Request Test Coverage Report for Build 3296791340

  • 263 of 422 (62.32%) changed or added relevant lines in 21 files are covered.
  • 2 unchanged lines in 1 file lost coverage.
  • Overall coverage decreased (-2.5%) to 81.161%

Changes Missing Coverage Covered Lines Changed/Added Lines %
internal/store/postgres/subscription.go 7 9 77.78%
core/notification/service.go 21 24 87.5%
plugins/queues/postgresq/model.go 39 43 90.7%
plugins/receivers/slack/notification_service.go 29 33 87.88%
pkg/secret/secret.go 0 5 0.0%
core/notification/handler_option.go 0 6 0.0%
plugins/receivers/httpreceiver/notification_service.go 0 6 0.0%
plugins/receivers/pagerduty/notification_service.go 0 6 0.0%
plugins/queues/inmemory/queue.go 0 7 0.0%
core/notification/handler.go 22 46 47.83%
Files with Coverage Reduction New Missed Lines %
core/notification/handler_option.go 2 0%
Totals Coverage Status
Change from base Build 3224101768: -2.5%
Covered Lines: 2951
Relevant Lines: 3636

💛 - Coveralls

@ravisuhag ravisuhag linked an issue Oct 13, 2022 that may be closed by this pull request
@mabdh mabdh marked this pull request as ready for review October 17, 2022 03:42
@mabdh mabdh requested review from rahmatrhd and ravisuhag October 17, 2022 03:42
@ravisuhag ravisuhag requested a review from spy16 October 19, 2022 23:38
@mabdh mabdh mentioned this pull request Oct 21, 2022
8 tasks
@ravisuhag ravisuhag merged commit c8dc654 into main Oct 21, 2022
@ravisuhag ravisuhag deleted the postgres-queue branch October 21, 2022 13:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Implement Postgres Queue between notification dispatcher and handler
3 participants