Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Re ingest missing packages #536

Closed
iliapolo opened this issue Nov 11, 2021 · 4 comments · Fixed by #1515
Closed

Re ingest missing packages #536

iliapolo opened this issue Nov 11, 2021 · 4 comments · Fixed by #1515
Labels
backlog effort/medium About 3 days enhancement New feature or request p1 risk/medium

Comments

@iliapolo
Copy link
Contributor

For various reasons, it might be the case that messages were lost from the pipeline before properly ingested.

Right now, if a package was discovered and staged, but was lost before it was ingested, we have no way to recover that package apart from doing a complete backfill from npm.

This is because the re-process workflow only operates on packages that were ingested.

We should have a way to detect packages that were discovered, but never ingested, and re-ingest them.
@iliapolo iliapolo self-assigned this Nov 11, 2021
@iliapolo iliapolo added effort/medium About 3 days p1 risk/medium enhancement New feature or request labels Nov 11, 2021
@Chriscbr
Copy link
Contributor

How does a package get lost after staging but before it is was ingested? Shouldn't such a package end up in the Staging DLQ?

@Chriscbr
Copy link
Contributor

@iliapolo are you still working on this issue?

@iliapolo
Copy link
Contributor Author

Nope. Never really started...unassigning

@iliapolo iliapolo removed their assignment Dec 21, 2021
@github-actions
Copy link
Contributor

This issue is now marked as stale because it hasn't seen activity for a while. Add a comment or it will be closed soon. If you wish to exclude this issue from being marked as stale, add the "backlog" label.

@github-actions github-actions bot added the stale label Feb 20, 2022
@mrgrain mrgrain closed this as not planned Won't fix, can't repro, duplicate, stale Aug 26, 2024
@mrgrain mrgrain reopened this Aug 26, 2024
@mrgrain mrgrain changed the title Identify missing packages Re ingest missing packages Oct 22, 2024
@mrgrain mrgrain mentioned this issue Oct 22, 2024
Merged
github-merge-queue bot pushed a commit that referenced this issue Oct 22, 2024
@mrgrain
feat: lambda function to manually re-stage a package version (#1515)
c9aeaa2 
Resolves #536

Adds a new lambda function intended for manual invocation. This function
will re-attempt to stage a specific package version. Documented usage in
the runbook.

I need to upgrade the node types to node18 for this, which required some
additional type only changes w.r.t the SDKv3 usage.

----

*By submitting this pull request, I confirm that my contribution is made
under the terms of the Apache-2.0 license*
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
backlog effort/medium About 3 days enhancement New feature or request p1 risk/medium
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

feat: lambda function to manually re-stage a package version cdklabs/construct-hub
4 participants
@mrgrain @iliapolo @Chriscbr @gabewomble