Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Prebuilds failing with "failed OTS download" #8595

Closed
Tracked by #7812
geropl opened this issue Mar 4, 2022 · 1 comment
Closed
Tracked by #7812

Prebuilds failing with "failed OTS download" #8595

geropl opened this issue Mar 4, 2022 · 1 comment
Labels
team: webapp Issue belongs to the WebApp team type: bug Something isn't working

Comments

@geropl
Copy link
Member

geropl commented Mar 4, 2022

Context: #8096

We see a lot of workspaces (prebuilds only?) failing to retrieve the Git one-time-secret (OTS) when trying to run the initializer. Affected workspaces can be identified in the DB with:

SELECT *
FROM d_b_prebuild_workspace AS pws
WHERE pws.error LIKE "%failed OTS download%"
LIMIT 50;

It's possible that there are more affected, who's state is later overwritten by #8096

@geropl geropl added type: bug Something isn't working team: webapp Issue belongs to the WebApp team labels Mar 4, 2022
@geropl geropl moved this to Scheduled in 🍎 WebApp Team Mar 4, 2022
@geropl geropl added the priority: highest (user impact) Directly user impacting label Mar 4, 2022
@jldec jldec removed the priority: highest (user impact) Directly user impacting label Mar 17, 2022
@geropl
Copy link
Member Author

geropl commented Apr 5, 2022

This issues is superseeded by two other issues:

  • workspaces in general have trouble to either a) download the OTS or b) use the materialized key (both symptoms of the same root cause, tracked in Accessing server API at gitpod.io from within workspaces might fail sometimes #8703 )
  • sometimes prebuilds hang for too long (<30mins) in the workspace cluster, resulting in start failures like these ☝️ While this is bad, both workspace and webapp teams have that our their radars, so this does not warrant a new issues.

@geropl geropl closed this as completed Apr 5, 2022
Repository owner moved this from Scheduled to Done in 🍎 WebApp Team Apr 5, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
team: webapp Issue belongs to the WebApp team type: bug Something isn't working
Projects
Archived in project
Development

No branches or pull requests

2 participants