Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[prebuild] Not starting workspace once the prebuild is done #8195

Closed
geropl opened this issue Feb 14, 2022 · 10 comments · Fixed by #10357 or #10646
Closed

[prebuild] Not starting workspace once the prebuild is done #8195

geropl opened this issue Feb 14, 2022 · 10 comments · Fixed by #10357 or #10646
Assignees

Comments

@geropl
Copy link
Member

geropl commented Feb 14, 2022

Instead, user is left handing with prebuild log, reading "exit" at the end. Reload starts the workspace properly.

Screenshot:
image

Idea: Might be triggered by us fixing the status updates recently.

/cc @jldec not the issue I had in mind during our meeting, but annoying nonetheless

@jldec
Copy link
Contributor

jldec commented Feb 15, 2022

@geropl can you clarify how to to repro this?

What do you have to do, to see a prebuild log (I assume in the dashboard) when you are expecting to start a workspace?

@geropl
Copy link
Member Author

geropl commented Feb 15, 2022

can you clarify how to to repro this?

@jldec This happened to me a couple of times last days (1 week) now on the gitpod-io/gitpod repo. I started a workspace, and was show the "Prebuild in progress" view with logs flowing in. I left the tab as this usually takes a couples of minutes.
When I came back the prebuild had clearly finished (noticable because of the last line saying "exit"). Now, because we don't show the actual status of that prebuild, I'm not sure if a) the prebuild was completed, or b) just the execution was finished, but it was still uploading the result.
But when I hit "F5", the workspace started immediately based on this (another?) prebuild.

@geropl geropl moved this to Scheduled in 🍎 WebApp Team Apr 26, 2022
@geropl geropl added the type: bug Something isn't working label Apr 26, 2022
@jankeromnes
Copy link
Contributor

jankeromnes commented Apr 26, 2022

Could it be that we don't properly handle websocket reconnections? (They are quite likely on long-running pages that you might leave open in the background.)

This is just a random guess, but I believe we might need something like a notifyDidOpenConnection listener to handle reconnections.

For example, this is how the ImageBuildView seems to handle websocket reconnections when you're starting a workspace and watching image build logs (i.e. it calls watchBuild again):

const toDispose = getGitpodService().registerClient({
notifyDidOpenConnection: () => {
registered = false; // new connection, we're not registered anymore
watchBuild();
},
onWorkspaceImageBuildLogs: (
info: WorkspaceImageBuild.StateInfo,
content?: WorkspaceImageBuild.LogContent,
) => {
if (!content) {
return;
}
logsEmitter.emit("logs", content.text);
},
});

The StartWorkspace.tsx page used to do something similar (calling watchHeadlessWorkspace again on reconnections):

    const toDispose = service.registerClient({
      notifyDidOpenConnection: () => watchHeadlessWorkspace(),
      onHeadlessWorkspaceLogs(event: HeadlessLogEvent): void {
        logsEmitter.emit('logs', event.text);
      },
    });

but the notifyDidOpenConnection call got removed in the refactor to watchHeadlessLogs: 71cc5eb#diff-1782bf62bd01382ce9f0a5f65a2d8ad7f2cc1b7c87f8d119dd3c818f91019266L349

I'm not entirely sure, but is it possible that the "new" watchHeadlessLogs might break on websocket reconnections?

@geropl
Copy link
Member Author

geropl commented Apr 29, 2022

Could it be that we don't properly handle websocket reconnections?

Exactly my suspicion, yes. Will try to fix. 🙃

Update: After looking through the code and checking blame I think it's sth else. The expectation was that the outer code (e.g., StartWorkspace) switches screens, but that seems to have gone missing. Now looking into re-using PrebuildLogs.

@AlexTugarev
Copy link
Member

I was just hitting the same issue. I forgot to take a screenshot, but I verified that the onPrebuildUpdated (available) event was received on 2nd or 3rd websocket connection and not on the initially created one.

@geropl geropl moved this from In Progress to Scheduled in 🍎 WebApp Team May 17, 2022
@geropl
Copy link
Member Author

geropl commented May 17, 2022

This did not see much work due to installer, and I'm on sick leave, so moving back to "scheduled".

@geropl geropl removed their assignment May 17, 2022
@josepjaume
Copy link

Hi! Sorry to chime in - this is a major blocker for us at Factorial, and probably the first source of frustration to transition to Gitpod. It gives people a sense of "unstability". In big repos with lots of activity, having to start over again makes people nervous 🙃. People end up understanding that they need to refresh at the end of the process, but that's far from ideal (especially given that from that moment on, everything works flawlessly).

That said, I love the product and I think you're doing an amazing job. Take care @geropl, and sorry to bother you while on sick leave!

@AlexTugarev AlexTugarev self-assigned this May 30, 2022
@AlexTugarev AlexTugarev moved this from Scheduled to In Progress in 🍎 WebApp Team May 30, 2022
@geropl geropl mentioned this issue May 31, 2022
11 tasks
@geropl
Copy link
Member Author

geropl commented May 31, 2022

@AlexTugarev Was just able to reprodue this on staging. 👍

@AlexTugarev
Copy link
Member

@geropl, I think I could reproduce it myself.

@AlexTugarev
Copy link
Member

Reopening this after realizing that the PR only solves it for one user – the webhook installer. For other users, no instance updates will be sent, thus the process breaks.

@AlexTugarev AlexTugarev reopened this Jun 9, 2022
Repository owner moved this from Done to In Progress in 🍎 WebApp Team Jun 9, 2022
@geropl geropl moved this from In Progress to Done in 🍎 WebApp Team Jun 15, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment