Gitpod is not loading the workspaces

[tommygonzaleza]

Bug description

When I try to open the workspace, this is what I get:

Steps to reproduce

It happens when I open the workspace, it just never opens it

Expected behavior

No response

Example repository

No response

Anything else?

No response

[jabuwu]

I was having the same issue until just now.. so maybe it's fixed?

[jelaniwoods]

This has been happening to me too with new and old workspaces. The workspace gets stuck in the preparing stage and then fails with the error

Oh, no! Something went wrong!

Error: 4 DEADLINE_EXCEEDED: Deadline exceeded

one new workspace did open eventually, but I'm unable to open old ones or create new ones.

[mikenikles]

Hi everyone, I'm looking into this but haven't been able to reproduce it just yet.

Does anyone have a workspace ID you could share? Also, I wonder if it may be happening in the Europe region 🤔 (I connect to the US based on my location).

[jelaniwoods]

@mikenikles I'm located in the US.

Some workspace IDs are:

• cyan-kite-5pv8xjk4
• teal-lemming-njnhd1ws
• olive-vicuna-o0b22ljz

[mikenikles]

Thank you @jelaniwoods 🙏.

A colleague of mine restarted one of the underlying systems. In his words, "I see activity now".

Could you all please check and see if you're back up and running?

[jelaniwoods]

@mikenikles Yes, my workspaces now start successfully!

[smacintyre]

@mikenikles No, The workspace start but with the default image and no prebuilds.

[aledbf]

@smacintyre prebuilds are being restarted now

[mikenikles]

☝️ he's the man of the hour 😄. Thank you @aledbf for getting us back up and running 🙏!

[smacintyre]

@mikenikles and @aledbf Also ports are stuck in detecting and are unreachable when I try open anything:

[smacintyre]

@mikenikles and @aledbf Still not working, just tried again. The workspace loads but is unusable.

• The incorrect Docker image is being loaded to the workspace, ignoring what's specified in .gitpod.yml
• Prebuilds aren’t working, or even running commands from the .gitpod.yml isn't working when the workspace loads
• Ports can’t be opened – Even if you manually install the software and do your builds

[aledbf]

@smacintyre please post the name of the workspace

[smacintyre]

@aledbf This applies to every workspace I try to create new; it's 100% reproducible from what I am experincing. I just created a new one: blush-quail-venl89wf

[smacintyre]

Hi @aledbf -- any update on this? Acknowledgement or ETA? It's the start of the workday here in Asia, so I'm concerned about this blocking my team.

[mikenikles]

Hey Shawn, we see some long-running prebuilds but nothing that points to the cause of what you experience yet. Investigation is ongoing at the moment.

[smacintyre]

@mikenikles Thank you!

[mikenikles]

You're welcome - and my apologies for the unnecessary added stress to start your day!

Ports look fine from a monitoring perspective now. Do you still see them stuck?

As for the incorrect Docker image that is used, I wonder if it for whatever reason falls back to the default gitpod/workspace-full...

[smacintyre]

@aledbf and @mikenikles

Here is our .gitpod.yml, only the extensions part seems to actually be used.

image:
  file: .gitpod.Dockerfile
tasks:
  - command: yarn gitpod:start
    init: yarn gitpod:prebuild
github:
  prebuilds:
    addCheck: true
    addBadge: true
    addLabel: true
    branches: true
    pullRequests: true
    pullRequestsFromForks: true
ports:
  - port:
      port: 3000
      onOpen: ignore
  - port:
      port: 4000
      onOpen: ignore
  - port:
      port: 8000
      onOpen: open-browser
vscode:
  extensions:
    - dbaeumer.vscode-eslint
    - bradlc.vscode-tailwindcss
    - csstools.postcss
    - github.vscode-pull-request-github
    - eamodio.gitlens
    - ms-azuretools.vscode-docker
    - AmazonWebServices.aws-toolkit-vscode
    - GraphQL.vscode-graphql
    - streetsidesoftware.code-spell-checker

And .gitpod.Dockerfile

FROM gitpod/workspace-full
USER gitpod

RUN curl "https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip" -o "/tmp/awscliv2.zip" \
  && unzip /tmp/awscliv2.zip -d /tmp \
  && sudo /tmp/aws/install

RUN python3 -m pip install --user pipx \
  && python3 -m pipx ensurepath \
  && python3 -m pipx install aws-sso-util

# Install Postrgresl Client
# Install Deps of Headless Chrome
#  See: https://github.com/puppeteer/puppeteer/blob/main/docs/troubleshooting.md#chrome-headless-doesnt-launch-on-unix
RUN sudo apt update \
  && sudo apt -y install \
  postgresql-client \
  ca-certificates \
  fonts-liberation \
  libappindicator3-1 \
  libasound2 \
  libatk-bridge2.0-0 \
  libatk1.0-0 \
  libc6 \
  libcairo2 \
  libcups2 \
  libdbus-1-3 \
  libexpat1 \
  libfontconfig1 \
  libgbm1 \
  libgcc1 \
  libglib2.0-0 \
  libgtk-3-0 \
  libnspr4 \
  libnss3 \
  libpango-1.0-0 \
  libpangocairo-1.0-0 \
  libstdc++6 \
  libx11-6 \
  libx11-xcb1 \
  libxcb1 \
  libxcomposite1 \
  libxcursor1 \
  libxdamage1 \
  libxext6 \
  libxfixes3 \
  libxi6 \
  libxrandr2 \
  libxrender1 \
  libxss1 \
  libxtst6 \
  lsb-release \
  wget \
  xdg-utils

[akosyakov]

Ports section does not look valid and entire .gitpod.yml is ignored. It should be:

ports:
  - port:3000
    onOpen: ignore
  - port: 4000
    onOpen: ignore
  - port: 8000
    onOpen: open-browser

[akosyakov]

There were an issue with invalid gitpod scheme lately: #4806 so an editor is not very helpful to validate .gitpod.yml file at the moment, but you should get proper errors after next deployments

@meysholdt @csweichel how hard is to deploy a change only for dashboard proxy: #4808

[smacintyre]

Thank you @akosyakov ... that seems to be it. We use a project generator for generating our config files. A recent update appears to have broken generating the ports.

[akosyakov]

I don't think we have an incident here, closing the issue. Please open a new one if something is wrong.

[DonnavanStaddon]

You need to push