Gitpod unusable: Very large load averages

[smacintyre]

Bug description

Workspace performance is unstable right now because very high load averages. Cluster: ws-us38

gitpod /workspace/nxity $ uptime
 08:55:29 up 1 day, 10:51,  0 users,  load average: 84.76, 67.62, 68.40

Can't even save reliably

Steps to reproduce

Open any repo

Workspace affected

all

Expected behavior

From past experience, I assume someone is doing crypto mining. There was a promise made towards the end of last year that paying customers would be isolated from free customers. Has this happened?

Example repository

No response

Anything else?

No response

[smacintyre]

The uptimes have come down and Gitpod is usable again. It should good to have some response from Gitpod. Is no one monitoring?

[geropl]

@smacintyre Thx for the confirmation. We're monitoring, but especially during night shifts rely on automation to detect abuse.
Every now and then there is a new specimen we have to adjust our detection on, though. 😕
As it was the case here. Purged all clusters, should be fine now.

[smacintyre]

@geropl My complaint here is this happened right at the start of the day. Not at night. My team is in crunch time right now and lost a big chunk of the morning because of this. If Gitpod is to be taken seriously, you need to respond to downtime when it happens. Not 10 hours later.