[code browser] measure all sessions vs errored sessions #12702

filiptronicek · 2022-09-06T16:23:25Z

Description

This PR advances on the changes made in #12222 and contributes a new metric field: gitpod_vscode_web_load_total, which will help in discovering how often (percentage-wise) errors happen in VS Code Browser, as described in #12107 (comment).

Related Issue(s)

Relates #12107

Changes done in the VS Code Browser workbench: gitpod-io/openvscode-server#428.

The query would look something like this:

sum(rate(gitpod_vscode_web_load_total{status='failed'}[2m]))/sum(rate(gitpod_vscode_web_load_total{status='loading'}[2m]))

How to test

Do a little bit of trolling (throw errors in VS Code Browser)

Open the preview environment of this PR
Select VS Code Browser (Insiders) as your IDE
Open a workspace within the preview environment

Verify with Grafana

Start a new workspace with this PR
Run ./dev/preview/portforward-monitoring-satellite.sh -c harvester
Open port 3000

Release Notes

NONE

Documentation

Werft options:

/werft with-preview

components/supervisor/frontend/src/index.ts

filiptronicek · 2022-09-06T20:20:17Z

/werft run

👍 started the job as gitpod-build-filiptronicek-vscode-in-browser-12107.2
(with .werft/ from main)

andreafalzetti · 2022-09-06T20:32:07Z

I am following the how-to-test steps, but on no. 3 I am confused on what should I see? Grafana?

At the moment, I get an empty page on :3000 🤔

My steps:

Open gitpod.io for this PR
Run ./dev/preview/portforward-monitoring-satellite.sh -c harvester
Open port 3000

And I get this:

What am I missing?

filiptronicek · 2022-09-06T20:44:23Z

Thanks for looking at the PR, @andreafalzetti! Not quite sure what is the problem here. I assumed Grafana would be accessible the same way it was on #12222. @mustard-mh did you do any special config to get it to work 🤔?

mustard-mh · 2022-09-07T03:45:04Z

@andreafalzetti @filiptronicek You need to visit Prometheus, which forwarded port is 9090

components/supervisor/frontend/src/ide/ide-metrics-service-client.ts

filiptronicek · 2022-09-08T17:06:16Z

components/supervisor/frontend/src/ide/ide-metrics-service-client.ts

@@ -8,7 +8,8 @@ import { serverUrl } from '../shared/urls';

 export enum MetricsName {
    SupervisorFrontendClientTotal = "gitpod_supervisor_frontend_client_total",
-    SupervisorFrontendErrorTotal = "gitpod_supervisor_frontend_error_total"
+    SupervisorFrontendErrorTotal = "gitpod_supervisor_frontend_error_total",
+    SupervisorFrontendLoadTotal = "gitpod_vscode_web_load_total",


We introduced a new metric: gitpod_vscode_web_load_total for measuring the percentage of failed loads.

gitpod/install/installer/pkg/components/ide-metrics/configmap.go

Lines 50 to 58 in bc47002

Name: "gitpod_vscode_web_load_total",

Help: "Total count of supervisor frontend client loading / errored windows",

Labels: []config.LabelAllowList{

{

Name: "status",

AllowValues: []string{"loading", "failed"},

},

},

},

this metric can have two different types: loading and error.

supervisor-frontend should be reverted, we should try to decouple metrics in VS Code Web workbench and here, and work on them separately. I think this PR should not have any changes in supervisor. cc @mustard-mh

filiptronicek · 2022-09-08T17:22:43Z

/werft run with-clean-slate

👍 started the job as gitpod-build-filiptronicek-vscode-in-browser-12107.8
(with .werft/ from main)

mustard-mh · 2022-09-15T07:36:31Z

Tested with block workbench/workbench.web.main.js

Before Block	After Block

And metrics data report works

mustard-mh · 2022-09-15T07:39:01Z

I will revert debug commit and approve this PR and openvscode-server one gitpod-io/openvscode-server#428. cc @akosyakov

install/installer/cmd/testdata/render/aws-setup/output.golden

akosyakov · 2022-09-15T12:46:09Z

@mustard-mh Can we add a new row to https://grafana.gitpod.io/d/oLzOteZ4z/vs-code-browser-overview?orgId=1 for VS Code Web Workbench with tables for failure ratio?

mustard-mh · 2022-09-16T05:33:11Z

Could you help test this PR? @iQQBot

iQQBot · 2022-09-16T05:38:40Z

components/ide/code/leeway.Dockerfile

@@ -71,7 +71,8 @@ RUN yarn --cwd extensions compile \
 # Check pkg/blobserve/blobserve.go, `inlineVars` method
 RUN cp /vscode-web/out/vs/gitpod/browser/workbench/workbench.html /vscode-web/index.html \
    && cp /vscode-web/out/vs/gitpod/browser/workbench/callback.html /vscode-web/callback.html \
-    && sed -i -e 's#static/##g' /vscode-web/index.html
+    && sed -i -e 's#static/##g' /vscode-web/index.html \
+    && sed -i -e "s/{{VERSION}}/$CODE_COMMIT/g" /vscode-web/index.html


Could we include CODE_QUALITY here? like stable-commitHash or insider-commitHash ?

mustard-mh · 2022-09-16T05:44:21Z

@mustard-mh Can we add a new row to https://grafana.gitpod.io/d/oLzOteZ4z/vs-code-browser-overview?orgId=1 for VS Code Web Workbench with tables for failure ratio?

Will do it in follow PR

Move remainder of IDE metrics to under OpenVSCode Server workbench Co-authored-by: Anton Kosyakov <anton@gitpod.io> Co-authored-by: mustard <mhqnwt@gmail.com>

iQQBot

Check it and it works like a charm 🎉

/hold
need remove debug commit

iQQBot · 2022-09-16T06:16:04Z

Next step I think we can

add instanceId and userId in window.gitpod object, then workbench page can use it.
modify loader.js to add onerror handler so that we can observe AMD lazy load script load error.

iQQBot · 2022-09-16T06:46:12Z

/unhold

mustard-mh · 2022-09-16T06:51:20Z

Thank you @iQQBot

akosyakov · 2022-09-16T08:03:37Z

components/supervisor/frontend/src/ide/ide-metrics-service-client.ts

@@ -10,7 +10,8 @@ const commit = require('../../config.json').commit;

 export enum MetricsName {
    SupervisorFrontendClientTotal = "gitpod_supervisor_frontend_client_total",
-    SupervisorFrontendErrorTotal = "gitpod_supervisor_frontend_error_total"
+    SupervisorFrontendErrorTotal = "gitpod_supervisor_frontend_error_total",
+    SupervisorFrontendLoadTotal = "gitpod_vscode_web_load_total",


@mustard-mh I don't think supervisor frontend should care about it.

filiptronicek requested a review from a team September 6, 2022 16:23

roboquat added release-note-none size/S labels Sep 6, 2022

github-actions bot added the team: IDE label Sep 6, 2022

filiptronicek changed the title ~~[code browser]: Measure all sessions vs errored sessions~~ [code browser] measure all sessions vs errored sessions Sep 6, 2022

filiptronicek mentioned this pull request Sep 6, 2022

[gp-code] measure all sessions vs errored sessions gitpod-io/openvscode-server#428

Merged

filiptronicek force-pushed the filiptronicek/vscode-in-browser-12107 branch from 061a53c to 5864861 Compare September 6, 2022 16:35

filiptronicek commented Sep 6, 2022

View reviewed changes

components/supervisor/frontend/src/index.ts Outdated Show resolved Hide resolved

filiptronicek commented Sep 6, 2022

View reviewed changes

components/supervisor/frontend/src/index.ts Outdated Show resolved Hide resolved

filiptronicek self-assigned this Sep 6, 2022

filiptronicek added the editor: code (browser) label Sep 6, 2022

filiptronicek marked this pull request as draft September 6, 2022 20:44

roboquat added the do-not-merge/work-in-progress label Sep 6, 2022

akosyakov reviewed Sep 7, 2022

View reviewed changes

components/supervisor/frontend/src/ide/ide-metrics-service-client.ts Outdated Show resolved Hide resolved

filiptronicek force-pushed the filiptronicek/vscode-in-browser-12107 branch from 5864861 to 5cbe872 Compare September 8, 2022 16:31

roboquat added size/M and removed size/S labels Sep 8, 2022

filiptronicek force-pushed the filiptronicek/vscode-in-browser-12107 branch from 5cbe872 to 7ee1dd9 Compare September 8, 2022 16:50

roboquat added size/L and removed size/M labels Sep 8, 2022

filiptronicek force-pushed the filiptronicek/vscode-in-browser-12107 branch from 7ee1dd9 to f806270 Compare September 8, 2022 17:05

filiptronicek commented Sep 8, 2022

View reviewed changes

filiptronicek force-pushed the filiptronicek/vscode-in-browser-12107 branch 2 times, most recently from 1c325b9 to bc47002 Compare September 8, 2022 17:08

filiptronicek force-pushed the filiptronicek/vscode-in-browser-12107 branch from bc47002 to 7fae1c5 Compare September 11, 2022 10:58

mustard-mh force-pushed the filiptronicek/vscode-in-browser-12107 branch from d7afb80 to df74f6b Compare September 14, 2022 18:07

mustard-mh marked this pull request as ready for review September 15, 2022 07:36

roboquat removed the do-not-merge/work-in-progress label Sep 15, 2022

akosyakov reviewed Sep 15, 2022

View reviewed changes

install/installer/cmd/testdata/render/aws-setup/output.golden Outdated Show resolved Hide resolved

mustard-mh force-pushed the filiptronicek/vscode-in-browser-12107 branch from df74f6b to f791252 Compare September 15, 2022 15:18

roboquat added size/S and removed size/L labels Sep 15, 2022

iQQBot reviewed Sep 16, 2022

View reviewed changes

mustard-mh force-pushed the filiptronicek/vscode-in-browser-12107 branch from ad1cb4f to 79171ad Compare September 16, 2022 05:53

filiptronicek and others added 3 commits September 16, 2022 05:55

[code browser]: Measure all sessions vs errored sessions

f0d8d97

Move remainder of IDE metrics to under OpenVSCode Server workbench Co-authored-by: Anton Kosyakov <anton@gitpod.io> Co-authored-by: mustard <mhqnwt@gmail.com>

[installer] add vscode-workbench to allowlist

bf23f39

[code] replace version during build time

a11cdfb

mustard-mh force-pushed the filiptronicek/vscode-in-browser-12107 branch from 79171ad to 92cb119 Compare September 16, 2022 05:55

iQQBot approved these changes Sep 16, 2022

View reviewed changes

roboquat added the do-not-merge/hold label Sep 16, 2022

mustard-mh force-pushed the filiptronicek/vscode-in-browser-12107 branch from 92cb119 to a11cdfb Compare September 16, 2022 06:13

roboquat removed the do-not-merge/hold label Sep 16, 2022

roboquat merged commit ac16572 into main Sep 16, 2022

roboquat deleted the filiptronicek/vscode-in-browser-12107 branch September 16, 2022 06:46

akosyakov reviewed Sep 16, 2022

View reviewed changes

roboquat added deployed: IDE IDE change is running in production deployed Change is completely running in production labels Sep 16, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[code browser] measure all sessions vs errored sessions #12702

[code browser] measure all sessions vs errored sessions #12702

filiptronicek commented Sep 6, 2022 •

edited by iQQBot

Loading

filiptronicek commented Sep 6, 2022 •

edited by werft-gitpod-dev-com bot

Loading

andreafalzetti commented Sep 6, 2022 •

edited by werft-gitpod-dev-com bot

Loading

filiptronicek commented Sep 6, 2022 •

edited by werft-gitpod-dev-com bot

Loading

mustard-mh commented Sep 7, 2022

filiptronicek Sep 8, 2022

filiptronicek Sep 9, 2022

akosyakov Sep 15, 2022

filiptronicek commented Sep 8, 2022 •

edited by werft-gitpod-dev-com bot

Loading

mustard-mh commented Sep 15, 2022

mustard-mh commented Sep 15, 2022 •

edited

Loading

akosyakov commented Sep 15, 2022

mustard-mh commented Sep 16, 2022

iQQBot Sep 16, 2022

mustard-mh commented Sep 16, 2022

iQQBot left a comment •

edited

Loading

iQQBot commented Sep 16, 2022

iQQBot commented Sep 16, 2022

mustard-mh commented Sep 16, 2022

akosyakov Sep 16, 2022

	Name: "gitpod_vscode_web_load_total",
	Help: "Total count of supervisor frontend client loading / errored windows",
	Labels: []config.LabelAllowList{
	{
	Name: "status",
	AllowValues: []string{"loading", "failed"},
	},
	},
	},

[code browser] measure all sessions vs errored sessions #12702

[code browser] measure all sessions vs errored sessions #12702

Conversation

filiptronicek commented Sep 6, 2022 • edited by iQQBot Loading

Description

Related Issue(s)

How to test

Do a little bit of trolling (throw errors in VS Code Browser)

Verify with Grafana

Release Notes

Documentation

Werft options:

filiptronicek commented Sep 6, 2022 • edited by werft-gitpod-dev-com bot Loading

andreafalzetti commented Sep 6, 2022 • edited by werft-gitpod-dev-com bot Loading

filiptronicek commented Sep 6, 2022 • edited by werft-gitpod-dev-com bot Loading

mustard-mh commented Sep 7, 2022

filiptronicek Sep 8, 2022

Choose a reason for hiding this comment

filiptronicek Sep 9, 2022

Choose a reason for hiding this comment

akosyakov Sep 15, 2022

Choose a reason for hiding this comment

filiptronicek commented Sep 8, 2022 • edited by werft-gitpod-dev-com bot Loading

mustard-mh commented Sep 15, 2022

mustard-mh commented Sep 15, 2022 • edited Loading

akosyakov commented Sep 15, 2022

mustard-mh commented Sep 16, 2022

iQQBot Sep 16, 2022

Choose a reason for hiding this comment

mustard-mh commented Sep 16, 2022

iQQBot left a comment • edited Loading

Choose a reason for hiding this comment

iQQBot commented Sep 16, 2022

iQQBot commented Sep 16, 2022

mustard-mh commented Sep 16, 2022

akosyakov Sep 16, 2022

Choose a reason for hiding this comment

filiptronicek commented Sep 6, 2022 •

edited by iQQBot

Loading

filiptronicek commented Sep 6, 2022 •

edited by werft-gitpod-dev-com bot

Loading

andreafalzetti commented Sep 6, 2022 •

edited by werft-gitpod-dev-com bot

Loading

filiptronicek commented Sep 6, 2022 •

edited by werft-gitpod-dev-com bot

Loading

filiptronicek commented Sep 8, 2022 •

edited by werft-gitpod-dev-com bot

Loading

mustard-mh commented Sep 15, 2022 •

edited

Loading

iQQBot left a comment •

edited

Loading