track git command initial

Add the TrackEvent type struct

Reference fixes

Only require one flag

Better function description

go fmt

Update components/gitpod-cli/cmd/git-track-command.go

Co-authored-by: bigint <bigint@icloud.com>

fixes

Log the correct vars and correct error messages

Set test data

Remove more unused code

go fmt

More go fmt

Mock `TrackEvent`

Fix destination of `gitCommand`

Fix function references

Spawn the git command tracker automatically

Correct the script name

Don't expect any results

Don't return on error with tracker

Use new connection syntax

`go fmt`

Set the correct type for params, rename args

`go fmt`

Fix param type names

Add resource scope

Move process release to an `else` block

Make `TrackEvent` types emulate the backend

Format

`go fmt`

Only use the one required scope

Close the client after returning

Change description

[changelog] updated changelog

Setup IdentitiesOnly as yes in ssh config

Fixes #7335

Handle "no data" by adding 'on() vector(0)' to each numerator
Relies on new variable $datasource
Also fixes legend for workspace startup panel
When exporting from Grafana, disable "export for sharing externally"

Helm is needed to support Observability

[changelog] updated changelog

add missing condition for ide-proxy heml chart

Improve label colors on dashboard integrations for the dark theme

Send headers when connecting to the supervisor

Create the version file during build

CHANGELOG.md

@@ -1,6 +1,10 @@
 # Change Log
 
 ## December 2021
+- Fix Team Workspace Success Criteria dashboard ([#7354](https://github.com/gitpod-io/gitpod/pull/7354)) - [@kylos101](https://github.com/kylos101)
+- Allow auth provider secrets to be passed in via a secret ([#7177](https://github.com/gitpod-io/gitpod/pull/7177)) - [@MrSimonEmms](https://github.com/MrSimonEmms)
+- Profile of the user who already added a project is linked. ([#7312](https://github.com/gitpod-io/gitpod/pull/7312)) - [@laushinka](https://github.com/laushinka)
+- Update GoLand IDE image to version 213.6461.23. ([#7327](https://github.com/gitpod-io/gitpod/pull/7327)) - [@roboquat](https://github.com/roboquat)
 - Allow setting a name and a description for each port on .gitpod.yml ([#7012](https://github.com/gitpod-io/gitpod/pull/7012)) - [@felladrin](https://github.com/felladrin)
 - Mention username who added project in a team ([#5128](https://github.com/gitpod-io/gitpod/pull/5128)) - [@AlexTugarev](https://github.com/AlexTugarev)
 - Enables bitbucket.org repositories in Teams & Projects ([#7251](https://github.com/gitpod-io/gitpod/pull/7251)) - [@AlexTugarev](https://github.com/AlexTugarev)

chart/templates/ide-proxy-deny-all-allow-explicit-networkpolicy.yaml

@@ -1,7 +1,7 @@
 # Copyright (c) 2020 Gitpod GmbH. All rights reserved.
 # Licensed under the MIT License. See License-MIT.txt in the project root for license information.
 
-{{ if .Values.installNetworkPolicies -}}
+{{ if and (not .Values.components.ideProxy.disabled) (.Values.installNetworkPolicies) -}}
 apiVersion: networking.k8s.io/v1
 kind: NetworkPolicy
 metadata:

chart/templates/ide-proxy-rolebinding.yaml

@@ -1,6 +1,7 @@
 # Copyright (c) 2020 Gitpod GmbH. All rights reserved.
 # Licensed under the MIT License. See License-MIT.txt in the project root for license information.
 
+{{ if not .Values.components.ideProxy.disabled -}}
 kind: RoleBinding
 apiVersion: rbac.authorization.k8s.io/v1
 metadata:
@@ -17,3 +18,4 @@ roleRef:
   kind: ClusterRole
   name: {{ .Release.Namespace }}-ns-psp:restricted-root-user
   apiGroup: rbac.authorization.k8s.io
+{{- end -}}

chart/templates/ide-proxy-serviceaccount.yaml

@@ -1,6 +1,7 @@
 # Copyright (c) 2020 Gitpod GmbH. All rights reserved.
 # Licensed under the MIT License. See License-MIT.txt in the project root for license information.
 
+{{ if not .Values.components.ideProxy.disabled -}}
 apiVersion: v1
 kind: ServiceAccount
 metadata:
@@ -10,4 +11,5 @@ metadata:
     component: ide-proxy
     kind: service-account
     stage: {{ .Values.installation.stage }}
-automountServiceAccountToken: false
+automountServiceAccountToken: false
+{{- end -}}

components/dashboard/src/settings/Integrations.tsx

@@ -291,15 +291,15 @@ function GitProviders() {
                     </ItemFieldIcon>
                     <ItemField className="w-4/12 xl:w-3/12 flex flex-col my-auto">
                         <span className="my-auto font-medium truncate overflow-ellipsis">{ap.authProviderType}</span>
-                        <span className="text-sm my-auto text-gray-400 truncate overflow-ellipsis">{ap.host}</span>
+                        <span className="text-sm my-auto text-gray-400 truncate overflow-ellipsis dark:text-gray-500">{ap.host}</span>
                     </ItemField>
                     <ItemField className="w-6/12 xl:w-3/12 flex flex-col my-auto">
-                        <span className="my-auto truncate text-gray-500 overflow-ellipsis">{getUsername(ap.authProviderId) || "–"}</span>
-                        <span className="text-sm my-auto text-gray-400">Username</span>
+                        <span className="my-auto truncate text-gray-500 overflow-ellipsis dark:text-gray-400">{getUsername(ap.authProviderId) || "–"}</span>
+                        <span className="text-sm my-auto text-gray-400 dark:text-gray-500">Username</span>
                     </ItemField>
                     <ItemField className="hidden xl:w-5/12 xl:flex xl:flex-col my-auto">
-                        <span className="my-auto truncate text-gray-500 overflow-ellipsis">{getPermissions(ap.authProviderId)?.join(", ") || "–"}</span>
-                        <span className="text-sm my-auto text-gray-400">Permissions</span>
+                        <span className="my-auto truncate text-gray-500 overflow-ellipsis dark:text-gray-400">{getPermissions(ap.authProviderId)?.join(", ") || "–"}</span>
+                        <span className="text-sm my-auto text-gray-400 dark:text-gray-500">Permissions</span>
                     </ItemField>
                     <ItemFieldContextMenu menuEntries={gitProviderMenu(ap)} />
                 </Item>
@@ -375,7 +375,7 @@ function GitIntegrations() {
         <div className="flex items-start sm:justify-between mb-2">
             <div>
                 <h3>Git Integrations</h3>
-                <h2 className="text-gray-500">Manage Git integrations for GitLab or GitHub self-hosted instances.</h2>
+                <h2>Manage Git integrations for GitLab or GitHub self-hosted instances.</h2>
             </div>
             {providers.length !== 0
                 ?

components/gitpod-cli/BUILD.yaml

@@ -9,7 +9,20 @@ packages:
       - CGO_ENABLED=0
       - GOOS=linux
     deps:
+      - :version
       - components/supervisor-api/go:lib
       - components/gitpod-protocol/go:lib
+    argdeps:
+      - version
     config:
       packaging: app
+    prep:
+      - ["cp", "_deps/components-gitpod-cli--version/version.txt", "pkg/gitpod/version.txt"]
+  - name: version
+    type: generic
+    argdeps:
+      - version
+    config:
+      commands:
+        - ["sh", "-c", "echo '${version}' > version.txt"]
+        - ["echo", "Gitpod-CLI Version: ${version}"]

components/gitpod-cli/cmd/credential-helper.go

@@ -57,6 +57,19 @@ var credentialHelper = &cobra.Command{
 		}()
 
 		repoURL, gitCommand := parseProcessTree()
+
+		// Starts another process which tracks the executed git event
+		gitCommandTracker := exec.Command("/proc/self/exe", "git-track-command", "--gitCommand", gitCommand)
+		err = gitCommandTracker.Start()
+		if err != nil {
+			log.WithError(err).Print("error spawning tracker")
+		} else {
+			err = gitCommandTracker.Process.Release()
+			if err != nil {
+				log.WithError(err).Print("error releasing tracker")
+			}
+		}
+
 		host := parseHostFromStdin()
 		if len(host) == 0 {
 			log.Println("'host' is missing")

components/gitpod-cli/cmd/git-track-command.go

@@ -0,0 +1,86 @@
+// Copyright (c) 2021 Gitpod GmbH. All rights reserved.
+// Licensed under the GNU Affero General Public License (AGPL).
+// See License-AGPL.txt in the project root for license information.
+
+package cmd
+
+import (
+	"context"
+	"io"
+	"os"
+	"time"
+
+	log "github.com/sirupsen/logrus"
+	"github.com/spf13/cobra"
+
+	gitpod "github.com/gitpod-io/gitpod/gitpod-cli/pkg/gitpod"
+	serverapi "github.com/gitpod-io/gitpod/gitpod-protocol"
+)
+
+var gitTrackCommandOpts struct {
+	GitCommand string
+}
+
+var gitTrackCommand = &cobra.Command{
+	Use:    "git-track-command",
+	Short:  "Gitpod's Git command tracker",
+	Long:   "Sending anonymous statistics about the executed git commands inside a workspace",
+	Args:   cobra.ExactArgs(0),
+	Hidden: true,
+	Run: func(cmd *cobra.Command, args []string) {
+		log.SetOutput(io.Discard)
+		f, err := os.OpenFile(os.TempDir()+"/gitpod-git-credential-helper.log", os.O_WRONLY|os.O_CREATE|os.O_APPEND, 0644)
+		if err == nil {
+			defer f.Close()
+			log.SetOutput(f)
+		}
+
+		log.Infof("gp git-track-command")
+
+		ctx, cancel := context.WithTimeout(context.Background(), 1*time.Minute)
+		defer cancel()
+		wsInfo, err := gitpod.GetWSInfo(ctx)
+		if err != nil {
+			fail(err.Error())
+		}
+
+		client, err := gitpod.ConnectToServer(ctx, wsInfo, []string{"function:trackEvent"})
+
+		if err != nil {
+			log.WithError(err).Fatal("error connecting to supervisor")
+		}
+
+		defer client.Close()
+
+		type GitEventParams struct {
+			Command             string `json:"command,omitempty"`
+			WorkspaceId         string `json:"workspaceId,omitempty"`
+			WorkspaceInstanceId string `json:"workspaceInstanceId,omitempty"`
+			Timestamp           int64  `json:"timestamp,omitempty"`
+		}
+
+		params := &GitEventParams{
+			Command:             gitTrackCommandOpts.GitCommand,
+			WorkspaceId:         wsInfo.WorkspaceId,
+			WorkspaceInstanceId: wsInfo.InstanceId,
+			Timestamp:           time.Now().Unix(),
+		}
+		event := &serverapi.RemoteTrackMessage{
+			Event:      "git_command",
+			Properties: *params,
+		}
+		log.WithField("command", gitTrackCommandOpts.GitCommand).
+			Info("tracking the GitCommand event")
+
+		err = client.TrackEvent(ctx, event)
+		if err != nil {
+			log.WithError(err).Fatal("error tracking git event")
+		}
+	},
+}
+
+func init() {
+	rootCmd.AddCommand(gitTrackCommand)
+	gitTrackCommand.Flags().StringVarP(&gitTrackCommandOpts.GitCommand, "gitCommand", "c", "", "The Git command to be recorded")
+	gitTrackCommand.MarkFlagRequired("gitCommand")
+}

components/gitpod-cli/pkg/gitpod/server.go

@@ -6,7 +6,9 @@ package server
 
 import (
 	"context"
+	_ "embed"
 	"os"
+	"strings"
 
 	serverapi "github.com/gitpod-io/gitpod/gitpod-protocol"
 	supervisor "github.com/gitpod-io/gitpod/supervisor/api"
@@ -15,6 +17,13 @@ import (
 	"google.golang.org/grpc"
 )
 
+var (
+	// Version : current version
+	Version string = strings.TrimSpace(version)
+	//go:embed version.txt
+	version string
+)
+
 func GetWSInfo(ctx context.Context) (*supervisor.WorkspaceInfoResponse, error) {
 	supervisorAddr := os.Getenv("SUPERVISOR_ADDR")
 	if supervisorAddr == "" {
@@ -50,10 +59,15 @@ func ConnectToServer(ctx context.Context, wsInfo *supervisor.WorkspaceInfoRespon
 	if err != nil {
 		return nil, xerrors.Errorf("failed getting token from supervisor: %w", err)
 	}
+
 	client, err := serverapi.ConnectToServer(wsInfo.GitpodApi.Endpoint, serverapi.ConnectToServerOpts{
 		Token:   clientToken.Token,
 		Context: ctx,
 		Log:     log.NewEntry(log.StandardLogger()),
+		ExtraHeaders: map[string]string{
+			"User-Agent":       "gitpod/cli",
+			"X-Client-Version": Version,
+		},
 	})
 	if err != nil {
 		return nil, xerrors.Errorf("failed connecting to server: %w", err)

components/gitpod-cli/pkg/version.txt

@@ -0,0 +1 @@
+set-during-build

components/gitpod-protocol/go/gitpod-service.go

@@ -82,6 +82,7 @@ type APIInterface interface {
 	InstallUserPlugins(ctx context.Context, params *InstallPluginsParams) (res bool, err error)
 	UninstallUserPlugin(ctx context.Context, params *UninstallPluginParams) (res bool, err error)
 	GuessGitTokenScopes(ctx context.Context, params *GuessGitTokenScopesParams) (res *GuessedGitTokenScopes, err error)
+	TrackEvent(ctx context.Context, event *RemoteTrackMessage) (err error)
 
 	InstanceUpdates(ctx context.Context, instanceID string) (<-chan *WorkspaceInstance, error)
 }
@@ -204,6 +205,8 @@ const (
 	FunctionUninstallUserPlugin FunctionName = "uninstallUserPlugin"
 	// FunctionGuessGitTokenScopes is the name of the guessGitTokenScopes function
 	FunctionGuessGitTokenScope FunctionName = "guessGitTokenScopes"
+	// FunctionTrackEvent is the name of the trackEvent function
+	FunctionTrackEvent FunctionName = "trackEvent"
 
 	// FunctionOnInstanceUpdate is the name of the onInstanceUpdate callback function
 	FunctionOnInstanceUpdate = "onInstanceUpdate"
@@ -1457,6 +1460,19 @@ func (gp *APIoverJSONRPC) GuessGitTokenScopes(ctx context.Context, params *Guess
 	return
 }
 
+// TrackEvent calls trackEvent on the server
+func (gp *APIoverJSONRPC) TrackEvent(ctx context.Context, params *RemoteTrackMessage) (err error) {
+	if gp == nil {
+		err = errNotConnected
+		return
+	}
+	var _params []interface{}
+
+	_params = append(_params, params)
+	err = gp.C.Call(ctx, "trackEvent", _params, nil)
+	return
+}
+
 // PermissionName is the name of a permission
 type PermissionName string
 
@@ -2030,6 +2046,11 @@ type GuessedGitTokenScopes struct {
 	Message string   `json:"message,omitempty"`
 }
 
+type RemoteTrackMessage struct {
+	Event      string      `json:"event,omitempty"`
+	Properties interface{} `json:"properties,omitempty"`
+}
+
 // BrandingLink is the BrandingLink message type
 type BrandingLink struct {
 	Name string `json:"name,omitempty"`

components/local-app/pkg/bastion/bastion.go

@@ -152,6 +152,7 @@ func (s *SSHConfigWritingCallback) InstanceUpdate(w *Workspace) {
 				&ssh_config.KV{Key: "User", Value: "gitpod"},
 				&ssh_config.KV{Key: "Port", Value: port},
 				&ssh_config.KV{Key: "IdentityFile", Value: ws.SSHPrivateFN},
+				&ssh_config.KV{Key: "IdentitiesOnly", Value: "yes"},
 			},
 		})
 	}

operations/observability/mixins/README.md

@@ -299,7 +299,10 @@ There are a couple of ways to trigger a werft job that will deploy a preview env
 
 1. You can open a Pull Request with the following line in the description:
 ```
-/werft with-observability
+# we deploy using K8s manifests generated by our new installer
+# observability only works with our old helm chart
+# to support observability, use with-helm will;
+/werft with-observability with-helm
 
 # Just in case your PR requires extra configuration on Prometheus side
 # (and you have a new branch on https://github.com/gitpod-io/observability with such changes)
@@ -309,13 +312,13 @@ There are a couple of ways to trigger a werft job that will deploy a preview env
 
 2. After opening a Pull Request, you can add a Github comment:
 ```
-/werft run with-observability
-/werft run withObservabilityBranch="<my-branch>"
+/werft run with-observability with-helm
+/werft run withObservabilityBranch="<my-branch>" with-observability
 ```
 
 3. Inside your workspace, run:
 ```
-werft run github -a with-observability="" -a withObservabilityBranch="<my-branch>"
+werft run github -a with-observability=true -a withObservabilityBranch="<my-branch>" -a with-helm=true
 ```
 
 As mentioned in [How is our mixin consumed?](#How-is-our-mixin-consumed), please remember that a commit must be done for us to update monitoring-satellite with the dashboards/alerts/recording rule changes.