Skip to content

[ws-manager] Add team and project to logs and traces #10632

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 4 commits into from
Jun 15, 2022
Merged

Conversation

Furisto
Copy link
Member

@Furisto Furisto commented Jun 13, 2022

Description

Add team and project to ws-manager logs and traces.

Logs:
logs

Traces:
traces

Related Issue(s)

Fixes #9374

How to test

  • Create a team and within that team a project
  • Open a workspace within the project
  • Check that the pod contains project and team labels
  • Search for team id/projectId in logs explorer
  • Search for team id/projectId in honeycomb

Release Notes

NONE

@Furisto Furisto requested a review from a team June 13, 2022 16:59
@Furisto Furisto requested review from aledbf and sagor999 as code owners June 13, 2022 16:59
@Furisto Furisto requested a review from a team June 13, 2022 16:59
@github-actions github-actions bot added team: webapp Issue belongs to the WebApp team team: workspace Issue belongs to the Workspace team labels Jun 13, 2022
@Furisto
Copy link
Member Author

Furisto commented Jun 13, 2022

/werft run

👍 started the job as gitpod-build-fo-log-context.5
(with .werft/ from main)

@@ -911,6 +911,14 @@ func (m *Manager) newStartWorkspaceContext(ctx context.Context, req *api.StartWo
workspaceClassLabel: clsName,
}

if req.Metadata.Project != "" {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: why not to just add it above into labels map as:
wsk8s.ProjectLabel: req.Metadata.Project ?
If it is empty, it will be an empty label. Which I think is fine.

// LogContext builds a structure meant for logrus which contains the owner, workspace and instance.
// Beware that this refers to the terminology outside of wsman which maps like:
// owner = owner, workspace = metaID, instance = workspaceID
func LogContext(owner, workspace, instance, project, team string) log.Fields {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we could extend this function

func OWI(owner, workspace, instance string) log.Fields {
to have the project and team as well.

Or to create another function

func PT(project, team string) log.Fields {
	return log.Fields{
		Project: project,
		Team: team,
	}
}

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The OWI function is used across many places where we do not currently log the team/project and Go does not support optional parameters, so that would require adapting all usages. I do not think that is a good trade-off.
Regarding your second suggestion we need all 5 fields, only project and team is not enough.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well, my idea was has a another function like this

func PT(project, team string) log.Fields {
	return log.Fields{
		Project: project,
		Team: team,
	}
}

So, the caller would be

owi := log.OWI(req.Metadata.Owner, req.Metadata.MetaId, req.Id)
pt := log.PT(project, team)
clog := log.WithFields(owi).WithFields(pt)

string team = 5;

// project the workspace belongs to
string project = 6;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shouldn't these be optional?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's a good question! Here is how I understand it: In contrast to proto2, proto3 uses no presence semantics as default (i.e. it just does not serialize values that are not present) and does not use explicit presence semantics (i.e. it stores if a field has explicitly been set to a value).

Tagging the fields with optional would switch it to explicit presence semantics. Explicit presence therefore increases message size, even when no value is present, because of the tracking requirement. In return additional methods are generated that allow you to check if a value has been set. I do not foresee a need to do this, as with no presence semantics we get an empty string for team/project which should be good enough for us. What do you think?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I do not foresee a need to do this, as with no presence semantics we get an empty string for team/project which should be good enough for us. What do you think?

Sometimes optional is nice for readability bc a) you get the has_ accessor and b) for making it explicit in the proto file. (Or if you need the extra bit in case you need the default value in the value domain - not the case here),
But searching our repo we mention optional mostly on comments, so fine with not using it! 👍 A comment would be nice, though. 🙂

@@ -100,7 +100,7 @@ func FromContext(ctx context.Context, name string) (opentracing.Span, context.Co

// ApplyOWI sets the owner, workspace and instance tags on a span
func ApplyOWI(span opentracing.Span, owi logrus.Fields) {
for _, k := range []string{log.OwnerField, log.WorkspaceField, log.InstanceField} {
for _, k := range []string{log.OwnerField, log.WorkspaceField, log.InstanceField, log.ProjectField, log.TeamField} {
Copy link
Member

@geropl geropl Jun 14, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not necessarily in-scope for this PR, but: We started writing up some naming conventions for tracing, incl. for OWI. (just extended it for projectId/teamId 😇 ).
Would be awesome to use on common scheme across all components! 🧡

Copy link
Member

@geropl geropl left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code LGTM and works! 👍

/hold in case you want to address this comment

@Furisto Furisto force-pushed the fo/log-context branch 2 times, most recently from d0ac4a3 to cf2983f Compare June 15, 2022 12:44
@Furisto
Copy link
Member Author

Furisto commented Jun 15, 2022

@aledbf You are configured as one of the reviewers. Can you please take a look?

@Furisto
Copy link
Member Author

Furisto commented Jun 15, 2022

/unhold

@roboquat roboquat merged commit 45b0b45 into main Jun 15, 2022
@roboquat roboquat deleted the fo/log-context branch June 15, 2022 16:13
@roboquat roboquat added deployed: webapp Meta team change is running in production deployed: workspace Workspace team change is running in production deployed Change is completely running in production labels Jun 16, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
deployed: webapp Meta team change is running in production deployed: workspace Workspace team change is running in production deployed Change is completely running in production release-note-none size/XXL team: webapp Issue belongs to the WebApp team team: workspace Issue belongs to the Workspace team
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add Team and Project to OWI metadata in ws-manager, so we can inspect metrics by customer
6 participants