Provide client info in argocd server errors

[jsoref]

This message:

time="2024-10-15T18:59:29Z" level=warning msg="Failed to verify token: failed to verify token: token verification failed for all audiences: error for aud \"argo-cd\": failed to verify signature: failed to verify id token signature, error for aud \"argo-cd-cli\": oidc: expected audience \"argo-cd-cli\" got [\"argo-cd\"]"
time="2024-10-15T18:59:29Z" level=warning msg="Failed to verify token: failed to verify token: token verification failed for all audiences: error for aud \"argo-cd\": failed to verify signature: failed to verify id token signature, error for aud \"argo-cd-cli\": oidc: expected audience \"argo-cd-cli\" got [\"argo-cd\"]"
time="2024-10-15T18:59:29Z" level=warning msg="Failed to verify token: failed to verify token: token verification failed for all audiences: error for aud \"argo-cd\": failed to verify signature: failed to verify id token signature, error for aud \"argo-cd-cli\": oidc: expected audience \"argo-cd-cli\" got [\"argo-cd\"]"
time="2024-10-15T18:59:30Z" level=warning msg="Failed to verify token: failed to verify token: token verification failed for all audiences: error for aud \"argo-cd\": failed to verify signature: failed to verify id token signature, error for aud \"argo-cd-cli\": oidc: expected audience \"argo-cd-cli\" got [\"argo-cd\"]"
time="2024-10-15T18:59:30Z" level=warning msg="Failed to verify token: failed to verify token: token verification failed for all audiences: error for aud \"argo-cd\": failed to verify signature: failed to verify id token signature, error for aud \"argo-cd-cli\": oidc: expected audience \"argo-cd-cli\" got [\"argo-cd\"]"
time="2024-10-15T18:59:30Z" level=warning msg="Failed to verify token: failed to verify token: token verification failed for all audiences: error for aud \"argo-cd-cli\": oidc: expected audience \"argo-cd-cli\" got [\"argo-cd\"], error for aud \"argo-cd\": failed to verify signature: failed to verify id token signature"
time="2024-10-15T18:59:31Z" level=warning msg="Failed to verify token: failed to verify token: token verification failed for all audiences: error for aud \"argo-cd-cli\": oidc: expected audience \"argo-cd-cli\" got [\"argo-cd\"], error for aud \"argo-cd\": failed to verify signature: failed to verify id token signature"
time="2024-10-15T18:59:31Z" level=warning msg="Failed to verify token: failed to verify token: token verification failed for all audiences: error for aud \"argo-cd\": failed to verify signature: failed to verify id token signature, error for aud \"argo-cd-cli\": oidc: expected audience \"argo-cd-cli\" got [\"argo-cd\"]"
time="2024-10-15T18:59:31Z" level=warning msg="Failed to verify token: failed to verify token: token verification failed for all audiences: error for aud \"argo-cd\": failed to verify signature: failed to verify id token signature, error for aud \"argo-cd-cli\": oidc: expected audience \"argo-cd-cli\" got [\"argo-cd\"]"
time="2024-10-15T18:59:32Z" level=warning msg="Failed to verify token: failed to verify token: token verification failed for all audiences: error for aud \"argo-cd\": failed to verify signature: failed to verify id token signature, error for aud \"argo-cd-cli\": oidc: expected audience \"argo-cd-cli\" got [\"argo-cd\"]"
time="2024-10-15T18:59:32Z" level=warning msg="Failed to verify token: failed to verify token: token verification failed for all audiences: error for aud \"argo-cd\": failed to verify signature: failed to verify id token signature, error for aud \"argo-cd-cli\": oidc: expected audience \"argo-cd-cli\" got [\"argo-cd\"]"
time="2024-10-15T18:59:32Z" level=warning msg="Failed to verify token: failed to verify token: token verification failed for all audiences: error for aud \"argo-cd-cli\": oidc: expected audience \"argo-cd-cli\" got [\"argo-cd\"], error for aud \"argo-cd\": failed to verify signature: failed to verify id token signature"
time="2024-10-15T18:59:33Z" level=warning msg="Failed to verify token: failed to verify token: token verification failed for all audiences: error for aud \"argo-cd\": failed to verify signature: failed to verify id token signature, error for aud \"argo-cd-cli\": oidc: expected audience \"argo-cd-cli\" got [\"argo-cd\"]"
time="2024-10-15T18:59:33Z" level=warning msg="Failed to verify token: failed to verify token: token verification failed for all audiences: error for aud \"argo-cd-cli\": oidc: expected audience \"argo-cd-cli\" got [\"argo-cd\"], error for aud \"argo-cd\": failed to verify signature: failed to verify id token signature"
time="2024-10-15T18:59:33Z" level=warning msg="Failed to verify token: failed to verify token: token verification failed for all audiences: error for aud \"argo-cd\": failed to verify signature: failed to verify id token signature, error for aud \"argo-cd-cli\": oidc: expected audience \"argo-cd-cli\" got [\"argo-cd\"]"
time="2024-10-15T18:59:34Z" level=warning msg="Failed to verify token: failed to verify token: token verification failed for all audiences: error for aud \"argo-cd\": failed to verify signature: failed to verify id token signature, error for aud \"argo-cd-cli\": oidc: expected audience \"argo-cd-cli\" got [\"argo-cd\"]"
time="2024-10-15T18:59:34Z" level=warning msg="Failed to verify token: failed to verify token: token verification failed for all audiences: error for aud \"argo-cd\": failed to verify signature: failed to verify id token signature, error for aud \"argo-cd-cli\": oidc: expected audience \"argo-cd-cli\" got [\"argo-cd\"]"
time="2024-10-15T18:59:34Z" level=warning msg="Failed to verify token: failed to verify token: token verification failed for all audiences: error for aud \"argo-cd\": failed to verify signature: failed to verify id token signature, error for aud \"argo-cd-cli\": oidc: expected audience \"argo-cd-cli\" got [\"argo-cd\"]"
time="2024-10-15T18:59:35Z" level=warning msg="Failed to verify token: failed to verify token: token verification failed for all audiences: error for aud \"argo-cd\": failed to verify signature: failed to verify id token signature, error for aud \"argo-cd-cli\": oidc: expected audience \"argo-cd-cli\" got [\"argo-cd\"]"

from

argo-cd/util/oidc/oidc.go

Line 378 in e80de49

log.Warnf("Failed to verify token: %s", err)

isn't actionable.

At the very least, it needs to include a client ip. If it knows about a "user" or some similar thing, it should include that too.

At the same time, there's a thing in the verify side which hoped that the go oidc provider would do something, and that pr was closed, so the comment should be removed:

argo-cd/util/oidc/provider.go

Lines 133 to 135 in e80de49

	// We store the error for each audience so that we can return a more detailed error message to the user.
	// If this gets merged, we'll be able to detect failures unrelated to audiences and short-circuit this loop
	// to avoid logging irrelevant warnings: https://github.com/coreos/go-oidc/pull/406

[jsoref]

In terms of information I think I want about the token itself, it's roughly:

• Token identity
  • jti token identity
• Bad proxies for token identity (when jti is missing)
  • iat token issued at
  • exp token expires at
• Identity of token issuer
  • iss Alleged issuer
• Even less trustworthy information about whom the issuer was allegedly identifying
  • sub Subject that someone wanted the token to identify (but which it didn't, which is why I'm seeing the logging...)
• Something that the current logging code felt was very important, but which I don't really understand why it thought was important...
  • aud "Audience"

Where each of them could be truncated to probably 64 chars (anyone using more than 64 for most of those fields is being evil).

• Note that the order I'm listing is, I think, more or less, the order in which i care about things (aud is a bit weird, i only care because the logging code felt it was important to tell me about in the first place).
• Note that on average, I expect that all iss fields will be the same, so showing that first isn't helpful.
• If I have a jti, I expect I can figure out everything I need w/ just that.
• If I don't have a jti, then an iat+exp should on average give me most of what I wanted from a jti. And iss+sub should give me the rest. Yes, I understand two iss could have the same jti and mean different objects, but, again, on average I'm only ever going to see a single iss...

(This is, of course, in addition to including the client IP address, and probably port.)

[andrii-korotkov-verkada]

We have to be careful with logging a potentially sensitive info like tokens, but ip seems to be fine.
What's the problem you are trying to solve? If some people in your company have issues with ArgoCD CLI, won't they just reach out to you or other people who maintain ArgoCD in the company?

[jsoref]

I'm reading logs and am trying to figure out who's flooding them with error messages. All I have is the logs, nothing else, and the logs are useless.

[andrii-korotkov-verkada]

Would logging a token sha be sufficient?

[jsoref]

dunno, what can i do with that?

[andrii-korotkov-verkada]

You can store a table mapping a token sha to users for example.

[andrii-korotkov-verkada]

Though other fields like issuer and expiration date are probably fine. But I also don't know if that's enough.
Let's start with an ip address and see if that helps.