de-duplication of key/value pairs

[pohly]

What this PR does / why we need it:

Previously, de-duplication was limited such that keys passed to a log call as
parameters overwrote keys passed earlier to WithValues. Using the same key in
different WithValues calls or multiple times in a WithValues call or the
parameters led to duplicates in the output.

Now all key/value pairs get checked for duplicates. The last one wins and is
printed at the place where it was added. If a second occurrence is the same as
a prior one, the first one is kept and the second is discarded.

This is partly motivated by the concern that adding OpenTelemetry will lead to
duplicate IDs (cc @dashpole), partly by prior discussions around instrumentation
not being able to avoid duplicates when using WithValue when the instrumentation
is happening in unrelated packages (cc @sbueringer, if I remember correctly).

Special notes for your reviewer:

Because the new implementation avoids memory allocations does very little extra work
when there is nothing which needs to be de-duplicated, the performance
impact is minimal. Some cases even become faster. See commit messages for
benchmark results.

Release note:

The klog text output now consistently de-duplicates key/value pairs such that any given key is  only printed once with its most recent value.

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: pohly

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• OWNERS [pohly]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[k8s-ci-robot]

This issue is currently awaiting triage.

If klog contributors determine this is a relevant issue, they will accept it by applying the triage/accepted label and provide further guidance.

The triage/accepted label can be added by org members by writing /triage accepted in a comment.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

[pohly]

This should be ready for a review.

@mengjiao-liu, @harshanarayana, who has time?

[harshanarayana]

Checking @pohly

[mengjiao-liu]

I debugged the log outputs, and these tests were passed.
Just a non-blocking comment, LGTM otherwise!

[mengjiao-liu]

Should the code in lines 844-850 also be changed? Their output is similar.

klog/test/output.go

Lines 844 to 850 in 7f02688

	if actual != expected {
	if expectedWithPlaceholder == test.output {
	t.Errorf("Output mismatch. Expected:\n%s\nActual:\n%s\n", expectedWithPlaceholder, actual)
	} else {
	t.Errorf("Output mismatch. klog:\n%s\nExpected:\n%s\nActual:\n%s\n", test.output, expectedWithPlaceholder, actual)
	}
	}

[pohly]

Yes, that makes sense. Changed.