Move KEP-2845 to implementable

[serathius]

Sending PR to move KEP-2845 to implementable to start discussion as there is not a lot of time before code freeze.
Proposal: https://github.com/kubernetes/enhancements/blob/master/keps/sig-instrumentation/2845-deprecate-klog-specific-flags-in-k8s-components/README.md

Assigning people that already volunteered to review the KEP. Thanks for offering help!
/assign @ehashman @pohly

Still looking for approver from SIG-Arch side:

@dims, @thockin, @liggitt, @johnbelamaric
Could one of you become KEP approver?

[pohly]

Looking at the proposal again from top to bottom again, I have one concern: -vmodule is a feature that will only be supported by some logging backends. klog has it, but current logr backends don't. That means we can keep the flag or something like it, but should define it in such a way that it is clear that its effect depends on the selected backend. For JSON output via zapr, implementing the feature would be doable.

Also, can you clarify what "follow standard component configuration mechanism" is? Make it a link perhaps?

[dims]

/assign @johnbelamaric @liggitt

(for approval)

[liggitt]

for klog, I'll defer to @dims or @thockin

[dims]

thanks @liggitt i will definitely look. will assign tim as well

/assign @thockin

[ehashman]

/sig architecture

[ehashman]

@serathius you have added me as PRR approver but the PRR questionnaire is not complete. Please fill it out, including the version skew sections (explain why they are not applicable).

[thockin]

I like @pohly comment, but otherwise I am happy to see this proceed.

[serathius]

@ehashman Sorry for that, verify complained that I need PRR file, so I added it forgetting the questionnaire.
PTAL

[serathius]

@pohly For vmodule I'm still not sure what's the best solution. I see options:

• Implement it in logr to make it a standard. logr already implements verbosity handling, it should not be a big stretch to also support vmodule to encapsulate verbosity handling in logr. Upsides: simple and clear solution from K8s perspective. Downsides: We don't have usage data to confirm that it's really needed (we got one issue kubelet config flag using "--logging-format" and "--vmodule" raises a FATAL error kubernetes#96314) also what if we open flood gates for users to request more verbosity controls?
• Implement it as a logr wrapper in K8s. No impact on logr, but introducing wrappers is also not a preferred approach as we want to minimize logging code on K8s side.
• Make it a feature that backend can implement. This feature is generic enough that it doesn't make sense to have each backend implement it by itself. Downsides: it might lead to multiple reimplementations of the feature.
• Drop the feature. The only reason to keep the feature is fact that it impacts log volume (part of verbosity checks), so it's a escape hatch for contributors making mistakes when picking verbosity. Problem could be mitigated by improving how we pick verbosity. For example give better guidelines, measure log volume based on verbosity to make adjustments etc.

My preferred option would be first as it makes it clearly delegates verbosity to logr. I don't think that any backend would ever have any verbosity specific logic. Only case I would backend would want to play with verbosity is when someone would want to write to two backends on different verbosity conditions. But I don't think loger composition is something we planned for.

I'm not very involved with logr community, so I would depend on @thockin @pohly opinion for best action here.

[serathius]

@pohly as for example of standard K8s flag configuration you can take a look here https://github.com/kubernetes/kubernetes/blob/4556873abf8aa88ce2ab40476267b0e9d4253366/staging/src/k8s.io/component-base/logs/options.go#L43
This example is a little outdated, but it represents whole idea in one file.

• All configuration should be stored in dedicated structs.
• Those structs should have methods: New, AddFlags, Validate, Apply
• Components can use those structs to build hierarchical configuration, where there is no global state.

Current file differs a little as idea was expanded a little since. Instead of Options object having fileds directly we use Configuration object. The difference is that Configuration is an API object, meaning it can be passed via an API request.

[pohly]

Implement it in logr to make it a standard. logr already implements verbosity handling, it should not be a big stretch to also support vmodule to encapsulate verbosity handling in logr.

That's not how logr is currently designed. All logr.LogSink implementations provide their own verbosity checks, with only tracking of the message's verbosity level handled by logr.Logger. In other words, -v is a parameter for the underlying backend. And so is -vmodule.

My preferred option would be first as it makes it clearly delegates verbosity to logr. I don't think that any backend would ever have any verbosity specific logic.

I don't think we should change the logr design. We can provide support code for implementing -vmodule in logr and then LogSink implementations can share the same code to implement it if they want to.

My preferred solution is to keep only "-v" as a mandatory option and implement "-vmodule" for output via klog and zapr. That means in practice it will remain available in Kubernetes.

[serathius]

If we cannot make any changes in klog to improve default behavior, I don't think it makes sense to introduce an optional flag to klog. I think it's still important for alternative logging formats. JSON and future new logging formats should follow stream separation. This way we avoid pushing implementing unnecessary flags to other logr implementations.

In that case implementing splitting stdout/stderr to klog would be out of the scope of this KEP. If users would like to use it they would need to start using JSON format.

[serathius]

Updated KEP to not introduce breaking change or --logtostdout flag. I still think that would be a better long term decision, but we don't have quorum of reviewers to make it (OOO).

[pohly]

Looks better to me now. One nit about vmodule and a question around what we want to include now for split routing, but not a blocker IMHO.

[pohly]

Can we add:

Support for "-vmodule" will depend on the selected log format and be supported at least for the traditional "text" format.

[serathius]

Added in section below

[pohly]

We could promise to implement that for JSON output. But I prefer to keep that out of the KEP for now if that helps to get it merged.

If we want to include it, then here we could replace "Feature removed" with "Move into JSON backend" and below under "beta graduation" add "configuration of JSON backend implemented, providing at least message routing based on type and verbosity".

[serathius]

Added to Alpha stage as I want to implement it in next release to unblock JSON graduation to Beta.

[pohly]

/lgtm

[ehashman]

I need to do a final review, but having skimmed the discussion, so long as log stream splitting is opt-in rather than the default, I think this is okay from a PRR perspective.

As this was originally written, that became the new default, with single-stream logging being entirely deprecated, which was a blocker. Will complete my review today.

[johnbelamaric]

/unassign

[ehashman]

A few small comments on PRR. Just waiting on a review/:+1: from SIG Arch.

Looks good from Instrumentation perspective.

[ehashman]

Question says "Can the feature be disabled once it has been enabled (i.e. can we roll back the enablement)?" and this doesn't answer it. It should specify how to enable/disable.

In this case, there is no way to enable/disable as the feature is deprecating flags.

[dims]

i support this effort and approve on behalf of sig-arch. @thockin if you get a chance please take a pass at the latest updates as well.

/approve

[ehashman]

/label tide/merge-method-squash

[ehashman]

/approve
/lgtm

for both SIG Instrumentation and PRR

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: dims, ehashman, pohly, serathius

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• keps/prod-readiness/OWNERS [ehashman]
• keps/sig-instrumentation/OWNERS [dims,ehashman]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment