proposal: testing/slogtest: Add RunWithSkip func

[arukiidou]

Proposal Summary

Allows skipping of tests with specific names

func RunWithSkip(t *testing.T, newHandler func(*testing.T) slog.Handler, result func(*testing.T) map[string]any, skipTestName ...string)

• It is behind of testing/slogtest: support sub-tests #61758

---

Problems

Several logging libraries trying to support log/slog, but seem to not much imported testing/slogtest

Background

This is just a guess based on my personal insight:

• Currently slogtest is all or nothing.
  • Pass or fail-all, or no slogtest
• Existing logging libraries may take time to adapt to all of slog, Or they may not be able to.
  • In particular, grouping logs.
    • zapslog: Handler should inline the Attrs of a group with an empty key uber-go/zap#1402
    • stdlib slog handler for zerolog rs/zerolog#571
  • Logging library should not be broken to support slog.
• Most OSS will not merge code that test failed.

Proposal

Add RunWithSkip, allows skipping of tests with specific test names.

Benefits

• Expecting more slogtest imports.
• It makes possible "partial slog support" approach
  • I think that partial approach is better than all-or-nothing.
  • For example, kubernetes gateway-api conformance supports skipping tests.

[ianlancetaylor]

CC @jba

[arukiidou]

If could not skip the test,
Future additions of slogtest cases that support edge cases, it might break the GHA pipeline and block Golang updates.

• relates: testing/slogtest: add additional tests #62280

[jba]

I think you can do what you want with the existing slogtest.Run function:

slogtest.Run(t, func(t *testing.T) slog.Handler {
    if skipNameMap[t.Name()] {
        t.Skip()
   }
   ...,
}, ...)

Please correct me if I've misunderstood.

[arukiidou]

you are right,
This proposal is retractable.

https://go.dev/play/p/nJY0XtnW6mz

newHandler := func(tt *testing.T) slog.Handler {
if slices.Contains([]string{"TestSlogtest/empty-attr", "TestSlogtest/inline-group", "TestSlogtest/empty-group-record"}, tt.Name()) {
tt.Skip("skipped.")
}
buff.Reset()
return zapslog.NewHandler(core, nil)
}
run := func(tt *testing.T) map[string]any {
var ent map[string]any
if err := json.Unmarshal(buff.Bytes(), &ent); err != nil {
t.Fatal(err)
}
return ent
}
slogtest.Run(t, newHandler, tf)