Allow "subtest at cursor" to work with table-driven test setup

[schandra157]

Is your feature request related to a problem? Please describe.

In the case of Table driven Test model in GO programming, Currently, it's possible to run/debug test at the Function level. It would be great if its possible to be executed at the individual test level

Describe the solution you'd like
Have an option to Run and Debug at each individual Test level, which would be great to debug the individual test in case of issue and would be very handy feature

Additional context

The below Screen shot describes the problem

[findleyr]

Hi, this feature is available:
https://github.com/golang/vscode-go/blob/master/docs/commands.md#go-subtest-at-cursor

Does that meet your needs?

[suzmue]

Unfortunately, "Go: subtest at cursor" is pretty limited at the moment and does not work with table driven tests. Where would you expect the code lens to appear? In the definition of tests? On t.Run (with some way to specify which subtest)?

It would be reasonable for Go: subtest at cursor to request a subtest name via an input box if it cannot find a simple test name.

Issue #1579 is tracking updates to testing support for this extension, which may add some support for easier subtest running.

[schandra157]

Hi, this feature is available:
https://github.com/golang/vscode-go/blob/master/docs/commands.md#go-subtest-at-cursor

Does that meet your needs?

Yes, it will be helping us a lot.

[schandra157]

Unfortunately, "Go: subtest at cursor" is pretty limited at the moment and does not work with table driven tests. Where would you expect the code lens to appear? In the definition of tests? On t.Run (with some way to specify which subtest)?

It would be reasonable for Go: subtest at cursor to request a subtest name via an input box if it cannot find a simple test name.

Issue #1579 is tracking updates to testing support for this extension, which may add some support for easier subtest running.

Glad to see some Ideas around this, My opinion is to have code lens appear at definition of the test

[findleyr]

We couldn't do this in generality (for example if the test cases were outside the function scope), but it wouldn't be hard for gopls to provide code lenses for the common patterns of a slice of test cases in the function scope, or ranging over a literal.

We could add this feature as part of our migration to use gopls for generating test code lens.

[schandra157]

okay

[hyangah]

I guess users will still want to enter or select the subtest name after clicking a codelens. Currently, I don't think LSP's codelens+command allows that level of UX control yet. So the extension will need a middleware hack or the actual codelens implementation (based on info from gopls) if we want to implement it soon.

[gopherbot]

Change https://golang.org/cl/333309 mentions this issue: src/goTest.ts: prompt for subtest name if there is no subtest at cursor

[firelizzard18]

@schandra157 Once VSCode's testing API is finalized and #1579 is merged, you will be able to see specific subtests, though there are some limitations. Discovering subtests statically is problematic, so at the moment, I add subtests while running a test, any time I see a go test event with a test name that indicates a subtest, such as TestFoo/Bar. Since the set of defined subtests can change without notice (from the extension's perspective), I wipe out the subtests of a test whenever it is run, or whenever VSCode reports the test function was edited.

So the test explorer will list subtests. But every feasible method I can think of for discovering where a subtest was defined has edge cases that wouldn't work. And the only method I can think of that would always work would involve executing the code, which is not really something we can feasibly do at the moment. go test does not tell you where a test or a subtest was defined, and that's the only 'execute the code' method I consider feasible. I opened golang/go#47101 to discuss adding a core package such as go/test that might facilitate this kind of thing.

What we can recognize (probably)

• t.Run("SubTest", func(*testing.T) { ... })
• Table driven tests using obvious patterns
• Test suites such as those defined by stretchr

What won't work

• t.Run(fmt.Sprint(foo, bar baz), ...) - without resorting to executing code or building an interpreter, how would we know what the test name was?
• Table driven tests using non-obvious patterns - again, without resorting to executing or interpreting code, it can be very difficult to determine subtest names etc.
• Other non-obvious patterns - same issue.

[pckhoi]

How about a simpler solution such as adding command Go: Test -run that display an input box where you can specify -run flag directly? That would work for all edge cases.

[firelizzard18]

@pckhoi It's not clear to me that such a command would provide value. The user can already run go test -run on the command line. What improvement does running that via a command provide?

[pckhoi]

Except I never run tests on the command line anymore but mostly rely on "Go: Test Package" command. Sure I could run things on the command line but I also lose any integration with vscode, including custom settings I have defined in go.testEnvVars and go.testFlags. I have attempted to put -run flag in go.testFlags but it still proves cumbersome and prone to be committed by mistake. Adding a command that allows you to define -run flag each time will allow people to easily run sub-tests reliably while still having things integrated with vscode such as code coverage.

[firelizzard18]

@pckhoi I suggest you open a separate feature request, including your explanation here. "Add a command that allows me to specify a custom value for the -run flag" is a different feature from "Add support for table-driven subtests to the 'subtest at cursor' command".

The can both be used for similar purposes, but they take a fundamentally different approach, and must be implemented differently. Plus, if I can find a way to detect table-driven tests, this issue will likely be closed as resolved.

[marjau]

I'm coming from Goland(JetBrains). Digging a bit, I realized that we can run a specific test case into a table-driven. This seems to be related to the topic:

1. Put the cursor inside the TestFunction
2. In the command line, look for: "Go: Debug Subset at Cursor"
3. Write the specific name of the test case

Ref: https://github.com/golang/vscode-go/blob/master/docs/commands.md#go-subtest-at-cursor

I hope it helps to someone.

[vec715]

any updates?

[gopherbot]

Change https://go.dev/cl/601015 mentions this issue: gopls/internal/cache/tests: detect simple subtests

[gopherbot]

Change https://go.dev/cl/548675 mentions this issue: gopls/internal: test discovery