Skip to content

Commit

Permalink
logictest: add TestLogic_tmp for testing untracked logic test files
Browse files Browse the repository at this point in the history
This commit improves the developer experience when working with logic
tests. The logic test generator no longer generates a dedicated test for
logic tests files prefixed with "_". The hard-coded `TestLogic_tmp` test
has been added which runs all such logic test files. This allows
developers to create and run temporary test files that are not checked
into the repository, without repeatedly regenerating and reverting
changes to the `generated_test.go` files.

For example, if you add a git-ignored file logic test file,
`pkg/sql/logictest/testdata/logic_test/__test`, you can run it with the
`local` config with the command:

    ./dev test pkg/sql/logictest/tests/local -f TestLogic_tmp

Previously, to facilitate running these types of tests, `./dev test`
would automatically regenerate logic tests every time tests were run.
This was primarily motivated by the desire to make testing temporary
logic test files easier - a developer would not have to run
`./dev gen logictest` every time they wanted to test a temporary file.

However, this was a poor experience because it made running tests
significantly slower, and logic tests generated for temporary logic test
files would have to be reverted repeatedly. `TestLogic_tmp` eliminates
the need for this automatic regeneration, so it has been removed. The
`--no-gen` flag, which disabled automatic regeneration, has also been
removed. When committing a new logic test file to the repository,
developers must now run `./dev gen testlogic` to generate logic tests
for it.

Release note: None
  • Loading branch information
mgartner committed Aug 1, 2022
1 parent 5fe0527 commit 1419d2e
Show file tree
Hide file tree
Showing 44 changed files with 541 additions and 34 deletions.

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

13 changes: 13 additions & 0 deletions pkg/ccl/logictestccl/tests/3node-tenant/generated_test.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

13 changes: 13 additions & 0 deletions pkg/ccl/logictestccl/tests/5node/generated_test.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

13 changes: 13 additions & 0 deletions pkg/ccl/logictestccl/tests/fakedist-disk/generated_test.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

13 changes: 13 additions & 0 deletions pkg/ccl/logictestccl/tests/fakedist-vec-off/generated_test.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

13 changes: 13 additions & 0 deletions pkg/ccl/logictestccl/tests/fakedist/generated_test.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

13 changes: 13 additions & 0 deletions pkg/ccl/logictestccl/tests/local-mixed-21.2-22.1/generated_test.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

13 changes: 13 additions & 0 deletions pkg/ccl/logictestccl/tests/local-vec-off/generated_test.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

13 changes: 13 additions & 0 deletions pkg/ccl/logictestccl/tests/local/generated_test.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

13 changes: 13 additions & 0 deletions pkg/ccl/sqlitelogictestccl/tests/3node-tenant/generated_test.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

24 changes: 0 additions & 24 deletions pkg/cmd/dev/test.go
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,6 @@ const (
stressArgsFlag = "stress-args"
raceFlag = "race"
ignoreCacheFlag = "ignore-cache"
noGenFlag = "no-gen"
rewriteFlag = "rewrite"
streamOutputFlag = "stream-output"
testArgsFlag = "test-args"
Expand Down Expand Up @@ -101,7 +100,6 @@ pkg/kv/kvserver:kvserver_test) instead.`,
testCmd.Flags().Bool(raceFlag, false, "run tests using race builds")
testCmd.Flags().Bool(ignoreCacheFlag, false, "ignore cached test runs")
testCmd.Flags().Bool(rewriteFlag, false, "rewrite test files using results from test run (only applicable to certain tests)")
testCmd.Flags().Bool(noGenFlag, false, "skip generating logic test files before running logic tests")
testCmd.Flags().Bool(streamOutputFlag, false, "stream test output during run")
testCmd.Flags().String(testArgsFlag, "", "additional arguments to pass to the go test binary")
testCmd.Flags().String(vModuleFlag, "", "comma-separated list of pattern=N settings for file-filtered logging")
Expand All @@ -123,7 +121,6 @@ func (d *dev) test(cmd *cobra.Command, commandLine []string) error {
stressCmdArgs = mustGetFlagString(cmd, stressArgsFlag)
timeout = mustGetFlagDuration(cmd, timeoutFlag)
verbose = mustGetFlagBool(cmd, vFlag)
noGen = mustGetFlagBool(cmd, noGenFlag)
changed = mustGetFlagBool(cmd, changedFlag)
showLogs = mustGetFlagBool(cmd, showLogsFlag)
count = mustGetFlagInt(cmd, countFlag)
Expand Down Expand Up @@ -219,27 +216,6 @@ func (d *dev) test(cmd *cobra.Command, commandLine []string) error {
}
args = append(args, "--test_env=GOTRACEBACK=all")

// We need to re-generate logictest files if we're testing a
// logictest target.
if !noGen {
var shouldGenerateLogicTestFiles bool
for _, testTarget := range testTargets {
for _, logicTestPath := range logicTestPaths {
if strings.Contains(testTarget, logicTestPath) {
shouldGenerateLogicTestFiles = true
break
}
}
}

if shouldGenerateLogicTestFiles {
err := d.generateLogicTest(cmd)
if err != nil {
return err
}
}
}

if rewrite {
if stress {
return fmt.Errorf("cannot combine --%s and --%s", stressFlag, rewriteFlag)
Expand Down
Loading

0 comments on commit 1419d2e

Please sign in to comment.