Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: test mode #123

Merged
merged 23 commits into from
May 10, 2024
Merged

feat: test mode #123

merged 23 commits into from
May 10, 2024

Conversation

hay-kot
Copy link
Owner

@hay-kot hay-kot commented May 7, 2024

No description provided.

@hay-kot hay-kot linked an issue May 7, 2024 that may be closed by this pull request
export SCAFFOLD_OUT="gen"
export SCAFFOLD_DIR=".scaffold,.examples"

go run main.go new --test cli

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we'll need more than one test case. Let's say this example had a couple of "features". Then I would need to run new to scaffold out several entries in the matrix like {featureA, featureB, featureA+B} to have test coverage for mistakes in the scaffold that can only be observed when those features interact in the generated app, as well as verify each feature works in isolation.

So maybe the yaml provides named test cases? Then I would reference the names here like --test=case1?

@hay-kot
Copy link
Owner Author

hay-kot commented May 8, 2024

@alexeagle, took another stab with a different API.

  • Instead of the test key we'd now use a tests key that supports multiple key entries
tests:
  default:
     Var1: val1
     Var2: val2
  test2: 
     Var1: val1
     Var2: val2
  • Different command for tests instead of using a flag, so scaffold test ...
  • Allow for rendering into an in memory file system
  • Allow for outputting an AST (this is how we do our internal testing)
  • Allow for running specific cases with --case flag

Example AST Output

scaffold test --log-level="panic" --case="default" --memfs --ast <scaffold-name>
scaffold-test-3811:  (type=dir)
        main.go:  (type=file)
                package main

                import (
                        "fmt"
                )

                func main() {
                        fmt.Println("Hello, World!")
                }

@alexeagle
Copy link

Neat! Thanks :)

An idea for generalizing just a bit:

Since the tests block appears in the same scaffold file that users might inspect to see "what prompts does this ask", do you think this feature might also be useful for users to discover the tests, but not use them to assert correctness. Instead the user could skip some of the prompts by choosing "canned" answers for some or all of them. In that case just renaming tests to presets would make this appear more useful for that use case.

Personally I wouldn't add a new test command here, because you're not giving a way for me to exercise the system-under-test nor make assertions. In the "Arrange, Act, Assert" model, this only provides "Arrange". I think the new command with a flag would keep complexity lower, with proposal above

scaffold new https://github.com/hay-kot/scaffold-go-cli --preset my-preset-1

Copy link

@alexeagle alexeagle left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Neat! excited to try this out :)

@hay-kot
Copy link
Owner Author

hay-kot commented May 10, 2024

I'm pretty happy with how the API turned out. Going to write some better documentation and write a few more tests this weekend so hopefully it'll be merged and released by Monday.

Thanks for all your feedback, it was very helpful!

@hay-kot hay-kot merged commit fb10bda into main May 10, 2024
1 check passed
@hay-kot hay-kot deleted the feat/test-mode branch May 10, 2024 19:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Guidance on how to test scaffolds
2 participants