Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add integration tests through GitHub Actions #178

Closed
tonowak opened this issue Nov 9, 2022 · 4 comments
Closed

Add integration tests through GitHub Actions #178

tonowak opened this issue Nov 9, 2022 · 4 comments
Assignees
Labels
C-enhancement Category: raise the bar on expectations

Comments

@tonowak
Copy link
Collaborator

tonowak commented Nov 9, 2022

Describe your use case

Some tests might require getting data about crates from e.g. crates.io. But it is better to not require internet access during cargo test (for example, I might get a failed test because of no internet access or crates.io being down, which is not the fault of my changes to the code). The solution is to run those tests through GitHub Actions. Those tests would check whether cargo-semver-check works on some examples.

Describe the solution you'd like

Described here

Alternatives, if applicable

No response

Additional Context

One of the tests would be checking whether #147 is fixed.
It is necessary to add this test before releasing #173

#174 lists two commands that could also be tested. Testing some different use-cases might help prevent such bugs in the future.

@tonowak tonowak added the C-enhancement Category: raise the bar on expectations label Nov 9, 2022
@tonowak
Copy link
Collaborator Author

tonowak commented Nov 9, 2022

@staniewzki do you want to work on that?

@tonowak
Copy link
Collaborator Author

tonowak commented Nov 9, 2022

@obi1kenobi are there other issues/PRs that could be tested in this way?

@obi1kenobi
Copy link
Owner

I think these three are a solid start for now, but there will probably be more in the future: #167 is a great candidate to add as well, for example, once we've figured out what's wrong with it.

Ideally, we'd like to be able to easily add more commands and more repos in the future. We want to balance simplicity and ease of maintenance against e.g. more features, like comparing stdout in addition to exit codes.

I'd recommend figuring out what sort of design would be good to have in the long run, then identifying the minimum set of things we can build today to satisfy as much of our needs as possible while not requiring that we build everything all at once. For example, we don't need to check stdout right now, and we care about being able to plug in these tests quickly and easily much more than stdout checking.

@tonowak
Copy link
Collaborator Author

tonowak commented Dec 20, 2022

I guess this can be closed? @obi1kenobi

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
C-enhancement Category: raise the bar on expectations
Projects
None yet
Development

No branches or pull requests

3 participants