-
Notifications
You must be signed in to change notification settings - Fork 85
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Exit notebook tests early on forks #1363
Conversation
Thanks for looking into this! I don't think the feature branch approach will work super well. People without write access cannot create the initial feature branch to merge into. We could set up the feature branch for them and always use the same one like Instead, if we wanted to support testing from a fork, I think we would need to use something nifty like https://github.com/imjohnbo/ok-to-test. That is a bit complex, though, and I'm skeptical it's worth the complexity because we so rarely get contributions from people without write access. Still, this PR is really valuable to error on forks because:
I'd recommend we take this approach for now - still error on forks, but reword the error message:
Another concern with erroring on forks is we won't execute the code that validates all notebooks are classified. Ideally we could validate that, even if we skip running the notebooks. The best idea I can think of is to add a CLI arg to the notebook test that will skip all execution and only validate the classification. Wdyt? |
…H/disable-tests-on-forks
I think we had similar ideas; I've edited the message to be clearer. I have an idea about the classification validation; will make a follow-up PR. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks!
Co-authored-by: Eric Arellano <14852634+Eric-Arellano@users.noreply.github.com>
Suggestion on how to handle notebook testing on forks. Resolves Qiskit#967. This PR proposes always failing the test on forks, with advice to merge to another branch in this repo to test it before merging to the main branch. While a little clunky, we very rarely get PRs from forks so this is probably simplest for now. We can review the process if it starts becoming a problem. --------- Co-authored-by: Eric Arellano <14852634+Eric-Arellano@users.noreply.github.com>
Suggestion on how to handle notebook testing on forks. Resolves #967.
This PR proposes always failing the test on forks, with advice to merge to another branch in this repo to test it before merging to the main branch. While a little clunky, we very rarely get PRs from forks so this is probably simplest for now. We can review the process if it starts becoming a problem.