Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[CI] Run failing tests in CI, without turning CI red #20775

Closed
wants to merge 3 commits into from

Conversation

hramos
Copy link
Contributor

@hramos hramos commented Aug 21, 2018

We have several disabled tests in Circle, and they are not running at all.
This prevents us from seeing when a disabled test might actually be fixed, as enabling the test requires uncommenting the correct line in Circle CI's config.

In this PR, we use the existing swallow_error script to run known-failing steps, without failing the job. This will let us see the step's output in CI, without polluting PRs that have not introduced new failures to CI.

@facebook-github-bot facebook-github-bot added CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. Import Started This pull request has been imported. This does not imply the PR has been approved. labels Aug 21, 2018
Copy link
Contributor

@facebook-github-bot facebook-github-bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hramos is landing this pull request. If you are a Facebook employee, you can view this diff on Phabricator.

@react-native-bot react-native-bot added Core Team Missing Test Plan This PR appears to be missing a test plan. Missing Changelog This PR appears to be missing a changelog, or they are incorrectly formatted. labels Aug 22, 2018
@facebook-github-bot facebook-github-bot added Import Failed and removed Import Started This pull request has been imported. This does not imply the PR has been approved. labels Aug 22, 2018
@facebook-github-bot
Copy link
Contributor

I tried to merge this pull request into the Facebook internal repo but some checks failed. To unblock yourself please check the following: Does this pull request pass all open source tests on GitHub? If not please fix those. Does the code still apply cleanly on top of GitHub master? If not can please rebase. In all other cases this means some internal test failed, for example a part of a fb app won't work with this pull request. I've added the Import Failed label to this pull request so it is easy for someone at fb to find the pull request and check what failed. If you don't see anyone comment in a few days feel free to comment mentioning one of the core contributors to the project so they get a notification.

@react-native-bot
Copy link
Collaborator

This pull request was closed by @hramos in 6af3b16.

Once this commit is added to a release, you will see the corresponding version tag below the description at 6af3b16. If the commit has a single master tag, it is not yet part of a release.

@facebook facebook locked as resolved and limited conversation to collaborators Aug 22, 2018
@react-native-bot react-native-bot added the Merged This PR has been merged. label Aug 22, 2018
hramos added a commit that referenced this pull request Aug 22, 2018
Summary:
We have several disabled tests in Circle, and they are not running at all.
This prevents us from seeing when a disabled test might actually be fixed, as enabling the test requires uncommenting the correct line in Circle CI's config.

In this PR, we use the existing swallow_error script to run known-failing steps, without failing the job. This will let us see the step's output in CI, without polluting PRs that have not introduced new failures to CI.
Pull Request resolved: #20775

Differential Revision: D9442412

Pulled By: hramos

fbshipit-source-id: 83c930811a559fdcf6d7b926b4073343e862d2b3
aleclarson pushed a commit to aleclarson/react-native that referenced this pull request Sep 16, 2018
Summary:
We have several disabled tests in Circle, and they are not running at all.
This prevents us from seeing when a disabled test might actually be fixed, as enabling the test requires uncommenting the correct line in Circle CI's config.

In this PR, we use the existing swallow_error script to run known-failing steps, without failing the job. This will let us see the step's output in CI, without polluting PRs that have not introduced new failures to CI.
Pull Request resolved: facebook#20775

Differential Revision: D9442412

Pulled By: hramos

fbshipit-source-id: 83c930811a559fdcf6d7b926b4073343e862d2b3
t-nanava pushed a commit to microsoft/react-native-macos that referenced this pull request Jun 17, 2019
Summary:
We have several disabled tests in Circle, and they are not running at all.
This prevents us from seeing when a disabled test might actually be fixed, as enabling the test requires uncommenting the correct line in Circle CI's config.

In this PR, we use the existing swallow_error script to run known-failing steps, without failing the job. This will let us see the step's output in CI, without polluting PRs that have not introduced new failures to CI.
Pull Request resolved: facebook#20775

Differential Revision: D9442412

Pulled By: hramos

fbshipit-source-id: 83c930811a559fdcf6d7b926b4073343e862d2b3
t-nanava pushed a commit to microsoft/react-native-macos that referenced this pull request Jun 17, 2019
Summary:
We have several disabled tests in Circle, and they are not running at all.
This prevents us from seeing when a disabled test might actually be fixed, as enabling the test requires uncommenting the correct line in Circle CI's config.

In this PR, we use the existing swallow_error script to run known-failing steps, without failing the job. This will let us see the step's output in CI, without polluting PRs that have not introduced new failures to CI.
Pull Request resolved: facebook#20775

Differential Revision: D9442412

Pulled By: hramos

fbshipit-source-id: 83c930811a559fdcf6d7b926b4073343e862d2b3
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. Merged This PR has been merged. Missing Changelog This PR appears to be missing a changelog, or they are incorrectly formatted. Missing Test Plan This PR appears to be missing a test plan.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants