Skip to content

Fix flaky Postgres tests #8613

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
4 tasks done
mtrezza opened this issue Jun 4, 2023 · 9 comments · Fixed by #9210
Closed
4 tasks done

Fix flaky Postgres tests #8613

mtrezza opened this issue Jun 4, 2023 · 9 comments · Fixed by #9210
Labels
bounty:$50 Bounty applies for fixing this issue (Parse Bounty Program) state:released Released as stable version state:released-alpha Released as alpha version state:released-beta Released as beta version type:ci CI related issue

Comments

@mtrezza
Copy link
Member

mtrezza commented Jun 4, 2023

New Issue Checklist

Issue Description

The flaky Postgres tests are currently the most significant bottleneck when it comes to merging PRs:

  • More reviewer effort required to restart failing tests
  • Longer wait time for contributors to see PRs getting merged
  • Increased resource costs (GitHub Action compute time)

While there were other CI/CD related bottlenecks in the past, they have largely been removed over time so that this issue became now the most impactful one.

Solution

Fix the flakiness.

@parse-github-assistant
Copy link

parse-github-assistant bot commented Jun 4, 2023

Thanks for opening this issue!

  • 🚀 You can help us to fix this issue faster by opening a pull request with a failing test. See our Contribution Guide for how to make a pull request, or read our New Contributor's Guide if this is your first time contributing.

@mtrezza mtrezza added type:ci CI related issue bounty:$50 Bounty applies for fixing this issue (Parse Bounty Program) labels Jun 4, 2023
@mtrezza
Copy link
Member Author

mtrezza commented Jul 17, 2024

@dplewis not sure if that helps with fixing the flaky tests for Postgres, but since we disabled test randomization, there seems to be a recurring pattern of Postgres failing exactly like this: https://github.com/parse-community/parse-server/actions/runs/9973005229/job/27557477165?pr=9209

@dplewis
Copy link
Member

dplewis commented Jul 17, 2024

Is this for a specific Postgres version? Can I pick up this bounty?

@mtrezza
Copy link
Member Author

mtrezza commented Jul 17, 2024

I believe I have seen the issue always occurring for the same Postgres version since we disabled the test randomization.

The issue has been created for all flaky tests that are specific to Postgres, but I'm unsure how vast that is, now that the randomizer is off. I guess if you could fix the flaky test from the log above we can close this issue and pay out the bounty, as w/o randomizer it seems to be the only flaky test.

@dplewis
Copy link
Member

dplewis commented Jul 17, 2024

This test has always been flaky since the introduction of transactions. I would have fixed it years ago if I knew how it worked. Probably @davimacedo knows how to fix it. Also this isn't postgres specific either this can fail for mongo as well.

Screenshot 2024-07-16 at 6 13 13 PM

@mtrezza
Copy link
Member Author

mtrezza commented Jul 17, 2024

Interestingly, since we disabled the randomizer it seems to be only that specific Postgres version where it fails.

@parseplatformorg
Copy link
Contributor

🎉 This change has been released in version 7.3.0-alpha.3

@parseplatformorg parseplatformorg added the state:released-alpha Released as alpha version label Jul 17, 2024
@parseplatformorg
Copy link
Contributor

🎉 This change has been released in version 7.3.0-beta.1

@parseplatformorg parseplatformorg added the state:released-beta Released as beta version label Oct 3, 2024
@parseplatformorg
Copy link
Contributor

🎉 This change has been released in version 7.3.0

@parseplatformorg parseplatformorg added the state:released Released as stable version label Oct 3, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bounty:$50 Bounty applies for fixing this issue (Parse Bounty Program) state:released Released as stable version state:released-alpha Released as alpha version state:released-beta Released as beta version type:ci CI related issue
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants