Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: make test analytics GQL queries use gte #919

Merged
merged 2 commits into from
Oct 22, 2024
Merged

Conversation

joseph-sentry
Copy link
Contributor

we want to include the last day of the interval when we fetch test results and calculate aggregates.

we also want to remove the interval
date from the calculation of the percent changes.

we want to include the last day of the interval when we fetch test
results and calculate aggregates.

we also want to remove the interval
date from the calculation of the percent changes.
@joseph-sentry joseph-sentry requested a review from a team as a code owner October 22, 2024 20:46
@codecov-staging
Copy link

codecov-staging bot commented Oct 22, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

✅ All tests successful. No failed tests found.

📢 Thoughts on this report? Let us know!

Copy link

codecov bot commented Oct 22, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 96.25%. Comparing base (f8ac094) to head (8087396).
Report is 1 commits behind head on main.

✅ All tests successful. No failed tests found.

Additional details and impacted files
@@           Coverage Diff           @@
##             main     #919   +/-   ##
=======================================
  Coverage   96.25%   96.25%           
=======================================
  Files         823      823           
  Lines       18999    18999           
=======================================
  Hits        18288    18288           
  Misses        711      711           
Flag Coverage Δ
unit 92.52% <100.00%> (ø)
unit-latest-uploader 92.52% <100.00%> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@codecov-qa
Copy link

codecov-qa bot commented Oct 22, 2024

❌ 1 Tests Failed:

Tests completed Failed Passed Skipped
2633 1 2632 6
View the top 1 failed tests by shortest run time
graphql_api.tests.test_flake_aggregates.TestResultTestCase test_fetch_test_result_total_runtime
Stack Traces | 0.562s run time
self = &lt;graphql_api.tests.test_flake_aggregates.TestResultTestCase testMethod=test_fetch_test_result_total_runtime&gt;

    def test_fetch_test_result_total_runtime(self) -&gt; None:
        query = """
            query {
               owner(username: "%s") {
                    repository(name: "%s") {
                        ... on Repository {
                            testAnalytics {
                                flakeAggregates {
                                    flakeRate
                                    flakeCount
                                    flakeRatePercentChange
                                    flakeCountPercentChange
                                }
                            }
                        }
                    }
                 }
            }
        """ % (self.owner.username, self.repository.name)
    
        result = self.gql_request(query, owner=self.owner)
    
        assert "errors" not in result
&gt;       assert result["owner"]["repository"]["testAnalytics"]["flakeAggregates"] == {
            "flakeRate": 0.1,
            "flakeCount": 30,
            "flakeRatePercentChange": -33.33333,
            "flakeCountPercentChange": 100.0,
        }
E       AssertionError: assert {'flakeCount'...hange': 31.25} == {'flakeCount'...e': -33.33333}
E         
E         Differing items:
E         {'flakeRatePercentChange': 31.25} != {'flakeRatePercentChange': -33.33333}
E         {'flakeCount': 31} != {'flakeCount': 30}
E         {'flakeRate': 0.140625} != {'flakeRate': 0.1}
E         {'flakeCountPercentChange': 106.66667} != {'flakeCountPercentChange': 100.0}
E         Use -v to get more diff

graphql_api/tests/test_flake_aggregates.py:81: AssertionError

To view individual test run time comparison to the main branch, go to the Test Analytics Dashboard

Copy link

Test Failures Detected: Due to failing tests, we cannot provide coverage reports at this time.

❌ Failed Test Results:

Completed 2639 tests with 1 failed, 2632 passed and 6 skipped.

View the full list of failed tests

pytest

  • Class name: graphql_api.tests.test_flake_aggregates.TestResultTestCase
    Test name: test_fetch_test_result_total_runtime

    self = <graphql_api.tests.test_flake_aggregates.TestResultTestCase testMethod=test_fetch_test_result_total_runtime>

    def test_fetch_test_result_total_runtime(self) -> None:
    query = """
    query {
    owner(username: "%s") {
    repository(name: "%s") {
    ... on Repository {
    testAnalytics {
    flakeAggregates {
    flakeRate
    flakeCount
    flakeRatePercentChange
    flakeCountPercentChange
    }
    }
    }
    }
    }
    }
    """ % (self.owner.username, self.repository.name)

    result = self.gql_request(query, owner=self.owner)

    assert "errors" not in result
    > assert result["owner"]["repository"]["testAnalytics"]["flakeAggregates"] == {
    "flakeRate": 0.1,
    "flakeCount": 30,
    "flakeRatePercentChange": -33.33333,
    "flakeCountPercentChange": 100.0,
    }
    E AssertionError: assert {'flakeCount'...hange': 31.25} == {'flakeCount'...e': -33.33333}
    E
    E Differing items:
    E {'flakeRatePercentChange': 31.25} != {'flakeRatePercentChange': -33.33333}
    E {'flakeCount': 31} != {'flakeCount': 30}
    E {'flakeRate': 0.140625} != {'flakeRate': 0.1}
    E {'flakeCountPercentChange': 106.66667} != {'flakeCountPercentChange': 100.0}
    E Use -v to get more diff

    graphql_api/tests/test_flake_aggregates.py:81: AssertionError

@joseph-sentry joseph-sentry added this pull request to the merge queue Oct 22, 2024
Merged via the queue into main with commit 8562c38 Oct 22, 2024
18 of 19 checks passed
@joseph-sentry joseph-sentry deleted the joseph/fix-interval branch October 22, 2024 21:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants