bundle analysis: compare with stored compareSha if applicable

[JerrySentry]

When doing a comparison, try to find the base commit of the head commit based on the stored custom compareSha in the bundle report first, if not found default back to using commit.parent.

Legal Boilerplate

Look, I get it. The entity doing business as "Sentry" was incorporated in the State of Delaware in 2015 as Functional Software, Inc. In 2022 this entity acquired Codecov and as result Sentry is going to need some rights from me in order to utilize my contributions in this PR. So here's the deal: I retain all rights, title and interest in and to my contributions, and by keeping this boilerplate intact I confirm that Sentry can use, modify, copy, and redistribute my contributions, under Sentry's choice of terms.

[JerrySentry]

To be updated once Shared PR commit is merged to main

[codecov-staging]

Codecov Report

All modified and coverable lines are covered by tests ✅

✅ All tests successful. No failed tests found.

📢 Thoughts on this report? Let us know!

[codecov-qa]

❌ 1 Tests Failed:

Tests completed	Failed	Passed	Skipped
2263	1	2262	6

View the top 1 failed tests by shortest run time

graphql_api.tests.test_commit.TestCommit�test_bundle_analysis_compare_with_compare_sha

Stack Traces | 0.335s run time

self = <graphql_api.tests.test_commit.TestCommit testMethod=test_bundle_analysis_compare_with_compare_sha>
get_storage_service = <MagicMock name='get_appropriate_storage_service' id='139805424024272'>

    @patch("graphql_api.dataloader.bundle_analysis.get_appropriate_storage_service")
    def test_bundle_analysis_compare_with_compare_sha(self, get_storage_service):
        """
        This tests creates 3 commits C1 -> C2 -> C3
        C1 uses Report1, C2 and C3 uses Report2
        Normally when doing a compare of C3, it would select C2 as its parent
        then it would show no change, as expected
        However the difference is that in C3's Report2 it has the compareSha set to C1.commitid
        Now when doing comparison of C3, it would now select C1 as the parent
        therefore show correct comparison in numbers between Report1 and Report2
        """
        storage = MemoryStorageService({})
        get_storage_service.return_value = storage
    
        commit_1 = CommitFactory(
            repository=self.repo,
            commitid="6ca727b0142bf5625bb82af2555d308862063222",
        )
        commit_2 = CommitFactory(
            repository=self.repo, parent_commit_id=commit_1.commitid
        )
        commit_3 = CommitFactory(
            repository=self.repo, parent_commit_id=commit_2.commitid
        )
    
        commit_report_1 = CommitReportFactory(
            commit=commit_1,
            report_type=CommitReport.ReportType.BUNDLE_ANALYSIS,
        )
    
        commit_report_2 = CommitReportFactory(
            commit=commit_2,
            report_type=CommitReport.ReportType.BUNDLE_ANALYSIS,
        )
    
        commit_report_3 = CommitReportFactory(
            commit=commit_3,
            report_type=CommitReport.ReportType.BUNDLE_ANALYSIS,
        )
    
        with open("..../tests/samples/base_bundle_report.sqlite", "rb") as f:
            storage_path = StoragePaths.bundle_report.path(
                repo_key=ArchiveService.get_archive_hash(self.repo),
                report_key=commit_report_1.external_id,
            )
            storage.write_file(get_bucket_name(), storage_path, f)
    
        with open("..../tests/samples/head_bundle_report.sqlite", "rb") as f:
            storage_path = StoragePaths.bundle_report.path(
                repo_key=ArchiveService.get_archive_hash(self.repo),
                report_key=commit_report_2.external_id,
            )
            storage.write_file(get_bucket_name(), storage_path, f)
    
>       with open(
            "..../tests/samples/head_bundle_report_with_compare_sha_6ca727b0142bf5625bb82af2555d308862063222.sqlite",
            "rb",
        ) as f:
E       FileNotFoundError: [Errno 2] No such file or directory: '..../tests/samples/head_bundle_report_with_compare_sha_6ca727b0142bf5625bb82af2555d308862063222.sqlite'

graphql_api/tests/test_commit.py:950: FileNotFoundError

To view individual test run time comparison to the main branch, go to the Test Analytics Dashboard

[codecov-public-qa]

Test Failures Detected: Due to failing tests, we cannot provide coverage reports at this time.

❌ Failed Test Results:

Completed 2269 tests with 1 failed, 2262 passed and 6 skipped.

View the full list of failed tests

pytest

• Class name: graphql_api.tests.test_commit.TestCommit
  Test name: test_bundle_analysis_compare_with_compare_sha
  
  

  self = <graphql_api.tests.test_commit.TestCommit testMethod=test_bundle_analysis_compare_with_compare_sha>
  get_storage_service = <MagicMock name='get_appropriate_storage_service' id='139805424024272'>
  
      @patch("graphql_api.dataloader.bundle_analysis.get_appropriate_storage_service")
      def test_bundle_analysis_compare_with_compare_sha(self, get_storage_service):
          """
          This tests creates 3 commits C1 -> C2 -> C3
          C1 uses Report1, C2 and C3 uses Report2
          Normally when doing a compare of C3, it would select C2 as its parent
          then it would show no change, as expected
          However the difference is that in C3's Report2 it has the compareSha set to C1.commitid
          Now when doing comparison of C3, it would now select C1 as the parent
          therefore show correct comparison in numbers between Report1 and Report2
          """
          storage = MemoryStorageService({})
          get_storage_service.return_value = storage
      
          commit_1 = CommitFactory(
              repository=self.repo,
              commitid="6ca727b0142bf5625bb82af2555d308862063222",
          )
          commit_2 = CommitFactory(
              repository=self.repo, parent_commit_id=commit_1.commitid
          )
          commit_3 = CommitFactory(
              repository=self.repo, parent_commit_id=commit_2.commitid
          )
      
          commit_report_1 = CommitReportFactory(
              commit=commit_1,
              report_type=CommitReport.ReportType.BUNDLE_ANALYSIS,
          )
      
          commit_report_2 = CommitReportFactory(
              commit=commit_2,
              report_type=CommitReport.ReportType.BUNDLE_ANALYSIS,
          )
      
          commit_report_3 = CommitReportFactory(
              commit=commit_3,
              report_type=CommitReport.ReportType.BUNDLE_ANALYSIS,
          )
      
          with open("..../tests/samples/base_bundle_report.sqlite", "rb") as f:
              storage_path = StoragePaths.bundle_report.path(
                  repo_key=ArchiveService.get_archive_hash(self.repo),
                  report_key=commit_report_1.external_id,
              )
              storage.write_file(get_bucket_name(), storage_path, f)
      
          with open("..../tests/samples/head_bundle_report.sqlite", "rb") as f:
              storage_path = StoragePaths.bundle_report.path(
                  repo_key=ArchiveService.get_archive_hash(self.repo),
                  report_key=commit_report_2.external_id,
              )
              storage.write_file(get_bucket_name(), storage_path, f)
      
  >       with open(
              "..../tests/samples/head_bundle_report_with_compare_sha_6ca727b0142bf5625bb82af2555d308862063222.sqlite",
              "rb",
          ) as f:
  E       FileNotFoundError: [Errno 2] No such file or directory: '..../tests/samples/head_bundle_report_with_compare_sha_6ca727b0142bf5625bb82af2555d308862063222.sqlite'
  
  graphql_api/tests/test_commit.py:950: FileNotFoundError

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 96.16%. Comparing base (c84212a) to head (baf0eff).
Report is 1 commits behind head on main.

✅ All tests successful. No failed tests found.

Additional details and impacted files

@@             Coverage Diff             @@
##               main       #754   +/-   ##
===========================================
  Coverage   96.16000   96.16000           
===========================================
  Files           814        814           
  Lines         18424      18425    +1     
===========================================
+ Hits          17717      17718    +1     
  Misses          707        707           

Flag	Coverage Δ
unit	91.91% <100.00%> (+<0.01%)	⬆️
unit-latest-uploader	91.91% <100.00%> (+<0.01%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[JerrySentry]

This never had a filter by repo ugh.

[adrian-codecov]

Lgtm

Once we finish the coding side of things, it might be useful documenting this with some diagrams or something so that it's pretty clear how the BA comparison logic would differ from the coverage logic.