Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

ci: use custom test-reporter action to upload job results #18004

Merged
merged 6 commits into from
Nov 1, 2022
Merged

ci: use custom test-reporter action to upload job results #18004

merged 6 commits into from
Nov 1, 2022

Conversation

c-p-b
Copy link
Contributor

@c-p-b c-p-b commented Oct 14, 2022 •
edited
Loading

What

The current iteration of test-reporter github action truncates the report entirely if it's above a certain length (~65k bytes). This is due to a limitation of github api (the checks field only supports this length). Rather than destroying the entire field, instead the purpose of this pr is to explore whether some of the information can be preserved (such as runtime and test name) and significantly reduce the total length of the report at the same time.

We want to use this (truncated) data to perform analysis on tests over time. The reason we like this approach is that we already ingest metrics for analytics using the github api connector - so it's simply a matter of adding check runs to the github api ingest if this approach works.

How

Use https://github.com/EnricoMi/publish-unit-test-result-action to write out case json and upload that json to gcs to be consumed by our analytics

Resolves half of https://github.com/airbytehq/airbyte-cloud/issues/3204

@c-p-b c-p-b changed the title ci: use custom test-reporter action to upload job results DO NOT MERGE: ci: use custom test-reporter action to upload job results Oct 14, 2022
@c-p-b c-p-b marked this pull request as ready for review October 14, 2022 18:09
@c-p-b c-p-b changed the title DO NOT MERGE: ci: use custom test-reporter action to upload job results WIP: ci: use custom test-reporter action to upload job results Oct 14, 2022
@c-p-b c-p-b force-pushed the update-test-report-action branch 2 times, most recently from 328466f to 0a87905 Compare October 14, 2022 18:19
@c-p-b c-p-b temporarily deployed to more-secrets October 14, 2022 18:32 Inactive
@c-p-b c-p-b temporarily deployed to more-secrets October 14, 2022 18:42 Inactive
@c-p-b c-p-b temporarily deployed to more-secrets October 14, 2022 19:12 Inactive
@c-p-b c-p-b temporarily deployed to more-secrets October 15, 2022 19:54 Inactive
@c-p-b c-p-b temporarily deployed to more-secrets October 15, 2022 20:43 Inactive
@c-p-b c-p-b temporarily deployed to more-secrets October 15, 2022 22:16 Inactive
@c-p-b c-p-b temporarily deployed to more-secrets October 15, 2022 22:32 Inactive
@c-p-b c-p-b temporarily deployed to more-secrets October 15, 2022 22:59 Inactive
@c-p-b c-p-b temporarily deployed to more-secrets October 15, 2022 23:21 Inactive
@c-p-b c-p-b temporarily deployed to more-secrets October 16, 2022 16:44 Inactive
@c-p-b c-p-b temporarily deployed to more-secrets October 17, 2022 20:13 Inactive
@c-p-b c-p-b force-pushed the update-test-report-action branch from ae231af to 5ada95a Compare October 17, 2022 20:21
@c-p-b c-p-b temporarily deployed to more-secrets October 17, 2022 20:23 Inactive
@c-p-b c-p-b temporarily deployed to more-secrets October 17, 2022 20:51 Inactive
@c-p-b c-p-b temporarily deployed to more-secrets October 18, 2022 15:05 Inactive
@c-p-b c-p-b temporarily deployed to more-secrets October 18, 2022 15:24 Inactive
@c-p-b c-p-b temporarily deployed to more-secrets October 18, 2022 15:40 Inactive
@c-p-b c-p-b temporarily deployed to more-secrets October 18, 2022 16:13 Inactive
@c-p-b c-p-b temporarily deployed to more-secrets October 18, 2022 16:29 Inactive
@c-p-b c-p-b temporarily deployed to more-secrets October 18, 2022 19:11 Inactive
@c-p-b c-p-b temporarily deployed to more-secrets October 18, 2022 19:32 Inactive
@c-p-b c-p-b force-pushed the update-test-report-action branch from dedc8af to 7c3a550 Compare October 18, 2022 23:15
@c-p-b c-p-b temporarily deployed to more-secrets October 18, 2022 23:17 Inactive
@c-p-b c-p-b temporarily deployed to more-secrets October 18, 2022 23:56 Inactive
@c-p-b c-p-b temporarily deployed to more-secrets October 19, 2022 01:46 Inactive
@c-p-b c-p-b temporarily deployed to more-secrets October 31, 2022 21:38 Inactive
@c-p-b c-p-b temporarily deployed to more-secrets October 31, 2022 22:25 Inactive
@c-p-b c-p-b temporarily deployed to more-secrets October 31, 2022 23:08 Inactive
@c-p-b c-p-b force-pushed the update-test-report-action branch from 1231392 to ba9b806 Compare November 1, 2022 15:37
@c-p-b c-p-b temporarily deployed to more-secrets November 1, 2022 15:39 Inactive
@c-p-b c-p-b force-pushed the update-test-report-action branch from ba9b806 to 59cc225 Compare November 1, 2022 15:59
@c-p-b c-p-b temporarily deployed to more-secrets November 1, 2022 16:02 Inactive
@c-p-b c-p-b force-pushed the update-test-report-action branch from 59cc225 to fc29dc1 Compare November 1, 2022 16:13
@c-p-b c-p-b temporarily deployed to more-secrets November 1, 2022 16:16 Inactive
@c-p-b c-p-b requested a review from gosusnp November 1, 2022 16:47
@c-p-b c-p-b temporarily deployed to more-secrets November 1, 2022 16:49 Inactive
gosusnp
gosusnp approved these changes Nov 1, 2022
View reviewed changes
tools/bin/prep_test_results_for_gcs.py Show resolved Hide resolved
.github/workflows/gradle.yml Outdated Show resolved Hide resolved
@c-p-b c-p-b temporarily deployed to more-secrets November 1, 2022 17:10 Inactive
@c-p-b c-p-b temporarily deployed to more-secrets November 1, 2022 17:23 Inactive
@c-p-b c-p-b temporarily deployed to more-secrets November 1, 2022 18:06 Inactive
@c-p-b c-p-b temporarily deployed to more-secrets November 1, 2022 18:12 Inactive
@c-p-b c-p-b merged commit dc15f56 into master Nov 1, 2022
@c-p-b c-p-b deleted the update-test-report-action branch November 1, 2022 18:59
@octavia-squidington-iii octavia-squidington-iii mentioned this pull request Nov 1, 2022
Merged
letiescanciano added a commit that referenced this pull request Nov 2, 2022
@letiescanciano
Merge branch 'master' into leti/experiment-add-source-in-signup
01be2ac 
* master: (59 commits)
  🪟🔧 Remove styled components (round 1) (#18766)
  Bump Airbyte version from 0.40.17 to 0.40.18 (#18827)
  ci: add job and run id to test reports (#18832)
  hide ConfigPersistence inside ConfigRepository to discourage use (#18803)
  Remove the bulk actions from ConfigPersistence (#18800)
  remove config persistence from seeding logic (#18749)
  Remove ConfigPersistence usage from SecretsMigrator (#18747)
  Add normalization changelog and bump normalization version in platform (#18813)
  🐛Source Exchange Rates: Fix handling error during check connection (#18726)
  🐛Destination Google Sheets: Fix empty headers list (#18729)
  Bump helm chart version reference to 0.40.40 (#18815)
  Use equalsIgnoreCase (#18810)
  [charts/airbyte-cron] Cleanup env vars (#18787)
  🐛 Source Facebook Marketing: reduce request limit after specific error (#18734)
  Parameterize test_empty_streams and test_stream_with_1_airbyte_column by destination (#18197)
  Correct coinmarket spec (#18790)
  ci: use custom test-reporter action to upload job results (#18004)
  Fix unit tests in source relational db (#18789)
  query to include data plane attributes (#18531)
  Mark/update notification settings design (#18159)
  ...
drewrasm pushed a commit to drewrasm/airbyte that referenced this pull request Nov 2, 2022
@drewrasm
ci: use custom test-reporter action to upload job results (airbytehq#…
562c99e 
…18004)

* ci: use custom action to upload job results
natalyjazzviolin pushed a commit that referenced this pull request Nov 3, 2022
ci: use custom test-reporter action to upload job results (#18004)
66a2d2d 
* ci: use custom action to upload job results
arsenlosenko pushed a commit that referenced this pull request Nov 8, 2022
@drewrasm @marcosmarxm
@xiaohansong @tuliren @octavia-squidington-iii @grishick @artem1205 @perangel @jdpgrailsdev @github-actions @xpuska513 @lazebnyi @cgardens @josephkmh @darynaishchenko @mfsiega-airbyte @tybernstein @YowanR @timroes @subodh1810 @cuyk @Amruta-Ranade
🎉 Source Pinterest: filters for entity statuses (#15074)
431ed0f 
* solve conflicts

* solve conflict in json schema

* bump to version 0.1.8 for the changes of this pr

* change ad account id in the schemas

* query to include data plane attributes (#18531)

* query to include data plane attributes

* rename functions

* fix java build

* more renaming fix

* Fix unit tests in source relational db (#18789)

* Fix unit tests

* Add extra test case for record count > 1

* Store record count in variable

* ci: use custom test-reporter action to upload job results (#18004)

* ci: use custom action to upload job results

* Correct coinmarket spec (#18790)

* correct coinmarket spec

* remove duplicate support normalization from source spec

* rollback coinmarketcap version in source def seed

* update connector version

* auto-bump connector version

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>

* Parameterize test_empty_streams and test_stream_with_1_airbyte_column by destination (#18197)

* Remove lines that always add Postgres to list of destinations
* Parameterize all tests in test_ephemeral by destination

* 🐛 Source Facebook Marketing: reduce request limit after specific error (#18734)

* 🐛 Source Facebook Marketing: reduce request limit after specific error

* 🐛 Source Facebook Marketing: bump version; update docs

* 🐛 Source Facebook Marketing: add test

* 🐛 Source Facebook Marketing: increase timeout

* [charts/airbyte-cron] Cleanup env vars (#18787)

* [charts/airbyte-cron] Cleanup env vars

* Remove unused env var

* Use equalsIgnoreCase (#18810)

* Bump helm chart version reference to 0.40.40 (#18815)

Co-authored-by: perangel <perangel@users.noreply.github.com>
Co-authored-by: Kyryl Skobylko <xpuska513@gmail.com>

* 🐛Destination Google Sheets: Fix empty headers list (#18729)

* Fix empty headers list

* Updated PR number

* Bumped version

* auto-bump connector version

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>

* 🐛Source Exchange Rates: Fix handling error during check connection (#18726)

* Fix handling error during check connection

* Updated PR number

* auto-bump connector version

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>

* Add normalization changelog and bump normalization version in platform (#18813)

* Remove ConfigPersistence usage from SecretsMigrator (#18747)

* remove config persistence from seeding logic (#18749)

* Remove the bulk actions from ConfigPersistence (#18800)

* hide ConfigPersistence inside ConfigRepository to discourage use (#18803)

* ci: add job and run id to test reports (#18832)

* Bump Airbyte version from 0.40.17 to 0.40.18 (#18827)

Co-authored-by: grishick <grishick@users.noreply.github.com>

* 🪟🔧 Remove styled components (round 1) (#18766)

* refactor EditorHeader (untested)

* refactor BaseClearView

* delete unused Subtitle

* refactor ConfirmationModal

* refactor Arrow

* refactor BulkHeader

* refactor CatalogTreeSearch

* refactor StreamFieldTable

* refactor StreamHeader

* refactor ConnectorIcon

* refactor TreeRowWrapper

* refactor DeleteBlock

* refactor EmptyResourceBlock

* revert unintended element change

* fixed acceptance tests (#18699)

* 🪟🔧 Reactor Breadcrumbs component to use anchors (#18764)

* refactor breadcrumbs to use actual links

* PR comments on styles

* increase test timeout for some webapp tests to prevent flakes (#18807)

* Remove "Filters and Segments" from Google Analytics v4 (#18508)

Filters and Segments info was incorrectly added to the Google Analytics v4 connector instead of the Google Analytics (Universal Analytics) Connector.

* Add notes about EU OAUth (#18835)

EU OAuth is not fully tested so adding a note to account for that.

* 🪟🐛 Fix: visual regression in ConnectorIcon (#18849)

* fix visual regression

* remove unused prop

* Add links to demo page (#18828)

* mysql-source:fix tinyint unsigned handling (#18619)

* mysql-source:fix tinyint unsigned handling

* update doc

* format

* upgrade version

* auto-bump connector version

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>

* 🪟 🎉 Allow environment specific sections in docs (#18829)

* Allow environment specific sections in docs

* Change syntax to lower case

* ci: replace GITHUB_OUTPUT with GITHUB_ENV on multiline variables (#18809)

* ci: replace GITHUB_OUTPUT with GITHUB_ENV on multiline variables

* ci: replace github set-ouput with new syntax in ./tools/bin/

* Add connection ID to span (#18852)

* edited connector docs (#18855)

* 🪟 🔧 Upgrade husky to 8.0.1 (#18719)

* Upgrade Husky

* Upgrade Husky

* Upgrade Husky

* Upgrade Husky

* Upgrade Husky

* ci: replace GITHUB_OUTPUT with GITHUB_ENV for multiline variables (#18857)

* Avoid NPE when adding connection ID to trace (#18856)

* Filter exit errors by operation name (#18850)

* add label

* auto-bump connector version

Co-authored-by: marcosmarxm <marcosmarxm@gmail.com>
Co-authored-by: Xiaohan Song <xiaohan@airbyte.io>
Co-authored-by: Liren Tu <tuliren@gmail.com>
Co-authored-by: Conor <cpdeethree@users.noreply.github.com>
Co-authored-by: Marcos Marx <marcosmarxm@users.noreply.github.com>
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
Co-authored-by: Greg Solovyev <grishick@users.noreply.github.com>
Co-authored-by: Artem Inzhyyants <36314070+artem1205@users.noreply.github.com>
Co-authored-by: perangel <perangel@gmail.com>
Co-authored-by: Jonathan Pearlin <jonathan@airbyte.io>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: perangel <perangel@users.noreply.github.com>
Co-authored-by: Kyryl Skobylko <xpuska513@gmail.com>
Co-authored-by: Serhii Lazebnyi <53845333+lazebnyi@users.noreply.github.com>
Co-authored-by: Charles <charles@airbyte.io>
Co-authored-by: Octavia Squidington III <90398440+octavia-squidington-iii@users.noreply.github.com>
Co-authored-by: Joey Marshment-Howell <josephkmh@users.noreply.github.com>
Co-authored-by: darynaishchenko <80129833+darynaishchenko@users.noreply.github.com>
Co-authored-by: Michael Siega <109092231+mfsiega-airbyte@users.noreply.github.com>
Co-authored-by: Tyler B <104733644+TBernstein4@users.noreply.github.com>
Co-authored-by: Yowan Ramchoreeter <26179814+YowanR@users.noreply.github.com>
Co-authored-by: Tim Roes <tim@airbyte.io>
Co-authored-by: Subodh Kant Chaturvedi <subodh1810@gmail.com>
Co-authored-by: Volodymyr Pochtar <v.pochtar@gmail.com>
Co-authored-by: Amruta Ranade <11484018+Amruta-Ranade@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@davinchia davinchia davinchia left review comments

@gosusnp gosusnp gosusnp approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@c-p-b @gosusnp @davinchia