Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

📖 Reference new build system docs #22226

Merged
merged 2 commits into from
Feb 1, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 7 additions & 5 deletions airbyte-integrations/builds-readme.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,8 @@
# How is the builds page generated?
[This github action](https://github.com/airbytehq/airbyte/blob/master/.github/workflows/connector_integration_tests.yml) runs once a day.
It triggers builds for all connectors. The result of this build is persisted on an S3 bucket in our AWS dev environment.
Ask for access rights from the Airbyte team if you need it to access the environment. This S3 bucket is hooked up to a CDN, AWS CloudFront.
# How is the builds page generated?
[This github action](https://github.com/airbytehq/airbyte/blob/master/.github/workflows/connector_integration_tests.yml) runs once a day.
It triggers builds for all connectors. The result of this build is persisted on an S3 bucket in our AWS dev environment.
Ask for access rights from the Airbyte team if you need it to access the environment. This S3 bucket is hooked up to a CDN, AWS CloudFront.
Then, [this Github action](https://github.com/airbytehq/airbyte/blob/master/.github/workflows/build-report.yml) reads from Cloudfront to send a daily
Slack message informing us of the build status of any connectors in need of attention.
Slack message informing us of the build status of any connectors in need of attention.

To understand this system in more depth read the documentation here: https://internal-docs.airbyte.io/Generated-Reports/Build-Status-Reports
1 change: 1 addition & 0 deletions airbyte-integrations/builds.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
# Build statuses
<!-- Docs on how this system works: https://internal-docs.airbyte.io/Generated-Reports/Build-Status-Reports -->

# Sources

Expand Down
5 changes: 5 additions & 0 deletions tools/bin/build_report.py
Original file line number Diff line number Diff line change
Expand Up @@ -3,8 +3,13 @@
#

"""
Report Connector Build Status to Slack

All invocations of this script must be run from the Airbyte repository root.

BEFORE RUNNING THIS SCRIPT:
- Ensure you have read the documentation on how this system works: https://internal-docs.airbyte.io/Generated-Reports/Build-Status-Reports
Comment on lines +10 to +11
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍 I think it's fine to reference internal docs in the OSS repo. This part of the codebase is tooling.


To Run tests:
pytest ./tools/bin/build_report.py

Expand Down
4 changes: 4 additions & 0 deletions tools/ci_connector_ops/ci_connector_ops/qa_engine/inputs.py
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,10 @@ def from_string(cls, string_value: Optional[str]) -> "BUILD_STATUSES":


def get_connector_build_output_url(connector_technical_name: str, connector_version: str) -> str:
"""
Get the connector build output url.
Documentation of the larger build output system can be found here: https://internal-docs.airbyte.io/Generated-Reports/Build-Status-Reports
"""
return f"{CONNECTOR_BUILD_OUTPUT_URL}/{connector_technical_name}/version-{connector_version}.json"

def fetch_latest_build_status_for_connector_version(connector_technical_name: str, connector_version: str) ->BUILD_STATUSES:
Expand Down
4 changes: 4 additions & 0 deletions tools/status/report.sh
Original file line number Diff line number Diff line change
@@ -1,5 +1,9 @@
#!/usr/bin/env bash

# This script is used to report the status of a connector build to S3.
# BEFORE RUNNING THIS SCRIPT:
# - Ensure you have read the documentation on how this system works: https://internal-docs.airbyte.io/Generated-Reports/Build-Status-Reports

set -e

. tools/lib/lib.sh
Expand Down