diff --git a/airbyte-ci/connectors/pipelines/README.md b/airbyte-ci/connectors/pipelines/README.md index 29ad5b73d7fc..1a48d92149dc 100644 --- a/airbyte-ci/connectors/pipelines/README.md +++ b/airbyte-ci/connectors/pipelines/README.md @@ -96,7 +96,7 @@ At this point you can run `airbyte-ci` commands from the root of the repository. #### Options | Option | Default value | Mapped environment variable | Description | -| --------------------------------------- | ------------------------------- | ----------------------------- | ------------------------------------------------------------------------------------------- | +|-----------------------------------------|---------------------------------|-------------------------------|---------------------------------------------------------------------------------------------| | `--is-local/--is-ci` | `--is-local` | | Determines the environment in which the CLI runs: local environment or CI environment. | | `--git-branch` | The checked out git branch name | `CI_GIT_BRANCH` | The git branch on which the pipelines will run. | | `--git-revision` | The current branch head | `CI_GIT_REVISION` | The commit hash on which the pipelines will run. | @@ -115,7 +115,7 @@ Available commands: #### Options | Option | Multiple | Default value | Description | -| ---------------------- | -------- | ------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +|------------------------|----------|---------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| | `--use-remote-secrets` | False | True | If True, connectors configuration will be pulled from Google Secret Manager. Requires the GCP_GSM_CREDENTIALS environment variable to be set with a service account with permission to read GSM secrets. If False the connector configuration will be read from the local connector `secrets` folder. | | `--name` | True | | Select a specific connector for which the pipeline will run. Can be used multiple time to select multiple connectors. The expected name is the connector technical name. e.g. `source-pokeapi` | | `--release-stage` | True | | Select connectors with a specific release stage: `alpha`, `beta`, `generally_available`. Can be used multiple times to select multiple release stages. | @@ -282,7 +282,7 @@ Publish all connectors modified in the head commit: `airbyte-ci connectors --mod ### Options | Option | Required | Default | Mapped environment variable | Description | -| ------------------------------------ | -------- | --------------- | ---------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +|--------------------------------------|----------|-----------------|------------------------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| | `--pre-release/--main-release` | False | `--pre-release` | | Whether to publish the pre-release or the main release version of a connector. Defaults to pre-release. For main release you have to set the credentials to interact with the GCS bucket. | | `--docker-hub-username` | True | | `DOCKER_HUB_USERNAME` | Your username to connect to DockerHub. | | `--docker-hub-password` | True | | `DOCKER_HUB_PASSWORD` | Your password to connect to DockerHub. | @@ -326,7 +326,7 @@ Validate all `metadata.yaml` files in the repo: #### Options | Option | Default | Description | -| ------------------ | ------------ | -------------------------------------------------------------------------------------------------------------------------- | +|--------------------|--------------|----------------------------------------------------------------------------------------------------------------------------| | `--modified/--all` | `--modified` | Flag to run validation of `metadata.yaml` files on the modified files in the head commit or all the `metadata.yaml` files. | ### `metadata upload` command @@ -338,7 +338,7 @@ Upload all the `metadata.yaml` files to a GCS bucket: #### Options | Option | Required | Default | Mapped environment variable | Description | -| ------------------- | -------- | ------------ | --------------------------- | ------------------------------------------------------------------------------------------------------------------------ | +|---------------------|----------|--------------|-----------------------------|--------------------------------------------------------------------------------------------------------------------------| | `--gcs-credentials` | True | | `GCS_CREDENTIALS` | Service account credentials in JSON format with permission to get and upload on the GCS bucket | | `--modified/--all` | True | `--modified` | | Flag to upload the modified `metadata.yaml` files in the head commit or all the `metadata.yaml` files to a GCS bucket. | @@ -376,7 +376,8 @@ This command runs the Python tests for a airbyte-ci poetry package. ## Changelog | Version | PR | Description | -| ------- | --------------------------------------------------------- | -------------------------------------------------------------------------------------------- | +|---------|-----------------------------------------------------------|----------------------------------------------------------------------------------------------| +| 0.2.1 | [#28897](https://github.com/airbytehq/airbyte/pull/28897) | Sentry: Ignore error logs without exceptions from reporting | | 0.2.0 | [#28857](https://github.com/airbytehq/airbyte/pull/28857) | Add the `airbyte-ci tests` command to run the test suite on any `airbyte-ci` poetry package. | | 0.1.1 | [#28858](https://github.com/airbytehq/airbyte/pull/28858) | Increase the max duration of Connector Package install to 20mn. | | 0.1.0 | | Alpha version not in production yet. All the commands described in this doc are available. | diff --git a/airbyte-ci/connectors/pipelines/pipelines/sentry_utils.py b/airbyte-ci/connectors/pipelines/pipelines/sentry_utils.py index 394663ec8ec0..5542398ebd92 100644 --- a/airbyte-ci/connectors/pipelines/pipelines/sentry_utils.py +++ b/airbyte-ci/connectors/pipelines/pipelines/sentry_utils.py @@ -12,10 +12,19 @@ def initialize(): if "SENTRY_DSN" in os.environ: sentry_sdk.init( dsn=os.environ.get("SENTRY_DSN"), + before_send=before_send, release=f"pipelines@{importlib.metadata.version('pipelines')}", ) +def before_send(event, hint): + # Ignore logged errors that do not contain an exception + if "log_record" in hint and "exc_info" not in hint: + return None + + return event + + def with_step_context(func): def wrapper(self, *args, **kwargs): with sentry_sdk.configure_scope() as scope: diff --git a/airbyte-ci/connectors/pipelines/pyproject.toml b/airbyte-ci/connectors/pipelines/pyproject.toml index a173cb5180c7..6ca7626be6e7 100644 --- a/airbyte-ci/connectors/pipelines/pyproject.toml +++ b/airbyte-ci/connectors/pipelines/pyproject.toml @@ -4,7 +4,7 @@ build-backend = "poetry.core.masonry.api" [tool.poetry] name = "pipelines" -version = "0.1.1" +version = "0.2.1" description = "Packaged maintained by the connector operations team to perform CI for connectors' pipelines" authors = ["Airbyte "]