Skip to content

Commit

Permalink
test: Qdrant integration tests and CI
Browse files Browse the repository at this point in the history
  • Loading branch information
Anush008 committed Jan 18, 2024
1 parent bc88099 commit 9245f24
Show file tree
Hide file tree
Showing 21 changed files with 260 additions and 19 deletions.
6 changes: 6 additions & 0 deletions .github/boring-cyborg.yml
Original file line number Diff line number Diff line change
Expand Up @@ -403,6 +403,12 @@ labelPRBasedOnFilePath:
- tests/providers/presto/**/*
- tests/system/providers/presto/**/*

provider:qdrant:
- airflow/providers/qdrant/**/*
- docs/apache-airflow-providers-qdrant/**/*
- tests/providers/qdrant/**/*
- tests/system/providers/qdrant/**/*

provider:redis:
- airflow/providers/redis/**/*
- docs/apache-airflow-providers-redis/**/*
Expand Down
3 changes: 3 additions & 0 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -1469,6 +1469,9 @@ jobs:
breeze testing integration-tests --integration kafka
breeze down
if: needs.build-info.outputs.is-airflow-runner != 'true'
- name: "Integration Tests Postgres: Qdrant"
run: breeze testing integration-tests --integration qdrant
if: needs.build-info.outputs.is-airflow-runner == 'true'
- name: "Integration Tests Postgres: all-testable"
run: breeze testing integration-tests --integration all-testable
if: needs.build-info.outputs.is-airflow-runner == 'true'
Expand Down
2 changes: 2 additions & 0 deletions TESTING.rst
Original file line number Diff line number Diff line change
Expand Up @@ -1232,6 +1232,8 @@ The following integrations are available:
+--------------+----------------------------------------------------+
| pinot | Integration required for Apache Pinot hooks. |
+--------------+----------------------------------------------------+
| qdrant | Integration required for Qdrant tests. |
+--------------+----------------------------------------------------+
| statsd | Integration required for Satsd hooks. |
+--------------+----------------------------------------------------+
| trino | Integration required for Trino hooks. |
Expand Down
9 changes: 9 additions & 0 deletions airflow/utils/db.py
Original file line number Diff line number Diff line change
Expand Up @@ -541,6 +541,15 @@ def create_default_connections(session: Session = NEW_SESSION):
),
session,
)
merge_conn(
Connection(
conn_id="qdrant_default",
conn_type="qdrant",
host="qdrant",
port=6333,
),
session,
)
merge_conn(
Connection(
conn_id="redis_default",
Expand Down
12 changes: 2 additions & 10 deletions dev/breeze/src/airflow_breeze/global_constants.py
Original file line number Diff line number Diff line change
Expand Up @@ -50,7 +50,7 @@
ALLOWED_BACKENDS = ["sqlite", "mysql", "postgres", "none"]
ALLOWED_PROD_BACKENDS = ["mysql", "postgres"]
DEFAULT_BACKEND = ALLOWED_BACKENDS[0]
TESTABLE_INTEGRATIONS = ["cassandra", "celery", "kerberos", "mongo", "pinot", "trino", "kafka"]
TESTABLE_INTEGRATIONS = ["cassandra", "celery", "kerberos", "mongo", "pinot", "trino", "kafka", "qdrant"]
OTHER_INTEGRATIONS = ["statsd", "otel", "openlineage"]
ALLOWED_DEBIAN_VERSIONS = ["bookworm", "bullseye"]
ALL_INTEGRATIONS = sorted(
Expand Down Expand Up @@ -365,15 +365,7 @@ def get_airflow_extras():


# Initialize integrations
AVAILABLE_INTEGRATIONS = [
"cassandra",
"kerberos",
"mongo",
"pinot",
"celery",
"statsd",
"trino",
]
AVAILABLE_INTEGRATIONS = ["cassandra", "kerberos", "mongo", "pinot", "celery", "statsd", "trino", "qdrant"]
ALL_PROVIDER_YAML_FILES = Path(AIRFLOW_SOURCES_ROOT, "airflow", "providers").rglob("provider.yaml")
PROVIDER_RUNTIME_DATA_SCHEMA_PATH = AIRFLOW_SOURCES_ROOT / "airflow" / "provider_info.schema.json"

Expand Down
2 changes: 1 addition & 1 deletion images/breeze/output-commands.svg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
2 changes: 1 addition & 1 deletion images/breeze/output_shell.svg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
2 changes: 1 addition & 1 deletion images/breeze/output_shell.txt
Original file line number Diff line number Diff line change
@@ -1 +1 @@
d0b4fa36683eaa51744c631715f9ffe5
027f3033da6265d62d91bfc37aa477a3
Loading

0 comments on commit 9245f24

Please sign in to comment.