Add BteqOperator for Executing Teradata BTEQ Scripts in Airflow #51807
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
BTEQ (Basic Teradata Query) is a command-line utility provided by Teradata for interacting with Teradata databases. The BteqOperator is designed to provide flexible and robust execution of Teradata SQL Statements and BTEQ script/SQL file using the BTEQ command-line utility within Airflow DAGs
potiuk
approved these changes
Jun 21, 2025
RoyLee1224
pushed a commit
to RoyLee1224/airflow
that referenced
this pull request
Jun 21, 2025
…he#51807) * Add BteqOperator for Executing Teradata BTEQ Scripts in Airflow (apache#58) BTEQ (Basic Teradata Query) is a command-line utility provided by Teradata for interacting with Teradata databases. The BteqOperator is designed to provide flexible and robust execution of Teradata SQL Statements and BTEQ script/SQL file using the BTEQ command-line utility within Airflow DAGs * system test failure fixed * unit tests issues fixed * unit tests issues fixed * As per airflow recommendation, removed usage of Caplog * airflow workflow issue fixed * Documentation is improved --------- Co-authored-by: Satish Ch <satishchinthanippu@gmail.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR introduces a new BteqOperator within the Airflow Teradata provider to support the execution of Teradata BTEQ (Basic Teradata Query) scripts as part of Airflow DAGs. The operator enhances integration with Teradata by enabling flexible orchestration of BTEQ-based workloads both locally and remotely via SSH.
🔧 Key Features:
✅ Support for inline SQL statements or external script files (.sql / .bteq).
✅ Local and remote execution modes with optional SSH integration.
✅ Configurable session and file encoding (e.g., UTF-8, UTF-16).
✅ Graceful handling of BTEQ return codes via bteq_quit_rc.
✅ Support for timeout handling, with optional continuation on timeout using timeout_rc.
✅ Built-in support for templated parameters via Jinja.
🛠️ Additional Enhancements:
Includes utility functions for:
File validation and encoding checks
Script preparation for remote/local execution
Robust error handling and comprehensive logging
Teradata Provider System Health Dashboard: https://teradata.github.io/airflow/
Teradata Provider documentation build status: https://github.com/Teradata/airflow/actions/runs/15688902771
Teradata Provider Unit tests build status: https://github.com/Teradata/airflow/actions/runs/15688585657
^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named
{pr_number}.significant.rstor{issue_number}.significant.rst, in airflow-core/newsfragments.