Add BteqOperator for Executing Teradata BTEQ Scripts in Airflow #58
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
snreddygopu
reviewed
Jun 13, 2025
| with ( | ||
| self.preferred_temp_directory() as tmp_dir, | ||
| ): | ||
| # Preparing lo |
Collaborator
There was a problem hiding this comment.
It looks like the above comment is incomplete. Could you please review and update it.
sc250072
added a commit
that referenced
this pull request
Jun 24, 2025
…he#51807) * Add BteqOperator for Executing Teradata BTEQ Scripts in Airflow (#58) BTEQ (Basic Teradata Query) is a command-line utility provided by Teradata for interacting with Teradata databases. The BteqOperator is designed to provide flexible and robust execution of Teradata SQL Statements and BTEQ script/SQL file using the BTEQ command-line utility within Airflow DAGs * system test failure fixed * unit tests issues fixed * unit tests issues fixed * As per airflow recommendation, removed usage of Caplog * airflow workflow issue fixed * Documentation is improved --------- Co-authored-by: Satish Ch <satishchinthanippu@gmail.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR introduces a new BteqOperator within the Airflow Teradata provider to support the execution of Teradata BTEQ (Basic Teradata Query) scripts as part of Airflow DAGs. The operator enhances integration with Teradata by enabling flexible orchestration of BTEQ-based workloads both locally and remotely via SSH.
🔧 Key Features:
✅ Support for inline SQL statements or external script files (.sql / .bteq).
✅ Local and remote execution modes with optional SSH integration.
✅ Configurable session and file encoding (e.g., UTF-8, UTF-16).
✅ Graceful handling of BTEQ return codes via bteq_quit_rc.
✅ Support for timeout handling, with optional continuation on timeout using timeout_rc.
✅ Built-in support for templated parameters via Jinja.
🛠️ Additional Enhancements:
Includes utility functions for:
File validation and encoding checks
Script preparation for remote/local execution
Robust error handling and comprehensive logging
Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named
{pr_number}.significant.rstor{issue_number}.significant.rst, in airflow-core/newsfragments.