Skip to content

Conversation

@kaxil
Copy link
Member

@kaxil kaxil commented Jan 8, 2026

A new deferrable operator for executing commands on remote hosts via SSH. Unlike SSHOperator, this operator runs jobs detached on the remote host, making them resilient to network interruptions and worker restarts.

Use this operator when:

  • Running long-running jobs (minutes to hours) on remote hosts
  • Network stability is a concern
  • Jobs should survive temporary disconnections
  • You want to free up worker slots during execution

Example:

from airflow.providers.ssh.operators.ssh_remote_job import SSHRemoteJobOperator

run_script = SSHRemoteJobOperator(
    task_id="run_remote_script",
    ssh_conn_id="my_ssh_connection",
    command="/path/to/script.sh",
    poll_interval=5,
    cleanup="on_success",
)

^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named {pr_number}.significant.rst or {issue_number}.significant.rst, in airflow-core/newsfragments.

Copy link
Contributor

@BasPH BasPH left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Took me some time to understand the Bash/Powershell magic, but overall looks good! Few small comments.

@kaxil kaxil requested a review from BasPH January 9, 2026 16:08
@kaxil kaxil force-pushed the remote-ssh-op branch 2 times, most recently from 9d12906 to 69c5189 Compare January 9, 2026 20:11
Copy link
Member

@gopidesupavan gopidesupavan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks kaxil, looking great really helps this feature, have just two comments .:)

A deferrable operator for executing commands on remote hosts via SSH.
Jobs run detached and survive network interruptions, with logs streamed
incrementally back to Airflow. Supports both POSIX and Windows hosts.
@kaxil kaxil merged commit 8d28fe8 into apache:main Jan 10, 2026
129 checks passed
@kaxil kaxil deleted the remote-ssh-op branch January 10, 2026 01:56
jason810496 pushed a commit to jason810496/airflow that referenced this pull request Jan 22, 2026
…he#60297)

A deferrable operator for executing commands on remote hosts via SSH.
Jobs run detached and survive network interruptions, with logs streamed
incrementally back to Airflow. Supports both POSIX and Windows hosts.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants