SqlSensor parameter fails to capture the replication lag

[amolsr]

Description

Hi,
We need to run the pipeline when there is no replication lag in MySQL database. for this we need to use SqlSensor with SHOW SLAVE STATUS command. The success and the failure callback takes the first cell only and there is no other Select query to get the Seconds_Behind_Master only. Please extend the Sensor for querying the complete dataframe.

Use case/motivation

No response

Related issues

No response

Are you willing to submit a PR?

• Yes I am willing to submit a PR!

Code of Conduct

• I agree to follow this project's Code of Conduct

[paulooctavio]

I would be happy to take this one!

[potiuk]

Assigned

[vVv-AA]

Hi @potiuk The issue seems to be abandoned. Is it ok if I work on it?

There are 2 ways we can do this change.
SqlSensor already takes a success and failure callable to which the first cell is passed.

1. A backward incompatible change and pass the whole row into the checker function. It passed checker function can then do its thing on the whole row.
2. Add a selector parameter Callable[[Tuple[Any]], bool] which would take the whole row, and its output will be passed to the checker functions. To keep backward compatibility selector can be be itemgetter(0) by default

Which one would you prefer?

[potiuk]

common.sql should be generally backwards compatible - because we use it by many operators. So 2.

[vVv-AA]

Thanks @potiuk
Have raised the PR accordingly. Please do review when time permits.