[AIRFLOW-4147] Operator to publish event to Redis #4967

mans2singh · 2019-03-24T18:27:46Z

Make sure you have checked all steps below.

Jira

My PR addresses the following Airflow Jira issues and references them in the PR title. For example, "[AIRFLOW-XXX] My Airflow PR"
- https://issues.apache.org/jira/browse/AIRFLOW-4147
- In case you are fixing a typo in the documentation you can prepend your commit with [AIRFLOW-XXX], code changes always need a Jira issue.
- In case you are proposing a fundamental code change, you need to create an Airflow Improvement Proposal (AIP).
- In case you are adding a dependency, check if the license complies with the ASF 3rd Party License Policy.

Description

Here are some details about my PR, including screenshots of any UI changes: Redis publish operator

Tests

My PR adds the following unit tests: tests/contrib/operators/test_redis_publish_operator.py

Commits

My commits all reference Jira issues in their subject lines, and I have squashed multiple commits if they address the same issue. In addition, my commits follow the guidelines from "How to write a good git commit message":
1. Subject is separated from body by a blank line
2. Subject is limited to 50 characters (not including Jira issue reference)
3. Subject does not end with a period
4. Subject uses the imperative mood ("add", not "adding")
5. Body wraps at 72 characters
6. Body explains "what" and "why", not "how"

Documentation

In case of new functionality, my PR adds documentation that describes how to use it.
- When adding new operators/hooks/sensors, the autoclass documentation generation needs to be added.
- All the public functions and the classes in the PR contain docstrings that explain what it does
- If you implement backwards incompatible changes, please leave a note in the Updating.md so we can assign it to a appropriate release

Code Quality

Passes flake8

tests/contrib/operators/test_redis_publish_operator.py

mans2singh · 2019-03-25T21:51:25Z

@XD-DENG - I've updated the test based on your feedback. Please let me know if you have any additional advice. Thanks

mans2singh · 2019-03-27T21:32:07Z

@XD-DENG @zhongjiajie - Please let me know if you have more comments/recommendations and should I resolve your comments. Thanks

airflow/contrib/operators/redis_publish_operator.py

codecov-io · 2019-03-28T17:03:10Z

Codecov Report

Merging #4967 into master will increase coverage by 2.11%.
The diff coverage is 100%.

@@            Coverage Diff             @@
##           master    #4967      +/-   ##
==========================================
+ Coverage   76.36%   78.48%   +2.11%     
==========================================
  Files         471      468       -3     
  Lines       30290    29782     -508     
==========================================
+ Hits        23130    23373     +243     
+ Misses       7160     6409     -751

Impacted Files	Coverage Δ
...irflow/contrib/operators/redis_publish_operator.py	`100% <100%> (ø)`
airflow/hooks/hdfs_hook.py	`30% <0%> (-62.5%)`	⬇️
airflow/utils/decorators.py	`85.41% <0%> (-7.69%)`	⬇️
airflow/contrib/kubernetes/kube_client.py	`73.91% <0%> (-5.4%)`	⬇️
airflow/utils/timeout.py	`75% <0%> (-3.58%)`	⬇️
airflow/contrib/hooks/pinot_hook.py	`80% <0%> (-2.5%)`	⬇️
airflow/task/task_runner/base_task_runner.py	`74.57% <0%> (-2.1%)`	⬇️
airflow/utils/timezone.py	`89.58% <0%> (-1.73%)`	⬇️
airflow/example_dags/example_xcom.py	`60.86% <0%> (-1.64%)`	⬇️
airflow/api/__init__.py	`72.22% <0%> (-1.47%)`	⬇️
... and 130 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update b93f264...e17a4fe. Read the comment docs.

mans2singh · 2019-04-03T20:36:25Z

Hi: Just checking if you have any additional review comments/advice. Thanks

zhongjiajie · 2019-04-04T09:23:55Z

Hi, @XD-DENG could you please take a look at this PR thanks, @mans2singh ask committer review.

mans2singh · 2019-04-07T02:23:42Z

Hi @Fokko @mik-laj @XD-DENG - Please let me know if there are any recommendations for this PR. Thanks

mans2singh · 2019-04-11T03:21:19Z

@Fokko @mik-laj @XD-DENG - Please let me know if you have any additional review recommendations for this PR. Mans

Fokko

One small comment, apart from that LGTM. Sorry @mans2singh for taking so long to response, I've been a bit busy lately.

airflow/contrib/operators/redis_publish_operator.py

mans2singh · 2019-04-14T20:31:19Z

@Fokko - I've updated the code based on your feedback. Please let me know if there is anything else required. Thanks for your advice/guidance.

Additional tests have been added

mans2singh · 2019-04-16T21:40:39Z

@zhongjiajie @XD-DENG @mik-laj - Please let me know if you have any more recommendations for this pull request. Thanks.

mans2singh · 2019-04-19T11:29:40Z

@Fokko, Airflow Team - Please let me know if there is any additional advice for this PR. Thanks.

airflow/contrib/operators/redis_publish_operator.py

mans2singh · 2019-04-24T06:06:54Z

@mik-laj @XD-DENG @Fokko - Please let me know if you have any additional advice/comments on this pull request. Thanks

zhongjiajie

2 cent, otherwise LGTM

airflow/contrib/operators/redis_publish_operator.py

mans2singh · 2019-04-24T16:32:20Z

@zhongjiajie - I've updated the code based on your feedback. Please let me know if you have any more advice. Thanks

zhongjiajie · 2019-04-24T16:34:48Z

LGTM

mans2singh · 2019-04-24T21:38:13Z

@zhongjiajie - thanks for your feedback.

mans2singh · 2019-04-29T10:30:54Z

@XD-DENG and Airflow team - Please let me know if there is anything else outstanding that I need to address for this PR. Thanks.

mans2singh · 2019-05-16T10:39:23Z

@zhongjiajie @XD-DENG @mik-laj @Fokko @ashb @r39132- Just checking if you have any additional recommendations/comments for this PR.

Thanks

Mans

kaxil · 2019-05-24T20:19:28Z

Thank You @mans2singh , well done.

(cherry picked from commit ad832af)

mans2singh · 2019-05-25T18:31:05Z

Thanks @kaxil @zhongjiajie @XD-DENG @mik-laj @Fokko @ashb @r39132 @potiuk for your advice and help.

(cherry picked from commit ad832af)

[AIRFLOW-4147] Operator to publish event to Redis

83f4623

XD-DENG previously requested changes Mar 25, 2019

View reviewed changes

tests/contrib/operators/test_redis_publish_operator.py Show resolved Hide resolved

tests/contrib/operators/test_redis_publish_operator.py Outdated Show resolved Hide resolved

[AIRFLOW-4147] Added validation for test

100c693

zhongjiajie reviewed Mar 28, 2019

View reviewed changes

airflow/contrib/operators/redis_publish_operator.py Outdated Show resolved Hide resolved

airflow/contrib/operators/redis_publish_operator.py Outdated Show resolved Hide resolved

[AIRFLOW-4147] Updated logging and removed return

f487d99

Fokko reviewed Apr 14, 2019

View reviewed changes

airflow/contrib/operators/redis_publish_operator.py Outdated Show resolved Hide resolved

[AIRFLOW-4147] Updated logging based on review feedback

a90520d

Fokko approved these changes Apr 14, 2019

View reviewed changes

mik-laj reviewed Apr 19, 2019

View reviewed changes

airflow/contrib/operators/redis_publish_operator.py Show resolved Hide resolved

[AIRFLOW-4147] Added empty line in docstring based on review feedback

041ae59

zhongjiajie reviewed Apr 24, 2019

View reviewed changes

airflow/contrib/operators/redis_publish_operator.py Outdated Show resolved Hide resolved

airflow/contrib/operators/redis_publish_operator.py Outdated Show resolved Hide resolved

[AIRFLOW-4147] Updated based on review comments

e17a4fe

zhongjiajie approved these changes Apr 24, 2019

View reviewed changes

mik-laj approved these changes Apr 25, 2019

View reviewed changes

kaxil merged commit ad832af into apache:master May 24, 2019

kaxil pushed a commit that referenced this pull request May 24, 2019

[AIRFLOW-4147] Add Operator to publish event to Redis (#4967)

cf65343

(cherry picked from commit ad832af)

kaxil pushed a commit that referenced this pull request May 25, 2019

[AIRFLOW-4147] Add Operator to publish event to Redis (#4967)

7ccde89

(cherry picked from commit ad832af)

andriisoldatenko pushed a commit to andriisoldatenko/airflow that referenced this pull request Jul 26, 2019

[AIRFLOW-4147] Add Operator to publish event to Redis (apache#4967)

4791ca9

wmorris75 pushed a commit to modmed/incubator-airflow that referenced this pull request Jul 29, 2019

[AIRFLOW-4147] Add Operator to publish event to Redis (apache#4967)

9ac40a7

dharamsk pushed a commit to postmates/airflow that referenced this pull request Aug 8, 2019

[AIRFLOW-4147] Add Operator to publish event to Redis (apache#4967)

0144fe9

(cherry picked from commit ad832af)

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[AIRFLOW-4147] Operator to publish event to Redis #4967

[AIRFLOW-4147] Operator to publish event to Redis #4967

mans2singh commented Mar 24, 2019

mans2singh commented Mar 25, 2019

mans2singh commented Mar 27, 2019

codecov-io commented Mar 28, 2019 •

edited

Loading

mans2singh commented Apr 3, 2019

zhongjiajie commented Apr 4, 2019

mans2singh commented Apr 7, 2019

mans2singh commented Apr 11, 2019

Fokko left a comment

mans2singh commented Apr 14, 2019

mans2singh commented Apr 16, 2019

mans2singh commented Apr 19, 2019 •

edited

Loading

mans2singh commented Apr 24, 2019

zhongjiajie left a comment

mans2singh commented Apr 24, 2019

zhongjiajie commented Apr 24, 2019

mans2singh commented Apr 24, 2019

mans2singh commented Apr 29, 2019

mans2singh commented May 16, 2019

kaxil commented May 24, 2019

mans2singh commented May 25, 2019

[AIRFLOW-4147] Operator to publish event to Redis #4967

[AIRFLOW-4147] Operator to publish event to Redis #4967

Conversation

mans2singh commented Mar 24, 2019

Jira

Description

Tests

Commits

Documentation

Code Quality

mans2singh commented Mar 25, 2019

mans2singh commented Mar 27, 2019

codecov-io commented Mar 28, 2019 • edited Loading

Codecov Report

mans2singh commented Apr 3, 2019

zhongjiajie commented Apr 4, 2019

mans2singh commented Apr 7, 2019

mans2singh commented Apr 11, 2019

Fokko left a comment

Choose a reason for hiding this comment

mans2singh commented Apr 14, 2019

mans2singh commented Apr 16, 2019

mans2singh commented Apr 19, 2019 • edited Loading

mans2singh commented Apr 24, 2019

zhongjiajie left a comment

Choose a reason for hiding this comment

mans2singh commented Apr 24, 2019

zhongjiajie commented Apr 24, 2019

mans2singh commented Apr 24, 2019

mans2singh commented Apr 29, 2019

mans2singh commented May 16, 2019

kaxil commented May 24, 2019

mans2singh commented May 25, 2019

codecov-io commented Mar 28, 2019 •

edited

Loading

mans2singh commented Apr 19, 2019 •

edited

Loading