Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[AIRFLOW-3950] Improve AirflowSecurityManager.update_admin_perm_view #4774

Merged
merged 1 commit into from
Feb 26, 2019

Conversation

XD-DENG
Copy link
Member

@XD-DENG XD-DENG commented Feb 25, 2019

Jira

Description

  • Here are some details about my PR, including screenshots of any UI changes:

This PR helps improve AirflowSecurityManager.update_admin_perm_view by

  • Simplifying the implementation: avoid unnecessary for-loop by using set.
  • Improving the performance (especially when there are many DAGs)

Regarding performance improvement, please refer to this simple toy benchmarking:

def fun1(a, b):
    a=set(a)

    for i in b:
        if i not in a:
            a.add(i)

    list(a)

def fun2(a, b):
    list(set(a) | set(b))


if __name__ == '__main__':
    import timeit
    print("-Test.1-")
    print(timeit.timeit("fun1([1,2,3], range(100))", setup="from __main__ import fun1"))
    print(timeit.timeit("fun2([1,2,3], range(100))", setup="from __main__ import fun2"))
    print("-Test.2-")
    print(timeit.timeit("fun1([1,2,3], range(500))", setup="from __main__ import fun1"))
    print(timeit.timeit("fun2([1,2,3], range(500))", setup="from __main__ import fun2"))

Result:

-Test.1-
12.912254190000112
5.487735479000094
-Test.2-
62.42928877399936
21.93449944399981

- simplify the implementation
- improve the performance
@XD-DENG
Copy link
Member Author

XD-DENG commented Feb 25, 2019

One test failed, likely due to transient/flaky error.

@XD-DENG
Copy link
Member Author

XD-DENG commented Feb 25, 2019

@ashb @feng-tao PTAL. A low-hanging improvement.

@feng-tao feng-tao merged commit 61aed43 into apache:master Feb 26, 2019
@XD-DENG XD-DENG deleted the patch-5 branch February 26, 2019 02:10
ashb pushed a commit that referenced this pull request Mar 21, 2019
…4774)

- simplify the implementation
- improve the performance
ashb pushed a commit that referenced this pull request Mar 22, 2019
…4774)

- simplify the implementation
- improve the performance
andriisoldatenko pushed a commit to andriisoldatenko/airflow that referenced this pull request Jul 26, 2019
wmorris75 pushed a commit to modmed/incubator-airflow that referenced this pull request Jul 29, 2019
…pache#4774)

- simplify the implementation
- improve the performance
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants