Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

fix(ingest/snowflake): add more reporting for usage aggregation, handle lineage errors #10279

Merged
merged 21 commits into from
May 7, 2024
Merged

fix(ingest/snowflake): add more reporting for usage aggregation, handle lineage errors #10279

merged 21 commits into from
May 7, 2024

Commits on Apr 12, 2024

  1. fix(ingest/snowflake): add timer and row count for usage aggregation … 

    …query
    @mayurinehate
    mayurinehate committed Apr 12, 2024
    Configuration menu
    Copy the full SHA
    6f90903 View commit details
    Browse the repository at this point in the history

  2. update usage query to ignore temporary tables

    @mayurinehate
    mayurinehate committed Apr 12, 2024
    Configuration menu
    Copy the full SHA
    4832601 View commit details
    Browse the repository at this point in the history

  3. fix lint

    @mayurinehate
    mayurinehate committed Apr 12, 2024
    Configuration menu
    Copy the full SHA
    203519f View commit details
    Browse the repository at this point in the history

Commits on Apr 15, 2024

  1. add more timers, perf test for snowflake

    @mayurinehate
    mayurinehate committed Apr 15, 2024
    Configuration menu
    Copy the full SHA
    6441ad0 View commit details
    Browse the repository at this point in the history

  2. Merge branch 'master' into feature/ing-580

    @mayurinehate
    mayurinehate authored Apr 15, 2024
    Configuration menu
    Copy the full SHA
    7851c5a View commit details
    Browse the repository at this point in the history

Commits on Apr 16, 2024

  1. Merge branch 'master' into feature/ing-580

    @mayurinehate
    mayurinehate authored Apr 16, 2024
    Configuration menu
    Copy the full SHA
    9eeeaf1 View commit details
    Browse the repository at this point in the history

  2. re-add accidentally removed continue statement

    @mayurinehate
    mayurinehate committed Apr 16, 2024
    Configuration menu
    Copy the full SHA
    e9b9c01 View commit details
    Browse the repository at this point in the history

Commits on Apr 17, 2024

  1. make timer not include yield time, improve perf test

    @mayurinehate
    mayurinehate committed Apr 17, 2024
    Configuration menu
    Copy the full SHA
    1d165fc View commit details
    Browse the repository at this point in the history

  2. Merge branch 'master' into feature/ing-580

    @mayurinehate
    mayurinehate authored Apr 17, 2024
    Configuration menu
    Copy the full SHA
    4646ff1 View commit details
    Browse the repository at this point in the history

  3. add details about row being processed

    @mayurinehate
    mayurinehate committed Apr 17, 2024
    Configuration menu
    Copy the full SHA
    8c78817 View commit details
    Browse the repository at this point in the history

  4. Merge remote-tracking branch 'refs/remotes/origin/feature/ing-580' in… 

    …to feature/ing-580
    @mayurinehate
    mayurinehate committed Apr 17, 2024
    Configuration menu
    Copy the full SHA
    cde5738 View commit details
    Browse the repository at this point in the history

Commits on Apr 19, 2024

  1. more timers, exception handling for lineage

    @mayurinehate
    mayurinehate committed Apr 19, 2024
    Configuration menu
    Copy the full SHA
    17c149f View commit details
    Browse the repository at this point in the history

  2. Merge branch 'master' into feature/ing-580

    @mayurinehate
    mayurinehate authored Apr 19, 2024
    Configuration menu
    Copy the full SHA
    e2414fa View commit details
    Browse the repository at this point in the history

  3. fix lint

    @mayurinehate
    mayurinehate committed Apr 19, 2024
    Configuration menu
    Copy the full SHA
    316c632 View commit details
    Browse the repository at this point in the history

Commits on Apr 21, 2024

  1. group usage report stats

    @mayurinehate
    mayurinehate committed Apr 21, 2024
    Configuration menu
    Copy the full SHA
    e7c4cc6 View commit details
    Browse the repository at this point in the history

Commits on Apr 22, 2024

  1. update sql formatter to sqlglot

    @mayurinehate
    mayurinehate committed Apr 22, 2024
    Configuration menu
    Copy the full SHA
    760c716 View commit details
    Browse the repository at this point in the history

Commits on Apr 23, 2024

  1. add lru cache

    @mayurinehate
    mayurinehate committed Apr 23, 2024
    Configuration menu
    Copy the full SHA
    d929148 View commit details
    Browse the repository at this point in the history

Commits on Apr 25, 2024

  1. Merge branch 'master' into feature/ing-580

    @mayurinehate
    mayurinehate authored Apr 25, 2024
    Configuration menu
    Copy the full SHA
    ce7ec24 View commit details
    Browse the repository at this point in the history

  2. add sql formatting perf test

    @hsheth2
    hsheth2 committed Apr 25, 2024
    Configuration menu
    Copy the full SHA
    e5e5098 View commit details
    Browse the repository at this point in the history

Commits on Apr 26, 2024

  1. Update metadata-ingestion/src/datahub/ingestion/source/snowflake/snow… 

    …flake_usage_v2.py

Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
    @mayurinehate @hsheth2
    mayurinehate and hsheth2 authored Apr 26, 2024
    Configuration menu
    Copy the full SHA
    5d85abe View commit details
    Browse the repository at this point in the history

  2. Update snowflake_lineage_v2.py

    @mayurinehate
    mayurinehate authored Apr 26, 2024
    Configuration menu
    Copy the full SHA
    c968852 View commit details
    Browse the repository at this point in the history