Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Read UC delta table #773

Merged
merged 63 commits into from
Jan 11, 2024
Merged

Read UC delta table #773

merged 63 commits into from
Jan 11, 2024

Commits on Jan 8, 2024

  1. initial commit

    @xiaohanzhan-db
    xiaohanzhan-db committed Jan 8, 2024
    Configuration menu
    Copy the full SHA
    ca70006 View commit details
    Browse the repository at this point in the history

  2. use databricks-sql to read delta table and convert to json

    @xiaohanzhan-db
    xiaohanzhan-db committed Jan 8, 2024
    Configuration menu
    Copy the full SHA
    34b1bb9 View commit details
    Browse the repository at this point in the history

  3. update

    @xiaohanzhan-db
    xiaohanzhan-db committed Jan 8, 2024
    Configuration menu
    Copy the full SHA
    5944314 View commit details
    Browse the repository at this point in the history

  4. update

    @xiaohanzhan-db
    xiaohanzhan-db committed Jan 8, 2024
    Configuration menu
    Copy the full SHA
    3cc004e View commit details
    Browse the repository at this point in the history

  5. update

    @xiaohanzhan-db
    xiaohanzhan-db committed Jan 8, 2024
    Configuration menu
    Copy the full SHA
    b22c4bb View commit details
    Browse the repository at this point in the history

  6. add mocked unittest

    @xiaohanzhan-db
    xiaohanzhan-db committed Jan 8, 2024
    Configuration menu
    Copy the full SHA
    b0dd43a View commit details
    Browse the repository at this point in the history

  7. Fix lints

    @xiaohanzhan-db
    xiaohanzhan-db committed Jan 8, 2024
    Configuration menu
    Copy the full SHA
    286650e View commit details
    Browse the repository at this point in the history

  8. update

    @xiaohanzhan-db
    xiaohanzhan-db committed Jan 8, 2024
    Configuration menu
    Copy the full SHA
    8a45576 View commit details
    Browse the repository at this point in the history

  9. update

    @xiaohanzhan-db
    xiaohanzhan-db committed Jan 8, 2024
    Configuration menu
    Copy the full SHA
    b452ec3 View commit details
    Browse the repository at this point in the history

  10. restructure code

    @xiaohanzhan-db
    xiaohanzhan-db committed Jan 8, 2024
    Configuration menu
    Copy the full SHA
    c902b67 View commit details
    Browse the repository at this point in the history

  11. Add timer for optimizing

    @xiaohanzhan-db
    xiaohanzhan-db committed Jan 8, 2024
    Configuration menu
    Copy the full SHA
    f57c21c View commit details
    Browse the repository at this point in the history

  12. Add db-connect

    @xiaohanzhan-db
    xiaohanzhan-db committed Jan 8, 2024
    Configuration menu
    Copy the full SHA
    7ff9e5a View commit details
    Browse the repository at this point in the history

  13. add wrapper

    @xiaohanzhan-db
    xiaohanzhan-db committed Jan 8, 2024
    Configuration menu
    Copy the full SHA
    6d17cef View commit details
    Browse the repository at this point in the history

  14. update

    @xiaohanzhan-db
    xiaohanzhan-db committed Jan 8, 2024
    Configuration menu
    Copy the full SHA
    8116804 View commit details
    Browse the repository at this point in the history

  15. add install dbconnect

    @xiaohanzhan-db
    xiaohanzhan-db committed Jan 8, 2024
    Configuration menu
    Copy the full SHA
    92cfeb2 View commit details
    Browse the repository at this point in the history

  16. update

    @xiaohanzhan-db
    xiaohanzhan-db committed Jan 8, 2024
    Configuration menu
    Copy the full SHA
    fa9743b View commit details
    Browse the repository at this point in the history

  17. update

    @xiaohanzhan-db
    xiaohanzhan-db committed Jan 8, 2024
    Configuration menu
    Copy the full SHA
    00c3f73 View commit details
    Browse the repository at this point in the history

  18. patch dbconnect to allow multiple return formats

    @xiaohanzhan-db
    xiaohanzhan-db committed Jan 8, 2024
    Configuration menu
    Copy the full SHA
    7304c4c View commit details
    Browse the repository at this point in the history

  19. update

    @xiaohanzhan-db
    xiaohanzhan-db committed Jan 8, 2024
    Configuration menu
    Copy the full SHA
    9a1916b View commit details
    Browse the repository at this point in the history

  20. add arrow

    @xiaohanzhan-db
    xiaohanzhan-db committed Jan 8, 2024
    Configuration menu
    Copy the full SHA
    d3defc1 View commit details
    Browse the repository at this point in the history

  21. use compression

    @xiaohanzhan-db
    xiaohanzhan-db committed Jan 8, 2024
    Configuration menu
    Copy the full SHA
    da820eb View commit details
    Browse the repository at this point in the history

  22. clean up

    @xiaohanzhan-db
    xiaohanzhan-db committed Jan 8, 2024
    Configuration menu
    Copy the full SHA
    60a4be7 View commit details
    Browse the repository at this point in the history

  23. Add cluster rt check

    @xiaohanzhan-db
    xiaohanzhan-db committed Jan 8, 2024
    Configuration menu
    Copy the full SHA
    382a8fa View commit details
    Browse the repository at this point in the history

  24. Fix lints

    @xiaohanzhan-db
    xiaohanzhan-db committed Jan 8, 2024
    Configuration menu
    Copy the full SHA
    f0286dd View commit details
    Browse the repository at this point in the history

  25. remove patch.py for CI

    @xiaohanzhan-db
    xiaohanzhan-db committed Jan 8, 2024
    Configuration menu
    Copy the full SHA
    09efbb0 View commit details
    Browse the repository at this point in the history

  26. update

    @xiaohanzhan-db
    xiaohanzhan-db committed Jan 8, 2024
    Configuration menu
    Copy the full SHA
    64214de View commit details
    Browse the repository at this point in the history

  27. update

    @xiaohanzhan-db
    xiaohanzhan-db committed Jan 8, 2024
    Configuration menu
    Copy the full SHA
    13ce55b View commit details
    Browse the repository at this point in the history

  28. updat

    @xiaohanzhan-db
    xiaohanzhan-db committed Jan 8, 2024
    Configuration menu
    Copy the full SHA
    5edc497 View commit details
    Browse the repository at this point in the history

  29. update

    @xiaohanzhan-db
    xiaohanzhan-db committed Jan 8, 2024
    Configuration menu
    Copy the full SHA
    ad06211 View commit details
    Browse the repository at this point in the history

  30. fix tests

    @xiaohanzhan-db
    xiaohanzhan-db committed Jan 8, 2024
    Configuration menu
    Copy the full SHA
    7a18e96 View commit details
    Browse the repository at this point in the history

  31. fix lint

    @xiaohanzhan-db
    xiaohanzhan-db committed Jan 8, 2024
    Configuration menu
    Copy the full SHA
    89e658a View commit details
    Browse the repository at this point in the history

  32. update

    @xiaohanzhan-db
    xiaohanzhan-db committed Jan 8, 2024
    Configuration menu
    Copy the full SHA
    acef77c View commit details
    Browse the repository at this point in the history

  33. update

    @xiaohanzhan-db
    xiaohanzhan-db committed Jan 8, 2024
    Configuration menu
    Copy the full SHA
    3e1b55f View commit details
    Browse the repository at this point in the history

  34. Add more tests

    @xiaohanzhan-db
    xiaohanzhan-db committed Jan 8, 2024
    Configuration menu
    Copy the full SHA
    6bd32a5 View commit details
    Browse the repository at this point in the history

  35. update

    @xiaohanzhan-db
    xiaohanzhan-db committed Jan 8, 2024
    Configuration menu
    Copy the full SHA
    7205063 View commit details
    Browse the repository at this point in the history

  36. update

    @xiaohanzhan-db
    xiaohanzhan-db committed Jan 8, 2024
    Configuration menu
    Copy the full SHA
    284b52f View commit details
    Browse the repository at this point in the history

  37. update

    @xiaohanzhan-db
    xiaohanzhan-db committed Jan 8, 2024
    Configuration menu
    Copy the full SHA
    6f8957a View commit details
    Browse the repository at this point in the history

  38. change to download_json

    @xiaohanzhan-db
    xiaohanzhan-db committed Jan 8, 2024
    Configuration menu
    Copy the full SHA
    7ce02e1 View commit details
    Browse the repository at this point in the history

  39. update

    @xiaohanzhan-db
    xiaohanzhan-db committed Jan 8, 2024
    Configuration menu
    Copy the full SHA
    aba0f4d View commit details
    Browse the repository at this point in the history

  40. fix lints

    @xiaohanzhan-db
    xiaohanzhan-db committed Jan 8, 2024
    Configuration menu
    Copy the full SHA
    040620f View commit details
    Browse the repository at this point in the history

  41. Add decompressed option for arrow

    @xiaohanzhan-db
    xiaohanzhan-db committed Jan 8, 2024
    Configuration menu
    Copy the full SHA
    51522f8 View commit details
    Browse the repository at this point in the history

  42. format json to jsonl

    @xiaohanzhan-db
    xiaohanzhan-db committed Jan 8, 2024
    Configuration menu
    Copy the full SHA
    7ae65e3 View commit details
    Browse the repository at this point in the history

  43. Add comments

    @xiaohanzhan-db
    xiaohanzhan-db committed Jan 8, 2024
    Configuration menu
    Copy the full SHA
    7f027fc View commit details
    Browse the repository at this point in the history

  44. Make cf_collect_type global option

    @xiaohanzhan-db
    xiaohanzhan-db committed Jan 8, 2024
    Configuration menu
    Copy the full SHA
    34c5b72 View commit details
    Browse the repository at this point in the history

  45. fix comments

    @xiaohanzhan-db
    xiaohanzhan-db committed Jan 8, 2024
    Configuration menu
    Copy the full SHA
    0e9c029 View commit details
    Browse the repository at this point in the history

  46. fix lints

    @xiaohanzhan-db
    xiaohanzhan-db committed Jan 8, 2024
    Configuration menu
    Copy the full SHA
    2007688 View commit details
    Browse the repository at this point in the history

  47. fix comments

    @xiaohanzhan-db
    xiaohanzhan-db committed Jan 8, 2024
    Configuration menu
    Copy the full SHA
    10fd85a View commit details
    Browse the repository at this point in the history

  48. Fix lints

    @xiaohanzhan-db
    xiaohanzhan-db committed Jan 8, 2024
    Configuration menu
    Copy the full SHA
    462f9b7 View commit details
    Browse the repository at this point in the history

  49. change to use workspaceclient

    @xiaohanzhan-db
    xiaohanzhan-db committed Jan 8, 2024
    Configuration menu
    Copy the full SHA
    b7d9b04 View commit details
    Browse the repository at this point in the history

Commits on Jan 9, 2024

  1. Add CPT support

    @xiaohanzhan-db
    xiaohanzhan-db committed Jan 9, 2024
    Configuration menu
    Copy the full SHA
    626eeb1 View commit details
    Browse the repository at this point in the history

  2. Rewire method assignment logic

    @xiaohanzhan-db
    xiaohanzhan-db committed Jan 9, 2024
    Configuration menu
    Copy the full SHA
    85cc21d View commit details
    Browse the repository at this point in the history

Commits on Jan 10, 2024

  1. Fix bug in stripping https

    @xiaohanzhan-db
    xiaohanzhan-db committed Jan 10, 2024
    Configuration menu
    Copy the full SHA
    d8c28ac View commit details
    Browse the repository at this point in the history

  2. Add tests for rewired method assignment logic

    @xiaohanzhan-db
    xiaohanzhan-db committed Jan 10, 2024
    Configuration menu
    Copy the full SHA
    3ff5d0e View commit details
    Browse the repository at this point in the history

  3. Fix lints

    @xiaohanzhan-db
    xiaohanzhan-db committed Jan 10, 2024
    Configuration menu
    Copy the full SHA
    fdc7b14 View commit details
    Browse the repository at this point in the history

  4. Fix lints

    @xiaohanzhan-db
    xiaohanzhan-db committed Jan 10, 2024
    Configuration menu
    Copy the full SHA
    a543b80 View commit details
    Browse the repository at this point in the history

  5. Removed logger set_level

    @xiaohanzhan-db
    xiaohanzhan-db committed Jan 10, 2024
    Configuration menu
    Copy the full SHA
    ba2dbc2 View commit details
    Browse the repository at this point in the history

  6. Remove pyspark. It conflicts with databricks-connect

    @xiaohanzhan-db
    xiaohanzhan-db committed Jan 10, 2024
    Configuration menu
    Copy the full SHA
    88cad19 View commit details
    Browse the repository at this point in the history

Commits on Jan 11, 2024

  1. Merge branch 'main' into delta_input

    @xiaohanzhan-db
    xiaohanzhan-db committed Jan 11, 2024
    Configuration menu
    Copy the full SHA
    ddc6d25 View commit details
    Browse the repository at this point in the history

  2. Update the comment

    @xiaohanzhan-db
    xiaohanzhan-db committed Jan 11, 2024
    Configuration menu
    Copy the full SHA
    21efd22 View commit details
    Browse the repository at this point in the history

  3. skip cluster version check when cluster_id is serverless

    @xiaohanzhan-db
    xiaohanzhan-db committed Jan 11, 2024
    Configuration menu
    Copy the full SHA
    90d77fd View commit details
    Browse the repository at this point in the history

  4. Add use_serverless flag

    @xiaohanzhan-db
    xiaohanzhan-db committed Jan 11, 2024
    Configuration menu
    Copy the full SHA
    c8f8307 View commit details
    Browse the repository at this point in the history

  5. update tests with use_serverless flag

    @xiaohanzhan-db
    xiaohanzhan-db committed Jan 11, 2024
    Configuration menu
    Copy the full SHA
    9a524be View commit details
    Browse the repository at this point in the history

  6. Fix lints

    @xiaohanzhan-db
    xiaohanzhan-db committed Jan 11, 2024
    Configuration menu
    Copy the full SHA
    00ca457 View commit details
    Browse the repository at this point in the history