Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Use Apache Feather for API object transfer instead of RDS #328

Open
wants to merge 15 commits into
base: v2.3.1
Choose a base branch
from
Open

Use Apache Feather for API object transfer instead of RDS #328

wants to merge 15 commits into from

Commits on Oct 16, 2024

  1. Use feather instead of rds for fetching objects

    @PietrH
    PietrH committed Oct 16, 2024
    Configuration menu
    Copy the full SHA
    4848d8d View commit details
    Browse the repository at this point in the history

  2. connection becomes invalid after reading and closes

    @PietrH
    PietrH committed Oct 16, 2024
    Configuration menu
    Copy the full SHA
    88c338a View commit details
    Browse the repository at this point in the history

Commits on Oct 17, 2024

  1. add rds helper back in for benchmarking

    @PietrH
    PietrH committed Oct 17, 2024
    Configuration menu
    Copy the full SHA
    6c37e94 View commit details
    Browse the repository at this point in the history

  2. explicitly set compression algorithm to lz4 on the server

    @PietrH
    PietrH committed Oct 17, 2024
    Configuration menu
    Copy the full SHA
    2ad62b1 View commit details
    Browse the repository at this point in the history

  3. use feather to fetch the result object

    @PietrH
    PietrH committed Oct 17, 2024
    Configuration menu
    Copy the full SHA
    f5271ff View commit details
    Browse the repository at this point in the history

  4. try protoBuff instead of feather

    @PietrH
    PietrH committed Oct 17, 2024
    Configuration menu
    Copy the full SHA
    45117ab View commit details
    Browse the repository at this point in the history

  5. protoBuff unsuitable for large transfers: page use, crashes

    @PietrH
    PietrH committed Oct 17, 2024
    Configuration menu
    Copy the full SHA
    5c7f95e View commit details
    Browse the repository at this point in the history

  6. Merge branch 'feather_instead_of_rds' of github.com:inbo/etn into fea… 

    …ther_instead_of_rds
    @PietrH
    PietrH committed Oct 17, 2024
    Configuration menu
    Copy the full SHA
    594a72f View commit details
    Browse the repository at this point in the history

  7. arrow 17 requires R4.0, previous versions require C++17 compiler

    @PietrH
    PietrH committed Oct 17, 2024
    Configuration menu
    Copy the full SHA
    802f0f1 View commit details
    Browse the repository at this point in the history

  8. bump version to dev

    @PietrH
    PietrH committed Oct 17, 2024
    Configuration menu
    Copy the full SHA
    55c608a View commit details
    Browse the repository at this point in the history

  9. document minimum R version requirement

    @PietrH
    PietrH committed Oct 17, 2024
    Configuration menu
    Copy the full SHA
    17b20bf View commit details
    Browse the repository at this point in the history

  10. remove rds helper used for benchmarking

    @PietrH
    PietrH committed Oct 17, 2024
    Configuration menu
    Copy the full SHA
    4772f64 View commit details
    Browse the repository at this point in the history

  11. use apache arrow for transfer: set compression explicitly 

    undo 45117ab
    @PietrH
    PietrH committed Oct 17, 2024
    Configuration menu
    Copy the full SHA
    e9ac8d6 View commit details
    Browse the repository at this point in the history

  12. Add comment to clarify setting compression via url

    @PietrH
    PietrH committed Oct 17, 2024
    Configuration menu
    Copy the full SHA
    37d68cf View commit details
    Browse the repository at this point in the history

  13. No longer use protolite

    @PietrH
    PietrH committed Oct 17, 2024
    Configuration menu
    Copy the full SHA
    93dba44 View commit details
    Browse the repository at this point in the history