Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

HADOOP-19105. Improve resilience in vector reads. #7105

Draft
wants to merge 4 commits into
base: trunk
Choose a base branch
from

Commits on Oct 9, 2024

  1. HADOOP-19105. Improve resilience in vector reads.

    Add a new releaser method, which then is invoked to release
    buffers on failure.
    
    It is a bit contrived how we try not break external
    implementations when adding a new default implementation
    to PositionedReadable: the releaser will be lost unless
    they do so.
    
    Change-Id: Ie2f95a89cfa79d3fdf33c1cc58f63e4ca00c6207
    steveloughran committed Oct 9, 2024
    Configuration menu
    Copy the full SHA
    311c2e0 View commit details
    Browse the repository at this point in the history

Commits on Oct 10, 2024

  1. HADOOP-19105. Improve resilience in vector reads.

    Trying to implement resilience in the s3a input stream;
    buffers of incomplete reads are now released on failures.
    
    Change-Id: I4b23c6a2ff417687fd19c1e8e9feef6cf733c031
    steveloughran committed Oct 10, 2024
    Configuration menu
    Copy the full SHA
    535bcfb View commit details
    Browse the repository at this point in the history
  2. HADOOP-19105. Improve resilience in vector reads.

    Trying to implement resilience in the s3a input stream;
    buffers of incomplete reads are now released on failures.
    
    Change-Id: I851779383f8d120fe42bc97c5e6ae533c5fe2e23
    steveloughran committed Oct 10, 2024
    Configuration menu
    Copy the full SHA
    29e2049 View commit details
    Browse the repository at this point in the history

Commits on Oct 29, 2024

  1. HADOOP-19105. s3a input stream codepath looks good.

    Now: how do I pull this out so it can be used with the prefetch
    stream?
    
    Change-Id: I98e350ab3b28d9adc7b458dee4ceee56a0562f17
    steveloughran committed Oct 29, 2024
    Configuration menu
    Copy the full SHA
    fef1762 View commit details
    Browse the repository at this point in the history