Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add the s3 retry config options for storage option #3268

Merged
merged 2 commits into from
Dec 18, 2024

Conversation

SaintBacchus
Copy link
Collaborator

@SaintBacchus SaintBacchus commented Dec 18, 2024

Add client_max_retries and client_retry_timeout of RetryConfig for S3 client.

If there are some server error of object store server, the object store module of arrow-rs will retry client_max_retries times and also the total execute time is not over client_retry_timeout.

Closes #3182

@github-actions github-actions bot added the enhancement New feature or request label Dec 18, 2024
Copy link
Contributor

@wjones127 wjones127 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for working on this!

A few minor changes requested (I can merge them).

docs/read_and_write.rst Outdated Show resolved Hide resolved
docs/read_and_write.rst Outdated Show resolved Hide resolved
docs/read_and_write.rst Outdated Show resolved Hide resolved
rust/lance-io/src/object_store.rs Outdated Show resolved Hide resolved
rust/lance-io/src/object_store.rs Outdated Show resolved Hide resolved
@wjones127 wjones127 merged commit 6cd6ae8 into lancedb:main Dec 18, 2024
26 of 27 checks passed
@codecov-commenter
Copy link

Codecov Report

Attention: Patch coverage is 81.81818% with 4 lines in your changes missing coverage. Please review.

Project coverage is 78.88%. Comparing base (f2906cf) to head (ab19994).
Report is 7 commits behind head on main.

Files with missing lines Patch % Lines
rust/lance-io/src/object_store.rs 81.81% 0 Missing and 4 partials ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #3268      +/-   ##
==========================================
- Coverage   78.97%   78.88%   -0.09%     
==========================================
  Files         246      246              
  Lines       86313    86548     +235     
  Branches    86313    86548     +235     
==========================================
+ Hits        68162    68272     +110     
- Misses      15328    15446     +118     
- Partials     2823     2830       +7     
Flag Coverage Δ
unittests 78.88% <81.81%> (-0.09%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Expose with_retry in storage options?
3 participants