Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(common): Synchronize file to disk in NativeIO #256

Merged
merged 1 commit into from
Jun 16, 2024

Conversation

clabby
Copy link
Collaborator

@clabby clabby commented Jun 15, 2024

Overview

Fixes a rare synchronization error in the NativeIO, where when fetching is happening very fast concurrently, the file's contents may not be synced to disk fast enough for the other end of the pipe to recognize. Notably, std::fs::File does not block on updating the metadata of the file to disk when it is being dropped. File::sync_all does exactly this, blocking until all metadata and in-memory contents have been flushed to disk.

@clabby clabby force-pushed the cl/oracle-blob-fetcher branch from cc2c125 to 27b9970 Compare June 15, 2024 23:57
@clabby clabby force-pushed the cl/host-sync-fixes branch 2 times, most recently from f8f9794 to 2dab14e Compare June 16, 2024 00:00
@clabby clabby self-assigned this Jun 16, 2024
@clabby clabby added A-services Area: service crates K-fix Kind: fix labels Jun 16, 2024
@clabby clabby added this to the Phase 3: Kona Client & Host milestone Jun 16, 2024
@clabby clabby force-pushed the cl/oracle-blob-fetcher branch from 27b9970 to fd50500 Compare June 16, 2024 17:55
@clabby clabby force-pushed the cl/host-sync-fixes branch from 2dab14e to 87e28c3 Compare June 16, 2024 17:55
@clabby
Copy link
Collaborator Author

clabby commented Jun 16, 2024

Just a note, this is a dud. Changed upstack.

@clabby clabby force-pushed the cl/oracle-blob-fetcher branch from fd50500 to 18d7194 Compare June 16, 2024 20:16
@clabby clabby force-pushed the cl/host-sync-fixes branch from 87e28c3 to 1acce4d Compare June 16, 2024 20:16
Fixes a rare synchronization error in the `NativeIO`, where when
fetching is happening very fast concurrently, the file's contents may
not be synced to disk fast enough for the other end of the pipe to
recognize.
@clabby clabby force-pushed the cl/oracle-blob-fetcher branch from 18d7194 to 877f22e Compare June 16, 2024 20:51
@clabby clabby force-pushed the cl/host-sync-fixes branch from 1acce4d to 03d3283 Compare June 16, 2024 20:51
@codecov-commenter
Copy link

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 71.3%. Comparing base (877f22e) to head (03d3283).

Additional details and impacted files

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@clabby clabby merged commit 99bec93 into cl/oracle-blob-fetcher Jun 16, 2024
14 checks passed
@clabby clabby deleted the cl/host-sync-fixes branch July 2, 2024 03:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-services Area: service crates K-fix Kind: fix
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants