Skip to content

Commit

Permalink
perf(store/sync): avoid syncing files (#3693)
Browse files Browse the repository at this point in the history
We don't need manual FS syncs. The underlying FS decides when to flush its internal buffers. If the FS fails to write to disk after we finish writing to it, it's a bug on the FS side. Also, during historical block sync, manually syncing files per every block to FS will apply a performance penalty for no good reason.

Also, after #3679, files become immutable, meaning we can safely do delayed writes in FS by avoiding `Sync`
  • Loading branch information
Wondertan authored Aug 26, 2024
1 parent dd7b5cc commit 568744f
Show file tree
Hide file tree
Showing 2 changed files with 1 addition and 11 deletions.
7 changes: 1 addition & 6 deletions store/file/ods.go
Original file line number Diff line number Diff line change
Expand Up @@ -42,7 +42,6 @@ type ODS struct {

// CreateODS creates a new file under given FS path and
// writes the ODS into it out of given EDS.
// It ensures FS is synced after writing finishes.
// It may leave partially written file if any of the writes fail.
func CreateODS(
path string,
Expand All @@ -52,7 +51,7 @@ func CreateODS(
mod := os.O_RDWR | os.O_CREATE | os.O_EXCL // ensure we fail if already exist
f, err := os.OpenFile(path, mod, filePermissions)
if err != nil {
return fmt.Errorf("creating file: %w", err)
return fmt.Errorf("creating ODS file: %w", err)
}

hdr := &headerV0{
Expand Down Expand Up @@ -92,10 +91,6 @@ func writeODSFile(f *os.File, axisRoots *share.AxisRoots, eds *rsmt2d.ExtendedDa
return fmt.Errorf("flushing ODS file: %w", err)
}

if err := f.Sync(); err != nil {
return fmt.Errorf("syncing file: %w", err)
}

return nil
}

Expand Down
5 changes: 0 additions & 5 deletions store/file/q4.go
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,6 @@ type Q4 struct {

// CreateQ4 creates a new file under given FS path and
// writes the Q4 into it out of given EDS.
// It ensures FS is synced after writing finishes.
// It may leave partially written file if any of the writes fail.
func CreateQ4(
path string,
Expand Down Expand Up @@ -74,10 +73,6 @@ func writeQ4File(f *os.File, eds *rsmt2d.ExtendedDataSquare, hdr *headerV0) erro
return fmt.Errorf("flushing Q4: %w", err)
}

if err := f.Sync(); err != nil {
return fmt.Errorf("syncing Q4 file: %w", err)
}

return nil
}

Expand Down

0 comments on commit 568744f

Please sign in to comment.