Skip to content

Commit

Permalink
downloader: docs on MMAP for data-files r/w and experiments with bufio (
Browse files Browse the repository at this point in the history
#10074)

Pros:
- it allows to not pre-alloc files:
#8688
- it allows to not "sig-bus" when no space left on disk (return
user-friendly error). see:
#8500 - but DB will be MMAP
anyway and may get "sig-bus"

FYI:
- seems no perf difference (but i tested only on cloud drives)
- erigon will anyway open it as mmap 
 
Cons: 
- i did implemented `fsync` for mmap (
anacrolix/torrent#755 ) - probably will need
implement it for bufio: anacrolix/torrent#937
- no zero-copy: more `alloc` memory will be holded by APP (PageCache
starvation). I see 2x mem usage (at `--torrent.download.slots=500` 20gb
vs 40gb)
- i see "10K threads exchaused" error earlier (on
`--torrent.download.slots=500`).
- what else?
  • Loading branch information
AskAlexSharov authored Apr 27, 2024
1 parent 382f881 commit 7ac8b10
Show file tree
Hide file tree
Showing 2 changed files with 11 additions and 0 deletions.
File renamed without changes.
11 changes: 11 additions & 0 deletions erigon-lib/downloader/downloader.go
Original file line number Diff line number Diff line change
Expand Up @@ -2568,7 +2568,18 @@ func openClient(ctx context.Context, dbDir, snapDir string, cfg *torrent.ClientC
if err != nil {
return nil, nil, nil, nil, fmt.Errorf("torrentcfg.NewMdbxPieceCompletion: %w", err)
}

//Reasons why using MMAP instead of files-API:
// - i see "10K threads exchaused" error earlier (on `--torrent.download.slots=500` and `pd-ssd`)
// - "sig-bus" at disk-full - may happen anyway, because DB is mmap
// - MMAP - means less GC pressure, more zero-copy
// - MMAP files are pre-allocated - which is not cool, but: 1. we can live with it 2. maybe can just resize MMAP in future
// See also: https://github.com/ledgerwatch/erigon/pull/10074
m = storage.NewMMapWithCompletion(snapDir, c)
//m = storage.NewFileOpts(storage.NewFileClientOpts{
// ClientBaseDir: snapDir,
// PieceCompletion: c,
//})
cfg.DefaultStorage = m

torrentClient, err = torrent.NewClient(cfg)
Expand Down

0 comments on commit 7ac8b10

Please sign in to comment.