feat: piecereader: Allow parallel access

[magik6k]

Related Issues

Closes #10532
Replaces #10895

Proposed Changes

• Allow pieceReader reader-getters to define exact read size
• Rewrite pieceReader.ReadAt to be called in parallel, while still caching small amounts ahead for parallel sequential reads
  • This is necessary, because carv1 reads byte-by-byte for the header/cid varints, saves ~9 requests per block read
• Kept the old more aggressive prefetching mechanism for ReadSeeker, which is only used by the indexing scan, and makes it wayy faster

Additional Info

• Make sure tests pass
• Test on a real setup, best in booster-bitswap
  • With a race detector

This PR is based on v1.23.0 so that it's possible to test with boost

Proposed Changes

Checklist

Before you mark the PR ready for review, please make sure that:

• Commits have a clear commit message.
• PR title is in the form of of <PR type>: <area>: <change being made>
  • example: fix: mempool: Introduce a cache for valid signatures
  • PR type: fix, feat, build, chore, ci, docs, perf, refactor, revert, style, test
  • area, e.g. api, chain, state, market, mempool, multisig, networking, paych, proving, sealing, wallet, deps
• New features have usage guidelines and / or documentation updates in
  • Lotus Documentation
  • Discussion Tutorials
• Tests exist for new functionality or change in behavior
• CI is green

[snadrus]

context.Context is never supposed to be stored in a struct, so:
can we have init(ctx context.Context) ?
Here and the next line, can tag.New take that Context and return it as well (since it should be non-destructive).

[magik6k]

Yeahh, moved the ctx into init params, but the MCtx ones will probably have to stay in that struct - opencensus sadly needs a context to record stats on (technically we could hack into it's internal packages, but that's even more gnarly)

[snadrus]

The idea of passing context around is that it follows the goroutine & opencensus is producing a stat on that. Is this stat on a shared thing that cannot be expressed individually by the callers?

[snadrus]

Looks good. Let me know when you're ready for me to approve.

…

On Thu, May 25, 2023 at 9:08 AM Łukasz Magiera ***@***.***> wrote: ***@***.**** commented on this pull request. ------------------------------ In storage/sealer/piece_reader.go <#10913 (comment)> : > } func (p *pieceReader) init() (_ *pieceReader, err error) { stats.Record(p.ctx, metrics.DagStorePRInitCount.M(1)) + p.seqMCtx, _ = tag.New(p.ctx, tag.Upsert(metrics.PRReadType, "seq")) Yeahh, moved the ctx into init params, but the MCtx ones will probably have to stay in that struct - opencensus sadly needs a context to record stats on (technically we could hack into it's internal packages, but that's even more gnarly) — Reply to this email directly, view it on GitHub <#10913 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAOU4LTM5OAWVQP575CMLXDXH5RURANCNFSM6AAAAAAYLYA77E> . You are receiving this because your review was requested.Message ID: ***@***.***>

[magik6k]

Merged master into here so that the previous commits can still be depended on in boost

Should be mergeable now.