Skip to content

Conversation

@lidel
Copy link
Member

@lidel lidel commented Nov 14, 2025

Builds on #11046 to make fast-provide more practical (default behavior configurable via Import.* config) and supported in both ipfs add and ipfs dag import.

Fixes

Combined provide strategies didn't work

Strategy checking used else if, which breaks bitflag combinations like pinned+mfs:

// Before (broken):
if strategy&ProvideStrategyPinned != 0 && dopin {
    shouldProvide = true
} else if strategy&ProvideStrategyMFS != 0 && toFilesSet {  // never reached!
    shouldProvide = true
}

// After (fixed):
if strategy&ProvideStrategyPinned != 0 && dopin {
    shouldProvide = true
}
if strategy&ProvideStrategyMFS != 0 && toFilesSet {  // now works
    shouldProvide = true
}

New Features

Config defaults for fast-provide flags

Set defaults in config instead of passing flags on every command. See docs/config.md for Import.FastProvideRoot and Import.FastProvideWait options.

This way everyone can disable fast provide globally if it somehow negatively impacts their setup. This allows people to update to 0.39 without being blocked by any potential regression.

dag import support

dag import now supports --fast-provide-root and --fast-provide-wait flags. See updated ipfs dag import --help.

Refactors

  • Extracted ExecuteFastProvide() to core/commands/cmdenv/env.go for code sharing
  • Added ResolveBoolFromConfig() helper for flag/config resolution

fast-provide-root should honor the same config settings as the regular
provide system:
- skip when Provide.Enabled is false
- skip when Provide.DHT.Interval is 0
- respect Provide.Strategy (all/pinned/roots/mfs/combinations)

This ensures fast-provide only runs when appropriate based on user
configuration and the nature of the content being added (pinned vs
unpinned, added to MFS or not).
Add Import.FastProvideRoot and Import.FastProvideWait configuration options
to control default behavior of fast-provide-root and fast-provide-wait flags
in ipfs add command. Users can now set global defaults in config while
maintaining per-command flag overrides.

- Add Import.FastProvideRoot (default: true)
- Add Import.FastProvideWait (default: false)
- Add ResolveBoolFromConfig helper for config resolution
- Update docs with configuration details
- Add log-based tests verifying actual behavior
Extract fast-provide logic from add command into reusable components:
- Add config.ShouldProvideForStrategy helper for strategy matching
- Add ExecuteFastProvide function reusable across add and dag import commands
- Move DefaultFastProvideTimeout constant to config/provide.go
- Simplify add.go from 72 lines to 6 lines for fast-provide
- Move fast-provide tests to dedicated TestAddFastProvide function

Benefits:
- cleaner API: callers only pass content characteristics
- all strategy logic centralized in one place
- better separation of concerns
- easier to add fast-provide to other commands in future
Adds --fast-provide-root and --fast-provide-wait flags to `ipfs dag import`,
mirroring the fast-provide functionality available in `ipfs add`.

Changes:
- Add --fast-provide-root and --fast-provide-wait flags to dag import command
- Implement fast-provide logic for all root CIDs in imported CAR files
- Works even when --pin-roots=false (strategy checked internally)
- Share ExecuteFastProvide implementation between add and dag import
- Move ExecuteFastProvide to cmdenv package to avoid import cycles
- Add logging when fast-provide is disabled
- Conditional error handling: return error when wait=true, warn when wait=false
- Update config docs to mention both ipfs add and ipfs dag import
- Update changelog to use "provide" terminology and include dag import examples
- Add comprehensive test coverage (TestDagImportFastProvide with 6 test cases)

The fast-provide feature allows immediate DHT announcement of root CIDs
for faster content discovery, bypassing the regular background queue.
@lidel lidel requested a review from gammazero November 14, 2025 21:14
@lidel lidel mentioned this pull request Nov 14, 2025
74 tasks
Refine documentation to better explain fast-provide and sweep provider working
together, and highlight the performance improvement.

Changelog:
- add fast-provide to sweep provider features list
- explain performance improvement: root CIDs discoverable in <1s vs 30+ seconds
- note this uses optimistic DHT operations (faster with sweep provider)
- simplify examples, point to --help for details

Config docs:
- fix: --fast-provide-roots should be --fast-provide-root (singular)
- clarify Import.FastProvideRoot focuses on root CIDs while sweep handles all blocks
- simplify Import.FastProvideWait description

Command help:
- ipfs add: explain sweep provider context upfront
- ipfs dag import: add fast-provide explanation section
- both explain the split: fast-provide for roots, sweep for all blocks
@lidel lidel force-pushed the fast-provide-rebased branch from fc8002e to 1b9cc82 Compare November 14, 2025 21:16
@lidel lidel marked this pull request as ready for review November 14, 2025 21:26
@lidel lidel requested a review from a team as a code owner November 14, 2025 21:26
add tests covering all provide strategy combinations with focus on
bitflag OR logic (the else-if bug fix). organized by behavior:
- all strategy always provides
- single strategies match only their flag
- combined strategies use OR logic
- zero strategy never provides
@lidel lidel force-pushed the fast-provide-rebased branch from 1288d95 to 5a7d0e1 Compare November 14, 2025 23:51
change ExecuteFastProvide() to return error, enabling proper error
propagation when --fast-provide-wait=true. in sync mode, provide
failures now error the command as expected. in async mode (default),
always returns nil with errors logged in background goroutine.

also remove duplicate ExecuteFastProvide() from provide.go (75 lines),
keeping single implementation in cmdenv/env.go for reuse across add
and dag import commands.

call sites simplified:
- add.go: check and propagate error from ExecuteFastProvide
- dag/import.go: return error from ForEach callback, remove confusing
  conditional error handling

semantics:
- precondition skips (DHT unavailable, etc): return nil (not failure)
- async mode (wait=false): return nil, log errors in goroutine
- sync mode (wait=true): return wrapped error on provide failure
@lidel lidel force-pushed the fast-provide-rebased branch from 5a7d0e1 to 21dea1e Compare November 15, 2025 00:13
@gammazero gammazero merged commit cec7432 into master Nov 15, 2025
16 checks passed
@gammazero gammazero deleted the fast-provide-rebased branch November 15, 2025 05:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants