-
-
Notifications
You must be signed in to change notification settings - Fork 3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat: built-in content blocking based on IPIP-383 #10161
Merged
Commits on Oct 3, 2023
-
plugin: Add support for content blocking in Kubo
Fixes #8492. This introduces "nopfs" as a preloaded plugin into Kubo. It automatically make Kubo watch *.deny files found in: - /etc/ipfs/denylists - $XDG_CONFIG_HOME/ipfs/denylists - $IPFS_PATH/denylists (files need to be present before boot in order to be watched). Debug logging can be enabled with `GOLOG_LOG_LEVEL="nopfs=debug"`. All blocks are logged to "nopfs-blocks", so logging requests for blocked content can be achieved with `GOLOG_LOG_LEVEL="nopfs-blocks=warn"`: ``` WARN (...) QmRFniDxwxoG2n4AcnGhRdjqDjCM5YeUcBE75K8WXmioH3: blocked (test.deny:9) ``` Interactive/gateway users will also receive errors as responses but with less details: ``` Error: /ipfs/QmQvjk82hPkSaZsyJ8vNER5cmzKW7HyGX5XVusK7EAenCN is blocked and cannot be provided ``` One particularity to keep in mind is that GetMany() will silently drop blocked blocks from the response (a warnings are logged). AddMany() will act similarly and avoid adding blocked blocks. The code implementing all this is actually in nopfs: - https://github.com/ipfs-shipyard/nopfs (main library) - https://github.com/ipfs-shipyard/nopfs/tree/master/ipfs (wrappers) The interpretation of the list rules and block detection is well tested, but a general review might be in order.
Configuration menu - View commit details
-
Copy full SHA for d09c1f9 - Browse repository at this point
Copy the full SHA d09c1f9View commit details
Commits on Oct 4, 2023
-
Configuration menu - View commit details
-
Copy full SHA for bcae8a4 - Browse repository at this point
Copy the full SHA bcae8a4View commit details
Commits on Oct 19, 2023
-
Configuration menu - View commit details
-
Copy full SHA for 51acfb9 - Browse repository at this point
Copy the full SHA 51acfb9View commit details -
Configuration menu - View commit details
-
Copy full SHA for da3cf31 - Browse repository at this point
Copy the full SHA da3cf31View commit details -
test: content_blocking_test.go
Includes dependency update from ipfs-shipyard/nopfs#25
Configuration menu - View commit details
-
Copy full SHA for 0a8f651 - Browse repository at this point
Copy the full SHA 0a8f651View commit details -
Configuration menu - View commit details
-
Copy full SHA for e689080 - Browse repository at this point
Copy the full SHA e689080View commit details -
test: /ipfs/cid/* rule on CLI and gateway
CLI works as expected, gateway does not respect the rule (needs investigation)
Configuration menu - View commit details
-
Copy full SHA for 511b670 - Browse repository at this point
Copy the full SHA 511b670View commit details -
Configuration menu - View commit details
-
Copy full SHA for 0c47250 - Browse repository at this point
Copy the full SHA 0c47250View commit details
Commits on Oct 20, 2023
-
Configuration menu - View commit details
-
Copy full SHA for fd842c2 - Browse repository at this point
Copy the full SHA fd842c2View commit details -
Configuration menu - View commit details
-
Copy full SHA for 2eed42d - Browse repository at this point
Copy the full SHA 2eed42dView commit details -
Configuration menu - View commit details
-
Copy full SHA for 04c1f68 - Browse repository at this point
Copy the full SHA 04c1f68View commit details -
Configuration menu - View commit details
-
Copy full SHA for e1689b5 - Browse repository at this point
Copy the full SHA e1689b5View commit details -
Configuration menu - View commit details
-
Copy full SHA for f7483a6 - Browse repository at this point
Copy the full SHA f7483a6View commit details -
Configuration menu - View commit details
-
Copy full SHA for f516fc5 - Browse repository at this point
Copy the full SHA f516fc5View commit details -
test: fix sharness of dag export
cosmetic change to error message, this should be more robust
Configuration menu - View commit details
-
Copy full SHA for 0129417 - Browse repository at this point
Copy the full SHA 0129417View commit details
Commits on Oct 25, 2023
-
Configuration menu - View commit details
-
Copy full SHA for 9f4fefe - Browse repository at this point
Copy the full SHA 9f4fefeView commit details -
Configuration menu - View commit details
-
Copy full SHA for b4b7ca7 - Browse repository at this point
Copy the full SHA b4b7ca7View commit details
Commits on Oct 26, 2023
-
test: double-hash Path block (blake3-multihash)
this ensures we test something other than default sha256 Ref. ipfs-shipyard/nopfs#28
Configuration menu - View commit details
-
Copy full SHA for c3bdfff - Browse repository at this point
Copy the full SHA c3bdfffView commit details
Commits on Oct 27, 2023
-
test: Gateway.NoFetch and GatewayOverLibp2p
adds missing tests for "no fetch" gateways one can expose, in both cases the offline mode is done by passing custom blockservice/exchange into path resolver, which means global path resolver that has nopfs intercept is not used, and the content blocking does not happen on these gateways. needs to be fixed, but at least now we have tests that fail until it is fixed.
Configuration menu - View commit details
-
Copy full SHA for c99068e - Browse repository at this point
Copy the full SHA c99068eView commit details
Commits on Oct 28, 2023
-
fix: use offline path resolvers where appropriate
this fixes the problem described in #10161 (comment) by adding explicit offline path resolvers that are backed by offline exchange, and using them in NoFetch gateways instead of the default online ones
Configuration menu - View commit details
-
Copy full SHA for da2dc47 - Browse repository at this point
Copy the full SHA da2dc47View commit details -
fix(gw): blocked content produces http error 410
requires ipfs/boxo#497 which is based on top of the boxo already used in kubo master to avoid issues caused by ilater commits in boxo main
Configuration menu - View commit details
-
Copy full SHA for 3d8feac - Browse repository at this point
Copy the full SHA 3d8feacView commit details -
Configuration menu - View commit details
-
Copy full SHA for d4766b6 - Browse repository at this point
Copy the full SHA d4766b6View commit details
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.