-
Notifications
You must be signed in to change notification settings - Fork 50
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
introduce LinkSystem #143
Merged
Merged
introduce LinkSystem #143
Commits on Feb 25, 2021
-
This significantly reworks how linking is handled. All of the significant operations involved in storing and loading data are extracted into their own separate features, and the LinkSystem just composes them. The big advantage of this is we can now add as many helper methods to the LinkSystem construct as we want -- whereas previously, adding methods to the Link interface was a difficult thing to do, because that interface shows up in a lot of places. Link is now *just* treated as a data holder -- it doesn't need logic attached to it directly. This is much cleaner. The way we interact with the CID libraries is also different. We're doing multihash registries ourselves, and breaking our direct use of the go-multihash library. The big upside is we're now using the familiar and standard hash.Hash interface from the golang stdlib. (And as a bonus, that actually works streamingly; go-mulithash didn't.) However, this also implies a really big change for downstream users: we're no longer baking as many hashes into the new multihash registry by default.
Configuration menu - View commit details
-
Copy full SHA for a1482fe - Browse repository at this point
Copy the full SHA a1482feView commit details -
Extract multi{codec,hash} registries better.
And, make a package which can be imported to register "all" of the multihashes. (Or at least all of them that you would've expected from go-multihash.) There are also packages that are split roughly per the transitive dependency it brings in, so you can pick and choose. This cascaded into more work than I might've expected. Turns out a handful of the things we have multihash identifiers for actually *do not* implement the standard hash.Hash contract at all. For these, I've made small shims. Test fixtures across the library switch to using sha2-512. Previously I had written a bunch of them to use sha3 variants, but since that is not in the standard library, I'm going to move away from that so as not to re-bloat the transitive dependency tree just for the tests and examples.
Configuration menu - View commit details
-
Copy full SHA for 8fef531 - Browse repository at this point
Copy the full SHA 8fef531View commit details
Commits on Mar 12, 2021
-
Configuration menu - View commit details
-
Copy full SHA for 45b8e9c - Browse repository at this point
Copy the full SHA 45b8e9cView commit details -
LinkSystem: detect direct byte access features on readers and attempt…
… to use it. I'm still not fully convinced this is always a win for performance, but as discussed in #143 (comment) , there was past evidence, and reproducing that takes work -- so, we're going to try to be relatively conservative here and keep this logical branch behavior in place for now. (The reason this may be worth re-examining is that the change to hashing interface probably gets rid of one big source of copies. That doens't answer the holistic performance question alone, though; it just reopens it.)
Configuration menu - View commit details
-
Copy full SHA for 08f13e5 - Browse repository at this point
Copy the full SHA 08f13e5View commit details -
Configuration menu - View commit details
-
Copy full SHA for 8fcc676 - Browse repository at this point
Copy the full SHA 8fcc676View commit details -
Merge branch 'master' into linksystem
Resolves conflicts in go.sum. (They were conflicts in a textual sense only, but semantically trivial; application of `go mod tidy` was sufficient.)
Configuration menu - View commit details
-
Copy full SHA for 705307f - Browse repository at this point
Copy the full SHA 705307fView commit details -
Configuration menu - View commit details
-
Copy full SHA for 8a7497f - Browse repository at this point
Copy the full SHA 8a7497fView commit details -
Update some comments regarding multihash.
(Some things got better. Several others in the area still have not.)
Configuration menu - View commit details
-
Copy full SHA for 6386588 - Browse repository at this point
Copy the full SHA 6386588View commit details -
Better document why some of the branches around direct byte slice acc…
…ess matter in LinkSystem. (It can be hard to intuit this just by reading the code, because some of the key relevance is actually in *other* functions that might not be in this repo at all!)
Configuration menu - View commit details
-
Copy full SHA for d9d5682 - Browse repository at this point
Copy the full SHA d9d5682View commit details -
Configuration menu - View commit details
-
Copy full SHA for 8ef5eab - Browse repository at this point
Copy the full SHA 8ef5eabView commit details
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.