-
Notifications
You must be signed in to change notification settings - Fork 161
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
fix: pass links to store when adding data #299
fix: pass links to store when adding data #299
Conversation
cf1fa7f
to
2c2309f
Compare
7a51f59
to
858ac6f
Compare
I am now passing through all the links, so at all places where there was a (Cid, Bytes) before there is a (Cid, Bytes, Vec> now. But I don't like it very much. It is a bit verbose and error prone. So will try making a struct and see how that goes... |
3424a56
to
0188ff1
Compare
0188ff1
to
d16ed98
Compare
@dignifiedquire had a bit of a fun time merging this with the changes from @ramfox in 6ba6b57 , but I think it turned out OK. Regarding the tests: I have added validate to block, but I am not quite sure where to put the tests. I can easily validate that the blocks that are produced by file.encode or dir.encode are correct by adding a call to validate in stream_to_resolver, which is used by the roundtrip tests. But I think what you might want is to actually test that links are properly passed to the store in add_file or add_dir, which would be big enough to do a separate issue. I think there is currently no test at all for unixfs_builder::add_dir or unixfs_builder::add_file. And this would have to be more like an integration test that creates a temp dir etc. If you want I can add it to this PR, but it will take some time. Edit: added the former because why not. |
Lets get this in and make a new issue for adding adding tests |
Made a followup issue for the proposed tests: n0-computer/beetle#152 |
5818b45
to
a95f3e5
Compare
This had to be merged with some other changes related to the size info. Introduce a Block struct that has data, hash and links for DRY.
Validate blocks in all roundtrip tests to make sure that the blocks that are returned from file.encode or dir.encode are internally consistent.
a95f3e5
to
1867a42
Compare
The old code was creating new ids even if we already knew the cid, which caused some havoc.
1867a42
to
9407d13
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nice catch
Note that this may lead to decreased performance, since we are doing more work than before. But the store needs this information.
The way links are parsed can be optimized, see n0-computer/beetle#162 . And I am not quite sure if finding links should not be done on the store side. But that is another discussion to be had.
Fixes n0-computer/beetle#167