Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

git/odb/pack: implement *ChainBase for unpacking base components #2551

Merged
merged 2 commits into from
Sep 12, 2017

Conversation

ttaylorr
Copy link
Contributor

@ttaylorr ttaylorr commented Sep 6, 2017

This pull request provides the first of two implementations of the Chain interface:

In the following two pull requests in this series, two implementations of Chain will be provided: *ChainBase and *ChainDelta, which will decompress and apply patch deltas respectively.

*ChainBase handles the first kind of component of the delta-base chains found in Git packfiles: the base. A base encodes some base object's data in a zlib-compressed format, to which delta operations can be applied (patched).

Since the *ChainBase must handle zlib-compressed data, we run into the challenge of converting the mmap(2)'d *os.File (given as an io.ReaderAt) into an io.Reader that package compress/zlib can read from. This is done using the *OffsetReaderAt type, which implements the io.Reader interface by encoding an offset into an io.ReaderAt, and updating the offset each time a read is performed.

Because we cannot readily determine the length of the compressed object data in a packfile, we must take into account the greedy behavior from package compress/zlib. One approach to dealing with this is to find the next object, and limit reads from going beyond that. However, since we are not sharing *OffsetReaderAt instances with more than one Chain implementation, we can read past the end of the compressed data, and compress/zlib will simply ignore the extraneous bytes. Here is an example.


/cc @git-lfs/core @peff
/cc #2415

@ttaylorr ttaylorr changed the base branch from git-odb-pack-scaffolding to master September 12, 2017 19:46
@ttaylorr ttaylorr merged commit 76b1fe0 into master Sep 12, 2017
@ttaylorr ttaylorr deleted the git-odb-pack-chain-base branch September 12, 2017 19:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants