Fix object not found error in OFS deltas

[zeripath]

Unfortunately the code in #303 was incorrect for large objects in OFS deltas.

The underlying fault is that the seeker in the packfile was seeking away whilst the
delta was being read.

This PR simply reads the delta into a buffer.

Fix #323

Signed-off-by: Andrew Thornton art27@cantab.net

[starkers]

I just tested this in my project and it appears to have resolved the issue, thanks for the quick turn around @zeripath

EDIT:
Actually.. its truncated some large files for some reason.. I'll have to investigate further

Don't know why these files but ye.. they've been simply wrecked.. I'll try to find some time to investigate further tomorrow

[puerco]

Hi @zeripath, thanks a lot for working on this fix! I'm writing on behalf of the Kubernetes Release Engineering team. It seems that the issue this PR fixes broke the Kubernetes release process and we are unable to cut new versions right now 🙃

I was wondering If I can help you to merge this faster. Do you need help testing the patch or can we assist in any other way from our side? Thanks!!

[zeripath]

@starkers agh. Ok so the packfile must still be being used elsewhere. I guess we need to duplicate the packfile fd for each of these. I'll take a look. Would you be willing to help test this functionality if I adjust this PR so that it switches this off by default?

@puerco - sorry about this. I'll adjust this pr to switch this off by default until we're sure it's working.

[hiddeco]

@zeripath we have been testing this PR in fluxcd/image-automation-controller#173 in the hope it would also solve another issue we have been running into around empty git-upload-pack given errors that are thrown since v5.4.0 (#328).

Is it possible there was another bug introduced somewhere that in some conditions sends empty packages? As this patch does not seem to have an effect.

[mcuadros]

I just created a new release with #329: https://github.com/go-git/go-git/releases/tag/v5.4.2

@zeripath The feature is still useful so feel free to commit it again, once the problems are solved

[zeripath]

@mcuadros I'm deeply sorry about this.

Yeah we do still need to do something about the large files - especially to do with stating of files causing these files to end up memory.

I'll have another go at this. @starkers do you have a simple test case that can be used to replicate the issues?

[puerco]

Thank you very much @zeripath and everyone involved, I can confirm we are able to clone large repos again. I am attaching here the test program I used to diagnose the problem when it hit us. I hope it helps. Let me know if we can help further :)

test-clone.go.gz