Skip to content

Commit

Permalink
server: proxy uploads in no-cache mode, allow rewriting of object bat…
Browse files Browse the repository at this point in the history 
…ch action href
  • Loading branch information
@saracen
saracen committed Apr 2, 2019
1 parent 62cb8a2 commit c7033ce
Showing 1 changed file with 10 additions and 2 deletions.
12 changes: 10 additions & 2 deletions server/server.go
View file
Original file line number Diff line number Diff line change
Expand Up @@ -81,6 +81,10 @@ type originalHost struct {
host string
}

func DefaultObjectBatchActionURLRewriter(href *url.URL) *url.URL {
return href
}

// Server is a LFS caching server.
type Server struct {
logger log.Logger
Expand All @@ -89,6 +93,8 @@ type Server struct {
cache *cache.FilesystemCache
client *http.Client
hmacKey [64]byte

ObjectBatchActionURLRewriter func(href *url.URL) *url.URL
}

// New returns a new LFS proxy caching server.
Expand Down Expand Up @@ -125,6 +131,7 @@ func newServer(logger log.Logger, upstream, directory string, cacheEnabled bool)
ExpectContinueTimeout: 1 * time.Second,
},
},
ObjectBatchActionURLRewriter: DefaultObjectBatchActionURLRewriter,
}

_, err = rand.Read(s.hmacKey[:])
Expand Down Expand Up @@ -217,7 +224,7 @@ func (s *Server) batch() *httputil.ReverseProxy {
// modify batch request urls
for _, object := range br.Objects {
for operation, action := range object.Actions {
if operation != "download" {
if operation != "download" && s.cache != nil {
continue
}
if action.Header == nil {
Expand All @@ -242,7 +249,7 @@ func (s *Server) batch() *httputil.ReverseProxy {
action.Header[UpstreamHeaderList] = strings.Join(list, ";")
action.Header[OriginalHrefHeader] = action.Href
action.Header[SizeHeader] = strconv.Itoa(int(object.Size))
action.Href = (&url.URL{
action.Href = s.ObjectBatchActionURLRewriter(&url.URL{
Scheme: scheme,
Host: host.host,
Path: ContentCachePathPrefix + object.OID,
Expand Down Expand Up @@ -303,6 +310,7 @@ func (s *Server) nocache() *httputil.ReverseProxy {
return
}

req.Host = originalURL.Host
req.URL = originalURL
req.Header = header
}
Expand Down

3 comments on commit c7033ce

@saikrish03
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@saracen - I am trying to use your cache proxy. But I am getting
batch response: Repository or object not found: http://my-cache-proxy:9876/my/cache/dir/lfs/objects/batch
Check that it exists and that you have proper access to it

This is when I run git lfs pull and have
./lfscache --url -directory /my/cache/dir/lfs --http-addr=:9876 running fine -
level=info ts=2019-08-27T22:39:32.723827678Z event=listening proxy-endpoint= transport=HTTP addr=:9876

Does this work for Gitlab LFS? Is there anyway you can point out what is happening.

@saracen
Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It should work fine with Gitlab LFS.

The lfscache command example you've provided is missing a value for the --url argument, was this just removed because it's private? The value is usually the address to your gitlab repo, including .git/info/lfs.

If that doesn't solve it, can you please open an Issue instead of replying here?

@saikrish03
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I removed the URL because its my companies private information. But per the README I used the right URL.
I am seeing some authorization issues. Also when I run the docker command, the container starts but I do not see a public port exposed. Is that an additional step?

Please sign in to comment.