image:push to private registries is broken when using hostname/repo

[dnephin]

I believe you should be able to push to a private registry by using the full name of the image that includes the host/port/path to the registry.

dobi supports the same auth config as the docker client, so authentication should work as well.

I need to test it out to confirm.

cc @jmmills (let's move the discussion here, out of docker/compose#1126)

[jmmills]

Personally I think it would be most useful as a parameter that can be carried into the build via environment variable.

I've always been a bit dissatisfied having to include a hostname (of a registry host) in the namespace declaration of an image.

[dnephin]

I suppose if image.image supported variables you would be able to do something like:

image=myimage:
    image: '{env.registry:}user/myimage'

[tedstirm]

That or if there was some way in the tags field to access the image name so you could do something like:

tags:
   - '{env.registry}/{image.name}'

I would like to be able to tag an image locally with a name, then prepend the private registry it belongs to later via an alias command that applies all my tags.

[dnephin]

Right now tags are only the "tag" part of the image name, so latest, 3.2, etc. It seems like it would be a reasonable addition to allow these to be full name:tag strings as well. I think that would be good.

I'm not sure about {image.name}. The value would have to change per-resource, so the variable resolution would need to include that context. It's certainly possible, but it would add some complexity to it. For now it seems like it would be best to just repeat the name. If it becomes a common pattern it could be added.

[jzakrzeski]

I've actually been evaluating dobi to include as part of our CI pipeline with Docker containers, and this is a big need. I've attempted a setup like this for a ruby app I maintain:

meta:
  project: cattle-prod

# Mounts #

mount=source:
  bind: ./cattle-prod
  path: /code
  description: "source code directory"

mount=project:
  bind: .
  path: '{fs.projectdir}'
  description: "root project directory"

# Images #

image=cattle-prod:
  image: my.private.registry.net/cattle-prod
  context: ./cattle-prod
  tags: ['{git.short-sha}']
  dockerfile: Dockerfile
  description: "image to be used for testing/development"

Running dobi cattle-prod:push always falls back to trying to push to the hub despite having named the image as I would normally doing docker push. It does handle the tagging (and tests) very nicely, but I'd prefer to be able to go end-to-end with it so I can define jobs in the git repository instead of having to manually manage jenkins jobs per git repo.

Here's what I get using dobi with this project:

[DEBUG] Using default registry "https://index.docker.io/v1/"
[WARN] Missing auth config for "https://index.docker.io/v1/"
The push refers to a repository [my.private.registry.net/cattle-prod]

I've tried some thing like setting mounts as depends: for the image that include my ~/.docker directory mapped (which has the auths for that registry cached), but to no avail--still times-out trying to push to the hub rather than my registry, though it is seen. Granted, I've been looking at this solidly for the past hour so I could just be missing something obvious unless dobi does, indeed, not support this properly.

[cescoferraro]

@jzakrzeski
image push parses the image like

func parseAuthRepo(image string) (string, error) {
    // This is the approximate logic from
    // github.com/docker/docker/reference.splitHostname(). That package is
    // conflicting with other dependencies, so it can't be imported at this time.
    parts := strings.SplitN(image, "/", 3)
    switch len(parts) {
    case 1, 2:
        logging.Log.Debugf("Using default registry %q", defaultRepo)
        return defaultRepo, nil
    default:
        logging.Log.Debugf("Using registry %q", parts[0])
        return parts[0], nil
    }
}

It splits by slashes. I would try http://my.private.registry.net/cattle-prod

[dnephin]

I thought I had this fixed in version 0.7. Are you using a recent
version? I think the logic assumes that there is always a
username/reponame after the host name. I guess that assumption is wrong?

On Nov 2, 2016 6:47 PM, "cesco" notifications@github.com wrote:

@jzakrzeski https://github.com/jzakrzeski
image push parses the image like

func parseAuthRepo(image string) (string, error) {
// This is the approximate logic from
// github.com/docker/docker/reference.splitHostname(). That package is
// conflicting with other dependencies, so it can't be imported at this time.
parts := strings.SplitN(image, "/", 3)
switch len(parts) {
case 1, 2:
logging.Log.Debugf("Using default registry %q", defaultRepo)
return defaultRepo, nil
default:
logging.Log.Debugf("Using registry %q", parts[0])
return parts[0], nil
}
}

It splits by slashes. I would try http://my.private.registry.
net/cattle-prod

—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
#6 (comment), or mute
the thread
https://github.com/notifications/unsubscribe-auth/AAa_RH1GTYGcN3kKI4Dxal_8GkGSn0eoks5q6RL5gaJpZM4JVxEJ
.

[jzakrzeski]

@dnephin Tried that, blows up as invalid repository/tag:

[DEBUG] [image:build cattle-prod] http://my.private.registry.net/cattle-prod time=2016-11-02 19:00:17.568532586 -0400 EDT Start
[DEBUG] [image:build cattle-prod] http://my.private.registry.net/cattle-prod Image does not exist
[DEBUG] [image:build cattle-prod] http://my.private.registry.net/cattle-prod is stale
[DEBUG] stopping tasks
[ERROR] Failed to execute task "cattle-prod:build": API error (500): Error parsing reference: "http://my.private.registry.net/cattle-prod:42475a955b" is not a valid repository/tag

In our case, yes, we use the convention: my.private.registry.net/{repository}/{service}:{short-sha}

Our current process involves doing docker build -t my.private.registry.net/{repository}/{service}:{short-sha} ./directory and docker push my.private.registry.net/{repository}/{service}:{short-sha} which works perfectly fine using the docker client locally and on our build server.

"cattle-prod" is done a bit differently (probably mistakenly), as you can see, with just the repository name followed by a version tag. I just tried this with: image: my.private.registry.net/cattle-prod/cattle-prod and it did work perfectly. The logic holds up for the "correct" use of slashes by any normal standard on our part, too. I think we just hit and edge case here.

I do want to plus the variable interpolation in the image field though, that could be very useful for DRYing some build/test jobs and finding commonality across different repositories.

[cescoferraro]

@dnephin this is definetly a bug, it does not happen on docker's splitHostname.
Check this out
https://play.golang.org/p/5w_AYvSgTf

Docker's code also breaks when using http syntax http://my.private.registry.net/cattle-prod.
So we need to inject it ourselfes, but how are we gonna know if the registry serves http or https ?

[jzakrzeski]

FWIW, my registry is https and works flawlessly with my.private.registry.net/cattle-prod/cattle-prod

I agree that this is a bug. Docker supports it well even using my.private.registry.net/cattle-prod:tag

[dnephin]

Ok, we definitely need to fix this bug.

What I'm not sure about is how it's able to tell the difference between say: dnephin/dobi and mycustomhostname/repo. They have the exact same structure, but one expands to a hub repo, and the other a private registry.

Maybe it makes some assumptions about the first part of the name before the slash? Maybe if it finds a . it assumes hostname?

I hope to have some time this weekend to get a fix out.

[dnephin]

It looks like I missed an important part of splitHostname() which is that if it sees any . or : it assumes hostname. That should be an easy fix.

[dnephin]

Fix in #54