Podman push to Docker Hub does not "finalize"?

[carlpett]

Is this a BUG REPORT or FEATURE REQUEST? (leave only one on its own line)
/kind bug

Description
Pushing an image to Docker hub does not seem to "finish all the parts". I'm not sure where exactly it misses something, but it seems the final manifest might not be uploaded?
Docker Hub seems a bit confused about it too - the image doesn't show up in their views, but when working with the special latest tag, it is possible to download it anyway. And pushing again shows that it already has the image

Steps to reproduce the issue:

1. podman build -t myusername/some-new-image:latest
2. podman push myusername/some-new-image:latest
3. (On another machine) pull myusername/some-new-image:latest works
4. podman tag myusername/some-new-image:latest myusername/some-new-image:v1.0.0
5. podman push myusername/some-new-image:v1.0.0
6. (On another machine) pull myusername/some-new-image:v1.0.0 does not work
7. docker push myusername/some-new-image:latest
8. (On another machine) pull myusername/some-new-image:v1.0.0 works

Describe the results you received:
Unless pushing with the docker client, tagged images aren't indexed properly by Docker Hub (I believe). And, excepting the latest tag which seems to get special treatment there, they thus cannot be downloaded.

Describe the results you expected:
Works as well as the docker client.

Additional information you deem important (e.g. issue happens only occasionally):
I started this by pushing to a brand new repo. But even after pushing new tags with podman after getting stuff working with the docker client seems to show the same issue.

Output of podman version:

Version:            1.5.1
RemoteAPI Version:  1
Go Version:         go1.12.7
OS/Arch:            linux/amd64

Output of podman info --debug:

debug:
  compiler: gc
  git commit: ""
  go version: go1.12.7
  podman version: 1.5.1
host:
  BuildahVersion: 1.10.1
  Conmon:
    package: podman-1.5.1-3.fc30.x86_64
    path: /usr/libexec/podman/conmon
    version: 'conmon version 2.0.0, commit: d728afa06cd2df86a27f32a4692c7099a56acc97-dirty'
  Distribution:
    distribution: fedora
    version: "30"
  MemFree: 13342265344
  MemTotal: 33598476288
  OCIRuntime:
    package: runc-1.0.0-93.dev.gitb9b6cc6.fc30.x86_64
    path: /usr/bin/runc
    version: |-
      runc version 1.0.0-rc8+dev
      commit: e3b4c1108f7d1bf0d09ab612ea09927d9b59b4e3
      spec: 1.0.1-dev
  SwapFree: 0
  SwapTotal: 0
  arch: amd64
  cpus: 8
  eventlogger: journald
  hostname: XXXXX
  kernel: 5.2.9-200.fc30.x86_64
  os: linux
  rootless: true
  uptime: 78h 9m 3.81s (Approximately 3.25 days)
registries:
  blocked: null
  insecure: null
  search:
  - docker.io
  - registry.fedoraproject.org
  - quay.io
  - registry.access.redhat.com
  - registry.centos.org
store:
  ConfigFile: /home/XXXXX/.config/containers/storage.conf
  ContainerStore:
    number: 49
  GraphDriverName: overlay
  GraphOptions:
  - overlay.mount_program=/bin/fuse-overlayfs
  GraphRoot: /home/XXXXX/.local/share/containers/storage
  GraphStatus:
    Backing Filesystem: extfs
    Native Overlay Diff: "false"
    Supports d_type: "true"
    Using metacopy: "false"
  ImageStore:
    number: 309
  RunRoot: /tmp/1000
  VolumePath: /home/XXXXX/.local/share/containers/storage/volumes

Package info (e.g. output of rpm -q podman or apt list podman):

podman-1.5.1-3.fc30.x86_64

Additional environment details (AWS, VirtualBox, physical, etc.):
Fedora 30 laptop

[mheon]

Hm. Are we pushing the image successfully, but not the tags, I wonder?

[mheon]

@vrothberg You're a lot more familiar with the image side of things than I am - mind taking a look?

[carlpett]

There's probably some sort of bug/usability issue with Docker Hub as well, but I couldn't find any way to report it to them... If you have any connections, might make sense to get them to look at it too :)

[vrothberg]

@carlpett, @mheon, thanks for opening the issue and the ping. I will have a look at this tomorrow morning (about to shut down machines now).

[TomSweeneyRedHat]

Is this related to: docker/hub-feedback#1871? Which first cropped up in containers/buildah#1826. It looks like Docker hub is having problems with OCI images at the moment and that should be fixed next week. I think this is the same.

[vrothberg]

Pushing non-latest tags to the Docker Hub works now. Closing the issue. Again thanks for opening the issue 👍