Failure when using the -tz flag or setting tz in containers conf on macos

[Andrewpk]

Is this a BUG REPORT or FEATURE REQUEST? (leave only one on its own line)

/kind bug

Description

When using the tz flag or setting something like tz = "local" in your containers.conf, I get the following error when starting up a container.

Error: unable to start container 533cb99b804d99c0f60a84d2e016d9edd06fbe91e72faaea7b375cb0f90d75b3: error preparing container 533cb99b804d99c0f60a84d2e016d9edd06fbe91e72faaea7b375cb0f90d75b3 for attach: error finding local timezone for container 533cb99b804d99c0f60a84d2e016d9edd06fbe91e72faaea7b375cb0f90d75b3: lstat /etc/localtime: no such file or directory
125

Steps to reproduce the issue:

1. Add tz = "local" to containers block in containers.conf

2. Run a container - the latest containers I tried to run using a compose file:

wget https://raw.githubusercontent.com/grafana/loki/v2.3.0/production/docker-compose.yaml -O docker-compose.yaml
podman-compose -f docker-compose.yaml up

You don't have to use a compose file though - this happens with any container I've tried so far.

Describe the results you received:

Error: unable to start container f030babd6835c7f42764f4256802797f9a7615b291f4f9c714b3d59ebb051916: error preparing container f030babd6835c7f42764f4256802797f9a7615b291f4f9c714b3d59ebb051916 for attach: error finding local timezone for container f030babd6835c7f42764f4256802797f9a7615b291f4f9c714b3d59ebb051916: lstat /etc/localtime: no such file or directory
125

Describe the results you expected:
Podman to start the container normally and the time to be set in the container accordingly.

Additional information you deem important (e.g. issue happens only occasionally):
When time is off, lots of weird problems occur so even if this isn't deemed a bug the documentation may need to be updated to include information as to how a user can get time to sync appropriately from host to child.

Output of podman version:

Client:
Version:      3.4.0
API Version:  3.4.0
Go Version:   go1.17.1
Built:        Thu Sep 30 14:44:31 2021
OS/Arch:      darwin/amd64

Server:
Version:      3.3.1
API Version:  3.3.1
Go Version:   go1.16.6
Built:        Mon Aug 30 16:46:36 2021
OS/Arch:      linux/amd64

Output of podman info --debug:

host:
  arch: amd64
  buildahVersion: 1.22.3
  cgroupControllers: []
  cgroupManager: systemd
  cgroupVersion: v2
  conmon:
    package: conmon-2.0.29-2.fc34.x86_64
    path: /usr/bin/conmon
    version: 'conmon version 2.0.29, commit: '
  cpus: 1
  distribution:
    distribution: fedora
    version: "34"
  eventLogger: journald
  hostname: localhost
  idMappings:
    gidmap:
    - container_id: 0
      host_id: 1000
      size: 1
    - container_id: 1
      host_id: 100000
      size: 65536
    uidmap:
    - container_id: 0
      host_id: 1000
      size: 1
    - container_id: 1
      host_id: 100000
      size: 65536
  kernel: 5.14.9-200.fc34.x86_64
  linkmode: dynamic
  logDriver: ""
  memFree: 1537736704
  memTotal: 2061860864
  ociRuntime:
    name: crun
    package: crun-1.0-1.fc34.x86_64
    path: /usr/bin/crun
    version: |-
      crun version 1.0
      commit: 139dc6971e2f1d931af520188763e984d6cdfbf8
      spec: 1.0.0
      +SYSTEMD +SELINUX +APPARMOR +CAP +SECCOMP +EBPF +CRIU +YAJL
  os: linux
  remoteSocket:
    exists: true
    path: /run/user/1000/podman/podman.sock
  security:
    apparmorEnabled: false
    capabilities: CAP_CHOWN,CAP_DAC_OVERRIDE,CAP_FOWNER,CAP_FSETID,CAP_KILL,CAP_NET_BIND_SERVICE,CAP_SETFCAP,CAP_SETGID,CAP_SETPCAP,CAP_SETUID,CAP_SYS_CHROOT
    rootless: true
    seccompEnabled: true
    seccompProfilePath: /usr/share/containers/seccomp.json
    selinuxEnabled: true
  serviceIsRemote: true
  slirp4netns:
    executable: /usr/bin/slirp4netns
    package: slirp4netns-1.1.12-2.fc34.x86_64
    version: |-
      slirp4netns version 1.1.12
      commit: 7a104a101aa3278a2152351a082a6df71f57c9a3
      libslirp: 4.4.0
      SLIRP_CONFIG_VERSION_MAX: 3
      libseccomp: 2.5.0
  swapFree: 0
  swapTotal: 0
  uptime: 3h 10m 7.26s (Approximately 0.12 days)
plugins:
  log: null
  network: null
  volume: null
registries:
  search:
  - docker.io
store:
  configFile: /var/home/core/.config/containers/storage.conf
  containerStore:
    number: 4
    paused: 0
    running: 1
    stopped: 3
  graphDriverName: overlay
  graphOptions: {}
  graphRoot: /var/home/core/.local/share/containers/storage
  graphStatus:
    Backing Filesystem: xfs
    Native Overlay Diff: "true"
    Supports d_type: "true"
    Using metacopy: "false"
  imageStore:
    number: 8
  runRoot: /run/user/1000/containers
  volumePath: /var/home/core/.local/share/containers/storage/volumes
version:
  APIVersion: 3.3.1
  Built: 1630356396
  BuiltTime: Mon Aug 30 20:46:36 2021
  GitCommit: ""
  GoVersion: go1.16.6
  OsArch: linux/amd64
  Version: 3.3.1

Package info (e.g. output of rpm -q podman or apt list podman):

$ brew info podman
podman: stable 3.4.0 (bottled), HEAD
Tool for managing OCI containers and pods
https://podman.io/
/usr/local/Cellar/podman/3.4.0_1 (170 files, 39.2MB) *
  Poured from bottle on 2021-10-04 at 16:40:32
From: https://github.com/Homebrew/homebrew-core/blob/HEAD/Formula/podman.rb
License: Apache-2.0
==> Dependencies
Build: go ✘, go-md2man ✘
Required: qemu ✔
==> Options
--HEAD
	Install HEAD version
==> Caveats
zsh completions have been installed to:
  /usr/local/share/zsh/site-functions
==> Analytics
install: 11,500 (30 days), 22,596 (90 days), 49,126 (365 days)
install-on-request: 11,502 (30 days), 22,593 (90 days), 48,994 (365 days)
build-error: 0 (30 days)

Have you tested with the latest version of Podman and have you checked the Podman Troubleshooting Guide? (https://github.com/containers/podman/blob/master/troubleshooting.md)

Yes

Additional environment details (AWS, VirtualBox, physical, etc.):
Intel macOS 11.6 (20G165)

[rhatdan]

@dustymabe are there limited timezones installed on fedora coreos?
What timezone are you on. If you ssh into the podman machine OS, what timezone does it say?

[ashley-cui]

Looks like fcos is missing /etc/localtime, which we're trying to read when setting timezone to "local"

[core@localhost ~]$ ls -l /etc/localtime
ls: cannot access '/etc/localtime': No such file or directory

[rhatdan]

Is this something we should add in the ignition file to match the MAC?

[rhatdan]

@dustymabe Thoughts?

[github-actions]

A friendly reminder that this issue had no activity for 30 days.

[dustymabe]

@rhatdan - all the timezones are there. we just don't set a local timezone by default (i.e. default to UTC). Maybe podman could/should be modified to assume UTC if no /etc/localtime exists?

[rhatdan]

Sounds reasonable to me.

[rhatdan]

@ashley-cui PTAL
A better solution might be to have ignition grab the timezone off of the MAC and inject it into Fedora CoreOS.

[github-actions]

A friendly reminder that this issue had no activity for 30 days.

[rhatdan]

@ashley-cui Any movement on this? @afbjorklund WDYT?

[ashley-cui]

Not yet, still mostly working on GUI stuff. l might get a chance to look at it soon

[afbjorklund]

No idea about CoreOS, on Cloud I see /etc/localtime -> /usr/share/zoneinfo/Etc/UTC

[lucab]

For reference, Fedora CoreOS sticks to the default on this and follows the specs for localtime:

If /etc/localtime is missing, the default "UTC" timezone is used.

[rhatdan]

So if someone who understood MAC and ignition, they could have the podman machine init command query the timezone on the MAC and then tell ignition to set it within the VM. Could someone take a stab at this. I can't do it because of lack of access to a MAC.

[rhatdan]

@flouthoc Is this something you could do?

[flouthoc]

@rhatdan I don't have a mac machine handy to play around with this.

[rhatdan]

@baude PTAL

[afbjorklund]

Podman machine works on Linux, as far as I know ?
Last time I rented a Mac in the cloud (24 hr minimum)

[rhatdan]

Could you open a PR to add the linux /etc/localtime settings to the ignition file so that the VM matches the localtime of the host? Would this same change work on a MAC?

[baude]

@rhatdan i've got this