Logical backup fails after major version upgrade unless manual removal of CronJobs

[chitoku-k]

• Which image of the operator are you using?
  • registry.opensource.zalan.do/acid/postgres-operator:v1.7.1
• Where do you run it - cloud or metal? Kubernetes or OpenShift?
  • Bare Metal K8s
• Are you running Postgres Operator in production?
  • Yes
• Type of issue?
  • Bug report or possibly a feature request

I have an instance of Postgres Operator running with the major version upgrade enabled:

apiVersion: acid.zalan.do/v1
kind: OperatorConfiguration
metadata:
  name: postgresql-operator-default-configuration
configuration:
  major_version_upgrade:
    major_version_upgrade_mode: full

When I change the spec.postgresql.version filed in a postgresql resource from 13 to 14, the PostgreSQL instance gets successfully updated; however, the CronJobs for logical backup that belongs to that PostgreSQL instance would not get updated and the next logical backup fails with the following error:

pg_dumpall: error: aborting because of server version mismatch
pg_dumpall: error: server version: 14.0 (Ubuntu 14.0-1.pgdg18.04+1); pg_dumpall version: 13.4 (Ubuntu 13.4-4.pgdg18.04+1)

It seems that it's because the PG_VERSION filed in the spec.jobTemplate.spec.template.spec.container[0].env is not updated after the major version upgrade. After a manual removal of such CronJobs, reconciliation automatically creates a new CronJob resource whose PG_VERSION has a new version and solves this issue. It would be nice if Postgres Operator has an automatic update for that field too.

Thanks in advance!

[mprimeaux]

Thanks for this detail. I just ran into this in production and can confirm the above.