pkgng provider on FreeBSD does not do BATCH=yes

[oogali]

The pkgng provider on FreeBSD does not use the "BATCH=yes" environment setting when installing new packages.

This is problematic when installing packages that have an interactive post-install script (e.g. mail/postfix). The lack of the batch flag causes pkg to forever wait for a user response (which it will never receive), which in turn causes the minion to "hang".

# pkg install -qy postfix
===> Creating users and/or groups.
Using existing group 'mail'.
Creating group 'maildrop' with gid '126'.
Creating group 'postfix' with gid '125'.
Creating user 'postfix' with uid '125'.
Adding user 'postfix' to group 'mail'.
Would you like to activate Postfix in /etc/mail/mailer.conf [n]? ^C

But, because the minion is still reachable by the master, your highstate never completes or times out, and you then have to log in, kill the minion child, manually install the package in question, and then restart your highstate run.

# salt --versions-report
           Salt: 2015.5.2
         Python: 2.7.9 (default, Aug 27 2015, 01:28:02)
         Jinja2: 2.7.3
       M2Crypto: 0.22
 msgpack-python: 0.4.2
   msgpack-pure: Not Installed
       pycrypto: 2.6.1
        libnacl: Not Installed
         PyYAML: 3.11
          ioflo: Not Installed
          PyZMQ: 14.6.0
           RAET: Not Installed
            ZMQ: 4.1.3
           Mako: Not Installed

[jfindlay]

@oogali, thanks for the report.

[oogali]

Any traction? This is preventing me from fully using salt-cloud to orchestrate hosts in Amazon EC2.

[jfindlay]

@oogali, not that I know of. You could ask @cedwards or @kim0 if they think this would be an easy fix.

[abednarik]

HI @oogali

I'm not an expert, but I'm trying to fix this. Would be great if you can test this, of course not in production. I do not have any FreeBSD around.,

Thanks

#29707

[abednarik]

Sorry for any inconvenience, but please read my last comment #29707
I was trying to fix the old pkd module and this one is related to the new one.

[oogali]

@abednarik Adding -y is not enough. -y simply tells pkg not to prompt to install the package, but has no effect on interactive post-install scripts.

See my original report in which I have -y in the arguments passed to pkg in a shell, but it still prompts me to take action.

[abednarik]

Thanks @oogali. In that case we also need ASSUME_ALWAYS_YES=YES is this correct?

Cheers

[ohauer]

No, ASSUME_ALWAYS_YES is he same as the already present "-y".
The issue is in the postfix installation script!

The script waits for an user input [y/n] without timeout (read -p).
In case the env. POSTFIX_DEFAULT_MTA=yes is set the default for the answer will be "yes" else "no".

In my opinion the solution is to set a timeout in the postfix install script read loop, say 5min and then go on with proceeding the install script using the default "no" or in case the env. POSTFIX_DEFAULT_MTA=yes was found with "yes" as the answer.

Another idea is to use additional "test -t" inside the postfix install script and detect this way if the package is installed without terminal...

A workaround would be to install "/etc/mail/mailer.conf" with postfix as default before installing the postfix package (salt-cp or managed file)

I suspect with a fixed fixed postfix install script, something like this will do the trick

postfix:
 - env:
  - POSTFIX_DEFAULT_MTA: yes

Maybe someone has an insight if there is an salt specific environment variable that can be used for further decisions inside a script?

Given the date I can only work next week on a good solution but I expect I can update the ports before next year.

[oogali]

@ohauer I think the high-level solution of BATCH=yes would more than suffice, and scale (in terms of development effort) for installing any and all packages on FreeBSD that have some sort of interactive post-install script.

The BATCH environment variable is a standard pattern within post-install scripts that instructs ports/pkg to not even ask the interactive question, so there's no need for the fragile logic of sleep-poll-decision.

See:

• https://www.freebsd.org/doc/en_US.ISO8859-1/books/porters-handbook/book.html#slow-user-input
• https://github.com/freebsd/freebsd-ports/blob/master/mail/postfix-current/files/pkg-install.in#L39

[rallytime]

It appears from the comments that this still needs a little bit of work. Removing the Fixed Pending Verification label.

[abednarik]

Ok. Thanks for that information.

I can add a new optional param to set that env variable when running this module.

[oogali]

Any update here?

[jfindlay]

@abednarik, are you still able to work on this?

[abednarik]

HI

I can have a look again. I'm using SaltStack in production under FreeBSD. In the next days will check this again.

Thanks.

[bgdnlp]

Hello. Any news on this issue?

[fzipi]

I have this exact problem when installing postfix using the postfix-formula.

First I tested using env vars and shell, with these results (using bash):

[root@salt]# export ASSUME_ALWAYS_YES=YES
[root@salt]# pkg install -y postfix
Updating FreeBSD repository catalogue...
FreeBSD repository is up to date.
Updating Perceptyx repository catalogue...
Perceptyx repository is up to date.
All repositories are up to date.
Checking integrity... done (0 conflicting)
The following 1 package(s) will be affected (of 0 checked):

New packages to be INSTALLED:
	postfix: 3.3.1_1,1 [FreeBSD]

Number of packages to be installed: 1

The process will require 7 MiB more space.
[1/1] Installing postfix-3.3.1_1,1...
===> Creating groups.
Using existing group 'mail'.
Using existing group 'maildrop'.
Using existing group 'postfix'.
===> Creating users
Using existing user 'postfix'.
[1/1] Extracting postfix-3.3.1_1,1: 100%
Would you like to activate Postfix in /usr/local/etc/mail/mailer.conf [n]?

So it keeps waiting there for interaction, even when explicitly adding ASSUME_ALWAYS_YES=YES.

Next step is using BATCH=true:

[root@salt]# export BATCH=true
[root@salt]# pkg install postfix
Updating FreeBSD repository catalogue...
FreeBSD repository is up to date.
Updating Perceptyx repository catalogue...
Perceptyx repository is up to date.
All repositories are up to date.
Checking integrity... done (0 conflicting)
The following 1 package(s) will be affected (of 0 checked):

New packages to be INSTALLED:
	postfix: 3.3.1_1,1 [FreeBSD]

Number of packages to be installed: 1

The process will require 7 MiB more space.
[1/1] Installing postfix-3.3.1_1,1...
===> Creating groups.
Using existing group 'mail'.
Using existing group 'maildrop'.
Using existing group 'postfix'.
===> Creating users
Using existing user 'postfix'.
[1/1] Extracting postfix-3.3.1_1,1: 100%

===============================================================
Postfix was *not* activated in /usr/local/etc/mail/mailer.conf! 

To finish installation run the following commands:

  mkdir -p /usr/local/etc/mail
  install -m 0644 /usr/local/share/postfix/mailer.conf.postfix /usr/local/etc/mail/mailer.conf
===============================================================

Message from postfix-3.3.1_1,1:

To use postfix instead of sendmail:
  - clear sendmail queue and stop the sendmail daemons

Run the following commands to enable postfix during startup:
  - sysrc postfix_enable="YES"
  - sysrc sendmail_enable="NONE"

If postfix is *not* already activated in /usr/local/etc/mail/mailer.conf
  - mv /usr/local/etc/mail/mailer.conf /usr/local/etc/mail/mailer.conf.old
  - install -m 0644 /usr/local/share/postfix/mailer.conf.postfix /usr/local/etc/mail/mailer.conf

Disable sendmail(8) specific tasks,
add the following lines to /etc/periodic.conf(.local):
  daily_clean_hoststat_enable="NO"
  daily_status_mail_rejects_enable="NO"
  daily_status_include_submit_mailq="NO"
  daily_submit_queuerun="NO"

If you are using SASL, you need to make sure that postfix has access to read
the sasldb file.  This is accomplished by adding postfix to group mail and
making the /usr/local/etc/sasldb* file(s) readable by group mail (this should
be the default for new installs).

If you are upgrading from prior postfix version, please see the README
files for recommended changes to your configuration and additional
http://www.postfix.org/COMPATIBILITY_README.html

So using that the package gets installed.

So for me working solution will be to force install using batch when you add 'Force=true' for example.

I created PR #48730 with this fix.