Module does not work on RHEL 7

[linuxdan]

Affected Puppet, Ruby, OS and module versions/distributions

• Puppet: 5.5.6 (PE 2018.1.4)
• Module version: 3.2.0

How to reproduce (e.g Puppet code you use)

dwhite2.sec.dte.cert.org.yaml.txt

What are you seeing

etc-rsyslog.conf.txt

and /etc/rsyslog.d/ is empty

What behaviour did you expect instead

Previous /etc/rsyslog.conf: (trimmed of comment lines)
etc-rsyslog.conf-orig.txt
Previous /etc/rsyslog.d/listen.conf:
etc-rsyslog.d-listen.conf-orig.txt

Output log

puppet-run-2018-09-07-09-48-15.txt
Use "less -r" to view without garbage

[dhollinger]

By default this module places all config in a file called /etc/rsyslog.d/50_rsyslog.conf (See: https://github.com/voxpupuli/puppet-rsyslog#config-file), have you checked for that file?

This was done as this module is used across a variety of distros and use cases (including managing a config file mounted as a volume into an Rsyslog docker container). While making this an option was considered, it was passed on as it is purely a preference rather than providing any additional/better functionality. That said, you can manually override the target_file for each config option by adding the target_file option to config block like so:

rsyslog::client::global_config:
  'WorkDirectory':
    value: '/var/lib/rsyslog'
    type: 'legacy'
    target_file: '/etc/rsyslog.d/10_global.conf

While we do support legacy Rsyslog config options, this module's primary target is the Rsyslog 7+ rainerscript style configuration.

[dhollinger]

@linuxdan any updates on this issue?

[linuxdan]

I have not had any spare brain cycles to investigate this.

[dhollinger]

I will leave it open then

[dhollinger]

@linuxdan Any updates? Are you still seeing this issue?

[linuxdan]

Brand new CentOS 7 server, loaded with PuppetServer 6.3.0 and Puppet Agent 6.4.1
(Latest / greatest on https://yum.puppet.com/puppet6/el/7/x86_64/ )
and v3.3.0 of this module.

All I have in the common.yaml is

classes:

• rsyslog::client

Running puppet agent still wipes out the contents of /etc/rsyslog.d and /etc/rsyslog.conf and inserts nothing.
The file that is deleted - /etc/rsyslog.d/listen.conf - is from the systemd RPM.

There are no instructions about basic client setup, so IMHO, it is still busted.

[dhollinger]

@linuxdan The docs need some updating to make this clear, but by default the module will manage and replace the /etc/rsyslog.conf file and clear the /etc/rsyslog.d directory:

defaults: https://github.com/voxpupuli/puppet-rsyslog/blob/master/data/common.yaml#L8-L9
config dir code: https://github.com/voxpupuli/puppet-rsyslog/blob/master/manifests/base.pp#L47-L55
config file (and one more config dir) code: https://github.com/voxpupuli/puppet-rsyslog/blob/master/manifests/base.pp#L64-L84

As for client configuration, the documentation addresses most of the options for client configuration, but this is designed for Rsyslog >= 8.x and expects mostly rainerscript to be used.

There is a large major release coming up, but as I'm largely the only person working on it, it is taking some time.

[fraenki]

@dhollinger Would you agree to add some typical default values to the module data? I'd be able to provide defaults for Redhat and Debian. This would ensure that the module would leave a system's log service in a working condition.

[dhollinger]

I'd be up for defaults for OS-types, though it'd probably be best to keep it to the OS defaults and it'd have to be in the data/ directory.

[linuxdan]

Hey, folks. I finally had time to revisit this. It works for me now mostly thanks to the examples/common.yaml file.

So, here is the default /etc/rsyslog.conf for rsyslog-8.24.0-41.el7_7.2.x86_64
I trimmed out much of the descriptive comments and uncommented the forwarding rules:

# rsyslog configuration file

#### MODULES ####
$ModLoad imuxsock # provides support for local system logging (e.g. via logger command)
$ModLoad imjournal # provides access to the systemd journal

#### GLOBAL DIRECTIVES ####
$WorkDirectory /var/lib/rsyslog
$ActionFileDefaultTemplate RSYSLOG_TraditionalFileFormat
$IncludeConfig /etc/rsyslog.d/*.conf
$OmitLocalLogging on
$IMJournalStateFile imjournal.state

#### RULES ####
*.info;mail.none;authpriv.none;cron.none                /var/log/messages
authpriv.*                                              /var/log/secure
mail.*                                                  -/var/log/maillog
cron.*                                                  /var/log/cron
*.emerg                                                 :omusrmsg:*
uucp,news.crit                                          /var/log/spooler
local7.*                                                /var/log/boot.log


# ### begin forwarding rule ###
$ActionQueueFileName fwdRule1
$ActionQueueMaxDiskSpace 1g
$ActionQueueSaveOnShutdown on
$ActionQueueType LinkedList
$ActionResumeRetryCount -1

*.* @@remote-host:514
# ### end of the forwarding rule ###

and /etc/rsyslog.d/listen.conf :

$SystemLogSocketName /run/systemd/journal/syslog

Here is the initial YAML file I worked out from looking at the example:

---
# Node: CentOS 7

classes:
  - rsyslog::client

rsyslog::client::global_config:
  WorkDirectory:
    type: legacy
    value: '/var/lib/rsyslog'
  ActionFileDefaultTemplate:
    type: legacy
    value: "RSYSLOG_TraditionalFileFormat"
  ActionQueueFileName:
    type: legacy
    value: 'fwdRule1'
  ActionQueueMaxDiskSpace:
    type: legacy
    value: '1g'
  ActionQueueSaveOnShutdown:
    type: legacy
    value: 'on'
  ActionQueueType:
    type: legacy
    value: 'LinkedList'
  ActionResumeRetryCount :
    type: legacy
    value: '-1'

rsyslog::client::modules:
  imjournal:
    config:
      StateFile : 'imjournal.state'
  imuxsock:
    config:
      'SysSock.Use' : 'on'
      socket: '/run/systemd/journal/syslog'

rsyslog::client::legacy_config:
  messages_rule:
    key: "*.info;mail.none;authpriv.none;cron.none" 
    value: "/var/log/messages"
  auth_priv_rule:
    key: "auth,authpriv.*"
    value: "/var/log/secure"
  mail_rule:
    key: "mail.*"
    value: "-/var/log/mail.log"
  cron_rule:
    key: "cron.*"
    value: "/var/log/cron"
  emergency_rule:
    key: "*.emerg"
    value: ":omusrmsg:*"
  news_rule:
    key: "uucp,news.crit"
    value: "/var/log/spooler"
  boot_rule:
    key: "local7.*"
    value: "/var/log/boot.log" 
  remote_host_rule:
    key: "*.*"
    value: "@@remote-host:514"

# vim: ft=yaml:sts=2:sw=2:et

And the resulting /etc/rsyslog.d/50_rsyslog.conf

$ActionFileDefaultTemplate RSYSLOG_TraditionalFileFormat
$ActionQueueFileName fwdRule1
$ActionQueueMaxDiskSpace 1g
$ActionQueueSaveOnShutdown on
$ActionQueueType LinkedList
$ActionResumeRetryCount -1
$WorkDirectory /var/lib/rsyslog
module(load="imjournal" 
          StateFile="imjournal.state"
     
)
module(load="imuxsock" 
          SysSock.Use="on"
          socket="/run/systemd/journal/syslog"
     
)
# auth_priv_rule
auth,authpriv.*     /var/log/secure

# boot_rule
local7.*     /var/log/boot.log

# cron_rule
cron.*     /var/log/cron

# emergency_rule
*.emerg     :omusrmsg:*

# mail_rule
mail.*     -/var/log/mail.log

# messages_rule
*.info;mail.none;authpriv.none;cron.none     /var/log/messages

# news_rule
uucp,news.crit     /var/log/spooler

# remote_host_rule
# *.*     @@remote-host:514

I thought that looks great.
However, I think the ordering matters, so I wanted to get it as close to the original as possible.
Using priority I created my v2 YAML file

---
# Node: CentOS 7 Console Workstation/Server lb-gsf14-console-05.lab.nascom.nasa.gov

classes:
  - rsyslog::client

rsyslog::client::global_config:
  WorkDirectory:
    priority: 12
    type: legacy
    value: '/var/lib/rsyslog'
  ActionFileDefaultTemplate:
    priority: 13
    type: legacy
    value: "RSYSLOG_TraditionalFileFormat"
  ActionQueueFileName:
    priority: 99
    type: legacy
    value: 'fwdRule1'
  ActionQueueMaxDiskSpace:
    priority: 99
    type: legacy
    value: '1g'
  ActionQueueSaveOnShutdown:
    priority: 99
    type: legacy
    value: 'on'
  ActionQueueType:
    priority: 99
    type: legacy
    value: 'LinkedList'
  ActionResumeRetryCount :
    priority: 99
    type: legacy
    value: '-1'

rsyslog::client::modules:
  imjournal:
    priority: 11
    config:
      StateFile : 'imjournal.state'
  imuxsock:
    priority: 10
    config:
      'SysSock.Use' : 'on'
      socket: '/run/systemd/journal/syslog'

rsyslog::client::legacy_config:
  messages_rule:
    key: "*.info;mail.none;authpriv.none;cron.none" 
    value: "/var/log/messages"
  auth_priv_rule:
    key: "auth,authpriv.*"
    value: "/var/log/secure"
  mail_rule:
    key: "mail.*"
    value: "-/var/log/mail.log"
  cron_rule:
    key: "cron.*"
    value: "/var/log/cron"
  emergency_rule:
    key: "*.emerg"
    value: ":omusrmsg:*"
  news_rule:
    key: "uucp,news.crit"
    value: "/var/log/spooler"
  boot_rule:
    key: "local7.*"
    value: "/var/log/boot.log" 
  remote_host_rule:
    priority: 99
    key: "*.*"
    value: "@@remote-host:514"

# vim: ft=yaml:sts=2:sw=2:et

And got this /etc/rsyslog.d/50_rsyslog.conf

module(load="imuxsock" 
          SysSock.Use="on"
          socket="/run/systemd/journal/syslog"
     
)
module(load="imjournal" 
          StateFile="imjournal.state"
     
)
$WorkDirectory /var/lib/rsyslog
$ActionFileDefaultTemplate RSYSLOG_TraditionalFileFormat
# auth_priv_rule
auth,authpriv.*     /var/log/secure

# boot_rule
local7.*     /var/log/boot.log

# cron_rule
cron.*     /var/log/cron

# emergency_rule
*.emerg     :omusrmsg:*

# mail_rule
mail.*     -/var/log/mail.log

# messages_rule
*.info;mail.none;authpriv.none;cron.none     /var/log/messages

# news_rule
uucp,news.crit     /var/log/spooler

$ActionQueueFileName fwdRule1
$ActionQueueMaxDiskSpace 1g
$ActionQueueSaveOnShutdown on
$ActionQueueType LinkedList
$ActionResumeRetryCount -1
# remote_host_rule
*.*     @@remote-host:514

[vchepkov]

Module doesn't work RHEL8 either. It wipes out existing configuration without providing any sensible defaults, same as RHEL7. That's not a normal behavior for a puppet module

[dhollinger]

@vchepkov I don't generally work within RHEl7 or RHEL8 anymore, so I'd love it if someone could contribute some default configurations for each OS. I'd do it, but I currently don't have time to spin up a container or VM of every OS and transcribe the defaults to the module.

[vchepkov]

From what I can see from the code, module doesn't set any sensible defaults for any OS, unless I miss something.
Ironically, the only configuration line it sets is also wrong.
instead of $IncludeConfig /etc/rsyslog.d/*.conf it should be include(file="/etc/rsyslog.d/*.conf" mode="optional") since it should be in new format, right?
I would be happy to provide defaults for RHEL, but I have to tell in advance, writing tests is not something that I am good at.

[dhollinger]

@vchepkov At the time of the writing of this module include() wasn't available in rainerscript yet, at least not to my knowledge. Additionally, any "defaults" would be set in the data/ directory as this module uses module data for defaults rather than params.pp. We'll just need to create defaults there and make sure the hiera.yaml is configured to look up

[vchepkov]

Correct, I was pointing out that lack of defaults in data/ is common for all OS, not just RHEL as this ticket reports

[mungo312]

Is there any progress for this topic ? Is there any interest from the maintainers to get working defaults ?

[andyholdaway]

The examples above worked for me - had to change
rsyslog::client
to
rsyslog::config
but it works for me using the latest version