How do i get pass this error. I recent upgrading cfengine to 3.23.0 - Line from package inventory 'py39-openssl,23.2.0,1,amd64,pkg' did not yield correct number of elements.

[gettoknowmii]

OS: FreeBSD
Version: 13.2
CFengine Version: 3.23.0

Errors
error: Line from package inventory 'tdb,1.4.7,1,amd64,pkg' did not yield correct number of elements.
error: Line from package inventory 'brotli,1.1.0,1,amd64,pkg' did not yield correct number of elements.
error: Line from package inventory 'dbus,1.14.10,1,amd64,pkg' did not yield correct number of elements.
error: Line from package inventory 'glib,2.78.0,2,amd64,pkg' did not yield correct number of elements.
error: Line from package inventory 'icu,73.2,1,amd64,pkg' did not yield correct number of elements.
error: Line from package inventory 'joe,4.6,1,amd64,pkg' did not yield correct number of elements.
error: Line from package inventory 'libarchive,3.7.1,1,amd64,pkg' did not yield correct number of elements.
error: Line from package inventory 'libedit,3.1.20221030,1,amd64,pkg' did not yield correct number of elements.
error: Line from package inventory 'liblz4,1.9.4,1,amd64,pkg' did not yield correct number of elements.
error: Line from package inventory 'libssh2,1.11.0,3,amd64,pkg' did not yield correct number of elements.
error: Line from package inventory 'lmdb,0.9.31,1,amd64,pkg' did not yield correct number of elements.
error: Line from package inventory 'lsof,4.97.0,8,amd64,pkg' did not yield correct number of elements.
error: Line from package inventory 'mg,20230501,1,amd64,pkg' did not yield correct number of elements.
error: Line from package inventory 'mpfr,4.2.1,1,amd64,pkg' did not yield correct number of elements.
error: Line from package inventory 'net-snmp,5.9.1_4,1,amd64,pkg' did not yield correct number of elements.
error: Line from package inventory 'open-vm-tools-nox11,12.2.5_2,2,amd64,pkg' did not yield correct number of elements.
error: Line from package inventory 'p5-TimeDate,2.33,1,amd64,pkg' did not yield correct number of elements.
error: Line from package inventory 'py39-cryptography,41.0.3_1,1,amd64,pkg' did not yield correct number of elements.
error: Line from package inventory 'py39-dnspython,2.4.2,1,amd64,pkg' did not yield correct number of elements.
error: Line from package inventory 'py39-openssl,23.2.0,1,amd64,pkg' did not yield correct number of elements.
error: Line from package inventory 'python,3.9_3,2,amd64,pkg' did not yield correct number of elements.
error: Line from package inventory 'ruby,3.1.4_1,1,amd64,pkg' did not yield correct number of elements.
error: Line from package inventory 'sqlite3,3.43.1,1,amd64,pkg' did not yield correct number of elements.
error: Line from package inventory 'tdb,1.4.7,1,amd64,pkg' did not yield correct number of elements

This started happening after i upgraded cfe to 3.23.0.

I'm not sure of which policy file to update so that this package format can also be supported

[craigcomstock]

It looks like the error comes from the packagesmatching() function.

https://github.com/cfengine/core/blob/master/libpromises/evalfunction.c#L1756

That code expects 4 comma separated items and I see you have five in all cases above.

For example: 'tdb,1.4.7,1,amd64,pkg'

You could examine /var/cfengine/state/software_packages.csv and I think you would find that file has these lines with one too many items.

The function ReportSoftware() at https://github.com/cfengine/core/blob/master/cf-agent/verify_packages.c#L795 is responsible for writing out that data and seems the format of that hasn't changed since 10 years ago: aaf6dae

I wonder if the data your system is/was getting for version is including a comma somehow?

Ah. I see, some package names do have commas in them!

# pkg info -a | grep ','
liblz4-1.9.4,1                 LZ4 compression library, lossless and very fast
libssh2-1.11.0,3               Library implementing the SSH2 protocol
lmdb-0.9.31,1                  OpenLDAP Lightning Memory-Mapped Database

and I see that in software_packages.csv

lmdb,"0.9.31,1",default,pkg

I suspect that the format of your software_packages.csv is "old" somehow and if you move it aside and let CFEngine regenerate it, it might include the quotes and resolve your issue.

mv /var/cfengine/state/software_packages.csv /var/cfengine/state/old-software-packages.csv
cf-agent -KI

The bundle cfe_autorun_inventory_packages should recreate the file, it is in masterfiles: https://github.com/cfengine/masterfiles/blob/master/inventory/any.cf#L1093

[craigcomstock]

Can you please include the version you upgraded from? I logged a ticket: https://northerntech.atlassian.net/browse/CFE-4341

as well as a ticket detailing some trouble I had getting setup to reproduce your issue: https://northerntech.atlassian.net/browse/CFE-4340

Thanks for the question!

[gettoknowmii]

i upgraded from 3.21.1 and 3.22.0. Upgrading from any of these versions to 3.23.0 produces that error. @craigcomstock the suggestion above is not working for me. I deleted the state file but when i restart cf-serverd, it still produces that error but doesn't create a new software.csv file. When i make the change on a cf agent clientit recreates the file and still produces that error

[craigcomstock]

Ok. I will try that migration and see if I can reproduce.

Can you include a few lines from the csv file so I can confirm if it has quotes around the version?

[gettoknowmii]

Yes it has quotes. The error is for all those packages with quotes.

sudo cat /var/cfengine/state/software_packages.csv

p5-GSSAPI,0.28_2,default,pkg^M
p5-File-Tail,1.3,default,pkg^M
p5-File-Finder,0.53_2,default,pkg^M
p5-File-Copy-Recursive,0.45,default,pkg^M
p5-Digest-HMAC,1.04,default,pkg^M
p5-Convert-ASN1,0.34,default,pkg^M
p5-Config-IniFiles,3.000003_1,default,pkg^M
p5-Class-Accessor,0.51,default,pkg^M
p5-Capture-Tiny,0.48,default,pkg^M
p5-BSD-Sysctl,0.12.3,default,pkg^M
p5-BSD-Process,0.07_2,default,pkg^M
p5-Authen-SASL,2.17,default,pkg^M
p11-kit,0.24.1_2,default,pkg^M
openldap26-client,2.6.6,default,pkg^M
open-vm-tools-nox11,"12.2.5_2,2",default,pkg^M
oniguruma,6.9.8_1,default,pkg^M
nss-pam-ldapd-sasl,0.9.12_1,default,pkg^M
nss,3.93,default,pkg^M
nspr,4.35,default,pkg^M
npth,1.6,default,pkg^M
nload,0.7.4_1,default,pkg^M
ngrep,1.45_4,default,pkg^M
nginx,"1.24.0_12,3",default,pkg^M
nettle,3.9.1,default,pkg^M
net-snmp,"5.9.1_4,1",default,pkg^M
ncftp,3.2.6_2,default,pkg^M
ncdu,1.18.1,default,pkg^M
nano,7.2,default,pkg^M
mysql80-client,8.0.33_3,default,pkg^M
msktutil,1.2.1,default,pkg^M
mpfr,"4.2.1,1",default,pkg^M
mpdecimal,2.5.1,default,pkg^M
mkjail,0.0.4,default,pkg^M
mhash,0.9.9.9_5,default,pkg^M
mg,"20230501,1",default,pkg^M
mbuffer,20230301,default,pkg^M
lzop,1.04,default,pkg^M
lzo2,2.10_1,default,pkg^M
luajit-devel,2.1.0.20230812,default,pkg^M
lua53,5.3.6,default,pkg^M
lsof,"4.97.0,8",default,pkg^M
lmdb,"0.9.31,1",default,pkg^M
llvm15,15.0.7_6,default,pkg^M
llvm13,13.0.1_3,default,pkg^M
libyaml,0.2.5,default,pkg^M

[craigcomstock]

Ok, good. Glad the file is as expected. I will dig into why things might be going wrong otherwise.

[craigcomstock]

It seems like since the errors happen both with cf-serverd and cf-agent that you have a packagesmatching function call in a common bundle.

Can you send some debug logs either here or to the ticket https://northerntech.atlassian.net/browse/CFE-4341 ?

That should give a good clue where the problem is in the policy.

cf-serverd -Fd > /tmp/cf-serverd-debug.log
cf-agent -Kd > /tmp/cf-agent-debug.log

[gettoknowmii]

cf-agent-debug.log
cf-serverd-debug.log

I have attached both files

[craigcomstock]

It does look like you have policy which uses packagesmatching function

   debug: Resolving classes and variables in 'bundle agent host_info_report_software'
   debug: DeRefCopyPromise(): promiser:'packages'
   debug: DeRefCopyPromise():     copying constraint: 'data'
   debug: DeRefCopyPromise():     copying constraint: 'comment'
   debug: Evaluating function: packagesmatching(".*",".*",".*",".*")
   debug: Reading packages (21) for package module [pkg]
   debug: Reading inventory from database: 5374
   error: Line from package inventory 'brotli,1.1.0,1,amd64,pkg' did not yield correct number of elements.
   error: Line from package inventory 'dbus,1.14.10,1,amd64,pkg' did not yield correct number of elements.
   error: Line from package inventory 'glib,2.78.0,2,amd64,pkg' did not yield correct number of elements.

If I add a bit of policy to a file which is automatically included: services/main.cf as follows:

bundle agent mpf_main
# User Defined Service Catalogue
{
  methods:
    # Activate your custom policies here

  vars:
    "installed" data => packagesmatching(".*",".*",".*",".*");

  reports:
    "hi";
    "installed: ${with}" with => storejson(installed);
}

then I get some versions with commas just fine.

  {                                                                                                                             
    "arch": "default",                                                                                                          
    "method": "pkg",                                                                                                            
    "name": "liblz4",                                                                                                           
    "version": "1.9.4,1"                                                                                                        
  }, 

Ah, I think I see the difference now.

I have

   debug: Evaluating function: packagesmatching(".*",".*",".*",".*")
   debug: Reading inventory from '/var/cfengine/state/software_packages.csv'

where you have

   debug: Evaluating function: packagesmatching(".*",".*",".*",".*")
   debug: Reading packages (21) for package module [pkg]
   debug: Reading inventory from database: 5374

I guess the default install from pkg doesn't get me the package modules but rather the legacy package methods which generate the csv file. So in your case the csv file has nothing to do with your issue.

Let's see what is in the database:

cf-check dump /var/cfengine/state/packages_installed_pkg.lmdb # or similarly named file > /tmp/packages_installed.dump

And then move that file aside and let it be regenerated by an agent run.

Another idea: please compare the file at /var/cfengine/modules/packages/pkg with this file in our masterfiles repo: https://github.com/cfengine/masterfiles/blob/master/modules/packages/vendored/pkg.mustache

That is the source of the information in the database and if not correct might be the cause of the problem.

[craigcomstock]

Ok. I have reproduced the error by making some changes to masterfiles that were needed to enable package modules fully.

   error: Line from package inventory 'lmdb,0.9.31,1,amd64,pkg' did not yield correct number of elements.
   error: Line from package inventory 'joe,4.6,1,amd64,pkg' did not yield correct number of elements.
   error: Line from package inventory 'liblz4,1.9.4,1,amd64,pkg' did not yield correct number of elements.
   error: Line from package inventory 'libssh2,1.11.0,3,amd64,pkg' did not yield correct number of elements.
   error: Line from package inventory 'lmdb,0.9.31,1,amd64,pkg' did not yield correct number of elements.

I will see where the fix is.

[craigcomstock]

Looks like the <inventory> key from which packagesmatching gets its data is not well formatted

        "<inventory>": "bash,5.2.15,amd64\ncfengine,3.23.0,amd64\ncfengine-masterfiles,3.23.0,amd64\ncurl,8.5.0,amd64\nexpat,2.5.0,amd64\nfirstboot-freebsd-update,1.3,amd64\nfirstboot-pkgs,1.6,amd64\ngettext-runtime,0.21.1,amd64\ngit,2.43.0,amd64\nindexinfo,0.3.1,amd64\njoe,4.6,1,amd64\nlibffi,3.4.4,amd64\nlibiconv,1.17,amd64\nlibidn2,2.3.4,amd64\nliblz4,1.9.4,1,amd64\nlibnghttp2,1.58.0,amd64\nlibpsl,0.21.2_4,amd64\nlibssh2,1.11.0,3,amd64\nlibunistring,1.1,amd64\nlibxml2,2.10.4_2,amd64\nlibyaml,0.2.5,amd64\nlmdb,0.9.31,1,amd64\nmpdecimal,2.5.1,amd64\np5-Authen-SASL,2.17,amd64\np5-CGI,4.60,amd64\np5-Clone,0.46,amd64\np5-Digest-HMAC,1.04,amd64\np5-Encode-Locale,1.05,amd64\np5-Error,0.17029,amd64\np5-GSSAPI,0.28_2,amd64\np5-HTML-Parser,3.81,amd64\np5-HTML-Tagset,3.20_1,amd64\np5-HTTP-Date,6.06,amd64\np5-HTTP-Message,6.45,amd64\np5-IO-HTML,1.004,amd64\np5-IO-Socket-IP,0.42,amd64\np5-IO-Socket-SSL,2.084,amd64\np5-LWP-MediaTypes,6.04,amd64\np5-Mozilla-CA,20231213,amd64\np5-Net-SSLeay,1.92,amd64\np5-TimeDate,2.33,1,amd64\np5-URI,5.21,amd64\npcre,8.45_3,amd64\npcre2,10.42,amd64\nperl5,5.36.3_1,amd64\npkg,1.20.9,amd64\npython39,3.9.18,amd64\nreadline,8.2.0,amd64\nrsync,3.2.7,amd64\nsudo,1.9.15p5,amd64\nvirtualbox-ose-additions-nox11,6.1.50,amd64\nxxhash,0.8.2,amd64\nzstd,1.5.5,amd64\n",

Here is a pull request with the fix. I have tested and it does fix the issue.

#5462