improve CPU cores affinity to NUMA nodes v7

[lukashino]

Followup of #11706

Redmine ticket:
https://redmine.openinfosecfoundation.org/issues/7036

This work allows more precise thread assignment and it is done either

• automatically - by picking assigned cores of the same NUMA locality as the interface from the worker-cpu-list
• manually - you can specify per-iface settings in the threading section

This works with AF-PACKET and DPDK (and should with other capture modes as well). The primary target is workers runmode but it also works with autofp receive threads.

To try this PR:

1. compile Suri from this branch
2. Modify the newly generated suricata.yaml according to your setup.
3. Run as usual.

Describe changes:

v7:

• hwloc is now an optional dependency - it can be enabled with --enable-hwloc in the configure step,
• autopinning / autooptimizing is configurable from threading.autopin suri.yaml option,
• making CI happy (and with that fixing a few bugs)

v6

• support for autofp mode supported - receive-cpu-set can now contain per interface CPU affinity settings - here comes a question - should worker-cpu-set in autofp be somehow configurable to the receive-cpu-set threads?
• YAML node "per-iface" changed to "interface-specific-cpu-set"
• list form of (management/worker/...)-cpu-set was changed to simple tree nodes
• cpu/prio/mode properties of (management/worker/...)-cpu-set nodes moved one YAML layer up
• DPDK (net_bonding / PCIe address / ... ) / AFP interfaces are supported

On YAML changes:

I tried other YAML designs such as:

Specifying the NUMA affinity in the individual capture modes

Example

dpdk:
  - interface: "3b:00.0"
    threads: [ 2, 4, 6, 8]

But management / other threads would still be required to configure elsewhere. I liked the proposed approach more because all affinity-related settings are centralized.

Not having a list nodes in the YAML

threading:
  worker-cpu-set:
    net_bonding0:
      cpu: [ 1, 3, 5 ]

YAML library changes net_bonding to net-bonding when YAML is loaded - so this is a nogo.

Have interface-related properties as children

threading:
  worker-cpu-set:
    - interface: net_bonding0
        cpu: [ 1, 3, 5 ]

This is a disallowed construct in YAML, so another nogo. A similar pattern is currently with *-cpu-set nodes but the different here is that the parent list item has a value (in this case net_bonding0). That is what is causing problems. Interface related therefore need to right under "interface".

PRO TIP: A good way to visualize different YAML structures is by using YAML to JSON converters. Currently it creates object like:

[
  {
    "interface": "3b:00.0",
    "mode": "exclusive",
  }
]

[suricata-qa]

ERROR:

ERROR: QA failed on ASAN_TLPR1_cfg.

Pipeline 24043

[jasonish]

This breaks compatibilty with a previous suricata.yaml where set-cpu-affinity is enabled.

Notice: suricata: This is Suricata version 8.0.0-dev (77147807f5 2024-12-20) running in SYSTEM mode [LogVersion:suricata.c:1152]
Error: affinity: Affinity with name "0" not found [GetOrAllocAffinityTypeForIfaceOfName:util-affinity.c:183]
Error: affinity: Unknown CPU affinity type: 0 [AffinitySetupLoadFromConfig:util-affinity.c:492]

I do appreciate the cleanup as lists should have never been used here, but at the very list Suricata shouldn't fail to start. I guess we need to decide:

• Do we need to keep backwards compatibility and respect the old config?
• Detect the old config is in use and display a warning that it needs to be updated?
  I guess either way upgrade notes would be required as well.

[jasonish]

Determine backwards compatibilty strategy and provide upgrade documentation.