Configure the packages build workflows to trigger automatically on GitHub events like pull requests or merges to the main branch.

[mjcr99]

Description

This issue enables a PR check to verify that a newly introduced change by a certain PR does not break the package generation process. To do so, different packages will be requested to be built automatically and the result will be evaluated as a PR check.

The following packages should be requested to be built as part of this check:

• DEB amd64
• DEB arm64
• RPM amd64
• RPM arm64
• macOS intel64
• macOs arm64
• windows

This issue will also require deciding where to place the building workflow files, to better organize them based on their requirements.

[mjcr99]

Update

(08/01/2025) Problems description.
(09/01/2025) Tested package building reusing previously compiled agent. It seems we need to recompile the agent to package it.
(10/01/2025) Proposed WF design.

Problem 1: Workflow location

To improve CI/CD procedures, we need to develop different checks that will be triggered under certain conditions. Currently, every time a PR is updated, compilation and UT are run in order to verify the changes the PR introduces do not break anything.

A similar approach should be followed regarding packaging. We should implement tests/workflows to build and test a package when a PR introduces changes, to verify packages will keep working as it's expected.

This can be done using the following two different approaches.

• Option 1: Use wazuh-agent-packages repository to request the tested package to be built, bring it to wazuh-agent repository, and test. This has the following implications:

  • This requires giving this repository the capability of requesting the wazuh-agent-packages private repository to trigger workflows. We have already done this before from the wazuh/wazuh repository and it requires adding the PAT token to this repository to use the WF hosted in the private one.
  • This will also require to use of the GitHub Action: return-dispatch to get the remote dispatched workflow ID and wait until it's finished, which should mean, the requested package has been successfully built and can be fetched from the private repository.
  • All the packages will be built from the same repository, no matter they are testing packages, development packages, or release packages. All the building workflows will be preserved in the private repository.

• Option 2: Migrate the package generation workflows to this repository to build the packages directly here.

  • This option simplifies the package creation from this repository since it would not imply dealing with the private repository request mechanism, but we will need to maintain different workflows with almost the same code in two different locations since the signature and notarization of some packages must be done in the private repository.
  • It also would require more work to move and adjust the migrated workflows from the private repository.
  • The release packages will be built from the private repository, and the rest will be built from this repository.

Problem 2: Split package-building procedure in different stages.

This development also aims to split the package building process into 3 different steps, as it's currently done in the Mac OS case, we should divide the workflows to deal with the binaries creating, packaging (and signature if required), and testing.

By doing so we can:

• Reuse the generated binaries to test the agent using direct binaries instead of the package or reuse the generated binaries to build 2 different packages, for example, build the binaries for amd64 and use them for both DEB and RPM packages if it's required, saving time and cost by reducing the number of required compilations.

• Have a clearer view of the different package-building stages, which will allow easier debugging.

Since the Mac OS packages-building process already implements this mechanism we have to focus on the Linux packages and Windows to check which changes should be done to be able to divide this procedure into the mentioned 3 stages.

• Linux packages: Currently the binaries are generated inside a Docker container, using Centos 7 or Debian 10 docker images, and this container also packages the binaries into the final package. Implementing the different stages under this platform would require the following points:

  • We currently support the -b option when the generate_package.sh script is called. This option would be deprecated since it clones the repository inside the Docker container using the given branch, instead of using the sources from the caller version shared in a Docker volume if no -b option is given. This should be done since we want to preserve the generated binaries using the Docker image as an interface to homogenize the building environment, so sharing the local repository with the Docker container would preserve the generated binaries and let us either test the binaries directly or build a package using this binaries. We could still preserve this option, and deal with the cloned repository inside the container fetching it after compiling, but this would complicate the process while preserving an option that is not usually used.

    • Doing so we should share the entire repository with the compiled agent binaries as a shared volume to the building Docker container, this way, we preserve the abstraction layer of the container to manage the required packaging dependencies, while also passing directly into the container the compiled binaries.

  • Currently both RPM and DEB use their receipt, their SPECS file, to be built. This file includes steps for building the binaries and installing them in a given scaffolding which is subsequently packaged, generating the final package.

    • The installation of the generated binaries uses the make install command, which calls the previously built Makefile. One of the points to take into account is if we want to reuse the generated binaries, Makefile needs to be rebuilt since the installation location for the agent straight from sources and the installation location it in the packaged folder are different.

    • The previously discussed point could be discarded since all the tests have needed the sources to be recompiled while trying to build the package using previously built binaries. This is because the data stored in the CMakeCache.txt refers to the path where the compilation has been made for the first time, since the installation paths are required to be changed, removing this CMakeCache.txt file is required otherwise, the following message appears: + cmake .. -DINSTALL_ROOT=/build_wazuh/rpmbuild/BUILDROOT/wazuh-agent-5.0.0-0.x86_64/ CMake Error: The current CMakeCache.txt directory /build_wazuh/rpmbuild/BUILD/wazuh-agent-5.0.0/src/build/CMakeCache.txt is different than the directory /home/wazuh-agent/src/build where CMakeCache.txt was created. This may result in binaries being created in the wrong place. If you are not sure, reedit the CMakeCache.txt CMake Error: The source "/build_wazuh/rpmbuild/BUILD/wazuh-agent-5.0.0/src/CMakeLists.txt" does not match the source "/home/wazuh-agent/src/CMakeLists.txt" used to generate cache. Re-run cmake with a different source directory.

First proposal

Keeping the last discussed point, the final workflow would look as follows:

flowchart TD
    A[PR Event] --> B[Trigger test]
    B -->|Packages Binaries Compilation| C[Packaging ready binaries]
    B -->|UT and IT Binaries Compilation| D[Sources installation ready binaries]
    C --> |Packages Built| E[Signed Package]
    C --> |Packages Built| F[Unsigned Package]
    E --> |Signature Test| G[Signature Tested Package]
    F --> |Install and Uninstall Test| H[Installation tested Package]
    E --> |Install and Uninstall Test| H[Installation tested Package]

Loading