docs(adr): [#17] propose migration to Meson/Perl automation toolchain

.gitignore

@@ -11,3 +11,15 @@ torrust-tracker/
 # - infrastructure/.gitignore: Infrastructure-specific files (Terraform, VMs, etc.)
 # - application/.gitignore: Application-specific files (Docker, logs, data, etc.)
 # - .gitignore (this file): Project-wide cross-cutting files only
+
+
+# Ignore log files
+*.log
+logs/
+*.log.*
+
+# Ignore report files
+*report.txt*
+*report.diff*
+*report.commit*
+

docs/README.md

@@ -16,12 +16,13 @@ This directory currently contains cross-cutting documentation:
 
 **Current ADRs:**
 
-- [ADR-001: Makefile Location](adr/001-makefile-location.md) - Decision to keep
-  Makefile at repository root level
-- [ADR-002: Docker for All Services](adr/002-docker-for-all-services.md) - Decision
-  to use Docker for all services including UDP tracker
-- [ADR-003: Use MySQL Over MariaDB](adr/003-use-mysql-over-mariadb.md) - Decision
-  to use MySQL instead of MariaDB for database backend
+- [ADR-001: Makefile Location](adr/001-makefile-location.md) - Decision to keep Makefile at repository root level (proposed to be superseded by ADR-005)
+- [ADR-002: Docker for All Services](adr/002-docker-for-all-services.md) - Decision to use Docker for all services including UDP tracker
+- [ADR-003: Use MySQL Over MariaDB](adr/003-use-mysql-over-mariadb.md) - Decision to use MySQL instead of MariaDB for database backend
+- [ADR-004: Adopt Modernized Automation Toolchain](adr/004-adopt-modern-toolchain.md) - *(Proposed)* - High-level strategy to migrate from `Makefile/sh` to `Meson/Perl`.
+- [ADR-005: Adopt Test-Driven Implementation and Structure](adr/005-test-driven-implementation.md) - *(Proposed)* - Implementation plan for a test-driven approach and atomic migration.
+- [ADR-006: Adopt Perl Coding and Security Standards](adr/006-perl-coding-standards.md) - *(Proposed)* - Quality and security standards for the new Perl codebase.
+- [ADR-007: Adopt Meson Build System Principles and Conventions](adr/007-meson-build-system-principles.md) - *(Proposed)* - Guiding principles and conventions for using Meson.
 
 ### 📅 [`plans/`](plans/) (Ongoing Plans and Roadmaps)
 

docs/adr/004-adopt-modern-toolchain.md

@@ -0,0 +1,55 @@
+### **ADR-004: Proposal to Adopt a Modernized Automation Toolchain**
+
+*   **Status:** Proposed for Review
+*   **Date:** 2025-07-09
+*   **Proposer:** Cameron
+*   **Reviewers:** [List of project maintainers/core contributors]
+
+#### **1. Context and Problem Statement**
+
+The project's current automation toolchain, based on `Makefile` and POSIX-compliant shell scripts, has provided a simple and conventional interface for developers. It is pragmatic and effective for its stated purpose.
+
+However, as we formally adopt the Twelve-Factor App methodology for the application itself, an analysis of our automation tooling reveals a philosophical misalignment. Specifically, the current tooling does not fully adhere to the principles it helps enforce:
+
+1.  **Implicit Dependencies (Violation of Factor II):** The toolchain implicitly relies on the existence and version of system-wide tools like `make`, `awk`, `grep`, and `curl`. Their presence is not explicitly declared or verified in a manifest, leading to a non-deterministic setup environment.
+2.  **Blurred Build/Run Stages (Violation of Factor V):** Running a command like `make apply` mixes infrastructure planning, building, and execution. There is no distinct "build" or "setup" stage for the automation logic that validates its own dependencies before running.
+3.  **Brittle Logic:** The current shell scripts rely on parsing the text output of command-line tools (e.g., `virsh domifaddr | awk ...`). This creates a fragile contract that is less stable across OS versions than using a formal, versioned API.
+
+This proposal seeks to address these shortcomings by evolving our automation toolchain to be a first-class, Twelve-Factor-compliant component of the project, thereby increasing its long-term robustness and maintainability.
+
+#### **2. Proposed Change**
+
+It is proposed that the project adopt a new technology stack for its automation, based on the following core decisions:
+
+1.  **Adopt Meson as the Unified Automation Interface:** Meson will serve as the primary user-facing tool for running all project tasks. It will provide a single, declarative entry point for developers and CI/CD systems.
+2.  **Adopt Modern Perl for Core Automation Logic:** The imperative logic currently in shell scripts will be migrated to a new, structured Perl codebase.
+3.  **Adopt a "System-Package-Only" Dependency Policy:** To ensure maximum stability and a simple, curated "supply chain," this proposal mandates that all Perl module dependencies be fulfilled **exclusively through the system's native package manager** (e.g., `apt install libsys-virt-perl`). The direct use of language-specific, uncurated package managers like `cpanm` will be explicitly disallowed in the project's setup and CI workflows.
+4.  **Establish a Stable Target Platform Benchmark:** All tooling and dependencies will target versions that are officially maintained for the **current Debian Stable release**. This includes packages from the official `backports` repository, as they are specifically compiled and certified for use with the stable base system. The critical guiding principle is: *Is the package officially provided for Debian Stable by the Debian maintainers?*
+    *   *As of this writing, this sets our target versions to **Perl `5.36.0`** (from `bookworm`) and **Meson `1.7.0`** (from `bookworm-backports`).*
+
+#### **3. Rationale for Proposed Change**
+
+This strategic evolution will bring our automation toolchain into stronger alignment with the Twelve-Factor methodology and yield significant long-term benefits:
+
+*   **Explicit, Verifiable Dependencies (Factor II):** The `meson.build` file will serve as an explicit dependency manifest. It will programmatically check for `perl`, `opentofu`, `libvirt-client`, and all required `perl-*` packages. This provides a single, deterministic command (`meson setup`) to validate a contributor's environment, failing hard and early with clear error messages if the environment is incorrect.
+*   **Clear Build/Run Separation (Factor V):** The Meson workflow naturally separates our processes. `meson setup` becomes our distinct "build" stage, which configures the project and validates all dependencies. `meson test` or `meson compile` becomes the "run" stage, executing tasks in a pre-validated environment.
+*   **Increased Robustness and Maintainability:** Migrating from brittle text-parsing in shell to using formal Perl modules (like `Sys::Virt`) allows us to depend on more stable, versioned APIs. Perl also provides superior error handling, data structures, and code organization, which will make the automation easier to maintain and extend.
+
+#### **4. Scope and Relationship to Other Proposals**
+
+This document's scope is strictly limited to the **strategic decision to adopt Meson and Perl**. It establishes the core "why" and "what" of the change.
+
+All further details are explicitly deferred to subsequent proposals, which will build upon this foundational decision:
+
+*   **Implementation Strategy & Structure (`ADR-005`):** Will detail the "how" of this migration, including the specific repository structure, the plan for an atomic cutover (a "flag day" migration), and the deprecation of the `Makefile`.
+*   **Coding & Quality Standards (`ADR-006` and `ADR-007`):** Will define the "rules of the road" for the new Perl and Meson codebase, including mandatory pragmas, security hardening, linting policies, and style conventions.
+
+#### **5. Consequences and Acknowledged Trade-offs**
+
+*   **Primary Benefit:** The project's automation tooling will become a more reliable, maintainable, and professionally engineered component, in full alignment with the principles it helps to deploy.
+*   **Acknowledged Trade-off 1: Loss of "Zero-Prerequisite" Setup.** The single greatest trade-off is sacrificing the convenience of `make` being pre-installed. A contributor's first action will now be to install the Meson/Perl toolchain via their system package manager. This is a conscious decision to prioritize explicit, verifiable correctness over "zero-setup" convenience.
+*   **Acknowledged Trade-off 2: Dependency on a Curated Ecosystem.** By committing to system packages benchmarked against Debian Stable (including its official backports), we gain stability at the cost of immediate access to bleeding-edge tool and library features. This is a deliberate choice to favor long-term stability and reliability.
+
+#### **6. Next Steps**
+
+If this strategic proposal is accepted, the maintainers will proceed with a formal review of `ADR-005`, `ADR-006`, and `ADR-007` to finalize the implementation plan and quality standards for the new toolchain.

docs/adr/005-test-driven-implementation.md

@@ -0,0 +1,65 @@
+### **ADR-005: Proposal for a Test-Driven Implementation and Structure**
+
+*   **Status:** Proposed for Review
+*   **Date:** 2025-07-09
+*   **Proposer:** Cameron
+*   **Reviewers:** [List of project maintainers/core contributors]
+*   **Depends On:** [ADR-004: Proposal to Adopt a Modernized Automation Toolchain](./004-adopt-modern-toolchain.md)
+*   **Supersedes:** [ADR-001: Keep Makefile at Repository Root Level](./001-makefile-location.md)
+
+#### **1. Context and Problem Statement**
+
+Following the strategic decision in `ADR-004` to adopt a Meson/Perl toolchain, we must now define a concrete implementation plan. A simple one-to-one replacement of scripts would fail to capitalize on the full benefits of the new toolchain and could lead to a disorganized codebase.
+
+This proposal addresses the "how" and "where" of the migration. It recommends a specific code structure and implementation methodology designed to maximize the reliability, maintainability, and testability of our new automation system. It also proposes an atomic migration strategy to ensure a clean and unambiguous transition.
+
+#### **2. Proposed Change**
+
+It is proposed that the new toolchain be implemented using the following structured approach:
+
+1.  **Centralize Automation Logic:** A new top-level `automation/` directory will be created. This directory will become the single, authoritative home for all imperative automation code (Perl scripts and modules) and their corresponding tests. This cleanly separates the project's declarative artifacts (in `infrastructure/` and `application/`) from the imperative code that acts upon them.
+
+2.  **Adopt a Test-Driven Standard:** All new Perl logic will be accompanied by a formal test suite located in `automation/t/`. The project's primary test command, `meson test`, will validate the correctness of the automation logic itself, enabling unit tests that can run without requiring a live, deployed environment.
+
+3.  **Execute an Atomic "Flag Day" Migration:** The transition from the legacy `Makefile`/`sh` system will be performed in a single, comprehensive changeset. This "clean break" approach avoids a confusing transitional period with two competing systems. The `Makefile` and old scripts will be removed entirely, and the Meson/Perl system will be introduced as the sole, official standard.
+
+#### **3. Proposed Repository Structure**
+
+The following structure is recommended to support this change. It is designed for clarity and a strong separation of concerns.
+
+```text
+torrust-tracker-demo/
+├── application/                # Application deployment artifacts (e.g., compose.yaml)
+├── infrastructure/             # Infrastructure-as-Code declarations (Terraform, cloud-init)
+├── docs/                       # Project documentation
+├── automation/                 # NEW: All automation logic and its tests
+│   ├── lib/                    # Reusable Perl modules (e.g., Torrust::Demo::*)
+│   ├── t/                      # Test suite for automation logic (*.t files)
+│   └── meson.build             # Defines all automation targets
+├── .gitignore
+└── meson.build                 # ROOT INTERFACE: Orchestrates all tasks
+```
+
+#### **4. Rationale for Proposed Change**
+
+This structured and test-driven approach is recommended because it allows us to build a truly professional-grade automation toolchain:
+
+*   **Reliability Through Testing:** Implementing a test suite for our automation code is a transformative step. It allows us to verify complex logic (e.g., parsing configuration, generating template files) in isolation, leading to fewer bugs and faster, more confident development of our tooling.
+*   **Improved Maintainability:** The proposed structure creates a clear "home" for all automation code. This makes the system easier for new contributors to understand and easier for maintainers to extend. Separating logic into reusable Perl modules (`.pm` files) will reduce code duplication and improve overall quality.
+*   **Clarity of Implementation:** An atomic migration, while disruptive, provides immediate and total clarity. There is no ambiguity about which system to use or how tasks should be run. All documentation and workflows can be updated to reflect a single, consistent standard from day one.
+
+#### **5. Scope and Relationship to Other Proposals**
+
+This document's scope is strictly limited to the **implementation strategy, repository structure, and testing philosophy** of the new toolchain. It defines the "how" and "where" of the migration.
+
+It explicitly defers the definition of specific coding conventions to the next document:
+*   **Coding & Quality Standards (`ADR-006`):** Will define the specific rules for the Perl code itself, such as mandatory pragmas (`use strict;`), security hardening (`-T`), and static analysis (`Perl::Critic`) policies.
+
+#### **6. Acknowledged Trade-offs**
+
+*   **One-Time Contributor Effort:** The primary trade-off is that an atomic migration requires every contributor to learn and adapt to the new `meson`-based workflow simultaneously. The legacy `make` commands they are familiar with will cease to exist.
+*   **Mitigation Strategy:** This cost is deemed acceptable for the long-term benefit of a single, superior system. The migration will be supported by a comprehensive update to all documentation (`README.md`, contributing guides) and clear project-level communication to prepare contributors for the change.
+
+#### **7. Next Steps**
+
+If this proposal for the implementation strategy is accepted, the maintainers will proceed with a final review of `ADR-006` to establish the formal coding standards before beginning the migration work outlined herein.