diff --git a/.gitignore b/.gitignore index 7d3d992..82f4549 100644 --- a/.gitignore +++ b/.gitignore @@ -11,3 +11,15 @@ torrust-tracker/ # - infrastructure/.gitignore: Infrastructure-specific files (Terraform, VMs, etc.) # - application/.gitignore: Application-specific files (Docker, logs, data, etc.) # - .gitignore (this file): Project-wide cross-cutting files only + + +# Ignore log files +*.log +logs/ +*.log.* + +# Ignore report files +*report.txt* +*report.diff* +*report.commit* + diff --git a/docs/README.md b/docs/README.md index 70cfbc3..d4b0850 100644 --- a/docs/README.md +++ b/docs/README.md @@ -16,12 +16,13 @@ This directory currently contains cross-cutting documentation: **Current ADRs:** -- [ADR-001: Makefile Location](adr/001-makefile-location.md) - Decision to keep - Makefile at repository root level -- [ADR-002: Docker for All Services](adr/002-docker-for-all-services.md) - Decision - to use Docker for all services including UDP tracker -- [ADR-003: Use MySQL Over MariaDB](adr/003-use-mysql-over-mariadb.md) - Decision - to use MySQL instead of MariaDB for database backend +- [ADR-001: Makefile Location](adr/001-makefile-location.md) - Decision to keep Makefile at repository root level (proposed to be superseded by ADR-005) +- [ADR-002: Docker for All Services](adr/002-docker-for-all-services.md) - Decision to use Docker for all services including UDP tracker +- [ADR-003: Use MySQL Over MariaDB](adr/003-use-mysql-over-mariadb.md) - Decision to use MySQL instead of MariaDB for database backend +- [ADR-004: Adopt Modernized Automation Toolchain](adr/004-adopt-modern-toolchain.md) - *(Proposed)* - High-level strategy to migrate from `Makefile/sh` to `Meson/Perl`. +- [ADR-005: Adopt Test-Driven Implementation and Structure](adr/005-test-driven-implementation.md) - *(Proposed)* - Implementation plan for a test-driven approach and atomic migration. +- [ADR-006: Adopt Perl Coding and Security Standards](adr/006-perl-coding-standards.md) - *(Proposed)* - Quality and security standards for the new Perl codebase. +- [ADR-007: Adopt Meson Build System Principles and Conventions](adr/007-meson-build-system-principles.md) - *(Proposed)* - Guiding principles and conventions for using Meson. ### πŸ“… [`plans/`](plans/) (Ongoing Plans and Roadmaps) diff --git a/docs/adr/004-adopt-modern-toolchain.md b/docs/adr/004-adopt-modern-toolchain.md new file mode 100644 index 0000000..6029e45 --- /dev/null +++ b/docs/adr/004-adopt-modern-toolchain.md @@ -0,0 +1,55 @@ +### **ADR-004: Proposal to Adopt a Modernized Automation Toolchain** + +* **Status:** Proposed for Review +* **Date:** 2025-07-09 +* **Proposer:** Cameron +* **Reviewers:** [List of project maintainers/core contributors] + +#### **1. Context and Problem Statement** + +The project's current automation toolchain, based on `Makefile` and POSIX-compliant shell scripts, has provided a simple and conventional interface for developers. It is pragmatic and effective for its stated purpose. + +However, as we formally adopt the Twelve-Factor App methodology for the application itself, an analysis of our automation tooling reveals a philosophical misalignment. Specifically, the current tooling does not fully adhere to the principles it helps enforce: + +1. **Implicit Dependencies (Violation of Factor II):** The toolchain implicitly relies on the existence and version of system-wide tools like `make`, `awk`, `grep`, and `curl`. Their presence is not explicitly declared or verified in a manifest, leading to a non-deterministic setup environment. +2. **Blurred Build/Run Stages (Violation of Factor V):** Running a command like `make apply` mixes infrastructure planning, building, and execution. There is no distinct "build" or "setup" stage for the automation logic that validates its own dependencies before running. +3. **Brittle Logic:** The current shell scripts rely on parsing the text output of command-line tools (e.g., `virsh domifaddr | awk ...`). This creates a fragile contract that is less stable across OS versions than using a formal, versioned API. + +This proposal seeks to address these shortcomings by evolving our automation toolchain to be a first-class, Twelve-Factor-compliant component of the project, thereby increasing its long-term robustness and maintainability. + +#### **2. Proposed Change** + +It is proposed that the project adopt a new technology stack for its automation, based on the following core decisions: + +1. **Adopt Meson as the Unified Automation Interface:** Meson will serve as the primary user-facing tool for running all project tasks. It will provide a single, declarative entry point for developers and CI/CD systems. +2. **Adopt Modern Perl for Core Automation Logic:** The imperative logic currently in shell scripts will be migrated to a new, structured Perl codebase. +3. **Adopt a "System-Package-Only" Dependency Policy:** To ensure maximum stability and a simple, curated "supply chain," this proposal mandates that all Perl module dependencies be fulfilled **exclusively through the system's native package manager** (e.g., `apt install libsys-virt-perl`). The direct use of language-specific, uncurated package managers like `cpanm` will be explicitly disallowed in the project's setup and CI workflows. +4. **Establish a Stable Target Platform Benchmark:** All tooling and dependencies will target versions that are officially maintained for the **current Debian Stable release**. This includes packages from the official `backports` repository, as they are specifically compiled and certified for use with the stable base system. The critical guiding principle is: *Is the package officially provided for Debian Stable by the Debian maintainers?* + * *As of this writing, this sets our target versions to **Perl `5.36.0`** (from `bookworm`) and **Meson `1.7.0`** (from `bookworm-backports`).* + +#### **3. Rationale for Proposed Change** + +This strategic evolution will bring our automation toolchain into stronger alignment with the Twelve-Factor methodology and yield significant long-term benefits: + +* **Explicit, Verifiable Dependencies (Factor II):** The `meson.build` file will serve as an explicit dependency manifest. It will programmatically check for `perl`, `opentofu`, `libvirt-client`, and all required `perl-*` packages. This provides a single, deterministic command (`meson setup`) to validate a contributor's environment, failing hard and early with clear error messages if the environment is incorrect. +* **Clear Build/Run Separation (Factor V):** The Meson workflow naturally separates our processes. `meson setup` becomes our distinct "build" stage, which configures the project and validates all dependencies. `meson test` or `meson compile` becomes the "run" stage, executing tasks in a pre-validated environment. +* **Increased Robustness and Maintainability:** Migrating from brittle text-parsing in shell to using formal Perl modules (like `Sys::Virt`) allows us to depend on more stable, versioned APIs. Perl also provides superior error handling, data structures, and code organization, which will make the automation easier to maintain and extend. + +#### **4. Scope and Relationship to Other Proposals** + +This document's scope is strictly limited to the **strategic decision to adopt Meson and Perl**. It establishes the core "why" and "what" of the change. + +All further details are explicitly deferred to subsequent proposals, which will build upon this foundational decision: + +* **Implementation Strategy & Structure (`ADR-005`):** Will detail the "how" of this migration, including the specific repository structure, the plan for an atomic cutover (a "flag day" migration), and the deprecation of the `Makefile`. +* **Coding & Quality Standards (`ADR-006` and `ADR-007`):** Will define the "rules of the road" for the new Perl and Meson codebase, including mandatory pragmas, security hardening, linting policies, and style conventions. + +#### **5. Consequences and Acknowledged Trade-offs** + +* **Primary Benefit:** The project's automation tooling will become a more reliable, maintainable, and professionally engineered component, in full alignment with the principles it helps to deploy. +* **Acknowledged Trade-off 1: Loss of "Zero-Prerequisite" Setup.** The single greatest trade-off is sacrificing the convenience of `make` being pre-installed. A contributor's first action will now be to install the Meson/Perl toolchain via their system package manager. This is a conscious decision to prioritize explicit, verifiable correctness over "zero-setup" convenience. +* **Acknowledged Trade-off 2: Dependency on a Curated Ecosystem.** By committing to system packages benchmarked against Debian Stable (including its official backports), we gain stability at the cost of immediate access to bleeding-edge tool and library features. This is a deliberate choice to favor long-term stability and reliability. + +#### **6. Next Steps** + +If this strategic proposal is accepted, the maintainers will proceed with a formal review of `ADR-005`, `ADR-006`, and `ADR-007` to finalize the implementation plan and quality standards for the new toolchain. diff --git a/docs/adr/005-test-driven-implementation.md b/docs/adr/005-test-driven-implementation.md new file mode 100644 index 0000000..c9a2f4a --- /dev/null +++ b/docs/adr/005-test-driven-implementation.md @@ -0,0 +1,65 @@ +### **ADR-005: Proposal for a Test-Driven Implementation and Structure** + +* **Status:** Proposed for Review +* **Date:** 2025-07-09 +* **Proposer:** Cameron +* **Reviewers:** [List of project maintainers/core contributors] +* **Depends On:** [ADR-004: Proposal to Adopt a Modernized Automation Toolchain](./004-adopt-modern-toolchain.md) +* **Supersedes:** [ADR-001: Keep Makefile at Repository Root Level](./001-makefile-location.md) + +#### **1. Context and Problem Statement** + +Following the strategic decision in `ADR-004` to adopt a Meson/Perl toolchain, we must now define a concrete implementation plan. A simple one-to-one replacement of scripts would fail to capitalize on the full benefits of the new toolchain and could lead to a disorganized codebase. + +This proposal addresses the "how" and "where" of the migration. It recommends a specific code structure and implementation methodology designed to maximize the reliability, maintainability, and testability of our new automation system. It also proposes an atomic migration strategy to ensure a clean and unambiguous transition. + +#### **2. Proposed Change** + +It is proposed that the new toolchain be implemented using the following structured approach: + +1. **Centralize Automation Logic:** A new top-level `automation/` directory will be created. This directory will become the single, authoritative home for all imperative automation code (Perl scripts and modules) and their corresponding tests. This cleanly separates the project's declarative artifacts (in `infrastructure/` and `application/`) from the imperative code that acts upon them. + +2. **Adopt a Test-Driven Standard:** All new Perl logic will be accompanied by a formal test suite located in `automation/t/`. The project's primary test command, `meson test`, will validate the correctness of the automation logic itself, enabling unit tests that can run without requiring a live, deployed environment. + +3. **Execute an Atomic "Flag Day" Migration:** The transition from the legacy `Makefile`/`sh` system will be performed in a single, comprehensive changeset. This "clean break" approach avoids a confusing transitional period with two competing systems. The `Makefile` and old scripts will be removed entirely, and the Meson/Perl system will be introduced as the sole, official standard. + +#### **3. Proposed Repository Structure** + +The following structure is recommended to support this change. It is designed for clarity and a strong separation of concerns. + +```text +torrust-tracker-demo/ +β”œβ”€β”€ application/ # Application deployment artifacts (e.g., compose.yaml) +β”œβ”€β”€ infrastructure/ # Infrastructure-as-Code declarations (Terraform, cloud-init) +β”œβ”€β”€ docs/ # Project documentation +β”œβ”€β”€ automation/ # NEW: All automation logic and its tests +β”‚ β”œβ”€β”€ lib/ # Reusable Perl modules (e.g., Torrust::Demo::*) +β”‚ β”œβ”€β”€ t/ # Test suite for automation logic (*.t files) +β”‚ └── meson.build # Defines all automation targets +β”œβ”€β”€ .gitignore +└── meson.build # ROOT INTERFACE: Orchestrates all tasks +``` + +#### **4. Rationale for Proposed Change** + +This structured and test-driven approach is recommended because it allows us to build a truly professional-grade automation toolchain: + +* **Reliability Through Testing:** Implementing a test suite for our automation code is a transformative step. It allows us to verify complex logic (e.g., parsing configuration, generating template files) in isolation, leading to fewer bugs and faster, more confident development of our tooling. +* **Improved Maintainability:** The proposed structure creates a clear "home" for all automation code. This makes the system easier for new contributors to understand and easier for maintainers to extend. Separating logic into reusable Perl modules (`.pm` files) will reduce code duplication and improve overall quality. +* **Clarity of Implementation:** An atomic migration, while disruptive, provides immediate and total clarity. There is no ambiguity about which system to use or how tasks should be run. All documentation and workflows can be updated to reflect a single, consistent standard from day one. + +#### **5. Scope and Relationship to Other Proposals** + +This document's scope is strictly limited to the **implementation strategy, repository structure, and testing philosophy** of the new toolchain. It defines the "how" and "where" of the migration. + +It explicitly defers the definition of specific coding conventions to the next document: +* **Coding & Quality Standards (`ADR-006`):** Will define the specific rules for the Perl code itself, such as mandatory pragmas (`use strict;`), security hardening (`-T`), and static analysis (`Perl::Critic`) policies. + +#### **6. Acknowledged Trade-offs** + +* **One-Time Contributor Effort:** The primary trade-off is that an atomic migration requires every contributor to learn and adapt to the new `meson`-based workflow simultaneously. The legacy `make` commands they are familiar with will cease to exist. +* **Mitigation Strategy:** This cost is deemed acceptable for the long-term benefit of a single, superior system. The migration will be supported by a comprehensive update to all documentation (`README.md`, contributing guides) and clear project-level communication to prepare contributors for the change. + +#### **7. Next Steps** + +If this proposal for the implementation strategy is accepted, the maintainers will proceed with a final review of `ADR-006` to establish the formal coding standards before beginning the migration work outlined herein. diff --git a/docs/adr/006-perl-coding-standards.md b/docs/adr/006-perl-coding-standards.md new file mode 100644 index 0000000..842332a --- /dev/null +++ b/docs/adr/006-perl-coding-standards.md @@ -0,0 +1,94 @@ +### **ADR-006: Proposal for Perl Coding and Security Standards** + +* **Status:** Proposed for Review +* **Date:** 2025-07-09 +* **Proposer:** Cameron +* **Reviewers:** [List of project maintainers/core contributors] +* **Depends On:** [ADR-005: Proposal for a Test-Driven Implementation and Structure](./005-test-driven-implementation.md) + +#### **1. Context and Problem Statement** + +Having established the strategy (`ADR-004`) and structure (`ADR-005`) for our new Meson/Perl automation toolchain, it is imperative that we define a clear and enforceable set of standards for the code itself. Without agreed-upon conventions, the new codebase could quickly accumulate technical debt, becoming inconsistent, insecure, or difficult to maintain. + +This proposal aims to establish a baseline of quality, security, and convention for all Perl code within the `automation/` directory, ensuring the long-term health and clarity of our internal tooling. + +#### **2. Proposed Standards** + +It is proposed that the project adopt the following set of standards for all Perl code contributed: + +1. **Mandatory Pragmas for Code Safety:** It is recommended that all Perl scripts (`.pl`) and modules (`.pm`) enable `use strict;`, `use warnings;`, and `use autodie;`. This combination is foundational for modern, safe Perl, as it catches common errors, enforces good scoping, and ensures that failed system calls result in a predictable, immediate script failure. + +2. **Minimum Perl Version Benchmark:** To ensure broad compatibility and stability, it is proposed that the codebase target the version of Perl shipped in the **current Debian Stable release**. This provides an objective benchmark that aligns development with common server environments. The corresponding `use vX.XX;` pragma should be included in all Perl files to enforce this minimum version. + * *As of this writing, Debian 12 ("Bookworm") ships with Perl 5.36.0. The current standard would therefore be **`use v5.36;`**.* + +3. **Static Analysis via Perl::Critic:** It is suggested that `Perl::Critic` be adopted as the official linter. By integrating it into the `meson test` suite, we can provide automated feedback on code style and quality for all contributions, ensuring consistency. + +4. **Security Hardening by Default:** It is recommended that all executable Perl scripts be run with taint mode (`-T`). This is a proven security feature that prevents insecure data passed from outside the program (e.g., environment variables, command-line arguments) from being used in commands that interact with the shell. + +5. **Consistent Command-Line Interface:** To create a predictable user experience for developers, it is proposed that all public-facing scripts adopt a standard CLI argument style, implemented using Perl's core `Getopt::Long` module. + +#### **3. Rationale for Proposed Standards** + +Adopting this comprehensive set of standards offers clear, long-term engineering advantages: + +* **Reliability:** The chosen pragmas create code that is robust by default. `autodie` ensures that a failed `open()` or `mkdir()` will halt execution immediately, which is critical for deterministic automation. +* **Security:** Taint mode (`-T`) provides a strong, language-level defense against a class of command-injection vulnerabilities, which is a professional standard for any tool that orchestrates system commands. +* **Maintainability:** A consistent style enforced by `Perl::Critic` reduces the cognitive overhead required to read, review, and maintain the codebase. Code reviews can focus on logic, not formatting. +* **Stability:** Pegging the language version to Debian Stable provides a durable and predictable platform, avoiding both rapid churn from chasing the latest features and stagnation from targeting old versions. + +#### **4. Recommended Implementation** + +**1. Standard Boilerplate for `.pl` and `.pm` files:** +The following header could serve as a template for all new Perl files. + +```perl +#!/usr/bin/env perl -T +# The -T flag enables taint mode for security hardening. + +use strict; +use warnings; +# The autodie pragma automatically promotes failed system calls into exceptions. +use autodie; + +# Enforce a minimum Perl version pegged to the current Debian Stable release. +# As of Q3 2025, this is Perl 5.36 (from Debian 12 'Bookworm'). +use v5.36; +``` + +**2. Centralized `Perl::Critic` Configuration:** +To manage our linting policy, a configuration file should be created at `automation/.perlcriticrc`. + +```ini +# A gentle starting point that focuses on the most important issues. +# It avoids being overly pedantic about minor style nits. +severity = gentle + +# Example of disabling a specific, often-controversial policy. +# Subroutine prototypes are used less often in modern Perl styles. +[-Subroutines::ProhibitSubroutinePrototypes] +``` + +**3. Proposed Meson Integration:** +The `meson.build` file can define clear, separate targets for testing and linting. + +```meson +# In meson.build + +perlcritic_prog = find_program('perlcritic', required: true) +prove_prog = find_program('prove', required: true) + +# Test Target 1: Code Style & Quality. Implicitly uses .perlcriticrc. +test('Perl::Critic Linting', perlcritic_prog, args: ['automation/']) + +# Test Target 2: Unit Test Execution via the 'prove' TAP harness. +test('Automation Unit Tests', prove_prog, args: ['-vr', 'automation/t/']) +``` + +#### **5. Acknowledged Trade-offs** + +* **Increased Formality:** This proposal introduces a more formal development process. Contributors will need to adhere to these standards, which is more restrictive than writing simple shell scripts. This is presented as a beneficial trade-off for the gains in long-term quality and security. +* **Exclusion of Newer Language Features:** The project would intentionally be unable to use Perl features newer than what is available in the benchmarked Debian Stable release. This is a deliberate choice prioritizing stability and compatibility. + +#### **6. Next Steps** + +If this proposal is accepted, it will serve as the official quality standard for all code developed during the migration outlined in `ADR-005`. All new Perl contributions will be expected to adhere to these rules, which will be enforced automatically by the CI pipeline. diff --git a/docs/adr/007-meson-build-system-principles.md b/docs/adr/007-meson-build-system-principles.md new file mode 100644 index 0000000..8df8286 --- /dev/null +++ b/docs/adr/007-meson-build-system-principles.md @@ -0,0 +1,41 @@ + +### **ADR-007: Proposal for Meson Build System Principles and Conventions** + +* **Status:** Proposed for Review +* **Date:** 2025-07-11 +* **Proposer:** Cameron +* **Reviewers:** [List of project maintainers/core contributors] +* **Depends On:** [ADR-005: Proposal for a Test-Driven Implementation and Structure](./005-test-driven-implementation.md) + +#### **1. Context and Problem Statement** + +With the adoption of Meson as our primary automation interface (`ADR-005`), we must establish a clear set of principles for its use. A build system without guiding conventions can become as unmaintainable as the scripting it replaces. This document proposes a set of high-level standards to ensure our Meson build system remains clean, readable, robust, and aligned with the project's long-term goals. + +Its scope is to define the *philosophy* and *usage expectations* for our Meson build files, not the specific implementation details. + +#### **2. Proposed Principles and Conventions** + +It is proposed that our use of Meson be governed by the following core principles: + +1. **Principle of a Stable Build Environment:** The project's build system shall target a stable, predictable, and publicly benchmarked version of Meson. We will adopt the version shipped in the **current Debian Stable release** as our official minimum required version. This ensures that the features available to our build system are consistent with common, stable server environments and change on a slow, predictable cycle. + +2. **Principle of Modularity:** The build logic must be organized into modular sub-projects using Meson's `subdir()` functionality. The root `meson.build` file shall serve only as a high-level orchestrator, defining global project options and including component-specific build definitions. This keeps the main entry point clean and delegates complexity to the relevant sub-systems (e.g., an `automation/meson.build`). + +3. **Principle of Explicit Dependency Management:** All external tools, libraries, or `pkg-config` dependencies required by the automation toolchain must be declared and located exclusively through Meson’s native functions (`dependency()` and `find_program()`). The use of `run_command()` or other brittle methods to manually locate dependencies is explicitly disallowed. The `meson setup` phase is the single, authoritative gatekeeper for all system dependencies. + +4. **Principle of a Unified Quality Gate:** All quality assurance tasksβ€”including static analysis, linting, and unit testsβ€”must be integrated as runnable targets within the Meson build definition. This establishes `meson test` as the single, canonical command for a contributor to validate the full spectrum of their changes against project standards. + +5. **Principle of Consistent Style:** All `meson.build` files must adhere to a consistent, documented style to ensure readability and maintainability across the project. The specific style guide will be maintained separately in a `CONTRIBUTING.md` document, but its existence and enforcement are mandated by this principle. + +#### **3. Rationale for Adopting These Principles** + +These principles are not arbitrary rules; they are designed to cultivate a professional and durable build system: + +* **Stability and Predictability:** The Debian Stable benchmark prevents dependency churn and ensures our automation is compatible with long-term support operating systems. +* **Readability and Scalability:** A modular structure prevents the build definition from becoming a monolithic file, making it easier for new contributors to understand and for maintainers to extend over time. +* **Robustness and Reliability:** By enforcing the use of Meson's native dependency functions, we get clear, immediate, and user-friendly error messages when a required tool is missing, which is vastly superior to a script failing mid-execution. +* **Developer Efficiency:** A unified quality gate (`meson test`) simplifies the contribution workflow. A developer knows that if `meson test` passes, their changes meet the project's quality standards. + +#### **4. Next Steps** + +If this proposal is accepted, it will serve as the guiding architectural standard for all Meson build system code contributed to the project. The implementation of the migration (as detailed in `ADR-005`) will be engineered to adhere to these principles. The specific style conventions will be documented in a relevant contributor guide.