feat: Add allocation logging and visualization

[pzittlau]

This pull request introduces an allocation logging feature to betree and a visualization script to analyze the generated logs.

Key changes:

• Allocation Logging:

  • Added an allocation_log feature that enables logging of allocation and deallocation events.
  • The log includes the type of operation (allocate/deallocate), the disk offset, the size of the allocation, and the number of tries for an allocation to be successful.
  • A global header containing storage configuration is included at the beginning of the log.
  • The allocation log is written to a binary file, the path to which can be set via a runtime configuration parameter: allocation_log_file_path.

• Visualization Script:

  • Created a Python script (scripts/visualize_allocation_log) to visualize the allocation and deallocation patterns using matplotlib.
  • The script provides interactive controls for:
    • Displaying allocation bitmaps for each layer.
    • Visualizing fragmentation over time, both locally (per layer) and globally.
    • Visualizing free space over time, both locally (per layer) and globally.
    • Visualizing allocation tries and sizes.
    • Moving through the log using a slider.
    • Enabling or disabling components using checkboxes.
  • The script has an optional video export functionality using ffmpeg, which uses multiple processes for faster video generation.
  • The script comes with a requirements.txt to manage dependencies.
  • Added documentation about how to use the script.

• Code Changes:

  • Modified Cargo.toml to include the new allocation_log feature.
  • Modified the SegmentAllocator to optionally return allocation tries alongside the segment offset.
  • Modified dmu.rs to write the allocation and deallocation events, as well as writing the global header.
  • Modified database/mod.rs to include the allocation_log_file_path config and call the write global header before the tree is build.
  • Small fix in tree/imp/mod.rs to use the correct return type of the apply function.

Purpose:

This provides insights into how betree utilizes its storage space, allowing for better understanding and optimization of the system's performance and future memory allocation strategies. It allows for better debugging of memory-related issues.

How to use:

1. Enable the allocation_log feature when compiling betree.
2. Run your application with the desired configuration.
3. Use the scripts/visualize_allocation_log to visualize the logs. The log file path can be passed as a CLI parameter or specified via the configuration.

[fia0]

Tested it on my system and everything worked fine. Thanks for your work on this feature^^