⚗️ Benchmark GDAL Libertiff driver vs async-tiff

[weiji14]

Provide instructions for downloading and converting a Sentinel-2 True Colour Image (TCI) file, and run benchmarks comparing read speeds with Libertiff driver (GDAL 3.11.5) and async-tiff (version @ py-v0.2.0)

Steps

1. Obtain Sentinel-2 TCI image and convert it (from 318.0MB to 272.2MB) using the following specs:
   • a tiled GeoTIFF
   • apply LZW compression
   • set Horizontal differencing predictor
2. Run benchmarks using 4 threads by calling cargo bench

Results

Ran on my laptop (12th Gen Intel® Core™ i5-12600HX), with 16 threads:

    Finished `bench` profile [optimized] target(s) in 0.08s
     Running benches/read_geotiff.rs (target/release/deps/read_geotiff-cd8205595006d753)
Benchmarking read_geotiff/1_gdal_CPU/Sentinel-2 TCI: Warming up for 3.0000 s
Warning: Unable to complete 30 samples in 5.0s. You may wish to increase target time to 8.0s, or reduce sample count to 10.
Benchmarking read_geotiff/1_gdal_CPU/Sentinel-2 TCI: Collecting 30 samples in estimated 8.0334 read_geotiff/1_gdal_CPU/Sentinel-2 TCI
                        time:   [268.08 ms 270.13 ms 273.27 ms]
                        thrpt:  [996.10 MB/s 1.0077 GB/s 1.0154 GB/s]
                 change:
                        time:   [-2.9896% -1.8265% -0.4887%] (p = 0.00 < 0.05)
                        thrpt:  [+0.4911% +1.8605% +3.0817%]
                        Change within noise threshold.
Found 2 outliers among 30 measurements (6.67%)
  1 (3.33%) high mild
  1 (3.33%) high severe
Benchmarking read_geotiff/2_async-tiff_CPU/Sentinel-2 TCI: Warming up for 3.0000 s
Warning: Unable to complete 30 samples in 5.0s. You may wish to increase target time to 10.2s, or reduce sample count to 10.
Benchmarking read_geotiff/2_async-tiff_CPU/Sentinel-2 TCI: Collecting 30 samples in estimated 1read_geotiff/2_async-tiff_CPU/Sentinel-2 TCI
                        time:   [334.24 ms 335.85 ms 337.64 ms]
                        thrpt:  [806.19 MB/s 810.50 MB/s 814.39 MB/s]
                 change:
                        time:   [-3.4316% -2.2507% -1.1614%] (p = 0.00 < 0.05)
                        thrpt:  [+1.1750% +2.3025% +3.5535%]
                        Performance has improved.
Found 3 outliers among 30 measurements (10.00%)
  3 (10.00%) high mild

So LiberTIFF can reach ~1.0GB/s throughput, compared to ~0.8GB/s for async-tiff. Looking at my system monitor while the benchmarks were running, I noticed that LiberTIFF was able to reach nearly 100% of CPU load (see 40-25s mark), compared to async-tiff with only used 80% (see 25-10s mark). Slowdown in async-tiff might be because rayon's multi-threading isn't optimized properly yet to use full CPU load.

TODO

• Initial code
• Use gdal docker image to get and process sample GeoTIFF on CI
• Ensure that CodspeedHq/action works to report benchmark results on CI
• Parametrize n_threads to run with 1, 4 or 16 threads

[codspeed-hq]

CodSpeed Performance Report

Merging #3 will create unknown performance changes

_{Comparing bench/gdal_vs_async-tiff (60596d9) with main (a12d7d9)}

Summary

🆕 2 new

Benchmarks breakdown

	Benchmark	BASE	HEAD	Change
🆕	1_gdal_CPU_threads=4[Sentinel-2 TCI]	N/A	7.9 s	N/A
🆕	2_async-tiff_CPU_threads=4[Sentinel-2 TCI]	N/A	12.4 s	N/A