v2.4.0

This release adds the ability to load and save test sets and results from Apache Parquet files. It also refactors the command-line tool with a progress bar (estimating the time remaining until completion) as well as a "very verbose" mode for per-call details.

$ python mpc_qpbenchmark.py run
[2024-12-16 10:27:18,154] [info] Loading existing results from '~/src/mpc_qpbenchmark/results/qpbenchmark_results.csv'... (results.py:56)
[2024-12-16 10:27:18,160] [info] Loaded 3840 rows from '~/src/mpc_qpbenchmark/results/qpbenchmark_results.csv' (results.py:63)
 23%|████████████▏                                       | 720/3072 [00:00<00:01, 1775.82it/s]

Added

• Add ParquetTestSet class for test sets loaded from Parquet files
• Add Problem.from_qpsolvers shorthand function
• Add ProblemList class that can be saved to Parquet files
• Allow strings or paths as arguments to the main function
• CLI: Add list_problems command
• New dependency: tqdm
• Optionally add a GPU summary line to reports
• Results can now be read and save from and to Parquet files
• Results property to get the number of result rows
• Support Python 3.12
• Very-verbose mode for extra debug messages

Changed

• Allow results file paths to be strings or pathlib paths
• Don't save results more frequently than every 10 seconds
• More informative solver exception warnings
• Refactor results loading and saving
• Save results to file after problem is done rather than at each solver call
• Update skip logic in CPU info table

Fixed

• CLI: Only allow --solver to be an available solver for the run command

v2.3.0

This release improves subset handling for QP test sets that have both sparse and dense problems, allowing using the same results file for both.

Added

• Add results_path argument to the main function
• Results: handle loading/saving CSV files with out-of-test-set problems
• Special handling to get CVXOPT version number properly

Changed

• CLI: make results path an argument to all commands
• Rename "include timeouts" argument to "rerun timeouts"

v2.2.3

This patch release fixes reporting of solver versions when some solvers are excluded.

Fixed

• Fix reporting of solver versions when some solvers are excluded

v2.2.2

This patch version fixes handling of no-op conversions in test sets (dense to dense or sparse to sparse).

Added

• Expose exceptions from top-level module

Fixed

• Bump Python version to actual requirement of 3.9 or above
• Handle no-op conversion from dense to dense properly
• Handle no-op conversion from sparse to sparse properly

v2.2.1

This patch release updates to qpsolvers v4.3.1 which fixes a bug in the Gurobi interface. Thanks @563925743 👍

Changed

• Update qpsolvers to v4.3.1 to fix the Gurobi interface

v2.2.0

The benchmark is now distributed on conda-forge:

conda install qpbenchmark

This release also adds a main-function hook to make it easier to run test sets directly from their repository as python my_test_set.py.

Added

• Distribute on conda-forge
• Test set path argument to the main function of the command-line interface

Changed

• Update environment file to install from conda-forge

v2.1.1

This patch release fixes a bug in the handling of known solver issues.

Changed

• Clarabel: set tol_gap_abs settings in set_eps_abs
• Clarabel: set tol_gap_rel settings in set_eps_rel
• Log a warning message when skipping known solver issue
• PIQP: set eps_duality_gap_abs settings in set_eps_abs
• PIQP: set eps_duality_gap_rel settings in set_eps_rel
• ProxQP: set eps_duality_gap_abs settings in set_eps_abs
• ProxQP: set eps_duality_gap_rel settings in set_eps_rel

Fixed

• Handling of known solver issues

v2.1.0

This release removes the cost error from benchmark metrics, and completes the spinning off of test sets to their own repositories. It also brings a few quality-of-use improvements.

Added

• Expose Problem from top-level module
• Link in report header to go to results tables directly
• Number of problems in report header
• Utility function to load a problem from file

Changed

• Breaking: Remove the cost error from benchmark metrics
• Include Python 3.8 in supported versions

Removed

• Move the GitHub Free-for-all test set to its own repository
• Move Maros-Meszaros test set to its own repository

v2.0.0

This release renames the benchmark to qpbenchmark and adds the QPALM solver.

Added

• New solver: QPALM

Changed

• Breaking: Rename the project "qpbenchmark"
• Don't install solvers by default from PyPI
• Move solver issue and timeout listings to test-set themselves

Removed

• Module-wide skip solver issue/timeout functions

v1.2.0

This release adds a mid_accuracy set of settings to the QP solvers benchmark.

Added

• Citation file to refer to the benchmark in scientific works
• Medium accuracy settings
• Note in the readme on CPU throttling