ASAPy is an analyzing tool for algorithm selection (AS) written in Python 3. Given a matrix of performance data and a matrix of instance features, ASAPy automatically generates a thorough web-based report with many diagrams to provide insights in the data. Hence, ASAPy addresses two kinds of users:
- Users of algorithm selection to get insights in their own data, to verify the correctness of the data and to understand how algorithm selection handles the data
- Developers of algorithm selection tools to get better insights on what kind of data they operate
This program is free software: you can redistribute it and/or modify it under the terms of the MIT license (please see the LICENSE file). This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. You should have received a copy of the MIT license along with this program (see LICENSE file). If not, see https://opensource.org/licenses/MIT.
ASAPy runs with '''Python 3.5''' and we recommend to use a Linux OS (e.g. Ubuntu) -- we cannot give support for MacOS or Windows. Many of its dependencies can be fulfilled by using Anaconda 3.4.
To install all requirements, please run:
pip install -r requirements.txt
If you do not use Anaconda or a virtual environment, you may want to add the option --user to install it in your user space.
If the installation of autofolio fails, please try to install autofolio first and then asapy.
If you use Anaconda as your Python environment, you have to install the following three packages:
conda install gxx_linux-64 gcc_linux-64 swig
ASAPy reads two input formats: CSV and ASlib. The CSV format is easier for new users but has some limitations to express all kind of input data. The ASlib format has a higher expressiveness -- please see www.aslib.net for all details on this input format.
For the CSV format, simply two files are required.
One file with the performance data of each algorithm on each instance (each row an instance, and each column an algorithm).
And another file with the instance features for each instance (each row an instance and each column an feature).
All other meta-data (such as runtime cutoff) has to be specified by command line options (see python scripts/asapy --help).
The performance matrix has to be passed by the option --performance_csv [file name] and the feature matrix by --feature_csv [file name].
Furthermore, you can specify the objective (running time or solution quality; --objective) , the running time cutoff (runtime_cutoff) and to maximize or minimize the objective (--maximize).
For example, a call could look like
python scripts/asapy --performance_csv [file name] --feature_csv [file name]
To use the full expressiveness of algorithm selection data, you can use the ASlib format by providing a folder with all files of an ASLib szenario.
python scripts/asapy --scenario [folder]
By default ASAPy writes into the current directory. But you can also redirect the output with the option --output. Please note that ASAPy generates a lot of plots. Depending on the number of algorithms in your data, ASAPy can generate more than 100MB.
If you want to disable certain plots (for example, the scatter plots needs a lot of time to be generated),
you can use a json-formatted configuration file. To get a template for the configuration file, please use the option --print_config.
@ARTICLE{bischl-aij16a,
author = {B. Bischl and P. Kerschke and L. Kotthoff and M. Lindauer and Y. Malitsky and A. Frech\'ette and H. Hoos and F. Hutter and K. Leyton-Brown and K. Tierney and J. Vanschoren},
title = {ASlib: A Benchmark Library for Algorithm Selection},
journal = {Artificial Intelligence Journal (AIJ)},
volume = {237},
year = {2016},
pages = {41-58}
}
- ASP-POTASSCO
- BNSL-2016
- CPMP-2015
- CSP-2010
- CSP-MZN-2013
- MAXSAT12-PMS
- MAXSAT15-PMS-INDU
- PROTEUS-2014
- QBF-2011
- QBF-2014
- SAT11-HAND
- SAT11-INDU
- SAT11-RAND
- SAT12-ALL
- SAT12-HAND
- SAT12-INDU
- SAT12-RAND
- SAT15-INDU
Marius Lindauer: lindauer@cs.uni-freiburg.de