Portable workflows
[ Home ]
Please, do not forget to check Getting Started Guides to understand CK concepts!
Researchers and engineers often hardwire names, paths and versions of libraries, tools, benchmarks, data sets and experimental results in their scripts for quick prototyping due to a constant lack of time.
However, rapidly evolving software and hardware make such scripts outdated and often useless very quickly. Worse, sharing such scripts with the community, places extra burden on other users to adapt such scripts to a different environment.
Docker and Virtual Machines may partially solve this problem by providing a snapshot of all user tools and libraries. However, Docker can not help in cases when researchers want to validate and improve shared techniques in their native environment, with their own or latest tools, libraries, benchmarks and data sets, and possibly even using a different OS and architecture.
Cmake can help users detect software dependencies and build a project in a user environment. However, cmake tries to be smart by selecting the most suitable version of a required software dependency and then produces a fixed Makefile for a given project which is vital for end-users. In contract, researchers often want a flexible project that can be tried with different versions of a given compiler (say GCC or LLVM), analytics tool (old R versus new R), or different libraries such as comparing cuBLAS vs OpenBLAS vs CLBlast, which may co-exist in parallel! Furthermore, the learning curve of cmake is quite steep and it is unlikely that researchers will use it for small prototypes.
We and our colleagues have been facing similar problems in our computer systems' research (particularly on collaborative benchmarking, universal multi-objective autotuning and machine learning) for more than a decade. We have also been observing the suffering of reviewers during Artifact Evaluation at major computer systems' conferences including PPoPP, CGO and PACT.
Our solution to above problems is to use CK modules as platform independent templates with JSON API for any required software. Such modules can detect whether a required software is installed, prepare environment scripts for all available versions of this software (that may co-exist in parallel), invoke software with a correct environment, parse output, and embed it into unified JSON output. In such case, researchers can quickly prototype their software and hardware independent experimental workflows with unified API from shared or newly created modules simply as LEGO (TM). The following figure conceptually shows a CK module program that can compile and run a given program using any installed compiler.
We've implemented this functionality using four CK modules soft, env, package and os available in the ck-env repository (you can obtain it from the command line via ck pull repo:ck-env) as conceptually shown in the following figure:
Soft entries are used to describe how to detect whether a required software is already installed in the system, automatically find all available versions, and generate a local script for each or selected version to set up all required environment variables such as paths to binaries, libraries, include files, scripts and auxiliary tools.
The community already shared many CK entries to detect various types of software in the ck-env repository. You can list all available software descriptions via
$ ck pull repo:ck-env $ ck list soft
When searching for a specific software description, you can use wildcards or tags separated by comma as show in the following example (flag --all tells CK to show full CID including a repository of a given entry):
$ ck list soft:compiler* $ ck search soft --tags=lib,blas --all
For example, CK entry soft:compiler.gcc has the following self-explanatory meta.json description:
{
"auto_detect": "yes",
"customize": {
"add_only_with_version": "yes",
"add_to_path": "yes",
"check_that_exists": "yes",
"env_prefix": "CK_ENV_COMPILER_GCC",
"languages": [
"C",
"CPP",
"F77",
"F90",
"F95"
],
"limit_recursion_dir_search": {
"linux": 3,
"win": 4
},
"only_for_target_os_tags": [
"mingw",
"linux"
],
"soft_file": {
"linux": "gcc",
"win": "gcc.exe"
},
"soft_file_from_host_os": "yes",
"soft_version_cmd": {
"linux": "--version > $#filename#$ 2>&1",
"win": " --version > $#filename#$ 2>&1"
}
},
"soft_name": "GCC compiler",
"tags": [
"compiler",
"gcc",
"lang-c"
]
}
Basically, it describes how to find GCC on various OS and obtain its version. It also has user-friendly tags which helps you find this software or later set a software dependency in your research projects.
After finding a required software entry in the CK such as GCC, you can attempt to automatically detect it on your system simply as following
$ ck detect soft:compiler.gccor via
$ ck detect soft --tags=compiler,gcc
If software is not found, a user will be shown a file install.txt from the same entry as well as either install.linux.txt on Linux or install.win.txt on Windows. These files describe how to install a required software on your system with all dependencies:
$ cat `ck find soft:compiler.gcc`/install.linux.txt
Above entry also has a Python script customize.py. You can view it via:
$ cat `ck find soft:compiler.gcc`/customize.py
It has several functions including limit to prune falsely detect software instances (for example, stripped GCC binary can be found in other projects such as X11), parse_version to obtain versions of each found GCC instance, and setup which sets up environment variables for a given version of a software such as PATH, LD_LIBRARY_PATH, CK_CC, and so on. These variables will be recorded to a special OS script using CK env module as describe in the next section.
Hints:
- Whenever you want to add and share description of a new software, we use a Wikipedia-like concept, i.e. you should find similar existing software entry, copy it to the new CK entry via ck cp soft:compiler.gcc my-repo:soft:new-soft, and then edit meta.json and customize.py files in the newly created entry.
- If you want to know which variables are passed to limit, parse_version and setup, simply print the input and stop the script. The reason is that we successfully use agile development methodology where input is continuously extended while keeping backward compatibility, so the API documentation may not be always up to date.
- If you are lost in the API and meta, simply send your question to the CK mailing list ;) ...
We would like to thank the community for sharing more and more of such software descriptions - any help is very welcome!
Env entries are used to store prepared environment for a detected software instance in an env.sh script on a Linux host or env.bat script on a Windows host. The meta description of these entries contain software tags, detect version and various auxiliary info (such as specific command line switches for compilers).
It is possible to see all software registered in the CK with their UID, versions and tags in a user friendly way via
$ ck show env
It is also possible to prune this list by tags via
$ ck show env --tags=gcc
You can see how a meta of one of the added env entries look like via
$ ck load `ck search env --tags=gcc` --min
You can also see a generated environment script on a Linux host as following
$ CID=`ck search env --tags=gcc` ; P=`ck find $CID`; cat $P/env.sh
The idea is that when your research project needs a compiler, you can query a CK to find such environment, call it, and then call a compiler, i.e.
$ CID=`ck search env --tags=gcc` ; P=`ck find $CID`; . $P/env.sh ; which $CK_CC ; $CK_CC --version
However, do not worry, CK automates this too as will be shown in this section below!
Whenever a required software is not found, rather then asking a user to install it, we've implemented a CK functionality to automatically attempt to install it using package module.
Package entries describe how to install a specific version of a given software using install.sh script on a Linux host or install.bat on a Windows host. Entries may even contain a package for this software or contain instructions about how to download it (using wget or git for example).
For example, you can list all available packages in the public ck-caffe GitHub repository (intended to unify benchmarking and optimization of a Caffe Deep Learning framework across multiple platforms) via
$ ck pull repo --url=https://github.com/dividity/ck-caffe $ ck list ck-caffe:package:*
You can also list only packages related to Caffe via
$ ck search package --tags=caffe
or related to Caffe models via
$ ck list package:*caffemodel*
You can then install a required packages simply as following:
$ ck install package:caffemodel-bvlc-googlenet $ ck install package:imagenet-2012-val $ ck install package:lib-caffe-bvlc-master-cuda
Note, that if a package has other software dependencies, CK will attempt to automatically detect and register or install them too!
You may see a meta description of a imagenet-2012-val package as following:
$ ck load package:imagenet-2012-val --min
Similarly to soft entries, we also attempted to use keys as self-explanatory as possible in the package entries:
{
"check_exit_status": "yes",
"customize": {
"extra_dir": "",
"force_ask_path":"yes",
"no_os_in_suggested_path": "yes",
"no_ver_in_suggested_path": "yes",
"install_env": {
"IMAGENET_VAL_URL":"http://www.image-net.org/challenges/LSVRC/2012/nnoupb/ILSVRC2012_img_val.tar",
"IMAGENET_VAL_MD5":"29b22e2961454d5413ddabcf34fc5622"
},
"version": "2012"
},
"deps": {
},
"need_cpu_info": "no",
"end_full_path": {
"linux": "ILSVRC2012_val_00000001.JPEG"
},
"process_script": "install",
"soft_uoa": "ff47fc6a33b4b03b",
"suggested_path": "dataset-imagenet-ilsvrc2012-val",
"tags": [
"dataset",
"imagenet",
"raw",
"val",
"caffe",
"ilsvrc2012"
]
}
Note, that soft_uoa points to the UID of the related soft entry for this package to let CK automatically set up and register environment after this package is installed. By default, packages are installed in the $HOME/CK-TOOLS directory on Linux or %USERPROFILE%\CK-TOOLS on Windows.
You can also see the installation script as following:
$ cat `ck find package:imagenet-2012-val`/install.sh
You can help the community by sharing more packages using above example as a template!
Sometimes we need to install software on our host while it is intended for another target device such as Android and Windows-based mobile phones (compilers, libraries, auxiliary tools and scripts).
Hence, we provided an option --target_os in the CK to let users specify a target OS/arch tuple stored using CK os module from the ck-env repository.
You can see available OS/arch tuples via
$ ck list os
For example, if you want to detect GCC from Android NDK for Android21 target with ARM64 (Samsung Galaxy S7), you can find the most close os entry and then add it to the ck detect soft command as following:
$ ck list os:android* $ ck detect soft:compiler.gcc.android.ndk --target_os=android21-arm64
You can see meta information of this os:android21-arm64 meta via
$ ck load os:android21-arm64 --min
Now, you can see all software registered for this target via
$ ck show env --target_os=android21-arm64
You can also install (and possibly build) packages (such as XOpenME interface to instrument programs and expose their parameters via JSON from the ck-autotuning repo) for this target via
$ ck pull repo:ck-autotuning $ ck install package:lib-rtl-xopenme --target_os=android21-arm64
Researchers often write ad-hoc scripts to detect various properties of the host and target machine. We decided to unify this process and provided several CK modules platform.os, platform.cpu, platform.gpu, platform.nn and platform with detect function to obtain OS,CPU,GPU,Neural Network Accelerator, and general platform properties respectively and record them in JSON format.
For example, you can obtain your host OS and CPU properties via
$ ck detect platform.os $ ck detect platform.gpu
or you can detect all available properties and output in JSON via
$ ck detect platform --out=json
You can also detect properties of target devices other than your host (such as Android) as following:
$ ck detect platform --target_os=android21-arm64
If you have more than one Android device connected, you can find its UID via adb devices and then specify it via --target_device_id as following:
$ ck detect platform --target_os=android21-arm64 --target_device_id={take from "adb devices"}
Finally, platform.init entries keep various auxiliary scripts and tools required for different platforms (for example, to read or set CPU and GPU frequency, set frequency governors, measure power consumption, etc). We plan to extend it in the future (including auto-detection of a correct platform). You can obtain a list of available ones via:
$ ck list platform.init
Note that other repositories may provide similar modules for a specialized hardware. For example, repository ck-autotuning has a module to detect GPGPU properties (currently CUDA and OpenCL capabilities):
$ ck pull repo:ck-autotuning $ ck detect platform.gpgpu
Our idea is to let the community collaboratively extend these modules or add new via CK and GitHub while unifying their output (properties) across various platforms including Linux, MacOS, BSD, Windows, Android, etc. In the future, we may be interested to add more tools such as hwloc to the CK.
The community can also extend and reuse related CK entries with platform properties. For example, we also started recording and publicly share properties of various platforms in the CK format during our program/compiler crowdtuning campaign at GitHub.
You can easily obtain this repository (continuously updated) and use it for your own research or to embed it to your tools via
$ ck pull repo:ck-crowdtuning-platforms
You can see the online list of the above platforms at http://cknowledge.org/repo (see "Participated Platforms").
Implementing portable and customizable workload template (example of a universal workload characterization)
Above approach provided all the basic blocks needed to quickly prototype various portable experimental workflows independent of software and hardware, while reusing and cross-linking shared CK entries via JSON meta and CK UIDs. It is vital to enable collaborative and reproducible computer systems' research.
For example, we used all above functionality to involve the community and make our machine learning based crowd-tuning, autotuning, run-time adaptation and SW/HW co-design techniques practical (1, 2, 3, 4, 5, 6) as following.
We've implemented module program from the ck-autotuning repository as a universal workflow template with JSON API to be able to share various programs, benchmarks, kernels, codelets in a platform-independent format, describe their software dependencies, compile and run them with different optimization choices for any target device, and record their behavior in a unified JSON.
For example, let us have a look at the popular program susan to detect image corners that we have shared in the CK format in the ctuning-programs repository. You can obtain, compile and run it on your machine simply as following:
$ ck pull repo:ctuning-programs $ ck compile program:cbench-automotive-susan --speed $ ck run program:cbench-automotive-susan
The module program will automatically detect available compiler on your host machine required to compile this program, set its environment, compile program, and ask you to select possible command line to run this program (since it can detect corners and edges of an image or smooth it).
Also note that you will be asked to select a dataset that can fit this program. Datasets are recorded and shared in the CK using dataset module from the same repository. You can see all available datasets in the CK via
$ ck list dataset
You can also list only jpeg datasets via
$ ck search dataset --tags=jpeg
Datasets are described using JSON meta. For example, dataset:image-jpeg-0001 has the following description including list of files, tags and publications where it was used:
$ ck load dataset:image-jpeg-0001 --min
You can list files in this entry as following:
$ ck list_files dataset:image-jpeg-0001
Let us have a closer look at the susan entry. It has susan.c source file and the following partially self-explanatory JSON meta:
$ ck load program:cbench-automotive-susan --min
{
"backup_data_uid": "ffbf51c23a91343c",
"build_compiler_vars": {
"XOPENME": ""
},
"compile_deps": {
"compiler": {
"local": "yes",
"name": "C compiler",
"sort": 10,
"tags": "compiler,lang-c"
},
"xopenme": {
"local": "yes",
"name": "xOpenME library",
"sort": 20,
"tags": "lib,xopenme"
}
},
"compiler_env": "CK_CC",
"data_name": "cbench-automotive-susan",
"extra_ld_vars": "$<<CK_EXTRA_LIB_M>>$",
"main_language": "c",
"process_in_tmp": "yes",
"program": "yes",
"run_cmds": {
"corners": {
"dataset_tags": [
"image",
"pgm",
"dataset"
],
"hot_functions": [
{
"name": "susan_corners",
"percent": "95"
}
],
"ignore_return_code": "no",
"run_time": {
"fine_grain_timer_file": "tmp-ck-timer.json",
"run_cmd_main": "$#BIN_FILE#$ $#dataset_path#$$#dataset_filename#$ tmp-output.tmp -c",
"run_cmd_out1": "tmp-output1.tmp",
"run_cmd_out2": "tmp-output2.tmp",
"run_correctness_output_files": [
"tmp-output.tmp",
"tmp-output2.tmp"
],
"run_output_files": [
"tmp-output.tmp",
"tmp-ck-timer.json"
]
}
},
"edges": {
"dataset_tags": [
"image",
"pgm",
"dataset"
],
"hot_functions": [
{
"name": "susan_edges",
"percent": "70"
},
{
"name": "susan_thin",
"percent": "18"
}
],
"ignore_return_code": "no",
"run_time": {
"fine_grain_timer_file": "tmp-ck-timer.json",
"run_cmd_main": "$#BIN_FILE#$ $#dataset_path#$$#dataset_filename#$ tmp-output.tmp -e",
"run_cmd_out1": "tmp-output1.tmp",
"run_cmd_out2": "tmp-output2.tmp",
"run_correctness_output_files": [
"tmp-output.tmp",
"tmp-output2.tmp"
],
"run_output_files": [
"tmp-output.tmp",
"tmp-ck-timer.json"
]
}
},
"smoothing": {
"dataset_tags": [
"image",
"pgm",
"dataset"
],
"hot_functions": [
{
"name": "susan_smoothing",
"percent": "98"
}
],
"ignore_return_code": "no",
"run_time": {
"fine_grain_timer_file": "tmp-ck-timer.json",
"run_cmd_main": "$#BIN_FILE#$ $#dataset_path#$$#dataset_filename#$ tmp-output.tmp -s",
"run_cmd_out1": "tmp-output1.tmp",
"run_cmd_out2": "tmp-output2.tmp",
"run_correctness_output_files": [
"tmp-output.tmp",
"tmp-output2.tmp"
],
"run_output_files": [
"tmp-output.tmp",
"tmp-ck-timer.json"
]
}
}
},
"run_vars": {
"CT_REPEAT_MAIN": "1"
},
"source_files": [
"susan.c"
],
"tags": [
"cbench",
"lang-c",
"susan",
"automotive",
"benchmark",
"program",
"small",
"crowd-tuning"
],
"target_file": "a"
}
The most important keys here are:
- source_files - program source files;
- tags - list of user-friendly tags to classify and find this program;
- compile_deps - software dependencies required to compile this program:
"compile_deps": {
"compiler": { # specifies that CK should find CK environment with a C compiler
# by default, we provided pre-installed environments for GCC
"local": "yes",
"sort": 10,
"tags": "compiler,lang-c"
},
"xopenme": { # specifies dependency on our xOpenME library (stripped plugin based framework)
# which is used to instrument programs and expose
# various run-time features in JSON format.
# You may keep this dependency even if you don't use
# this library (since we use it in most of CK benchmarks and
# you may want to use and extend it in the future)
# or you can remove this dependency.
"local": "yes",
"sort": 20,
"tags": "lib,xopenme"
}
},
- build_compiler_vars - build variables which will be added to the compilation command line. They will be likely prefixed with -D on Linux and with /D on Windows (this information is set by the compiler CK environment).
- run_cmds - available command lines that may invoke various algorithms inside a given program (such as detecting corners, edges, etc):
"run_cmds": {
"corners": { # User key describing a given execution command line
"dataset_tags": [ # Data set tags - will be used to query CK
"image", # and automatically find related entries such as images
"pgm",
"dataset"
],
"run_time": { # Next is the execution command line format
# $#BIN_FILE#$ will be automatically substituted with the compiled binary
# $#dataset_path#$$#dataset_filename#$ will be substituted with
# the first file from the CK data set entry (see above example
# of adding new data sets to CK).
# tmp-output.tmp is and output file of a processed image.
# Basically, you can shuffle below words to set your own CMD
"run_cmd_main": "$#BIN_FILE#$ $#dataset_path#$$#dataset_filename#$ tmp-output.tmp -c",
"run_cmd_out1": "tmp-output1.tmp", # If !='', add redirection of the stdout to this file
"run_cmd_out2": "tmp-output2.tmp", # If !='', add redirection of the stderr to this file
"run_output_files": [ # Lists files that are produced during
# benchmark execution. Useful when program
# is executed on remote device (such as
# Android mobile) to pull necessary
# files to host after execution
"tmp-output.tmp",
"tmp-ck-timer.json"
],
"run_correctness_output_files": [ # List files that should be used to check
# that program executed correctly.
# For example, useful to check benchmark correctness
# during automatic compiler/hardware bug detection
"tmp-output.tmp",
"tmp-output2.tmp"
],
"fine_grain_timer_file": "tmp-ck-timer.json" # If XOpenME library is used, it dumps run-time state
# and various run-time parameters (features) to tmp-ck-timer.json.
# This key lists JSON files to be added to unified
# JSON program workflow output
},
"hot_functions": [ # Specify hot functions of this program
{ # to analyze only these functions during profiling
"name": "susan_corners", # or during standalone kernel extraction
"percent": "95" # with run-time memory state (see "codelets"
# shared in CK repository from the MILEPOST project
# and our recent papers for more info)
}
]
"ignore_return_code": "no" # Some programs have return code >0 even during
# successful program execution. We use this return code
# to check if benchmark failed particularly during
# auto-tuning or compiler/hardware bug detection
# (when randomly or semi-randomly altering code,
# for example, see Grigori Fursin's PhD thesis with a technique
# to break assembler instructions to detect
# memory performance bugs)
},
...
},
- run_vars - environment variables required before running this program.
Furthermore, program module can also take --target_os and --target_device_id as input to build and run this program on a remote device.
For example, it is possible to run it on your Android mobile phone or tablet connected via ADB as following:
$ ck compile program:cbench-automotive-susan --speed --target_os=android19-arm $ ck run program:cbench-automotive-susan --target_os=android19-arm
Above approach allows computer systems' researchers avoid wasting considerable amount of time writing their own ad-hoc experimental workflows for various architectures. Instead, they can reuse CK program module with shared programs and datasets, and focus on new experiments and innovation.
It is possible to easily integrate program with other scripts or CK modules due to a unified JSON input/output. It is possible to obtain API for each module's action via
$ ck compile program --help $ ck run program --help
It is also possible to prepare input.json as following
{
"data_uoa":"cbench-automotive-susan",
"speed":"yes",
"target_os":"android19-arm",
"out":"json_file",
"out_file":"output.json"
}
and then compile program with this input file, while producing output.json:
$ ck compile program @input.json
It is also possible to compile program from another module or python script via access function from the CK kernel
r=ck.access({"action":"compile",
"module_uoa":"b0ac08fe1d3c2615",
"data_uoa":"cbench-automotive-susan",
"speed":"yes",
"target_os":"android19-arm"})
if r['return']>0: return r
print (r)
Note, that instead of program we used its alias b0ac08fe1d3c2615 (obtained via ck info module:program) since we strongly recommend to reference other modules or data from within CK via their UID and not alias, as described here.
You may be interested to have a look at the implementation of the program module to understand above concepts better:
$ cat `ck find module:program`/module.py
Next, we've created a higher-level function pipeline in this module to enable reproducible experimentation by chaining other CK modules to compile and run code several times while setting and monitoring CPU and GPU frequency between experiments, profiling code, extracting program and dataset features, performing basic statistical analysis of results, and so on. This experimental pipeline is continuously extended by the community to chain new tools when they become available such as dividiti's OpenCL profiler. You can see all currently available options via
$ ck pipeline program --help
This function also unifies the input and output API according to our concept of collaborative and machine-learning based program auto-tuning and crowd-tuning as a natural science (see PDF). Basically we expose four main keys in the API:
- design and optimization choices
- multiple characteristics
- hardware and software features
- run-time state
Such unification, in turn, allowed us to implement a universal and multi-objective autotuner that takes as input any other module with a pipeline function, start exploring exposed choices, applies Pareto filter and complexity reduction (if needed), and records behavior in a local CK repository using experiment module as conceptually shown in the following figure:
For example, you can invoke compiler flags autotuning of the previously mentioned susan program with a selected data set on your machine simply as following:
$ ck autotune program:cbench-automotive-susan
CK will then automatically detect available compilers, ask you to select a specific version, will select its flags, embed it to the "choices" input key, and will perform 30 iterations with random selection of flags. At the end, CK will report you which CK entry your results are in, and how to visualize results using CK module graph from the ck-analytics repository.
Again, you can easily retarget autotuning via
$ ck autotune program:cbench-automotive-susan --target_os=android19-arm $ ck autotune program:cbench-automotive-susan --target_os=mingw-64 ...
You may check available compiler descriptions via
$ ck list compiler
You can also see an example of compiler choices for GCC 6.1.0 in CK format via
$ ck load compiler:gcc-6.1.0-auto --min
We expect that users will be gradually add more compiler versions or provide more optimization flags and passes to the meta.
We've also implemented a CK machine learning module model with our JSON API and with 3 functions build (to build a predictive model), validate (to cross-validate new models) and use to predict values or classes (such as optimizations). It is available in the ck-analytics repository and attempts to unify various available regression and classification algorithms which users can embed into their pipelines.
You can find several related demos (including decision trees, unified statistical analysis and graph capabilities in the CK) in the following demo entries:
$ ck pull repo:ck-analytics $ ck list ck-analytics:demo:*
Just navigate to any of the above directories and run demo scripts there with json inputs:
$ cd `ck find demo:ml-decision-tree` $ ./model-sklearn-dtc-build.bat $ ./model-sklearn-dtc-validate.bat $ ./model-sklearn-dtc-use.bat
Note, that model module may require sklearn, math, matplotlib, numpy Python packages as well as dot tool installed on your system. If you use Anaconda Scientific Python, you will have most of those Python packages already installed.
After our approach of accessing various machine learning algorithms via a web service provided in the [cTuning.org] since 2009 gained considerable popularity, we extended this concept using simple web server integrated with CK. For example, our live repo provides an access to all above ML techniques via CK web service with unified JSON API! You can see if this module is available at the remote server using CK simply as following:
$ ck list remote-ck:module:model*
At the same time, we work with the community to gradually add more machine learning algorithms including recent DNN frameworks (Caffe, TensorFlow and others) which can be accessible in a unified way via CK web services with JSON API, and can be easily integrated with your own AI-based projects.
Integrated CK web-service with JSON API, also makes it relatively straightforward to record benchmarking and autotuning results in some remote server rather than in the local repository of the user machine. This should then be enough to start collaboratively benchmarking, tuning and learning behavior of shared workloads across multiple systems provided by volunteers, exchange results via public (or private) repositories of knowledge, and even classify and predict optimizations when enough knowledge is collected.
We've implemented several several crowd-tuning scenarios such as GCC and LLVM compiler flag multi-objective autotuning in a ck-crowdtuning repository. You can obtain it and start immediately participating in collaborative performance tuning via
$ ck pull repo:ck-crowdtuning $ ck crowdtune program
All the results are publicly shared in the CK-based public repository at http://cknowledge.org/repo. This dashboard is implemented in a module program.optimization via show function which returns prepared HTML while cross-linking all available information in the repository about hardware, compilers, workloads and optimizations.
Furthermore, unified JSON API of the CK web service allowed us to implement a simple Android application that can let volunteers use their mobile phones and tablets to participate in crowd-tuning. You can download this application via Google Play Store and get its sources at GitHub. In fact, this application is a universal framework for crowdsourcing experiments since it simply queries available public servers for available experiment packs, downloads and runs them, and then send required statistics back to the server.
We now continue adding more collaborative optimization scenarios such as OpenCL/CUDA/CPU-based library crowd-tuning (see our CK-based CAFFE DNN framework, CK-GEMMBENCH) and compiler bug detection (see CK-powered CLSMITH) with the help of the community, dividiti and the non-profit cTuning foundation.
We also started converting all our past research techniques to CK to help the community not only reproduce our past research, but also build upon it (see CK-powered MILEPOST GCC and our initiative on reproducible research).
The community also helps us add more realistic workloads (programs, benchmarks, kernels and datasets), collaboratively crowd-benchmark and crowd-tune them across as many platforms as possible, continuously exchange optimization knowledge via our open repository http://cknowledge.org/repo, and then extrapolate it to suggest how to build next generation of efficient self-tuning computer systems while dramatically reducing development costs (see our publications for more details) - please, join our collaborative effort!
If you have any questions, do not hesitate to contact us or ask the community via CK mailing list!