FLiT v2.1.0

FLiT is now very stable, so we can exit the beta releases. Honestly, we kept it in the beta release not because the tool was unreliable, but rather because we were unsure if the interface would undergo breaking changes with design reworks.

The interface has stabilized. We may do small breaks of interface in the future, but so far nothing major until FLiT 3.0.

Highlights of Major Changes:

• Place items from flit::fsutil namespace into the flit namespace (breaking change)
• MPI
  • Add MPICH support
  • Change MPI usage to be with a main-like function as subprocesses
• flit bisect
  • Bisect uses information from flit-config.toml if the given compiler matches one specified there.
  • Improve speed of the generated Makefile
  • Add --compiler-type flag
  • Add --ldflags, --add-ldflags, and --use-linker
• Autogenerated Makefile
  • Use standard GNU Makefile variable names
  • Brief colored output
  • Source files can have any suffix (instead of just .cpp)
  • Removed the automatic insertion of --gcc-toolchain and --gxx-name for the Clang and Intel compilers respectively -- user can now add this from flit-config.toml (documentation added on how to do this)
  • Experimental support for a Ninja build file instead (through flit experimental ninja)
  • Improve speed and responsiveness using more self-recursion
  • Faster make clean by deleting output directories rather than all files
  • Mitigate C++ standard library incompatibility by compiling FLiT source code into each test executable
• flit-config.toml
  • Add fixed_compile_flags and fixed_link_flags fields for each compiler specified in flit-config.toml
• Return std::vector of float, double, long double, or std::string from tests. Add a comparison function for each.
• Add functions to FLiT's C++ interface (can be used in tests): rsplit(), lsplit(), split(), trim(), rstrip(), lstrip(), strip(), basename(), and dirname()
• Bash-completion

All Issues Addressed:

• #255 (PR #260): Added --compiler-type flag for flit bisect
• #222 (PR #264): Added MPICH support
• #253 (PR #266): Handle different file endings for source files
• #267 (PR #268): Makefile recursion on dry runs too
• #269 (PR #270): Faster make clean by deleting destination directories. Also, reorganize destination directories. Add a make dirs target.
• #49 (PR #271): Return std::vector of floating-point or of std::string from a test
• #273 (PR #274): Fix flit bisect when calling main as a subprocess
• #165 (PR #280): Fixed the SimpleCHull litmus test to not use relative paths
• #272 (PR #275): Add ability to call a main-like function from your test as a subprocess, with or without MPI
• #216 (PR #281): Use standard Makefile Variable names in autogenerated Makefiles
• #276 (PR #282): Brief colored output from Makefiles
• #283 (PR #284): Fix --delete flag behavior for flit bisect
• #289 (PR #290): Fix automated test for flit bisect involving templates
• #286 (PR #288): Add helper functions for std::string such as flit::split() and flit::trim(). Remove flit::fsutil namespace
• #292 (PR #295): Add DESTDIR variable to FLiT's top-level Makefile
• #296 (PR #297): Fix Makefile color output for older systems and OSX
• #125 (PR #301): Add fixed_compile_flags and fixed_link_flags to the compiler section in flit-config.toml
• #302 (PR #303): Experimental support for Ninja build files instead of an autogenerated Makefile
• #293 (PR #304): Improve usage and testing of mkstemp() and mkdtemp(). Add functions strip(), basename(), and dirname() to FLiT
• #307 (PR #309): Fix use of fixed_compile_flags and fixed_link_flags from flit bisect
• #310 (PR #311): Significantly improve responsiveness of autogenerated Makefiles by using more self-recursion
• #96 (PR #314): All flit functions in the flit namespace
• #227 (PR #315): Remove hostname from generated files
• #210 (PR #318): Mitigate C++ standard library incompatibility by compiling FLiT source code into each test executable
• #133 (PR #322): Create a good default compare() method that handles NaN and inf in a way that makes sense
• #316 (PR #324): Do not copy Empty.cpp when calling flit init --litmus-tests
• #320 (PR #321): Update the issue template on GitHub
• #121 (PR #319): Implement bash-completion for FLiT
• #308 (PR #325): Add --ldflags, --add-ldflags, and --use-linker to flit bisect

FLiT v2.0-beta.2

Highlights of Major Changes

• TravisCI continuous integration fixed and working
• flit bisect
  • outputs results incrementally.
  • optimizations, such as memoizing and search space reduction
  • order findings by the magnitude of variability
  • add flags --compile-only and --precompile-fpic
  • additional bisect assertion for finding disjoint sets
  • add dependency of pyelftools
• Users specify the FLiT search space in flit-config.toml instead of being hard-coded
• flit import: add --dbfile flag to create an import database without a flit-config.toml file
• Remove hosts section from flit-config.toml
• Many bug fixes

All Issues Addressed:

• #174 (PR #179): Have bisect output results incrementally
• #175 (PR #181): Have function passed to bisect_search only take one argument
• #182 (PR #185, #186, #187): Fix TravisCI continuous integration
• #180 (PR #190): Memoize the bisect test function
• #143 (PR #195): Fix NaN return values from litmus tests
• #172 (PR #197): Order bisect findings by magnitude of variability
• #200 (PR #201): Fix CSV file parsing
• #120 (PR #206): Allow user-provided compilers in flit-config.toml
• (PR #209): Add --compile-only and --precompile-fpic to flit bisect
• #211 (PR #215, #223): Add --dbfile to flit import to import without a flit-config.toml file
• #220 (PR #221): Have the test harness delete temporary directories even when exceptions are thrown
• #217 (PR #219): Create a template for issues and pull requests (a GitHub-specific update)
• #212 (PR #213): For bisect, if the Makefile errors out, print out the Makefile output for debugging
• #225 (PR #226): For bisect, add assertion for finding disjoint sets (cannot continue if they overlap)
• #230 (PR #232): Fix compilation if header files go missing
• #229 (PR #231): Fix bisect deadlock where multiple updates to ground-truth.csv were attempted
• #194 (PR #236): Fix the slow bisect search due to an explosion of templated function definitions
• #238 (PR #241): Fix bisect search space problems. Specifically, only search over strong and publicly exported function symbols. This pull request added the dependency of pyelftools.
• #233 (PR #243): Update the required version of binutils in the documentation
• #239 (PR #242): Fix support for older versions of the Clang compiler (version 4)
• #244 (PR #246); Remove the hosts section from flit-config.toml. It was not used.
• #249 (PR #250): Update documentation about how FLiT handles MPI tests. Specifically how the test return values are handled.
• #119 (PR #251): Allow the user to specify the list of optimization levels and flags in flit-config.toml for the FLiT search space. No longer hard-coded.
• #256 (PR #258): Fix bisect crash when an object file has an empty dwarf info
• #257 (PR #259): Fix bisect crash when an object file has no dwarf info section

FLiT v2.0-beta.1

Since FLiT has been stable in the alpha phase for some time and some other things have stabilized more (such as documentation and interface), we are happy to announce the move to beta status.

In this release, we have included the following pull requests:

#115, #116, #128, #130, #131, #132, #135, #139, #142, #145, #147, #150, #154, #155, #156, #157, #158, #159, #160, #161, #167, #170, #176

This includes the following features:

• Bisect
  • Fully implemented and supported - assigns variability blame to individual source files and functions
  • Works with Intel linker problems
  • Autodelete intermediate generated files to reduce disk usage from going out of control
  • Autorun all differences found in a given database file
  • Unfortunately, this functionality is not adequately documented (see issue #136)
• Add MPI support within flit-config.toml
• Add timing parameters to flit-config.toml
• Add --info flag to test executable to show how it was compiled
• Rename results and database columns to be more understandable
• Add plot_timing_histogram.py for auto-generating histogram plots and update plot_timing.py
• Add benchmarks: polybench and random - benchmark examples of FLiT usage
• Add uninstall target to the top-level Makefile
• Remove CUDA completely, it was broken and cumbering the space
• Remove old unused files. The repository is more lean with all files relevant
• License notices on files
• Fix support for GCC 4.8 (had a compiler error in FLiT)

Hotfix on install of v2.0-alpha.2

There was reported a bug in the installation making flit import not work. This has been fixed and a new alpha release is ready.

Updated alpha release of v2.0

Many fixes in this update:

• Remove TestInput and CUTestInput types, replace with arrays and std::vector
• Disable unfinished flit cli commands
• Remove sections of the settings file that are unused
• Fix incremental build
• Add version argument
• Recursive makefile allows for more scalability with project size and number of tests
• Add some tests for FLiT's infrastructure - not comprehensive, but some automated tests
• Fix timing functionality
• Add a time plot

Alpha release of new version

Not everything for this new release is finished. The tool has been rearchitected to be easier to use within existing code bases. Many things have changed. Here are a few of them:

1. A command-line tool called flit with subcommands
2. Storage is into an SQLite database rather than Postgres (for simplicity and ease of installation)
3. A configuration file where you can specify your ground-truth compilation and your developer compilation.
   Some of the values in this configuration file are not yet implemented to be used.
4. An included makefile to help you specify how to build your source files with the tests
5. Turn FLiT into a shared library to be linked

First official release

This release provides litmus tests, a reproducibility test framework, and analysis tools for the results of the tests.