Benchmark FFTW planning

[chrisbrahms]

Do we need FFTW.PATIENT or is MEASURE or even ESTIMATE good enough?

[chrisbrahms]

So I did some benchmarks for rFFT. Code can be found here.

First, all the data (starting at 1024 samples). For each plot, lower is faster. Colours present different numbers of modes and dashes different planning modes. The number of threads here is set with FFTW.set_num_threads, JULIA_NUM_THREADS is always 4.

I think the first conclusion here is that PATIENT only has a small advantage over MEASURE for a single thread. For more threads, it seems that ESTIMATE and MEASURE get it wrong somehow and PATIENT just reproduces the single-thread performance. So for big parallel but single-threaded simulations on the workstation we can safely switch to MEASURE (I still need to run this benchmark on the workstation though).

Second, a comparison of single- and multi-threaded performance:

For single-mode, multi-threading is useless (as we'd expect). For 12 modes, the 16-threaded version (still using 4 threads for Julia though) can be up to twice as fast, but this is still only useful when we're not running many of these simultaneously.

Thoughts?

[jtravs]

For more threads, it seems that ESTIMATE and MEASURE get it wrong somehow and PATIENT just reproduces the single-thread performance.

Yes, PATIENT switches threads off when it doesn't help: http://www.fftw.org/fftw3_doc/How-Many-Threads-to-Use_003f.html#How-Many-Threads-to-Use_003f

[chrisbrahms]

Ah I misremembered and thought that was the case for MEASURE as well.

[jtravs]

For 12 modes, the 16-threaded version (still using 4 threads for Julia though) can be up to twice as fast, but this is still only useful when we're not running many of these simultaneously.

This is useful quite often, dropping say a 12 h sim to 6 h. But in those cases I expect most of the effort is actually in the nonlinear operator.

[jtravs]

What about free space?

[chrisbrahms]

What about free space?

Next on the list, but needs to run overnight.

[jtravs]

Note that FFTW now does not set threads by default: JuliaMath/FFTW.jl#151