Infer benchmark data to use from positional arguments

[eriknw]

It is very common to run the same benchmark code with different input. Currently, a separate benchmark is required for each input. I would like to be able to define a single benchmark function, and the prefix of the input data is given by the name of the positional argument:

data_empty = 0
data_small = 10
data_large = 10000

def bench_zeros(data):
    zeros(data)

The results need to be displayed cleanly in the output table. It should also be possible to provide multiple arguments, and all combinations of input should (probably) be benchmarked. I would be happy to begin with only supporting a single positional argument if it makes things easier.

[mrocklin]

Maybe keyword arguments that are lists are assumed to be different cases?

def bench_zeros(data=[0, 10, 10000]):
    zeros(data)  # actually gets called with 0, then 10, then 10000

I think IPython's interact might be a relevant thing to copy here.

[eriknw]

Which IPython interact are you referring to? A quick search reveals several interact occurrences in IPython's API and code.

[mrocklin]

There is in interact function commonly used as a decorator. I guess it's
kinda described here
http://nbviewer.ipython.org/github/ipython/ipython/blob/2.x/examples/Interactive%20Widgets/Index.ipynb
but honestly you can't describe it well in a static link.

I recommend opening the IPython notebook and then running a single cell
with these contents

from IPython.html.widgets import interact

@interact
def foo(name=['Alice', 'Bob', 'Charlie']):
print "Hello, %s!" % name

On Sat, Jun 21, 2014 at 11:38 AM, Erik Welch notifications@github.com
wrote:

Which IPython interact are you referring to? A quick search reveals
several interact occurrences in IPython's API and code.

—
Reply to this email directly or view it on GitHub
#6 (comment).