[JOSS Review]: Performance

[jbytecode]

Due to paper submission in JOSS

There is an item in reviewing list that states

Performance: If there are any performance claims of the software, have they been confirmed? (If there are no claims, please check off this item.)

Since, the paper does not directly include any performance claims, the README.md file includes

...c-lasso has a default algorithm setting that proved to be the fastest in our numerical experiments...

If this claim is current, please add a small micro-benchmark script that reports performances of implemented methods so people can reproduce the results and make comparisons.

[jbytecode]

Note:

It is also mentioned in paper

Note that the run time for this d = 100-dimensional example for a single path computation
is about 0.5 seconds on a standard Laptop.

and this is definitely a performance claim. Please follow the instructions commented above. microbenchmark script is the solution. The standard Laptop is also an ambiguous term. Please specify.

[muellsen]

Thanks for the suggestion! Yes, absolutely. Most of our computational work was actually to test the performance of the different methods. We will add a benchmark and include it, and clarify the run time statements.

[jbytecode]

Dear authors,

This is the last issue. Whenever you finish this work, I will finalize my review.

Thank you for your great effort.

[muellsen]

Update regarding performance benchmarks: We will provide this in the upcoming days, including a comparison to cvx. Will ping you once done!

[muellsen]

We ran a comprehensive run time comparison now... Currently in the process of providing the result in a presentable format (and including it in the paper); sneak peak here: https://github.com/Leo-Simpson/c-lasso/tree/master/benchmark

[jbytecode]

@muellsen does your output underpin the performance claims in the paper?

Will you make a change in the paper? Please let me know, so I can re-compile it in the review page.

[jbytecode]

@muellsen can you please, update your status of progress?

Thank you.

[muellsen]

Happy New Year! Here is a heads-up. We updated the Paper.md and included a section on numerical benchmarks. We also finalized a README in the benchmark folder https://github.com/Leo-Simpson/c-lasso/tree/master/benchmark that shows the micro-benchmarks. We think, now our performance statements in the paper are backed up by these experiments now.

[jbytecode]

seems okay. Thank you. I am now closing this issue.