[RFC]: Enhancing Benchmark Visualization with TAP Parser and Plot Frontend

[Jai0401]

Description

As part of enhancing benchmark visualization capabilities, I propose implementing a Test Anything Protocol (TAP) parser and a plot frontend for consuming parsed TAP results. This feature will allow users to visualize and compare benchmark results more effectively, enhancing the overall usability and utility of the benchmarking tool.

Related Issues

Related issues: stdlib-js/google-summer-of-code#6

Questions

No.

Other

No.

Checklist

• I have read and understood the Code of Conduct.
• Searched for existing issues and pull requests.
• The issue name begins with RFC:.

[Jai0401]

For reference, here is the link to the latest TAP 13 specifications documentation: TAP 13 Specifications

[stdlib-bot]

👋 Hi there! 👋

And thank you for opening your first issue! We will get back to you shortly. 🏃 💨

[Jai0401]

Regarding the visualization aspect for our project, I'd like to discuss the approach we should take. Are we more inclined towards:
Browser-Based: Optimal for real-time interaction, dynamic updates, and detailed visualizations but requires browser capabilities and may introduce dependencies.

Command Line Output: Ideal for simplicity, fast feedback, and lightweight visualizations without interactivity, suitable for quick insights and comparisons without cluttering the workspace.
for example:
Namespace: Math
+-------------------+
| ██████████████ |
| ██████▒▒▒▒▒▒▒▒ |
+-------------------+

Namespace: Array
+-------------------+
| ▒▒▒▒▒▒▒▒▒▒▒▒▒▒ |
| ██████████████ |
+-------------------+

[kgryte]

@Jai0401 I think both are potentially interesting. I think as a first pass having a unicode/ascii plotter could be nice. We have some existing Unicode plotters in https://github.com/stdlib-js/stdlib/tree/develop/lib/node_modules/%40stdlib/plot (see sparklines and unicode folders). We'd want whatever plotter we do use to be written in-house. And, in fact, having an ASCII plotter has been desired for a while, but has been low priority.

I think longer term we'd also want the ability to visualize benchmark results in a browser/electron plot. Our current plot capabilities are somewhat limited, so, barring a complete refactor, we'd want to consider relying on a 3rd party library, such as observable-plot or vega.

[kgryte]

If you're interested in the ASCII plotter aspect, I'd suggest opening a new issue dedicated to discussing that particular aspect. Benchmarks could be a concrete use case for using such a plotter.