Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Create standard benchmark interface #2231

Merged
merged 5 commits into from
Mar 14, 2016
Merged

Create standard benchmark interface #2231

merged 5 commits into from
Mar 14, 2016

Conversation

lucaswoj
Copy link
Contributor

@lucaswoj lucaswoj commented Mar 8, 2016

This pull request defines a standard benchmark interface.

Having a consistent interface for benchmarks allows us to use the same test runner and, in the future, collect benchmark results in an automated way.

I haven't yet moved the fps benchmark over to this new interface. Once I get some 👀 and 👍 on the spec I'll tackle that.

cc @ansis @mourner @jfirebaugh

@jfirebaugh
Copy link
Contributor

Looks good, I especially like capturing motivation and quality criteria in the README.

Do you think browser-based benchmarks are the way to go? Would we ever want headless / pure node benchmarks?

@lucaswoj
Copy link
Contributor Author

lucaswoj commented Mar 9, 2016

Do you think browser-based benchmarks are the way to go? Would we ever want headless / pure node benchmarks?

We should explore headless and pure node benchmarks (especially #1362). The bench interface was designed to accommodate use in headless and pure node environments. For this PR, sticking to browser environments makes the most sense because it is simple and most representative of real-world usage.

@ansis
Copy link
Contributor

ansis commented Mar 9, 2016

It would be good to document how access tokens can be set and provided

Other than that, this looks good!

@lucaswoj
Copy link
Contributor Author

One remaining task: I just realized that the env variables won't work for the debug page because it isn't run through browserify. I think it's time to combine the debug server and bench server into one.

@lucaswoj lucaswoj force-pushed the bench-interface branch 4 times, most recently from ce4796b to 6d504f1 Compare March 10, 2016 21:35
@lucaswoj
Copy link
Contributor Author

Alrightey. This is ready to 🚢. Since the last round of 👀 I

  • combined the debug server and benchmark server into one
  • allowed the use of environment access tokens for the debug server (and documented that fact)
  • moved debug urls from /debug to /
  • moved the existing fps benchmark over to the new interface

@lucaswoj
Copy link
Contributor Author

fixes #2007

@mourner
Copy link
Member

mourner commented Mar 11, 2016

Still needs a few minor ESLint fixes.

@lucaswoj
Copy link
Contributor Author

Oops. I could've sworn I had fixed and pushed those.

👀?

@lucaswoj
Copy link
Contributor Author

fixes #954

@lucaswoj lucaswoj merged commit bbcc6bd into master Mar 14, 2016
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants