Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[FEATURE Beta] Benchmarks #3903

Closed
wants to merge 1 commit into from
Closed

[FEATURE Beta] Benchmarks #3903

wants to merge 1 commit into from

Conversation

twokul
Copy link
Member

@twokul twokul commented Dec 10, 2013

This PR introduces benchmarks for Ember.js. Originally, it came out of Composable CP PR.

Benchmarks are powered by ES6 modules, grunt and benchmark.js.

TODOs:

  • Benchmarks run in the browser
  • Benchmarks run in the command line
  • Add an indicator of overall perf difference (percentage)
  • Grunt should pick up different version of Ember.js automatically
    (right now benchmarks run against only two versions of Ember.js)
  • Output results of the benchmarks
  • Add README
  • Add more benchmark tests (real life scenarios)
  • Leverage ember-performance tests
  • Use bower for dependency management

/cc @jdalton @stefanpenner @hjdivad @kselden

@twokul
Copy link
Member Author

twokul commented Dec 10, 2013

ember benchmarks 2013-12-10 13-21-20

@jdalton
Copy link

jdalton commented Dec 10, 2013

Wow! It would be great to have an indicator of overall perf difference (percentage). I do this with smth like this. To get a handle on the overage perf increase/decrease as ops/sec can be hard read into:

ss

@twokul
Copy link
Member Author

twokul commented Dec 10, 2013

@jdalton good catch, adding to the list

@rwjblue
Copy link
Member

rwjblue commented Dec 10, 2013

@twokul - Great job on this, it looks good. I'll try to check it out in more detail later, but here are some initial observations:

  • The Handlebars source is included twice (benchmarks/lib/dependencies/hbs.js and vendor/hbs.js).
  • Could we just use the CDN/S3 versions of jQuery and Handlebars? This way they don't need to be included in the repo.
  • It would be nice to be able to use the current build system to load the comparison version. You could use the direct SHA targets (like http://builds.emberjs.com.s3.amazonaws.com/canary/shas/43354100e92a386bcc6d44291b13ebcbb060fe6d/ember.js) or just allow testing (via simple drop-down) against the three channels (release, beta, and canary). That would allow us to compare previous versions without having to build manually each time.

This PR introduces benchmarks for Ember.js. Originally, it came out of
[Composable CP PR](#3696).

Benchmarks are powered by ES6 modules, grunt and
[benchmark.js](http://benchmarkjs.com/).

TODOs:
- [x] Benchmarks run in the browser
- [ ] Benchmarks run in the command line
- [x] Grunt should pick up different version of Ember.js automatically
  (right now benchmarks run against two versions of Ember.js)
- [ ] Output results of the benchmarks
- [x] Add README
- [ ] Add more benchmark tests (real life scenarios)
- [ ] Leverage [ember-performance](https://github.com/eviltrout/ember-performance) tests
- [ ] Use [bower](http://bower.io/) for dependency management
@mixonic
Copy link
Member

mixonic commented Dec 11, 2013

I am curious what others have to say, but I really like being able to run the test suite offline. Fwiw.

@rwjblue
Copy link
Member

rwjblue commented Dec 11, 2013

@mixonic - Yeah, @twokul and I talked a bit more about it, and came to that same conclusion. My only objection is storing the actual assets in the repo, but that is easily solved by either bower or NPM. I do think that having a simple way to pull the comparisons from the published builds makes a lot of sense, but it should still be possible to generate your own comparison locally....

@mixonic
Copy link
Member

mixonic commented Dec 11, 2013

👍 👍 you guys are great.

@hjdivad
Copy link
Member

hjdivad commented Dec 11, 2013

Really nice stuff. The awesome followup will be ember-dev building a graph of operation perf / time (commit).

@rwjblue
Copy link
Member

rwjblue commented Dec 11, 2013

@hjdivad - Definitely on the TODO list once this is finished…

benchmarks = {};

window.addEventListener('message', function(event) {
if (event.origin === 'http://localhost:8000') {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this callback should be wrapped in a runloop.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

good catch

libNames: Ember.A(),
name: null,
duration: 0,
sortedByHerz: function() {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

typo: Herz -> Hertz

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could shorten to sortedByHz :D

@stefanpenner
Copy link
Member

This is not a beta feature. We should stop sneaking everything into beta, this should really just enter on canary and stay for the appropriate cycles.

@wagenet
Copy link
Member

wagenet commented Apr 15, 2014

@twokul can you rebase this and retag it as just [FEATURE]?

@wagenet
Copy link
Member

wagenet commented Jul 28, 2014

This appears abandoned, closing.

@wagenet wagenet closed this Jul 28, 2014
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

8 participants