Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactor website JS and add model comparison tool #1473

Merged
merged 6 commits into from
Oct 31, 2017
Merged

Conversation

ines
Copy link
Member

@ines ines commented Oct 30, 2017

This PR mainly includes two improvements to the website and the interactive features and JavaScript layer. I was a little stubborn about this and decided to do it entirely in Vanilla JS and with very few dependencies. But hey, it's still very lightweight and I made an effort to document it, so it should be easy to maintain (which will mostly be done by "future me" anyways).

Refactor JavaScript

  • Use ES6 modules to split up the individual components into modules, e.g. GithubEmbed, ModelLoader, and import and initialise them in in the templates if needed. This is important, since the spaCy website initialises the JS components with stringified JSON data from the static CMS to avoid duplicate data – for example, the available models and languages. This needs to be populated by the template in _includes/_scripts.jade. It looks a bit hacky, but it's very convenient.

  • On build, a rollup.js is compiled using rollup, which includes a bundle of all modules and makes them available to the window object. The file is included as <script nomodule>, which is only fetched if the browser does not yet support JS modules. All module-related JS is loaded as <script type="module"> and only interpreted if the browser supports modules.

See this blog post for more details on ES6 modules. In general, JS is only supposed to enhance the spaCy website – not to provide crucial information. While interactive components and auto-loaded code/model details from GitHub are nice, they're not a must. In all cases, fallbacks (e.g. links and the most important info) are provided. The site should always be fully functional with JS disabled, so this PR provides some fixes to make this work more smoothly as well.

Add experimental model comparison tool

The tool will be available at /models/comparison and lets the user compare two models by features, accuracy and speed. Model data is fetched from GitHub and presented in a chart (accuracy) and table (feature and accuracy/speed figures). The main goal is to make it easier to show and visualise the differences and trade-offs between models of the same type – for example, en_core_web_lg and en_core_web_sm.

models_comparison

Website build process

This is only relevant for deployment and not for users running the site locally, as harp serve takes care of this and is all you need.

All steps of the build process are also available as short commands via npm run, set in the package.json. There's one main problem with the full build process (which has always existed, but only came up now): when installing the dependencies from the package.json, npm will create a node_models directory within the website directory. There's currently no way to prevent Harp to building this directory (see sintaxi/harp#275) – and no way to prefix it with an underscore, as this won't work with npm (which is by design and makes sense in context). So for now, all dev dependencies (babel-cli, harp, rollup, uglify-js) have to be installed locally via npm install -g. A solution would be to move the site into another subdirectory, but that's quite ugly and inconvenient.

Description

Types of changes

documentation

Checklist

  • I have submitted the spaCy Contributor Agreement.
  • I ran the tests, and all new and existing tests passed.
  • My changes don't require a change to the documentation, or if they do, I've added all required details.

ines added 5 commits October 30, 2017 14:06
rollup.js will be compiled by the rollup package and Babel on build, and will be loaded if a browser doesn't yet support JS modules
ModelLoader will now output "not compatible" if no compatible version of model is found for a spaCy version
User can select two model and their meta is fetched from GitHub. Features, accuracy figures and speed benchmarks are displayed in a table, with an additional chart comparing the accuracy scores if available. Main use case: demonstrating and visualising trade-offs between larger and smaller models of the same type.
@ines ines added docs Documentation and website meta Meta topics, e.g. repo organisation and issue management models Issues related to the statistical models labels Oct 30, 2017
@ines ines merged commit 3c8db3e into develop Oct 31, 2017
@ines ines deleted the refactor-javascript branch October 31, 2017 13:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
docs Documentation and website meta Meta topics, e.g. repo organisation and issue management models Issues related to the statistical models
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant