A Node.js framework for API-consuming web apps that unifies AJAX client-side behavior and graceful degradation. Apps built on webpages.js should work with or without javascript in the browser, which means
- initial page loads are snappy,
- user actions (navigation, form submission) can precede js enhancement, and
- pages are search-engine ready,
all through a single javascript API.
Features include:
- Automatic AJAX for both
<a/>and<form/>actions - Session management and CSRF protection
- Easy integration with dynapack
npm install webpages
First, a brief overview, then the minimal example!
You write page prototypes, which are just javascript objects with the following lifecycle methods:
- read: Get data from APIs, save it in page state.
- recover: Extract data from the DOM.
- render: Render the data to the DOM.
- run: Attach DOM event listeners.
- write: Send user data to APIs.
Then, you write a layout page prototype, whose render method returns an html string
that includes <head>, <link>, etc. Its render method overwrites the render
method of your regular page prototypes for the server.
Finally, you match urls to page prototypes in a routes file, and tie it all together with a webpages() instance. Bundling, AJAX, server-side rendering, and batteries included!
A simple example describes it best, so here is the minimal set of files we need to get going.
routes.js
layout.js
page.js
user-page.js
server.js
The contents of each are printed below; let's start with routes.js
as it should look the most familiar:
var routes = {
// Only one route, b/c we have only one page.
'user': {
path: '/users/<username>',
params: {
username: /^[a-z]+$/
}
}
};
module.exports = routes;Each entry in the route map is a config object for an osh-route.
The file layout.js exports an object that will be merged into all page
prototypes on the server (i.e., layout attributes overwrite page prototype
attributes). The layout should export a render function that returns the full
html string:
var Layout = {
render: function(pages) {
return (
'<!DOCTYPE html>' +
'<head>' +
'<title>' + this.state.title + '</title>' +
'</head>' +
'<body>' +
this.state.body +
this.renderAjax() +
'</body>' +
'</html>'
);
}
};
module.exports = Layout;The file page.js defines the base page prototype and takes care of
rendering (and rerendering) page state to the browser document.
var Page = {
/**
* Render on the browser. Use a DOM renderer
* with diffing, like ReactJS, rather than what is done here.
*/
render: function(pages) {
document.body.innerHTML = this.state.body;
document.title = this.state.title;
}
};
module.exports = Page;The module user-page.js exports a complete page prototype, grabbing data
from an API server using SuperAgent
(which works on the client and server--important!)
and preparing the state for rendering.
var request = require('superagent'); // isomorphic!..mostly
var merge = require('xtend/immutable');
var escape = require('escape-html');
var Page = require('./page');
/**
* Extend our basic rendering page prototype with a read method.
*/
var UserPage = merge(Page, {
/**
* Read data from various APIs. In this case, we
* pull some user data from a fictitious API.
*/
read: function(pages, render) {
var page = this;
request.get('https://api.mysite.com/users/' + this.props.username)
.end(function(err, res) {
var user = res && res.body;
page.setState({
title: page.props.username, // see routes.js
body: '<h1>' + escape(err ? err.message : user && user.fullname) + '</h1>'
});
render();
});
}
});
module.exports = UserPage;Finally, you serve the app using Express, and optionally use the built-in
bundler (based on dynapack) to
enable client-side AJAX rendering and navigation. The following is
server.js:
var express = require('express');
var serveStatic = require('serve-static');
var webpages = require('webpages');
var dynapack = require('dynapack');
var dest = require('vinyl-fs').dest;
var __bundles = __dirname + '/bundles';
var pack = dynapack({prefix: '/js/'});
var app = express();
var pages = webpages({
basedir: __dirname, // all paths are relative to basedir
routes: './routes',
layout: './layout',
scripts: __bundles
});
pages.set('user', './user-page');
pages.entries().pipe(pack).pipe(dest(__bundles));
pack.scripts().pipe(dest(__bundles));
app.use(pages);
app.use('/js', serveStatic(__bundles));
pack.on('end', function() {
app.listen(3333);
});Call this to create a new pages instance on the server; it returns an express middleware function augmented with the following setup methods.
For example:
var webpages = require('webpages');
var pages = webpages({
basedir: __dirname,
routes: './routes',
layout: './layout',
scripts: __dirname + '/bundles'
});- default:
process.cwd() - required: yes
- type:
String
All configuration parameters that are specified as relative paths are assumed relative to this directory.
- required: yes
- type: String
Path to a module that exports an object mapping route ids to
Route config objects.
The path can be relative to basedir set in constructor.
Example routes.js:
module.exports = {
'user': {
path: '/users/<username>',
params: {
username: /^[a-z]+$/
}
},
'article': {
path: '/articles/<articleId>',
params: {
articleId: /^\w+$/
}
}
};Path to a module that exports a page prototype with a render method that returns a string of HTML. This object is merged into all page prototypes for rendering on the server.
Path to the directory that contains one html file for each route/page. Each html file should contain the scripts you want to load for the corresponding page; webpages.js places the contents of this file directly in the rendered page HTML at the base of the body element.
Webpages.js relies on a naming scheme to locate each page's scripts file. For
example, if you registered the route names "home" and "user", then you
should have the files "home.main.html" and "user.main.html" in the scripts
directory.
This behavior allows webpages.js to work almost seamlessly with dynapack and gulp.js.
The following methods are available on a webpages instance:
var pages = webpages(opts);name: The id of a route exported by the routes module.- type: String
- required: yes
page: Path to a module that exports a page prototype. Can be relative to basedir.- type: String
- required: yes
Register page logic with a route. You need to register by way of a module path so that webpages can bundle your javascript.
name: The name of the server function.fn: The server function.- Signature:
fn(opts, done)whereoptsis POJO data anddoneis a callback. Pass error and result as first and second arg, respectively, todone.
- Signature:
Register a server function (or remote procedure) with the webpages instance. A server function is callable within a page's read/write methods and runs on only the server (when called in the browser, an AJAX request handles the function call for you, being careful to send and check a CSRF token for security).
Use server functions when a task needs to be performed privately, like authenticating with an OAuth2-capable API server.
Errors returned to the done callback will have only their err.message property
sent back to the client if the server function call came from the browser.
Within the server function, this has the following properties:
this.session: The current Session instance.
Example:
pages.fn('refreshAccessToken', function(opts, done) {
// For persisting the refresh token.
var session = this.session;
request.post('https://api.api.api/oauth/token')
.auth('thewebs', 'sshh')
.send({
grant_type: 'refresh_token',
refresh_token: session.secrets.refreshToken
})
.end(function(err, res) {
if (err) done(err);
else {
session.setSecrets({refreshToken: res.body.refresh_token});
done(null, res.body.access_token);
}
});
});Returns a readable stream that outputs an "entry" javascript file (in vinyl File format) for each route/page in your app. These files were designed for piping into Dynapack in a gulp.js workflow.
var webpages = require('webpages');
var dynapack = require('dynapack');
var dest = require('vinyl-fs').dest;
var __scripts = __dirname + '/js';
var pages = webpages({
scripts: __scripts
// ...
});
var pack = dynapack();
pages.entries().pipe(pack).pipe(dest(__scripts));
pack.scripts().pipe(dest(__scripts));
pack.on('end', function() {
var app = express();
app.use(pages);
app.listen(8080);
});A Page prototype registered with the pages.set() method should implement the following API. Lifecycle methods are required to do anything useful.
These methods should be defined on a Page prototype.
Called on a GET request for the Page.
Using information in this.props and the given pages object, gather data
from APIs and make calls to this.setState(state) to prepare the page for
rendering. Call the render callback without arguments when ready to render
the page.
The pages object contains the following properties to help with optimization
and managing session state:
pages.session: The current Session instance.pages.current: The currently rendered page. If you are not managing your own caching, use this to migrate state from the old page to the new page without requerying an API. The only properties available are:pages.current.namepages.current.propspages.current.state
It also houses every server function registered with pages.fn().
The render callback doubles as a redirector; passing it either a URI or a
name/props pair will skip rendering of the current page and either send a 302
response (if running on the server) or begin an AJAX GET of the indicated page
(if running in the browser). For example, if there was an error fetching data
from an API, you can redirect to a not-found page via:
module.exports = {
read: function(pages, render) {
var page = this;
var session = pages.session;
var current = pages.current;
if (current && current.props.username === 'beatrix') {
this.setState({
user: current.state.user
});
render();
}
else {
api.getUser('beatrix', function(err, user) {
if (err) {
render('error', {code: 404, msg: err.message});
// Assuming the 'error' route path is simply: '/error', the
// following would be equivalent:
//render('/error?code=404&msg=' + encodeURIComponent(err.message));
}
else {
page.setState({user: user});
render();
}
});
}
}
// ...
};where 'error' is the name of a route.
If stashing was disabled in Page.read(), the state that was not stashed should be recovered from the server-rendered HTML in this method. This is called once per browser session, on initial page load.
For example, an API might return a large chunk of raw HTML. Rather than use the automatic webpages.js stashing and recovery feature, it would be more efficient to read the HTML from the document on initial page load.
For example, given the layout prototype:
var Layout = {
render: function(pages) {
return (
'<div id="post">' + this.state.blogPost + '</div>'
);
}
};Custom state stashing and recovery could be acheived as follows:
var MyPage = {
read: function(pages, render) {
var page = this;
// Important... see this later in the docs. Turns off all
// stashing.
page.stash(false);
request.get('https://api.blog.com/posts/42')
.end(function(res) {
page.setState({
blogPost: res.text
});
render();
});
},
// Called on only the browser.
recover: function(pages) {
this.setState({
blogPost: document.getElementById('post').innerHTML
});
},
run: function(pages) {
// and we have it...
console.log(this.state.blogPost);
}
};This method should exist on all page prototypes that have a read() method
and on the special layout prototype.
In both versions, the pages object has the following
properties:
pages.csrf: Properties required by webpages.js when submitting forms to protect against cross-site request forgeries. The following strings should be set as attributes on a hidden<input>element that appears first in any<form>groups. Each property name matches the<input>attribute name on which it should be set.pages.csrf.name: Field name recognized by osh-pages.pages.csrf.value: The csrf token.
pages.uri(name, props): Get a URI from route name/props pairs for creating links.
The render method on the layout prototype is called on the server for
initial renders. This version should
return the entire page html, including <!DOCTYPE html>, <head>, and
whatnot.
Example:
var escape = require('escape-html');
var Layout = {
// ...
render: function(pages) {
return (
'<!DOCTYPE html>' +
'<head>' +
escape(this.state.title) +
'</head>' +
'<body>' +
this.state.body +
this.renderAjax() +
'</body>'
'</html>'
);
}
};
module.exports = Layout;Update the browser document to show the current page. A very basic implementation (that would defeat the purpose of AJAX navigation) might be:
var MyPage = {
// ...
render: function(pages) {
document.body.innerHTML = this.state.body;
document.title = this.state.title;
}
};A more performant version would find the smallest difference between the currently rendered page and the page to render, and update only those elements of the document that need it. React.js provides automatic DOM diffing and is a good choice here (in fact, this library was built with React rendering in mind); in principle, any DOM diffing/rendering tool would work.
Event handlers should not be attached to the document in this step. Instead they should be attached in the run() lifecycle method, which gets called both on initial page load and after each AJAX render. In general, it is okay to push rendering into the run method (at the risk of re-rendering your initial page), but not okay to push progressive enhancement into the render method.
Note: In the case of a view library like React, which provides rendering and progressive enhancement, simply defer rendering to run() (although setting some DOM, like document.title, might be more appropriate for render()). Up to you.
Attach event handlers to the DOM. Or use a view library (like ReactJS) that handles progressive enhancement, DOM diffing/rendering, and event handling.
Called when the page is POSTed to. This method stands alone; render methods are not called after write, because pages should not be returned from POST requests, only redirects (see this wonderful treatise on the topic). It is possible to have a page prototype that consists only of a write method (do this to create a route that serves only POST requests).
The pages object contains the following properties to help with
managing session state:
pages.session: The current Session instance.
It also houses every server function registered with pages.fn().
Inside the write method, this.payload is used to access the data that was
POSTed from the form. Standard urlencoded forms will result in a simple
this.payload object, where keys are form input names. For example, submission
of the form:
<form>
<input name="greeting" type="text" value="hello"/>
<input type="submit"/>
</form>would result in a payload object (shown as json):
{
"greeting": "hello"
}If the form encoding was multipart/form-data (for
file uploads), then the this.payload object will be a readable stream which
can be piped to a superagent request to some API. If not piping, the payload
can be split up by listening for 'field' events (this.payload is also an
event emitter in this case).
The given redirect method should be called with a name and props object like,
redirect('view-user', {username: 'tory'});or with a uri
redirect('/users/tory');In the following contrived example, the write method is enacting a POST request that will attempt to change the full name of a user:
var Page = module.exports = {
// ...
write: function(session, redirect) {
request.post('https://api.mysite.com/users/' + session.state.username)
.set('x-api-key', session.state.apiKey)
.send({
fullname: this.payload.fullname
})
.end(function(res) {
if (res.ok) {
redirect('view-user', {
username: session.state.username
});
}
else {
redirect('update-user-form', {
// Some error message from the API server:
msg: res.body.message
});
}
});
}
};These methods are used from within the lifecycle methods described above.
Call this in Page.read() and Page.recover() to set downloaded or recovered state on the page instance.
Toggle stashing of state set with setState(). By default, stashing is turned on, so that all the state set in a Page's read lifecycle method is available in the browser on initial page load (without the need for requerying APIs).
Call this in your layout render method to enable AJAX/progressive enhancement. It
includes <script> elements for the javascript bundles generated by Dynapack
and a <span> for transporting state that was stashed in a Page's read method
for reuse in the browser.
This is passed in to the read/write lifecycle methods by way of the pages object. Use it to
set small chunks of data in cookies that are persisted between pages.
There are two types of state that can be set to a session, public and secret. Public
state is available on pages.session.state whereas secret state is only available on
this.session.secrets from within a server function.
Store public data in a browser cookie. This function can be called anywhere (browser
or server) and does not set the http-only flag. This state persis
Store secret session data in a http-only browser cookie. This method is only callable
from within a server function, where the session instance is
found at this.session.
MIT