blog/website: choose the new hosting

[iAdramelk]

UPDATE: See summary of options in #992 (comment)

This is a ticket to discuss and compare possible solutions, based on the criteria listed below.

We plan to convert dvc.org to gatsby and merge it with the blog. Right now, dvc.org is hosted on Heroku, and the blog is hosted on Netlify. We need to choose one hosting that will work for both of those services' needs.

What we want:

API endpoints

dvc.org and blog both use some custom API endpoints to fetch and transform data from github and discourse. We also need to be able to cache the results of these requests because they are not very fast and shouldn't be updated more often than once in 15 minutes.

Right now, this is implemented as Node.js server on Heroku with in-memory cache, but we wouldn't necessarily have a server after we migrate to gatsby. Also, our current implementation has a problem - for the first user that will try to access the page until the cache is created, load time can be quite significant (~10s). Ideally, we should perform cache update ourselves with something like cron and always sent cached results for all of our users.

To solve this, we can use Netlify functions, Cloudflare workers, or something else.

Redirects

We have a large list of redirects that we need to support https://github.com/iterative/dvc.org/blob/master/redirects-list.json

New hosting should allow it.

Build time

Our current build time for the blog on Netlify is long, 7m now, and will only get longer after we merge it with the main site. We can speed it up by preserving .cache dir, yarn module cache between builds. A new hosting/build option should allow us to preserve them.

Demo stands

Right now, both Netlify and Hero allow us to automatically created preview stands from github PRs. We want to have this functionality in the future too.

--

Our new hosting solution may be not one server, but a combination of the few different ones, e. g. CircleCI + Cloudflare Workers + Netlify/Now.sh, but it should be able to all of the things listed above.

[shcheklein]

@fabiosantoscode please, take a look. Let's discuss it here.

[fabiosantoscode]

I agree with you, in that the holy grail is to have our API be just a bunch of static data updated by a cron. It's similar to the architecture I left behind at a paper, where events would wake up a lambda function which rendered a react page, and then push the result to S3. This was called the reactive manifesto back in the day.

However in our situation, where DX is key, I don't really want to write and maintain a bunch of scripts to support a per-PR environment and work differently in production. Ideally we don't have any code handling this. The reason is, that both websites are so static that it's not really worth it to even have a load balancer, and a single server with a memcached instance could certainly cut the mustard when you hide it behind cloudflare or another caching CDN.

I've talked with @shcheklein a lot about this a lot yesterday and one thing we touched on a lot of times was running workers at the edge. However I've given it more thought, and I don't think we need this capability if our API is made up of mostly static things. Instead, it can just slowly spread across the CDN and be served from the edge. It's OK even if it takes a few minutes to update. If it's not OK, we can invalidate the cache.

Heroku is pretty flexible. It doesn't have functions at the edge (that I know of), but I don't think we need them. It also has a nice DX.

---

My proposal:

I think we can use just heroku. We put a CDN in front of it for production, I don't think it matters which (cloudflare or cloudfront).

We stay away from edge functions or pushing to the CDN, and for simplicity allow it to fetch from our poor server. If we find that it is not enough, we can start pushing to the edge. Cloudflare allows for this with workers + KV, and cloudfront allows this by having an S3 bucket as an origin and pushing to it.

We will need a memory cache store. This is because we will need to preserve the content and etags we get from the Github API. The reason we store the etags as well is that if we do a request to github with an if-none-match: {etag} header, and the content is the same, we don't use up our rate limit.

There's no reason we can't serve our gatsby files from an express server. Zeit's serve-handler can handle that. And it removes trailing slashes to boot.

  (req, res) => {
    // do redirects if needed and return early

    serve(req, res, serveOptions);
  }

So this way, we can have a local development server (yarn dev runs nodemon server.js) and a production server (yarn start runs node server.js) which are the same server. We can also serve API requests from this server. PR environments will simply run this server without a CDN, and prod would have a CDN.

We will still have the flexibility of setting nice cache headers, which are read both by the browser (and cloudflare as well as your typical run-of-the-mill cache server or CDN). For example, public/static is filled with content-addressible files. We can set expires headers far into the future (or use cache-control: immutable) because they will never change. Static JS/CSS files too, since webpack places an MD4 hash into their filename.

I guess this is not in-scope, but just in case, for dynamic page content like comments, we have a few options:

• Place comments in a cache key and serve them from the API, and render them only in the clientside
• Employ an edge strategy known as edge-side includes, which are instructions to copy the string from another URL (or cloudflare KV) into the place they are found. Basically it's like an iframe, but it copies HTML into it in a dumb way, and the browser just reads it as HTML (tutorial: https://blog.cloudflare.com/edge-side-includes-with-cloudflare-workers/).
• Place comments in an iframe with a known location and just point to it. This means our comments can be dynamically rendered at the server and not expire the cache of the larger document. It also allows for pagination. It's easier to do this if we don't add clientside react to comments, but it's possible to do this using webpack DLLs.

[shcheklein]

@fabiosantoscode thanks, good summary and good points.

what about the CD part if we stick to Heroku - we need a fast way to build and deliver Gatsby to it.

What do you think about deploying to both - Heroku (APIs + cache) and Netlify (static stuff)?

[jorgeorpinel]

Just a few comments from me for now:

On redirects, Netlify seems to have this built in 🙂 https://docs.netlify.com/routing/redirects/

this way, we can have a local development server (yarn dev runs nodemon server.js) and a production server

I think this is pretty important for the docs process (for local sanity checks). Demo stands also help but its faster, easier, and cheaper to run the site locally.

public/static is filled with content-addressible files. We can set expires headers far into the future (or use cache-control: immutable) because they will never change

MD files in public/static/docs/ are the ones that change the most. Very often. (But we don't serve them directly to users atm, the web app "proxies" them via react-markdown rendering).

edge-side includes, which are instructions to copy the string from another URL (or cloudflare KV) into the place they are found

ESI is meant more for personalization I think i.e. if we had user logins. If this is probable in the future road map then I think it's a good option to incorporate now.

---

In general I just have the impression we may be trying to use too many technologies? Could it all be plain Gatsby (whatever that means) or all microservices or all serverless functions + cdn?

[fabiosantoscode]

@shcheklein

what about the CD part if we stick to Heroku - we need a fast way to build and deliver Gatsby to it.

What do you think about deploying to both - Heroku (APIs + cache) and Netlify (static stuff)?

I say we take advantage of their built-in offering. They also have a cache folder where we can cache the gatsby folder and the public folder, such that things like generating thumbnails is done incrementally. Remember that there will always be a build step. Locally (on an intel i7) a build on a hot cache takes 19 seconds.

The real bottleneck will be saving and restoring cache, so the more images we have, the slower the build will be. We can try to tackle this in the future when it becomes a problem. It will be complicated, yet unavoidable.

Deploying to both Netlify and Heroku would kind of kill the development experience of sending a PR and getting a nice link to a temporary environment: we would have to find a way to connect netlify environment to the heroku API. Additionally, netlify is running a node process anyways :)

---

@jorgeorpinel

In general I just have the impression we may be trying to use too many technologies? Could it all be plan Gatsby (whatever that means) or all microservices or all serverless functions + cdn?

You make a good point here. I had another look and found this middleware from gatsby. We can use it to embed gatsby's logic into our express server. This is pretty close to plain gatsby. It could get closer if there was a way to use middleware in production like it's possible in dev. They might not allow middleware in production because they mean for gatsby sites to be statically hosted.

I would like to argue though, that having a single node process with express do everything, and put a CDN on top of it for DDOS protection, speed and edge caching, shouldn't be too many technologies. Even if it's under the hood, this is mostly what we have now (gatsby serve is express, and netlify puts a cache in front of it, it's not a static server). We do need to run a gatsby build before we serve, but when the build is finished that's it.

---

After having written all of this, I think we have a real option to kick the can down the road by continuing to use netlify. Depends on the price of their cdn though.

We would still write an express server to serve gatsby and our APIs, but it wouldn't have any memory cache. It wouldn't be ideal, but I have no reason to believe it wouldn't be webscale.

[shcheklein]

MD files in public/static/docs/ are the ones that change the most. Very often. (But we don't serve them directly to users atm, the web app "proxies" them via react-markdown rendering).

@jorgeorpinel it's not the case with Gatsby - we serve pre-built static HTMLs that include processed MD in them

On redirects, Netlify seems to have this built in 🙂 https://docs.netlify.com/routing/redirects/

@jorgeorpinel I don't think that is flexible enough. I would prefer to keep the redirects logic that we have - including tests, etc.

We would still write an express server to serve gatsby and our APIs, but it wouldn't have any memory cache.

@fabiosantoscode for some reason I had an idea that it's not possible with Netlify - running your own server with in memory cache that serves APIs externally.

---

Heroku alone sounds like a good option (+some cache like CDN like Cloudflare). Obviously with some CD (if Heroku can do it - fine, if not - Gatsby as a business has something?). And we have all the flexibility we need, up to having databases if needed.

This solution should be very simple to deploy, runs locally, has previews, edge caching is done by Cloudlfare ... any real downsides to this? cc @iAdramelk @jorgeorpinel @fabiosantoscode

[iAdramelk]

Sorry for the long answer guys. I think we are overengineering it a little. In the perfect world I prefer not to have our own Express server at all:

1. Gatsby generates bunch of static files that we can just expose to the world with either Netlify or heroku-buildpack-static.
2. We don't need to care about correct headers for the files in the public/static folder. We need it now because we include them by filename now, but it is not a good way to do it anyway. With gatsby I plan to switch to including them with webpack loader for images. This way we can store images in corresponding components, have automatic cache busting with unique filenames and will be able to automatically optimize images at the build time. Same with the images linked inside the md files. This way we can just set inifinite cache time for such images and forget about them.
3. Both Netlify and heroku-buildpack-static allows to set redirects using json-file config in the root directory without the need for the custom servers.

The only problem that we have with static approach is hosting and updating API functions and caching their results. And, for example, Netlify allows us to solve this as well using Netlify Functions. Here is an example of using Netlify Functions to fetch remote API. It's not that different from our current API implementation and can be deployed and updated as a part of our normal deploy process to Netlify.

I'm not sure that going with Netlify is the best option because I'm not sure that we can optimize our build time on Netlify and I'm not sure from the get go how to cache results of such serverless functions between calls, so I'd like to check other options too, like Heroky, now.sh, etc.

But ideally what I would like to have in the result:

1. Static hosting that just exposes our public folder without the need to manually set up the server.
2. Redirets that are set up as a json files in the root without the need for a server.
3. API as a bunch of serverless functions written in the node.js and deployed as a part of our standard deploy process.

For the local development we can either mock these functions or use already deployed one's. I doubt that we will be updating them this often. We can even place them in the other repository and deploy them separately.

[shcheklein]

@iAdramelk

Static hosting that just exposes our public folder without the need to manually set up the server.

What do you mean by manually, what are the benefits you see in not using an express or something else before Gatsby?

We don't need to care about correct headers for the files in the public/static folder.

most likely Cloudflare/Netlify do proper headers already?

Redirects that are set up as a json files in the root without the need for a server.

I doubt the Netlify redirect's config is flexible enough to handle what we need. Probably, heroku's one is the same, but I haven't checked (but of we go all static Heroku does not make much sense anyway)

API as a bunch of serverless functions written in the node.js and deployed as a part of our standard deploy process.

I like, but it feels like it might complicate the workflow, deployment, local experience .. would love to try before we jump into this.

For the local development we can either mock these functions or use already deployed one's. I doubt that we will be updating them this often. We can even place them in the other repository and deploy them separately.

Sounds like a complicated setup to me. Would love to see something like yarn develop locally that can handle everything.