Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Have a CDN with all API responses statically generated #309

Open
wilzbach opened this issue Mar 30, 2018 · 2 comments
Open

Have a CDN with all API responses statically generated #309

wilzbach opened this issue Mar 30, 2018 · 2 comments
Labels

Comments

@wilzbach
Copy link
Member

Similar to the new Netlify preview, we could have a DUB mirror that updates itself daily (Travis cron) and is at the end of the chain and the last resort.

The idea: prefetch all responses with wget/curl and deploy them to a CDN (GitHub Pages, Netlify, ...).
Netlify can handle a few GBs quite fine (the docarchives.dlang.io currently take 3G), but I doubt that it
will be that much. Currently the uncompressed JSON of all packages is 38M
Netlify does support redirects, so the zips don't need to be downloaded.

Downsides/problems:

  • search won't work (ok, because this is last resort, s.t. building always works)
@bausshf
Copy link
Contributor

bausshf commented Apr 5, 2018

You could possibly make search work by generating a static javascript object, then search through that with javascript, instead of using an api call.

Of course that would be additional functionality to add on top.

@MartinNowak
Copy link
Member

I'd rather argue for migrating the registry to properly hosted servers and a simple HA setup instead of complicating the architecture with such a strong requirement (or uglifying it with a hack).
We could possibly funnel through cloudfront instead of using curl/wget, but I have bad experience with long 503s with unreliable backend servers.

In fact I'm preparing migration to a couple of VPS instances atm., let's try the proved approach first.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

3 participants