Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix modsn.us URL shortening script #3212

Merged
merged 7 commits into from
Jan 29, 2021
Merged

Fix modsn.us URL shortening script #3212

merged 7 commits into from
Jan 29, 2021

Conversation

taneliang
Copy link
Member

@taneliang taneliang commented Jan 28, 2021

Context

Implements a fix for #2057 behind a new enableShortUrl feature flag.

NOTE: This PR doesn't actually fix the issue for end users as the flag is disabled. See the TODO below for more info.

This PR reimplements the 6-year-old short_url.php in a Vercel serverless function using TypeScript. short_url.php had been lying unused since around the time we moved to a Dockerized setup (in #1937). The Dockerized setup was abandoned recently (#3098), and gives us the opportunity to deploy this in a serverless manner.

Also lays groundwork for adding more serverless functions to /website, required by #3035.

TODO: because the serverless function imposes a huge delay on URL shortening (it takes 2-3s), we'll want to display the unshortened URL immediately while the shortener loads. We can alternatively make it shorten on demand (i.e. a shorten button) or optionally (i.e. have a toggle). (EDIT: as this requires some UI work and iteration, we'll punt this to a future PR to unblock the urgent-ish #3035)

Implementation

  • TS used instead of PHP for consistency with the rest of NUSMods V3 (NUSMods R #294). short_url.php dates back to V1/2 times.
  • Using TS also allows us to integrate Sentry.
  • Using a serverless function so that we can version and deploy the script with the rest of /website.
  • One downside is that we're now storing a secret in Vercel, so external PRs won't have automatic /website deployments on Vercel until one of us goes in to approve it.

In the future, we could investigate implementing our own URL shortener using Cloudflare Workers, or folding URL shortening into the new GraphQL server (#1726). YOURLS seems overcomplicated for what's essentially just a persisted dictionary.

@vercel
Copy link

vercel bot commented Jan 28, 2021

This pull request is being automatically deployed with Vercel (learn more).
To see the status of your deployments, click below or on the icon next to each commit.

nusmods-website – ./website

🔍 Inspect: https://vercel.com/nusmodifications/nusmods-website/k3tidkwwk
✅ Preview: https://nusmods-website-git-eliang-shorturl.nusmodifications.vercel.app

nusmods-export – ./export

🔍 Inspect: https://vercel.com/nusmodifications/nusmods-export/h6p88dvz7
✅ Preview: https://nusmods-export-git-eliang-shorturl.nusmodifications.vercel.app

@codecov
Copy link

codecov bot commented Jan 28, 2021

Codecov Report

Merging #3212 (fe46b95) into master (886c3c2) will decrease coverage by 0.06%.
The diff coverage is 37.50%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master    #3212      +/-   ##
==========================================
- Coverage   55.44%   55.37%   -0.07%     
==========================================
  Files         254      255       +1     
  Lines        5317     5320       +3     
  Branches     1227     1228       +1     
==========================================
- Hits         2948     2946       -2     
- Misses       2369     2374       +5     
Impacted Files Coverage Δ
website/src/views/timetable/ShareTimetable.tsx 68.42% <28.57%> (-7.95%) ⬇️
website/src/featureFlags.ts 100.00% <100.00%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 886c3c2...fe46b95. Read the comment docs.

@taneliang taneliang marked this pull request as draft January 28, 2021 07:47
@nusmods-deploy-bot
Copy link

nusmods-deploy-bot bot commented Jan 28, 2021

@ZhangYiJiang
Copy link
Member

ZhangYiJiang commented Jan 28, 2021

Curious about this living under /website. It seems logically these serverless functions should live under /serverless at the repo root or something (and we can merge in https://github.com/nusmodifications/serverless-functions/). Does this enable better Vercel deployment?

Also I can't believe I've not noticed the shorturl endpoint is literally just a proxy to modn.us

@taneliang
Copy link
Member Author

Unfortunately no. By default, Vercel expects serverless functions to be placed under an /api directory at the root of the project, and the Vercel nusmods-website project is rooted at /website. I think we could try to configure that, but iirc support for that was a little sketchy, and I'm pretty sure it won't accept functions located outside the project root.

Alternatively, we could create a new nusmods-serverless Vercel project rooted at a new /serverless directory. We'll just need to host it on a different domain, e.g. maybe serverless.nusmods.com or similar. I'm not sure if there's any benefit to doing that though.

Also I can't believe I've not noticed the shorturl endpoint is literally just a proxy to modn.us

Yeah ikr, I realized that when working on Dockerizing it.

It also seems pretty dumb because that script/endpoint is publicly accessible, so it basically allows anyone to shorten a URL with modsn.us 🤷 I'm actually surprised nobody has abused it yet.

@taneliang
Copy link
Member Author

@nusmodifications/nusmods-developers This is ready for review. TL;DR I've fixed but disabled URL shortening as shortening is a bit too slow. But we need to merge this asap to unblock the MPE work (#3035)

@@ -125,6 +126,7 @@
"@material/fab": "0.40.1",
"@material/snackbar": "0.40.1",
"@sentry/browser": "5.30.0",
"@sentry/node": "6.0.2",
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This uses a different major version from our website lol

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes... yes it does 😆 I didn't want the overhead of upgrading packages at this time but looks like v6 doesn't actually change any APIs. I'll bump all @sentry/* before merging.

Copy link
Member

@ZhangYiJiang ZhangYiJiang left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we should try to get the Sentry packages on the same version just to reduce the amount of duplication in node_modules, but otherwise this looks good. Also a nice test run for serverless functions.

* - https://github.com/nusmodifications/nusmods/issues/2057
* - https://github.com/nusmodifications/nusmods/pull/3212
*/
export const enableShortUrl = false;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some way to override this at runtime would be nice, maybe reuse the debug flags code by reading them from query params

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we should actually move all the debug.ts stuff into dynamic feature flags that check query params on module init, which will prevent pages that change the params (e.g. Venues, Module search) from flipping the debug flags. This can also replace useGlobalDebugValue, with the downside of losing the ability to flip flags after our JS loads.

I'll move this into another issue.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@taneliang taneliang merged commit 940058b into master Jan 29, 2021
@taneliang taneliang deleted the eliang/shorturl branch January 29, 2021 05:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants