Migrating to nextjs

[arielsvn]

Migrating create react app to nextjs

yarn add next

and so it begins

Add .next/ to .gitignore.

Initially I thought I would have to change the directory structure, but next.js supports having the pages directory in src/pages vercel/next.js#8451

Styles and Images

Started to get errors about importing .sass files which are used heavily on refine.bio. This is suported with a plugin

• instructions here https://github.com/zeit/next-plugins/tree/master/packages/next-sass#without-css-modules

It was also needed for CSS imports https://github.com/zeit/next-plugins/tree/master/packages/next-css

For the svg files the recommended approach is to use babel-plugin-inline-react-svg and import the SVG as react components. However in refine.bio we have been importing them as images, which is maybe more convenient but less efficient.

After some research, I found next-images which seems to address our specific case.

With that the first error appeared :)

Other projects/demos

• Similar migration Updates to design + next.js implementation nteract/nteract.io#7
• Hackernews with nextjs demo https://github.com/clintonwoo/hackernews-react-graphql/tree/master/src

[arielsvn]

TODO

• Remove packages that are no longer needed

• Minify resources

• Replace react-helpmet with nextjs' Head and add hotjar and social network tags there

• Include google analytics

• Passing environment variables to both the NODE process and the client one. https://nextjs.org/docs/api-reference/next.config.js/environment-variables

• load initial state for redux loadInitialState

• Remove All localStorage usages + windows

• Adapt react-modal

• NoSSR component

  function NoSSR({ children }) {
    if (isServer) return null;
    return children;
  }
  

• Sentry initialization

• Replace usages of fetch with https://www.npmjs.com/package/isomorphic-unfetch

• Custom 404 page https://nextjs.org/docs/advanced-features/custom-error-page

• Remove all flow typings

Nice to have

• VSCode debugging. Recipe here https://github.com/microsoft/vscode-recipes/tree/master/Next-js but it didn't work for me

• Usages of localStorage that might need to be moved into cache

• Find a way to import TweenLite for the compendia page, looks like if some libraries only work in the browser they can be imported dynamically https://nextjs.org/docs/advanced-features/dynamic-import#with-no-ssr

[arielsvn]

Interesting details:

• When nextjs sees that some pages don't have any blocking data, it generates static versions of them https://nextjs.org/docs/advanced-features/automatic-static-optimization to avoid server computations. I noticed this with an indicator that they had in the bottom corner.
  

• It's possible to use modules only in server side rendering, but they have to be imported correctly https://arunoda.me/blog/ssr-and-server-only-modules

[arielsvn]

Deploying to Lambda@Edge https://github.com/danielcondemarin/serverless-next.js/tree/master/packages/serverless-nextjs-component

[arielsvn]

One of my biggest concerns was the number of URLs that google has discovered but not indexed from our sitemap:

Out of the 60,585 that we have there, 37k are being excluded from search results, more than 50%. This is not good, especially because I want to add another million urls with sample pages. That would be the real long tail 🔥

Even with all that, we are getting some traffic from Google which is also our #1 traffic source.

Other advantages

• Real 404 errors, google complained often about soft 404. Also real 301 redirects.

• Code splitting https://codeburst.io/next-js-ssr-vs-create-react-app-csr-7452f71599f6

• Performance comparison https://codeburst.io/next-js-ssr-vs-create-react-app-csr-7452f71599f6

Pain Points

• Routing. It doesn't support state

• Limited control of code splitting?

[arielsvn]

Redux

Using Redux is tricky, I would have loved to remove it by this point but we have a bunch of code that depends on it. Mainly for our search page and storing the current state of the user's datasets.

From the examples, there seem to be two approaches to include it. Both create higher order components, in one there's a <Provider and a store for each page and the other one creates a single store for the entire App.

One disadvantage of the latter approach is that it disables automatic static optimization for nextjs.

[arielsvn]

History/router

We were using react-router initially on the client side. Next.js has plans to support it, but it's not a priority.

---

• replace usages of withRouter https://nextjs.org/docs/api-reference/next/router#userouter

• replace Link with nextjs version
  - links can only have a single children <a>
  - scrolling with links also needs to be modified https://nextjs.org/docs/api-reference/next/link#disable-scrolling-to-the-top-of-the-page
  - dynamic routes need to have both href and as https://nextjs.org/docs/api-reference/next/link#dynamic-routes

• usages of redirect for direct uses of Router https://sergiodxa.com/articles/redirects-in-next-the-good-way/

• Properly handle 301 redirects and 404. Workaround for 404 Redirect to 404 page in getInitialProps()? vercel/next.js#746 (comment)

• Major issue restoring scroll position. Some solutions at Scroll restoration happens too early before the page gets rendered after hitting browser back button vercel/next.js#3303
  Caching getInitialProps results https://levelup.gitconnected.com/6-tips-using-next-js-for-your-next-web-app-e3f056fa46

• Any component inside of /pages is a possible route.

• Passing state as part of navigation. This is not supported Possible to pass state to next/router? vercel/next.js#771

  Posible solution using Router.push(href, as) https://spectrum.chat/next-js/general/add-query-params-to-router-push-same-way-as-link-does-it~105ec690-5734-4ff9-8175-0303af5c04e6?m=MTU0NzEyMDg0MzIwNQ==

  • Signal on experiment page when the previous page was the search
  • Signal on dataset page that user wants to regenerate the current dataset
  • pass user email_address on downloads page
  • Messages on Alert component

• Loading indicator when navigating between pages Placeholder loading app while fetching data vercel/next.js#2985 (comment)

• Router doesn't support hashes <Link /> doesn't work with hashes vercel/next.js#5161 [idea] Router.hash vercel/next.js#5163 (comment)

Notes

• Nextjs incorporating react router Use with React Router 4 vercel/next.js#1632

[arielsvn]

Additional SEO considerations

• Filters in search should be links
• Page items in search should be links

[arielsvn]

CSS Styles

• CRA puts all styles in a single stylesheet while Next.js splits them.

Can global styles only be imported on _app https://nextjs.org/blog/next-9-2#built-in-css-support-for-global-stylesheets RFC: vercel/next.js#8626

Nextjs doesn't work with CSS files in node_modules vercel/next-plugins#70

Started to get this warnings: webpack-contrib/mini-css-extract-plugin#250
facebook/create-react-app#5372 (comment)

They recommend CSS in JS https://nextjs.org/learn/basics/styling-components

Consequently, there are a bunch of practical issues to consider with traditional CSS-file-based styling (especially with SSR), so we suggest avoiding this method when styling for Next.js.
Instead we recommend CSS in JS, which you can use to style individual components rather than importing CSS files.

chunk styles [mini-css-extract-plugin]
Conflicting order between:
 * css ./node_modules/css-loader??ref--6-2!./node_modules/sass-loader/lib/loader.js??ref--6-3!./src/components/Footer/Footer.scss
 * css ./node_modules/css-loader??ref--6-2!./node_modules/sass-loader/lib/loader.js??ref--6-3!./src/components/Button/Button.scss

Also, pages were not rendering correctly when navigating on the client-side:

[arielsvn]

Packages to remove

• react-helmet
• react-scripts

[arielsvn]

Sitemap sample

• Generate sitemap and robots.txt files via function Add an example how to add robots.txt or sitemap.xml vercel/next.js#751 (comment)

<?xml version="1.0" encoding="UTF-8"?>
<sitemapindex xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
    <sitemap>
        <loc>https://www.refine.bio/sitemap-0.xml</loc>
    </sitemap>
    <sitemap>
        <loc>https://www.refine.bio/sitemap-1.xml</loc>
    </sitemap>
</sitemapindex>

<?xml version="1.0" encoding="UTF-8"?>
<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9" 
    xmlns:news="http://www.google.com/schemas/sitemap-news/0.9" 
    xmlns:xhtml="http://www.w3.org/1999/xhtml" 
    xmlns:mobile="http://www.google.com/schemas/sitemap-mobile/1.0" 
    xmlns:image="http://www.google.com/schemas/sitemap-image/1.1" 
    xmlns:video="http://www.google.com/schemas/sitemap-video/1.1">
    <url>
        <loc>https://www.refine.bio/</loc>
        <priority>0.5</priority>
    </url>
    <url>
        <loc>https://www.refine.bio/about</loc>
        <priority>0.5</priority>
    </url>
    <url>
        <loc>https://www.refine.bio/experiments/SRP068364/transcriptional-profiling-through-rna-seq-of-zebrafish-larval-liver-after-exposure-to-biliatresone-a-biliary-toxin</loc>
        <priority>0.8</priority>
    </url>
    <url>
        <loc>https://www.refine.bio/experiments/GSE24528/expression-analysis-of-zebrafish-melanoma-and-skin-from-the-mitf-brafv600e-p53-line</loc>
        <priority>0.8</priority>
    </url>
    <url>
        <loc>https://www.refine.bio/experiments/SRP069839/marker-gene-pathway-discovery-for-polystyrene-particle-toxicity-in-zebrafish-larvae</loc>
        <priority>0.8</priority>
    </url>
</urlset>

[arielsvn]

Tests

Had some issues mocking fetch

Bigger problems with the integration tests. I need to find a way to https://spectrum.chat/next-js/general/end-to-end-integration-test~74921ed4-2632-410f-93c1-8e3e6407dc1b

[arielsvn]

Optimizing build

Chunking strategy changed with vercel/next.js#7631
Announcement here https://nextjs.org/blog/next-9-2#improved-code-splitting-strategy

By default with Nextjs vs with custom config

with custom config

cost of javscript suggest splitting large bundles https://v8.dev/blog/cost-of-javascript-2019

[arielsvn]

Page speed scores before

Landing page

Search page

https://www.refine.bio/search?q=Medulloblastoma

Experiments page

https://www.refine.bio/experiments/GSE116436/drug-induced-change-in-gene-expression-across-nci-60-cell-lines-after-exposure-to-15-anticancer-agents-for-2-6-and-24h

[arielsvn]

Google coverage after deploy