feat(gatsby): Adapters (#38232)

* initial - cache restoration * snapshot * cache.restore() return comment * redirect status * note about named wildcard paths * don't pass named wildcards to function manifest * fix status code resolution for redirects * scaffold initial gatsby-adapter-netlify package and use if for dev purposes (hardcoded for now) * build adapter package as ESM and load it as such, so we can use ESM-only utils - like @netlify/cache-utils * no require in esm world * add redirect headers * start scaffolding autoinstallation of adapters * webpack assets + unmanaged assets (start) * static queries, app-data.json, minor refactors and initial setup for having routes sorted by specificity * move adapter version to 1.0.0 * generalize get-latest-apis for adapters * handle JS files in get-latest-gatsby-files * set peerDep * use other testing pkg * get installation, discovery, re-using working * update versions * move adapter init into its own file * add version checking * adjust comment * move stuff around * feat: add headers to gatsby-config * misc stuff * initial engine lambda * start headers utils * update deps * rewrite util * linting * convert to obj args * remove todo comment * add requiredFiles to functions manifest * make headers default to [] * move constants to own file * export rankRoute * delete unneeded util * createHeadersMatcher initial impl * use createHeadersMatcher * fix types * add http status code type * improve createHeadersMatcher and add tests * move adapterManager init to initialize func * adjust func args to move reporter and allow adapter options * add "adapter" option to gatsby-config * put netlify adapter first in the list - it will only match when env var is set, so won't change default behavior of using testing one * kebabcase function name as function id * export FunctionDefinition type * req.path -> req.url * initial functions wrapping/bundling in gatsby-adapter-netlify * remove netlify adapter from gatsby deps, add gatsby as devDep to adapter (to access types) * fix bundling function files containing [ ] * unify tsconfig for adapter * add joi testing for adapter setting * typescript: make bootstrap work again * generate redirect/rewrite rules, generate 2 variants of ssr-engine (odb and regular) * move routes manifest handling into its own module * generate _headers rules * add sorting to routesManifest * adjust graphql-engine bundling to not leave unreasolvable imports * ssr lambda handling when it executes in read-only dir (use tmpdir() then) * inject functions matchPath into function bundle and generate req.params inside of it * serve api from path prefixed path as well * add path prefix stripping in function wrapper * add cache store and restore in gatsby-adapter-netlify * adjust internal 'lambda' name to 'function' * format lambda-handler * misc changes * missing rename * compile gatsby-adapter-netlify to cjs * add generator field * use netlify adpter when NETLIFY or NETLIFY_LOCAL env var is defined * use headers from config for ssg/dsg * allow specyfing different lmdb binary than current process, use abi83 if adapters are used (it works on node14, 16 and 18) * get-route-path tests * manager refactoring + typo fix * initialize adapters e2e test * cypress: remove viewPort configs * use next preid for netlify adapter * remove test adapter from adapters manifest * don't log errors when testing for user installed adapters * update adapters manifest * cleanup gatsby-adapter-netlify a bit, add more public adapter related types to gatsby * e2e: update gitignore * resolve lmdb binary from lmdb package and not hardcode the forced path * gatsby-plugin-image add downlevelIteration * fix persisted redux keys * update snapshot and mocks * fix: only run adapters during gatsby build * resolve netlify functions runtime deps from adapter context * improve public typings * adding adapter to e2e test so that dev-cli copies stuff over * update babel-preset-gatsby-package dep * e2e: test functions and assets * e2e: client-only WIP * e2e: improve basics * e2e: improve client-only * e2e: redirects * merge _headers and _redirects instead of overwriting it * apply trailing slash option + pass through trailingSlash & pathPrefix * add unit tests for manager * improve manager tests * update types * improve e2e tests * add excludeDatastoreFromEngineFunction flow * normalize path after globbing * mock shouldBundleDatastore * rename adapter config types to be less confusing with gatsby-config * keep same obfuscated path between builds * normalize more paths * support custom 404/500 page for serverless functions * update snapshot * generate relative imports in function * skip trying to copy data to tmp if we are downloading from cdn * improve TS types * mock uuid * put requiredFiles in correct place heh * typo * snapshot * handle GATSBY_EXCLUDE_DATASTORE_FROM_BUNDLE env var * improve README & update types * code block * handle partytown routes * handle slices (html and slice-data) * handle chunk-map and webpack.stats * feat: add name to functionsManifest & displayName to Netlify * update snapshot * rename headers constants * handle image-cdn and file-cdn * update routesManifest test fixture and snapshot * don't log unmanaged static assets anymore * handle some TODOs * add 'adapters' to feature list * tmp: make peer dependency allow to use with canaries * feat: add 'supports' to config to let adapters provide some capabilities and potentially fail builds with clear explanation instead of producing faulty deploy * adjust some text * apply trailingSlash to tests and other stuff, add utils * readme and package.json update * fix ts versions * enable typecheck for adapters.js * don't try to use install adapter if no version matches, install version specified in adapters.js * allow adapter to disable prior deployment plugins * disable gatsby-plugin-netlify when using gatsby-adapter-netlify * fix ts * handle non-alpha-numerc paths * verbose log adapter package that is being installed * only try to restore cache if payload provided * memoize cache utils * log adapter version * handle body parsing in produced api functions * properly handle cases when body parsing already happened and when there is no function handler export * handle body parsing config in generated function * drop unused * remove redirects created by previous deployment plugins * properly handle default body for status responses in api functions * put some dev logs as verbose to limit regular terminal output * put some dev logs as verbose to limit regular terminal output vol2 * maybe deflake function tests * maybe precompile api functions in develop function tests --------- Co-authored-by: Michal Piechowiak <misiek.piechowiak@gmail.com>
gatsbyjs · Jul 24, 2023 · b2d4aef · b2d4aef
1 parent 7a2778b
commit b2d4aef
Show file tree

Hide file tree

Showing 138 changed files with 5,816 additions and 284 deletions.
diff --git a/.circleci/config.yml b/.circleci/config.yml
@@ -444,6 +444,15 @@ jobs:
       - store_test_results:
           path: e2e-tests/trailing-slash/cypress/results
 
+  e2e_tests_adapters:
+    <<: *e2e-executor
+    steps:
+      - run: echo 'export CYPRESS_RECORD_KEY="${CY_CLOUD_ADAPTERS}"' >> "$BASH_ENV"
+      - e2e-test:
+          test_path: e2e-tests/adapters
+      - store_test_results:
+          path: e2e-tests/adapters/cypress/results
+
   starters_validate:
     executor: node
     steps:
@@ -594,6 +603,8 @@ workflows:
           <<: *e2e-test-workflow
       - e2e_tests_trailing-slash:
           <<: *e2e-test-workflow
+      - e2e_tests_adapters:
+          <<: *e2e-test-workflow
       - e2e_tests_development_runtime_with_react_18:
           <<: *e2e-test-workflow
       - e2e_tests_production_runtime_with_react_18:

diff --git a/e2e-tests/adapters/.gitignore b/e2e-tests/adapters/.gitignore
@@ -0,0 +1,13 @@
+node_modules/
+.cache/
+public
+
+# Local Netlify folder
+.netlify
+
+# Cypress output
+cypress/videos/
+cypress/screenshots/
+
+# Custom .yarnrc file for gatsby-dev on Yarn 3
+.yarnrc.yml
diff --git a/e2e-tests/adapters/README.md b/e2e-tests/adapters/README.md
@@ -0,0 +1,27 @@
+# adapters
+
+E2E testing suite for Gatsby's [adapters](http://www.gatsbyjs.com/docs/how-to/previews-deploys-hosting/adapters/) feature.
+If possible, run the tests locally with a CLI. Otherwise deploy the site to the target platform and run Cypress on the deployed URL.
+
+Adapters being tested:
+
+- [gatsby-adapter-netlify](https://github.com/gatsbyjs/gatsby/tree/master/packages/gatsby-adapter-netlify)
+
+## Usage
+
+- To run all tests, use `npm run test`
+- To run individual tests, use `npm run test:%NAME` where `test:%NAME` is the script, e.g. `npm run test:netlify`
+
+If you want to open Cypress locally as a UI, you can run the `:debug` scripts. For example, `npm run test:netlify:debug` to test the Netlify Adapter with Cypress open.
+
+### Adding a new adapter
+
+- Add a new Cypress config inside `cypress/configs`
+- Add a new `test:` script that should run `start-server-and-test`. You can check what e.g. `test:netlify` is doing.
+- Run the Cypress test suites that should work. If you want to exclude a spec, you can use Cypress' [excludeSpecPattern](https://docs.cypress.io/guides/references/configuration#excludeSpecPattern)
+
+## External adapters
+
+As mentioned in [Creating an Adapter](https://gatsbyjs.com/docs/how-to/previews-deploys-hosting/creating-an-adapter/#testing) you can use this test suite for your own adapter.
+
+Copy the whole `adapters` folder, and follow [adding a new adapter](#adding-a-new-adapter).
diff --git a/e2e-tests/adapters/constants.ts b/e2e-tests/adapters/constants.ts
@@ -0,0 +1,2 @@
+export const title = "Adapters"
+export const siteDescription = "End-to-End tests for Gatsby Adapters"
diff --git a/e2e-tests/adapters/cypress.config.ts b/e2e-tests/adapters/cypress.config.ts
@@ -0,0 +1,11 @@
+import { defineConfig } from "cypress"
+
+export default defineConfig({
+  e2e: {
+    baseUrl: `http://localhost:9000`,
+    projectId: `4enh4m`,
+    videoUploadOnPasses: false,
+    experimentalRunAllSpecs: true,
+    retries: 2,
+  },
+})
diff --git a/e2e-tests/adapters/cypress/configs/netlify.ts b/e2e-tests/adapters/cypress/configs/netlify.ts
@@ -0,0 +1,13 @@
+import { defineConfig } from "cypress"
+
+export default defineConfig({
+  e2e: {
+    baseUrl: `http://localhost:8888`,
+    // Netlify doesn't handle trailing slash behaviors really, so no use in testing it
+    excludeSpecPattern: [`cypress/e2e/trailing-slash.cy.ts`,],
+    projectId: `4enh4m`,
+    videoUploadOnPasses: false,
+    experimentalRunAllSpecs: true,
+    retries: 2,
+  },
+})
diff --git a/e2e-tests/adapters/cypress/e2e/basics.cy.ts b/e2e-tests/adapters/cypress/e2e/basics.cy.ts
@@ -0,0 +1,44 @@
+import { title } from "../../constants"
+
+describe('Basics', () => {
+  beforeEach(() => {
+    cy.intercept("/gatsby-icon.png").as("static-folder-image")
+    cy.intercept("/static/astro-**.png").as("img-import")
+
+    cy.visit('/').waitForRouteChange()
+  })
+
+  it('should display index page', () => {
+    cy.get('h1').should('have.text', title)
+    cy.title().should('eq', 'Adapters E2E')
+  })
+  // If this test fails, run "gatsby build" and retry
+  it('should serve assets from "static" folder', () => {
+    cy.wait("@static-folder-image").should(req => {
+      expect(req.response.statusCode).to.be.gte(200).and.lt(400)
+    })
+
+    cy.get('[alt="Gatsby Monogram Logo"]').should('be.visible')
+  })
+  it('should serve assets imported through webpack', () => {
+    cy.wait("@img-import").should(req => {
+      expect(req.response.statusCode).to.be.gte(200).and.lt(400)
+    })
+
+    cy.get('[alt="Gatsby Astronaut"]').should('be.visible')
+  })
+  it(`should show custom 404 page on invalid URL`, () => {
+    cy.visit(`/non-existent-page`, {
+      failOnStatusCode: false,
+    })
+
+    cy.get('h1').should('have.text', 'Page not found')
+  })
+  it('should apply CSS', () => {
+    cy.get(`h1`).should(
+      `have.css`,
+      `color`,
+      `rgb(21, 21, 22)`
+    )
+  })
+})
diff --git a/e2e-tests/adapters/cypress/e2e/client-only.cy.ts b/e2e-tests/adapters/cypress/e2e/client-only.cy.ts
@@ -0,0 +1,89 @@
+Cypress.on('uncaught:exception', (err) => {
+  if (err.message.includes('Minified React error')) {
+    return false
+  }
+})
+
+describe('Sub-Router', () => {
+  const routes = [
+    {
+      path: "/routes/sub-router",
+      marker: "index",
+      label: "Index route"
+    },
+    {
+      path: `/routes/sub-router/page/profile`,
+      marker: `profile`,
+      label: `Dynamic route`,
+    },
+    {
+      path: `/routes/sub-router/not-found`,
+      marker: `NotFound`,
+      label: `Default route (not found)`,
+    },
+    {
+      path: `/routes/sub-router/nested`,
+      marker: `nested-page/index`,
+      label: `Index route inside nested router`,
+    },
+    {
+      path: `/routes/sub-router/nested/foo`,
+      marker: `nested-page/foo`,
+      label: `Dynamic route inside nested router`,
+    },
+    {
+      path: `/routes/sub-router/static`,
+      marker: `static-sibling`,
+      label: `Static route that is a sibling to client only path`,
+    },
+  ] as const
+
+  routes.forEach(({ path, marker, label }) => {
+    it(label, () => {
+      cy.visit(path).waitForRouteChange()
+      cy.get(`[data-testid="dom-marker"]`).contains(marker)
+
+      cy.url().should(
+        `match`,
+        new RegExp(`^${Cypress.config().baseUrl + path}/?$`)
+      )
+    })
+  })
+})
+
+describe('Paths', () => {
+  const routes = [
+    {
+      name: 'client-only',
+      param: 'dune',
+    },
+    {
+      name: 'client-only/wildcard',
+      param: 'atreides/harkonnen',
+    },
+    {
+      name: 'client-only/named-wildcard',
+      param: 'corinno/fenring',
+    },
+  ] as const
+
+  for (const route of routes) {
+    it(`should return "${route.name}" result`, () => {
+      cy.visit(`/routes/${route.name}${route.param ? `/${route.param}` : ''}`).waitForRouteChange()
+      cy.get("[data-testid=title]").should("have.text", route.name)
+      cy.get("[data-testid=params]").should("have.text", route.param)
+    })
+  }
+})
+
+describe('Prioritize', () => {
+  it('should prioritize static page over matchPath page with wildcard', () => {
+    cy.visit('/routes/client-only/prioritize').waitForRouteChange()
+    cy.get("[data-testid=title]").should("have.text", "client-only/prioritize static")
+  })
+  it('should return result for wildcard on nested prioritized path', () => {
+    cy.visit('/routes/client-only/prioritize/nested').waitForRouteChange()
+    cy.get("[data-testid=title]").should("have.text", "client-only/prioritize matchpath")
+    cy.get("[data-testid=params]").should("have.text", "nested")
+  })
+})
diff --git a/e2e-tests/adapters/cypress/e2e/dsg.cy.ts b/e2e-tests/adapters/cypress/e2e/dsg.cy.ts
@@ -0,0 +1,14 @@
+import { title } from "../../constants"
+
+describe("Deferred Static Generation (DSG)", () => {
+  it("should work correctly", () => {
+    cy.visit("/routes/dsg/static").waitForRouteChange()
+
+    cy.get("h1").contains("DSG")
+  })
+  it("should work with page queries", () => {
+    cy.visit("/routes/dsg/graphql-query").waitForRouteChange()
+
+    cy.get(`[data-testid="title"]`).should("have.text", title)
+  })
+})
diff --git a/e2e-tests/adapters/cypress/e2e/functions.cy.ts b/e2e-tests/adapters/cypress/e2e/functions.cy.ts
@@ -0,0 +1,27 @@
+const routes = [
+  {
+    name: 'static',
+    param: '',
+  },
+  {
+    name: 'param',
+    param: 'dune',
+  },
+  {
+    name: 'wildcard',
+    param: 'atreides/harkonnen'
+  },
+  {
+    name: 'named-wildcard',
+    param: 'corinno/fenring'
+  }
+] as const
+
+describe('Functions', () => {
+  for (const route of routes) {
+    it(`should return "${route.name}" result`, () => {
+      cy.request(`/api/${route.name}${route.param ? `/${route.param}` : ''}`).as(`req-${route.name}`)
+      cy.get(`@req-${route.name}`).its('body').should('contain', `Hello World${route.param ? ` from ${route.param}` : ``}`)
+    })
+  }
+})
diff --git a/e2e-tests/adapters/cypress/e2e/redirects.cy.ts b/e2e-tests/adapters/cypress/e2e/redirects.cy.ts
@@ -0,0 +1,57 @@
+import { applyTrailingSlashOption } from "../../utils"
+
+Cypress.on("uncaught:exception", (err) => {
+  if (err.message.includes("Minified React error")) {
+    return false
+  }
+})
+
+const TRAILING_SLASH = Cypress.env(`TRAILING_SLASH`) || `never`
+
+// Those tests won't work using `gatsby serve` because it doesn't support redirects
+
+describe("Redirects", () => {
+  it("should redirect from non-existing page to existing", () => {
+    cy.visit(applyTrailingSlashOption(`/redirect`, TRAILING_SLASH), {
+      failOnStatusCode: false,
+    }).waitForRouteChange()
+      .assertRoute(applyTrailingSlashOption(`/routes/redirect/hit`, TRAILING_SLASH))
+
+    cy.get(`h1`).should(`have.text`, `Hit`)
+  })
+  it("should respect that pages take precedence over redirects", () => {
+    cy.visit(applyTrailingSlashOption(`/routes/redirect/existing`, TRAILING_SLASH), {
+      failOnStatusCode: false,
+    }).waitForRouteChange()
+      .assertRoute(applyTrailingSlashOption(`/routes/redirect/existing`, TRAILING_SLASH))
+
+    cy.get(`h1`).should(`have.text`, `Existing`)
+  })
+  it("should support hash parameter on direct visit", () => {
+    cy.visit(applyTrailingSlashOption(`/redirect`, TRAILING_SLASH) + `#anchor`, {
+      failOnStatusCode: false,
+    }).waitForRouteChange()
+
+    cy.location(`pathname`).should(`equal`, applyTrailingSlashOption(`/routes/redirect/hit`, TRAILING_SLASH))
+    cy.location(`hash`).should(`equal`, `#anchor`)
+    cy.location(`search`).should(`equal`, ``)
+  })
+  it("should support search parameter on direct visit", () => {
+    cy.visit(applyTrailingSlashOption(`/redirect`, TRAILING_SLASH) + `?query_param=hello`, {
+      failOnStatusCode: false,
+    }).waitForRouteChange()
+
+    cy.location(`pathname`).should(`equal`, applyTrailingSlashOption(`/routes/redirect/hit`, TRAILING_SLASH))
+    cy.location(`hash`).should(`equal`, ``)
+    cy.location(`search`).should(`equal`, `?query_param=hello`)
+  })
+  it("should support search & hash parameter on direct visit", () => {
+    cy.visit(applyTrailingSlashOption(`/redirect`, TRAILING_SLASH) + `?query_param=hello#anchor`, {
+      failOnStatusCode: false,
+    }).waitForRouteChange()
+
+    cy.location(`pathname`).should(`equal`, applyTrailingSlashOption(`/routes/redirect/hit`, TRAILING_SLASH))
+    cy.location(`hash`).should(`equal`, `#anchor`)
+    cy.location(`search`).should(`equal`, `?query_param=hello`)
+  })
+})
diff --git a/e2e-tests/adapters/cypress/e2e/slices.cy.ts b/e2e-tests/adapters/cypress/e2e/slices.cy.ts
@@ -0,0 +1,9 @@
+import { siteDescription } from "../../constants"
+
+describe("Slices", () => {
+  it("should work correctly", () => {
+    cy.visit('/').waitForRouteChange()
+
+    cy.get(`footer`).should("have.text", siteDescription)
+  })
+})
diff --git a/e2e-tests/adapters/cypress/e2e/ssr.cy.ts b/e2e-tests/adapters/cypress/e2e/ssr.cy.ts
@@ -0,0 +1,39 @@
+const staticPath = "/routes/ssr/static"
+const paramPath = "/routes/ssr/param"
+
+describe("Server Side Rendering (SSR)", () => {
+  it(`direct visit no query params (${staticPath})`, () => {
+    cy.visit(staticPath).waitForRouteChange()
+    cy.get(`[data-testid="query"]`).contains(`{}`)
+    cy.get(`[data-testid="params"]`).contains(`{}`)
+  })
+
+  it(`direct visit with query params (${staticPath})`, () => {
+    cy.visit(staticPath + `?foo=bar`).waitForRouteChange()
+    cy.get(`[data-testid="query"]`).contains(`{"foo":"bar"}`)
+    cy.get(`[data-testid="params"]`).contains(`{}`)
+  })
+
+  it(`direct visit no query params (${paramPath})`, () => {
+    cy.visit(paramPath + `/foo`).waitForRouteChange()
+    cy.get(`[data-testid="query"]`).contains(`{}`)
+    cy.get(`[data-testid="params"]`).contains(`{"param":"foo"}`)
+  })
+
+  it(`direct visit with query params (${paramPath})`, () => {
+    cy.visit(paramPath + `/foo` + `?foo=bar`).waitForRouteChange()
+    cy.get(`[data-testid="query"]`).contains(`{"foo":"bar"}`)
+    cy.get(`[data-testid="params"]`).contains(`{"param":"foo"}`)
+  })
+
+  it(`should display custom 500 page`, () => {
+    const errorPath = `/routes/ssr/error-path`
+
+    cy.visit(errorPath, { failOnStatusCode: false }).waitForRouteChange()
+
+    cy.location(`pathname`)
+      .should(`equal`, errorPath)
+      .get(`h1`)
+      .should(`have.text`, `INTERNAL SERVER ERROR`)
+  })
+})