Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(gatsby): open lmdb instances in writeable locations in generated ssr/dsg function #38631

Merged
merged 1 commit into from
Oct 13, 2023

Conversation

pieh
Copy link
Contributor

@pieh pieh commented Oct 12, 2023

Description

This makes caches-lmdb AND gatsby-core-utils getStorage open lmdb cache/storage instances in writeable location (in case CWD in lambda environment is not writeable). The FS rewrites we have there now doesn't apply to LMDB (fs handling of lmdb is not in Node.js land) so we have to explicitly deal with "fs rewrite" when LMDB is concerned. We were already addressing this for main LMDB instance (datastore), but this expands it to cache and storage usage of lmdb as well.

gatsby-core-utils is already using global.__GATSBY.root -

export function getDatabaseDir(): string {
const rootDir = global.__GATSBY?.root ?? process.cwd()
return path.join(rootDir, `.cache`, `data`, `gatsby-core-utils`)
}

This PR adds similar usage to lmdb cache usage, as well as actually sets the root to writeable location in lambda

Documentation

Tests

This is example showing segfault problem caused by trying to open LMDB in non-writeable location - https://6527cb3ca4fc76285774d547--exquisite-pavlova-cc785c.netlify.app/

https://6527d2acfad30e0085a57db2--exquisite-pavlova-cc785c.netlify.app/ this one is using changes from this PR showing it fixing the problem.

Related Issues

Fixes FRA-10
https://github.com/netlify/pillar-support/issues/724

@gatsbot gatsbot bot added the status: triage needed Issue or pull request that need to be triaged and assigned to a reviewer label Oct 12, 2023
Comment on lines +157 to -152
console.log(`Downloaded datastore from CDN`)
}
console.log(`Downloaded datastore from CDN`)
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is unrelated to rest of changes, just noticed this was logged even if LMDB wasn't excluded from the bundle (so when it shouldn't be logged)

@pieh pieh added topic: render-mode Related to Gatsby's different rendering modes and removed status: triage needed Issue or pull request that need to be triaged and assigned to a reviewer labels Oct 12, 2023
@pieh pieh marked this pull request as ready for review October 12, 2023 13:15
Copy link

@MarcL MarcL left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍🏻

@pieh pieh merged commit 3af35ae into master Oct 13, 2023
1 check passed
@pieh pieh deleted the fix/ssr-dsg-lambda-non-writeable-lmdb-location branch October 13, 2023 10:44
@madhums
Copy link

madhums commented Oct 14, 2023

@pieh Can we do a release for this fix? Having this issue in production version of code hosted on netlify

pieh added a commit that referenced this pull request Oct 16, 2023
pieh added a commit that referenced this pull request Oct 17, 2023
…ssr/dsg function (#38631) (#38638)

(cherry picked from commit 3af35ae)

Co-authored-by: Michal Piechowiak <misiek.piechowiak@gmail.com>
@pieh
Copy link
Contributor Author

pieh commented Oct 17, 2023

gatsby@5.12.7 was released with this change

cc @madhums

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
topic: render-mode Related to Gatsby's different rendering modes
Projects
Status: Published
Development

Successfully merging this pull request may close these issues.

3 participants