Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Access Parameter store config locally via Dotenv #4745

Merged
merged 25 commits into from
May 24, 2022
Merged

Conversation

jamesgorrie
Copy link
Contributor

@jamesgorrie jamesgorrie commented Apr 27, 2022

Part of #4823

Generates a .env file from parameter store as part of our make dev command.

gen-dotenv has been added to the build, build-ci and dev targets as it is only needed when running the server and is a file that is part of the build.

I've added a check to the see if we're running NODE_ENV=production to fail if the file cannot be generated, and then set that for the general targets build and build-ci rather than just when running the node script.

  • check env
  • gen env
  • read generated .env

Dev messaging

Screenshot 2022-05-04 at 21 47 37

Co-authored-by: Olly Namey olly.namey@guardian.co.uk
Co-authored-by: Ash Carr ashleigh.carr@guardian.co.uk
Co-authored-by: Ioanna Kokkini ioanna.kokkini@guardian.co.uk

@github-actions
Copy link

github-actions bot commented Apr 27, 2022

Size Change: -944 B (0%)

Total Size: 1.68 MB

Filename Size Change
dotcom-rendering/dist/frontend.server.js 432 kB -944 B (0%)
ℹ️ View Unchanged
Filename Size
dotcom-rendering/dist/259.legacy.********************.js 3.24 kB
dotcom-rendering/dist/2947.********************.js 2.6 kB
dotcom-rendering/dist/2947.legacy.********************.js 2.68 kB
dotcom-rendering/dist/3307.********************.js 3.11 kB
dotcom-rendering/dist/3584.********************.js 1.8 kB
dotcom-rendering/dist/3584.legacy.********************.js 1.8 kB
dotcom-rendering/dist/4019.********************.js 3.44 kB
dotcom-rendering/dist/4019.legacy.********************.js 3.48 kB
dotcom-rendering/dist/4135.legacy.********************.js 6.76 kB
dotcom-rendering/dist/5030.legacy.********************.js 2.76 kB
dotcom-rendering/dist/5248.********************.js 6.62 kB
dotcom-rendering/dist/53.********************.js 4.15 kB
dotcom-rendering/dist/53.legacy.********************.js 4.15 kB
dotcom-rendering/dist/5436.legacy.********************.js 2.68 kB
dotcom-rendering/dist/6166.********************.js 23.3 kB
dotcom-rendering/dist/6166.legacy.********************.js 24 kB
dotcom-rendering/dist/6400.********************.js 21.5 kB
dotcom-rendering/dist/6400.legacy.********************.js 21.5 kB
dotcom-rendering/dist/7148.********************.js 5.97 kB
dotcom-rendering/dist/7148.legacy.********************.js 6.41 kB
dotcom-rendering/dist/7197.********************.js 2.71 kB
dotcom-rendering/dist/7576.********************.js 3.71 kB
dotcom-rendering/dist/7576.legacy.********************.js 4.32 kB
dotcom-rendering/dist/7800.********************.js 11.3 kB
dotcom-rendering/dist/7829.********************.js 2.78 kB
dotcom-rendering/dist/7829.legacy.********************.js 2.89 kB
dotcom-rendering/dist/8129.legacy.********************.js 11.8 kB
dotcom-rendering/dist/8177.legacy.********************.js 30.8 kB
dotcom-rendering/dist/8344.legacy.********************.js 6.37 kB
dotcom-rendering/dist/8612.********************.js 3.72 kB
dotcom-rendering/dist/8612.legacy.********************.js 4.21 kB
dotcom-rendering/dist/8684.********************.js 32.8 kB
dotcom-rendering/dist/9130.********************.js 5.92 kB
dotcom-rendering/dist/9380.legacy.********************.js 2.83 kB
dotcom-rendering/dist/AlreadyVisited-importable.********************.js 4.52 kB
dotcom-rendering/dist/AlreadyVisited-importable.legacy.********************.js 4.52 kB
dotcom-rendering/dist/atomIframe.********************.js 1.76 kB
dotcom-rendering/dist/atomIframe.legacy.********************.js 2.03 kB
dotcom-rendering/dist/AudioAtomWrapper-importable.********************.js 460 B
dotcom-rendering/dist/AudioAtomWrapper-importable.legacy.********************.js 517 B
dotcom-rendering/dist/bootCmp.********************.js 8.01 kB
dotcom-rendering/dist/bootCmp.legacy.********************.js 11.6 kB
dotcom-rendering/dist/Branding-importable.********************.js 3.99 kB
dotcom-rendering/dist/Branding-importable.legacy.********************.js 4 kB
dotcom-rendering/dist/braze-web-sdk-core.********************.js 36.1 kB
dotcom-rendering/dist/braze-web-sdk-core.legacy.********************.js 36.1 kB
dotcom-rendering/dist/BrazeMessaging-importable.********************.js 8.98 kB
dotcom-rendering/dist/BrazeMessaging-importable.legacy.********************.js 9.66 kB
dotcom-rendering/dist/CalloutBlockComponent-importable.********************.js 7.96 kB
dotcom-rendering/dist/CalloutBlockComponent-importable.legacy.********************.js 8.32 kB
dotcom-rendering/dist/ChartAtomWrapper-importable.********************.js 262 B
dotcom-rendering/dist/ChartAtomWrapper-importable.legacy.********************.js 271 B
dotcom-rendering/dist/CommentCount-importable.********************.js 7.13 kB
dotcom-rendering/dist/CommentCount-importable.legacy.********************.js 7.28 kB
dotcom-rendering/dist/CommercialMetrics-importable.********************.js 6.51 kB
dotcom-rendering/dist/CommercialMetrics-importable.legacy.********************.js 7.04 kB
dotcom-rendering/dist/CoreVitals-importable.********************.js 6.06 kB
dotcom-rendering/dist/CoreVitals-importable.legacy.********************.js 6.27 kB
dotcom-rendering/dist/DiscussionContainer-importable.********************.js 7.19 kB
dotcom-rendering/dist/DiscussionContainer-importable.legacy.********************.js 7.49 kB
dotcom-rendering/dist/DiscussionMeta-importable.********************.js 7.55 kB
dotcom-rendering/dist/DiscussionMeta-importable.legacy.********************.js 7.71 kB
dotcom-rendering/dist/DocumentBlockComponent-importable.********************.js 2.98 kB
dotcom-rendering/dist/DocumentBlockComponent-importable.legacy.********************.js 3.1 kB
dotcom-rendering/dist/dynamicImport.********************.js 2.9 kB
dotcom-rendering/dist/dynamicImport.legacy.********************.js 3.18 kB
dotcom-rendering/dist/EditionDropdown-importable.********************.js 3.7 kB
dotcom-rendering/dist/EditionDropdown-importable.legacy.********************.js 3.76 kB
dotcom-rendering/dist/EmbedBlockComponent-importable.********************.js 2.87 kB
dotcom-rendering/dist/EmbedBlockComponent-importable.legacy.********************.js 2.97 kB
dotcom-rendering/dist/embedIframe.********************.js 1.76 kB
dotcom-rendering/dist/embedIframe.legacy.********************.js 2.04 kB
dotcom-rendering/dist/EnhancePinnedPost-importable.********************.js 6.21 kB
dotcom-rendering/dist/EnhancePinnedPost-importable.legacy.********************.js 6.79 kB
dotcom-rendering/dist/FilterKeyEventsToggle-importable.********************.js 3.31 kB
dotcom-rendering/dist/FilterKeyEventsToggle-importable.legacy.********************.js 3.37 kB
dotcom-rendering/dist/FocusStyles-importable.********************.js 4.71 kB
dotcom-rendering/dist/FocusStyles-importable.legacy.********************.js 4.77 kB
dotcom-rendering/dist/ga.********************.js 3.76 kB
dotcom-rendering/dist/ga.legacy.********************.js 4.02 kB
dotcom-rendering/dist/GetCricketScoreboard-importable.********************.js 7.64 kB
dotcom-rendering/dist/GetCricketScoreboard-importable.legacy.********************.js 7.87 kB
dotcom-rendering/dist/GetMatchNav-importable.********************.js 12.8 kB
dotcom-rendering/dist/GetMatchNav-importable.legacy.********************.js 13.1 kB
dotcom-rendering/dist/GetMatchStats-importable.********************.js 9.96 kB
dotcom-rendering/dist/GetMatchStats-importable.legacy.********************.js 10.8 kB
dotcom-rendering/dist/GetMatchTabs-importable.********************.js 6.71 kB
dotcom-rendering/dist/GetMatchTabs-importable.legacy.********************.js 6.93 kB
dotcom-rendering/dist/guardian-braze-components-banner.********************.js 11.3 kB
dotcom-rendering/dist/guardian-braze-components-banner.js 10.3 kB
dotcom-rendering/dist/guardian-braze-components-banner.legacy.********************.js 11.4 kB
dotcom-rendering/dist/guardian-braze-components-end-of-article.********************.js 9.28 kB
dotcom-rendering/dist/guardian-braze-components-end-of-article.js 8.76 kB
dotcom-rendering/dist/guardian-braze-components-end-of-article.legacy.********************.js 9.36 kB
dotcom-rendering/dist/GuideAtomWrapper-importable.********************.js 262 B
dotcom-rendering/dist/GuideAtomWrapper-importable.legacy.********************.js 274 B
dotcom-rendering/dist/initDiscussion.********************.js 10.3 kB
dotcom-rendering/dist/initDiscussion.legacy.********************.js 10.6 kB
dotcom-rendering/dist/InstagramBlockComponent-importable.********************.js 3 kB
dotcom-rendering/dist/InstagramBlockComponent-importable.legacy.********************.js 3.09 kB
dotcom-rendering/dist/InteractiveBlockComponent-importable.********************.js 7.86 kB
dotcom-rendering/dist/InteractiveBlockComponent-importable.legacy.********************.js 8.09 kB
dotcom-rendering/dist/islands.********************.js 10.5 kB
dotcom-rendering/dist/islands.legacy.********************.js 11.3 kB
dotcom-rendering/dist/KnowledgeQuizAtomWrapper-importable.********************.js 268 B
dotcom-rendering/dist/KnowledgeQuizAtomWrapper-importable.legacy.********************.js 282 B
dotcom-rendering/dist/LabsHeader-importable.********************.js 8.29 kB
dotcom-rendering/dist/LabsHeader-importable.legacy.********************.js 8.45 kB
dotcom-rendering/dist/Links-importable.********************.js 5.58 kB
dotcom-rendering/dist/Links-importable.legacy.********************.js 5.66 kB
dotcom-rendering/dist/LiveBlogEpic-importable.********************.js 5.45 kB
dotcom-rendering/dist/LiveBlogEpic-importable.legacy.********************.js 6.12 kB
dotcom-rendering/dist/Liveness-importable.********************.js 5.7 kB
dotcom-rendering/dist/Liveness-importable.legacy.********************.js 5.84 kB
dotcom-rendering/dist/MapEmbedBlockComponent-importable.********************.js 8.72 kB
dotcom-rendering/dist/MapEmbedBlockComponent-importable.legacy.********************.js 9.02 kB
dotcom-rendering/dist/MostViewedFooterData-importable.********************.js 11.1 kB
dotcom-rendering/dist/MostViewedFooterData-importable.legacy.********************.js 11.3 kB
dotcom-rendering/dist/MostViewedRightWrapper-importable.********************.js 10.6 kB
dotcom-rendering/dist/MostViewedRightWrapper-importable.legacy.********************.js 8.54 kB
dotcom-rendering/dist/newsletterEmbedIframe.********************.js 1.92 kB
dotcom-rendering/dist/newsletterEmbedIframe.legacy.********************.js 2.18 kB
dotcom-rendering/dist/OnwardsLower-importable.********************.js 15.8 kB
dotcom-rendering/dist/OnwardsLower-importable.legacy.********************.js 16.2 kB
dotcom-rendering/dist/OnwardsUpper-importable.********************.js 20.5 kB
dotcom-rendering/dist/OnwardsUpper-importable.legacy.********************.js 20.9 kB
dotcom-rendering/dist/ophan.********************.js 7.1 kB
dotcom-rendering/dist/ophan.legacy.********************.js 7.3 kB
dotcom-rendering/dist/PersonalityQuizAtomWrapper-importable.********************.js 270 B
dotcom-rendering/dist/PersonalityQuizAtomWrapper-importable.legacy.********************.js 283 B
dotcom-rendering/dist/ProfileAtomWrapper-importable.********************.js 265 B
dotcom-rendering/dist/ProfileAtomWrapper-importable.legacy.********************.js 276 B
dotcom-rendering/dist/PulsingDot-importable.********************.js 1.66 kB
dotcom-rendering/dist/PulsingDot-importable.legacy.********************.js 1.76 kB
dotcom-rendering/dist/QandaAtomWrapper-importable.********************.js 260 B
dotcom-rendering/dist/QandaAtomWrapper-importable.legacy.********************.js 273 B
dotcom-rendering/dist/ReaderRevenueDev-importable.********************.js 4.58 kB
dotcom-rendering/dist/ReaderRevenueDev-importable.legacy.********************.js 4.6 kB
dotcom-rendering/dist/readerRevenueDevUtils.********************.js 3.13 kB
dotcom-rendering/dist/readerRevenueDevUtils.js 2.31 kB
dotcom-rendering/dist/readerRevenueDevUtils.legacy.********************.js 4.16 kB
dotcom-rendering/dist/ReaderRevenueLinks-importable.********************.js 5.05 kB
dotcom-rendering/dist/ReaderRevenueLinks-importable.legacy.********************.js 5.45 kB
dotcom-rendering/dist/relativeTime.********************.js 2.3 kB
dotcom-rendering/dist/relativeTime.legacy.********************.js 2.57 kB
dotcom-rendering/dist/RichLinkComponent-importable.********************.js 9.31 kB
dotcom-rendering/dist/RichLinkComponent-importable.legacy.********************.js 9.48 kB
dotcom-rendering/dist/sentry.********************.js 684 B
dotcom-rendering/dist/sentry.legacy.********************.js 691 B
dotcom-rendering/dist/sentryLoader.********************.js 9.28 kB
dotcom-rendering/dist/sentryLoader.legacy.********************.js 12.6 kB
dotcom-rendering/dist/SetABTests-importable.********************.js 8.29 kB
dotcom-rendering/dist/SetABTests-importable.legacy.********************.js 8.82 kB
dotcom-rendering/dist/ShareCount-importable.********************.js 7.27 kB
dotcom-rendering/dist/ShareCount-importable.legacy.********************.js 7.44 kB
dotcom-rendering/dist/shimport.********************.js 2.78 kB
dotcom-rendering/dist/shimport.legacy.********************.js 2.79 kB
dotcom-rendering/dist/SignInGateMain.********************.js 4.46 kB
dotcom-rendering/dist/SignInGateMain.js 2.88 kB
dotcom-rendering/dist/SignInGateMain.legacy.********************.js 4.56 kB
dotcom-rendering/dist/SignInGateSelector-importable.********************.js 5.19 kB
dotcom-rendering/dist/SignInGateSelector-importable.legacy.********************.js 5.54 kB
dotcom-rendering/dist/SlotBodyEnd-importable.********************.js 8.99 kB
dotcom-rendering/dist/SlotBodyEnd-importable.legacy.********************.js 10.2 kB
dotcom-rendering/dist/SpotifyBlockComponent-importable.********************.js 8.63 kB
dotcom-rendering/dist/SpotifyBlockComponent-importable.legacy.********************.js 8.94 kB
dotcom-rendering/dist/StickyBottomBanner-importable.********************.js 13.3 kB
dotcom-rendering/dist/StickyBottomBanner-importable.legacy.********************.js 15 kB
dotcom-rendering/dist/SubNav-importable.********************.js 6.89 kB
dotcom-rendering/dist/SubNav-importable.legacy.********************.js 7.06 kB
dotcom-rendering/dist/TimelineAtomWrapper-importable.********************.js 263 B
dotcom-rendering/dist/TimelineAtomWrapper-importable.legacy.********************.js 275 B
dotcom-rendering/dist/TopRightAdSlot-importable.********************.js 2.05 kB
dotcom-rendering/dist/TopRightAdSlot-importable.legacy.********************.js 2.08 kB
dotcom-rendering/dist/TweetBlockComponent-importable.********************.js 1.8 kB
dotcom-rendering/dist/TweetBlockComponent-importable.legacy.********************.js 1.79 kB
dotcom-rendering/dist/UnsafeEmbedBlockComponent-importable.********************.js 2.98 kB
dotcom-rendering/dist/UnsafeEmbedBlockComponent-importable.legacy.********************.js 3.08 kB
dotcom-rendering/dist/VideoFacebookBlockComponent-importable.********************.js 8.73 kB
dotcom-rendering/dist/VideoFacebookBlockComponent-importable.legacy.********************.js 9.02 kB
dotcom-rendering/dist/VineBlockComponent-importable.********************.js 2.8 kB
dotcom-rendering/dist/VineBlockComponent-importable.legacy.********************.js 2.9 kB
dotcom-rendering/dist/YoutubeBlockComponent-importable.********************.js 9.31 kB
dotcom-rendering/dist/YoutubeBlockComponent-importable.legacy.********************.js 9.69 kB

compressed-size-action

@jamesgorrie jamesgorrie changed the title Server side dotenv Access Parameter store config locally via Dotenv May 4, 2022
@jamesgorrie jamesgorrie mentioned this pull request May 4, 2022
3 tasks
@github-actions
Copy link

github-actions bot commented May 4, 2022

⚡️ Lighthouse report for the changes in this PR

Lighthouse tested 2 URLs

⚠️ Budget exceeded for 3 of 10 audits.

Report for Article

tested url https://www.theguardian.com/commentisfree/2020/feb/08/hungary-now-for-the-new-right-what-venezuela-once-was-for-the-left

Category Status Expected Actual
First Contentful Paint 1500 1132
Largest Contentful Paint 3000 1715
Time to Interactive 3500 2300
Cumulative Layout Shift ⚠️ 0.002 0.004428
accessibility 0.97 1.000000

Report for Front

tested url https://www.theguardian.com/uk

Category Status Expected Actual
First Contentful Paint 1500 1201
Largest Contentful Paint ⚠️ 3000 3028
Time to Interactive 3500 2527
Cumulative Layout Shift 0.002 0.000000
accessibility ⚠️ 0.97 0.950000

@jamesgorrie jamesgorrie force-pushed the server-side-dotenv branch 2 times, most recently from fab115d to 86d695d Compare May 4, 2022 15:42
@jamesgorrie jamesgorrie marked this pull request as ready for review May 18, 2022 09:37
@jamesgorrie jamesgorrie added the Fronts 📰 Related to the migration of front pages to DCR label May 18, 2022

### Lines in the sand

#### `.env` shouldn't be required
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1 An important point not to be forgotten

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you think it's worth enforcing in some or other way? That might be how we access to config is the only way I can think of.

Copy link
Contributor

@OllysCoding OllysCoding left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks really good!

Should we document somewhere how this all works, and I think an important thing to cover is that right now we only pass our .env variables into the server build - meaning that they're not available on the client.

It's important that if that were to ever change appropriate considerations were taking (like requiring a CLIENT_ prefix) to avoid any accidental security leaks.

@@ -0,0 +1,81 @@
const { CredentialsProviderError } = require('@aws-sdk/property-provider');
const path = require('path');
const { prompt, log, warn } = require('../env/log');
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I personally don't like the colours in this library, but at least using it will make things consistent.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I didn't actually know this library existed! Are there some other places we should be using it?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It’s been there … a while #80 😆

Copy link
Contributor

@mxdvl mxdvl left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why are we using floating promises and commonJS?

  • All the file system APIs now exist as synchronous methods.
  • top-level await is supported since node 13
  • ES modules are enabled by default since node 14 (using .mjs)

jamesgorrie and others added 3 commits May 19, 2022 10:25
Co-authored-by: Olly <9575458+OllysCoding@users.noreply.github.com>
@jamesgorrie
Copy link
Contributor Author

@mxdvl "floating promises" Could you point this out - if you're talking about in aws-parameters, this was a full copy/paste. Happy to refine, and maybe test this script?

"common-js" <- same it was just copy/paste.

@OllysCoding nice idea, I'll add an ADR.

@OllysCoding OllysCoding linked an issue May 20, 2022 that may be closed by this pull request
jamesgorrie and others added 2 commits May 23, 2022 11:20
Co-authored-by: Ioanna Kokkini <ioannakok@users.noreply.github.com>
@jamesgorrie jamesgorrie merged commit 2375fbd into main May 24, 2022
@jamesgorrie jamesgorrie deleted the server-side-dotenv branch May 24, 2022 09:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
dotcom-rendering Fronts 📰 Related to the migration of front pages to DCR Platform
Projects
No open projects
Status: Done
Development

Successfully merging this pull request may close these issues.

Solution for storing secrets in DCR
4 participants