Enhancement/issue 971 refactor bundling and optimizations

[thescientist13]

Related Issue

resolves #971 and #581 (related to #970 and also ties into #550 )

• also seems to resolve avoid Rollup double bundling for pre-bundled static assets for server routes #883 too?
• and also resolves remove hardcoded directory references in rollup configuration #542 ??

Summary of Changes

General strategy revolved around reducing duplicate work and make more of the business logic "first party"

1. Refactoring rollup.config.js to do more work of the in bundle lifecycle re-using the optimization lifecycles of resource plugins
2. Consolidate used resources into a single data structure as part of the compilation
3. Now for example, no special Rollup plugin needed for transforming JSON, CSS
4. Resolves inline optimization of styles

Some of the "breaking" changes actually help fix bugs too

• Consistent minification now of all JS / CSS (fixes ensure inline <script> and <style> tags are getting minified / optimized when running build command #581 )
• "Harden" the timing of when intercept and optimize lifecycles run
• Solved for the double bundle since we don't rely on Rollup for crawling HTML anymore (also, we can / should be statically generating all this ahead of time anyway)

TODO

1. Need to handle optimization configurations and specs
2. Validate that we can re-use all our resources for support resource plugin based transformations for standard module formats (ex: import JSON, CSS) for SSR #878 as ESM, and so maybe wouldn't need plugins for Rollup at all in that case?
   • Got this working for Import JSON plugin ✅
3. Get all CLI specs * (not including SSR specs, to be refactored 👇 )
4. Refactor SSR bundling (see avoid Rollup double bundling for pre-bundled static assets for server routes #883 )
5. Get all plugin specs passing
6. Refactor all "import" related plugins to not need specialized Rollup plugins, update package.json etc
7. Make sure website is building correctly (develop / build / serve)
8. Static router not loading
9. Missed postcssImport optimizing in standard CSS plugin
10. hundreds of inputs for Rollup for www build, need to dedupe what goes into resources before it goes into Rollup?
11. handle missing optimizedFileContents, like from lib/router.js (restore local project work around)
12. Netlify build failing?
13. Update documentation / plugin READMEs as needed
    • ResourceInterface lifecycles and best practices -
    • Rollup plugins use cases (not needed for most* resources) - Rollup plugin is a leaky abstraction #550 (comment)
14. Fix Windows build / normalize cross-platform key / pathname setting (e.g. /C:/)
15. Getting extra output bundles compared to prod with puppeteer prerendering?
    
16. Static router not loading (again)
17. Windows build failing (again 😅 )
18. Refactor / TODOs
    • Need to align on how Lit polyfill bundling is handled (once in CLI and once in the plugin it seems...) - see create Lit specific polyfill plugin #728
    • clean up all gwd-data-opt=* markers from HTML
    • optimize should reuse body
    • add node_modules specific tests to import CSS and JSON plugins
    • have HUD check both valid app and page body roots
    • have plugins filter on contentType
    • "real" hashString function
    • misc code TODOs
    • clean up scratch files in public folder
    • add smoke tests for nested head tags (e.g. there should be no <script> ... <script>...</script></script>
19. Start this as the base of a new release branch (release/0.27.0)
    • add performance Benchmarking check-in as part of release TODOs - performance benchmarking / optimizing Greenwood #970
20. Track Hacktoberfest Good First Issues issue from leftover TODOs

Future Feature Smoke Testing (Nice To Have)

1. Test against support resource plugin based transformations for standard module formats (ex: import JSON, CSS) for SSR #878 (also confirm version of NodeJS we should use)
2. Validate PostCSS -> Parcel POC (CSS Bundling (and dropping PostCSS as core dependency) #763) - Enhancement/issue 763 parcel css bundling #977
3. Test CSS Import Assertions - import attributes for standard module formats; CSS and JSON  #923

Thoughts / Questions

• Should we use Map / Set more?
• Re: Rollup, maybe it could be an undocumented API? Perhaps Alias plugins could also be solved by resource plugins?
• Should we rethink default values / methods for ResourceInterface, maybe as part of Standardize on Web APIs (Request / Response / URL / etc) #948 ? Maybe intercept should be more clearly documented as more of a middleware style?
• Continue to use string concat or lean into htmlparser (AST)? - conventional wisdom seems to be string manipulation is ideal
• Call it a manifest.json instead of graph.json?
• should optimize resource lifecycles take a url or body, or both? with a "hard" URL, you can get to a body, right? but what about chained optimizations? I think it should it always get a body from the calling point. So then it would need URL at all then?
• Rename compile lifecycle?
• rename imports field to resources in frontmatter?
• Deprecate Rollup plugin? (maybe a grab bag RFC for sprint 9 to downshift away from a bundler per se, just use terser directly as the default?)
  • might still need it for things like replace, resolve, etc?
  • Move Rollup to JS plugin optimize lifecycle?
  • should we defer .js files to Greenwood resources too?