Remove build products from this repository

[cpcallen]

This is a tracking bug for work to update our build system so that we can remove all checked-in build products from the google/blockly repository.

Background

The blockly repository contains not only source files but also build products, including the *_compressed.js (and .map.js) files. This is very convenient for being able to quickly fire up (e.g.) the Blockly Playground after doing a git clone, but creates some problems:

• Previously, every build would potentially modify these files, creating a lot of confusion about what should and shouldn't be included in any commit.
  • This was fixed (for Closure Compiler output) by building to build/ and adding the npm run checkin command.
• Now, we are in a situation where it is very easy to inadvertently be using stale checked-in build output rather than the freshly-generated copies.
  • This has occasionally caused quite a lot of confusion, particularly around running the playground in compiled mode (which uses the checked-in copies at least until chore(tests): Use freshly-build files in compressed mode. #6218 is finished.
  • Additionally, there has been some desire to check in additional build product, such as the wrapped language files.

It is probably preferable not to have any build output checked into the repository, but to generate it when required.

Outline Proposal

• Remove all generated files from the repository.
• Have build/ contain only ~temporary build products that are not part of the final release—e.g.:
  • The .js files output from tsc.
  • The un-wrapped language files currently checked in at msg/js/.
• Have dist/ contain (the only copy of) each "final" build product (e.g. blockly_compressed.js). The contents of dist/ would be the canonical place to load blockly from:
  • Test/playground code in the Blockly repository would load Blockly from this directory directly.
  • The contents of this directory will (continue to) be packaged as an NPM package.
  • The contents of this directory could additionally be attached to the GitHub release, to provide an easy way for non-NPM users to download a ready-to-use Blockly as a .zip file.
• The only exception will be test/playgrounds running in uncompiled mode (for edit/compile/test velocity) that would continue to to use files in build/ where appropriate.
• npm run build would write some things that it currently writes to build/ directly to dist/.
• npm run package would add whatever additional files (e.g. package.json) that are needed for release but not for testing.
• npm run prepare, which happens automatically when doing npm install, would be configured to do npm run build (or a suitable subset thereof) to ensure that previously checked-in build products have been built, allowing a "clone and go" approach to continue to work (just now "clone, install and go").
• Additionally, to reduce the potential for human error when building, packaging and publishing Blockly, each step of the build / package / publish process would be configured to run all the previous steps. So for example in the following list each target includes all the previous targets (this is not a comprehensive list):
  • build:js (runs tsc)
  • build:deps (runs closure-make-deps)
  • build:compiled (runs google-closure-compiler; this target will thus become synonymous with build and be removed.)
  • package (copies files into dist/)
  • publish (publishes contents of dist/)
    This is already the case for build:js / build:deps / build targets, so this is just extending the same principle to the full set of primary targets.

To Do list

• Modify gulp targets so that each runs all the ones it depends on, to avoid any inconsistency in build/packaged files.
  • Consider whether to rename gulp scripts to reflect the new arrangements.
    • For example, originally the build:xyz targets were all separate non-overlapping parts of the build target, but that will no longer be the case, so a different naming system might be advisable; perhaps e.g. just tsc / deps / bundle.
• Modify package target to run the cleanBuildDir and cleanReleaseDir targets (i.e. the same as npm run clean) before any of the other build steps to ensure that no extraneous files are inadvertently included in dist/ and thus the package.
• Modify publish script to run npm ci before invoking the gulp script to ensure that dependencies are at the specified version.
• Modify tests/run_all_tests.sh to just run npm run package once at the start (for efficiency, since package will now invoke build).
• Modify gulp targets to write to dist/ instead of build/ where appropriate.
  • Remove obsolete copying code from package gup targets.
• Modify tests to load dist/msg/en.js instead of msg/messages.js.
• Modify tests/bootstrap.js to load from dist/ (chore(tests): Use freshly-build files in compressed mode. #6218).
• Remove build products from the repository:
  • *_compressed.js and .js.map
  • msg/js/*
  • (any others?)
• Update the procedures for pushing our GitHub Pages to include checking in the relevant build products to the gh_pages branch so that our GitHub Pages will continue to work.

[cpcallen]

Reopening this until I have finished the requisite documentation changes.

[cpcallen]

Closed by cl/503198373.