Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Do not download vendor JS files in released plugin versions #1025

Merged
merged 1 commit into from
Jun 12, 2017

Conversation

nylen
Copy link
Member

@nylen nylen commented Jun 5, 2017

This PR introduces the first example of logic needing to vary between the development and release versions of the plugin. In development mode, we download and cache "nightly" vendor JavaScript files, but this isn't appropriate for release - we should bundle these with the plugin instead.

This is accomplished by generating a slightly different gutenberg.php file during the plugin release process, then only downloading vendor files if the GUTENBERG_DEVELOPMENT_MODE constant is truthy.

We respect the SCRIPT_DEBUG constant for these vendor scripts, but not yet for our own JavaScript. I'll create a separate issue for this (#1026).

Follow-up to #985.

Addresses #953 (comment), after this we should be ready to release Gutenberg to the WP plugins directory.

Fixes #921 at the same time, since I was making changes in that area.

Copy link
Member

@aduth aduth left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not going to pretend I'm not hopelessly lost trying to understand some of the Bash wizardry here. A bit worrying from maintainability perspective, but then again, probably not a permanent solution I'd imagine. Ideally we'd move off the nightlies. I think we can already for TinyMCE. React 16's latest alpha was released back in February, so not as promising there.

Should we add some of the temporary files to gitignore in case the script is aborted or otherwise exists mid-execution? Thinking *.tmp.js.

php bin/get-vendor-scripts.php debug
)
while IFS='|' read -u 3 url filename; do
wget -nv "$url" -O "vendor/$$.tmp.js"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

wget is not stock Mac friendly:

./bin/build-plugin-zip.sh: line 41: wget: command not found

Maybe just curl: https://stackoverflow.com/a/4572158

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

TIL $$, but also TIL apparently it's unwise for temporary files?

https://stackoverflow.com/questions/78493/what-does-mean-in-the-shell

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The issues with $$ and other "weak" methods of temporary filename generation are related to privileged processes that can be tricked into writing unexpected files that a user would not otherwise be able to write to. Thankfully, this set of issues does not apply to us.

Still, I've removed the use of $$ to avoid promoting this practice.

# Run the build
npm install
npm run build

# Remove any existing zip file
rm -f gutenberg.zip

# Temporarily modify `gutenberg.php` with production constants defined
php bin/generate-gutenberg-php.php > gutenberg.$$.php
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we write directly into gutenberg.php here?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wouldn't recommend it, since the php script is reading line-by-line from the same file at the same time. This might work due to kernel buffering or something like that, but it seems like undefined behavior to me. Added a comment to clarify.

@nylen nylen force-pushed the add/vendor-js-to-plugin-zip branch from 9f9d49b to 7be1f75 Compare June 8, 2017 14:16
@nylen
Copy link
Member Author

nylen commented Jun 8, 2017

I'm not going to pretend I'm not hopelessly lost trying to understand some of the Bash wizardry here. A bit worrying from maintainability perspective, but then again, probably not a permanent solution I'd imagine.

I added a couple more comments to try to address this - but yes, I agree this is not a permanent solution, and the way we handle vendor scripts needs some reworking. Nightlies are part of the issue, our current setup does what we need it to do without having to be configured, but could probably be redesigned a bit.

Should we add some of the temporary files to gitignore in case the script is aborted or otherwise exists mid-execution? Thinking *.tmp.js.

Everything in vendor/ is already ignored.

@nylen nylen force-pushed the add/vendor-js-to-plugin-zip branch from 7be1f75 to 67d6c42 Compare June 8, 2017 14:25
@nylen
Copy link
Member Author

nylen commented Jun 9, 2017

I'd like to merge this today so that we can move forward with a release to the plugin directory ahead of WCEU.

Copy link
Member

@aduth aduth left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should I expect that this downloads both minified and unminified versions of dependencies?

https://unpkg.com/react@next/umd/react.production.min.js
 > vendor/react.min.c7397a88.js ... done!
https://unpkg.com/react-dom@next/umd/react-dom.production.min.js
 > vendor/react-dom.min.912dc6cf.js ... done!
https://unpkg.com/react-dom@next/umd/react-dom-server.production.min.js
 > vendor/react-dom-server.min.538e07c8.js ... done!
https://unpkg.com/moment@2.18.1/min/moment.min.js
 > vendor/moment.min.0fc04ee9.js ... done!
https://fiddle.azurewebsites.net/tinymce/nightly/tinymce.min.js
 > vendor/tinymce.min.04a1d751.js ... done!
https://fiddle.azurewebsites.net/tinymce/nightly/plugins/lists/plugin.min.js
 > vendor/tinymce-plugin-lists.min.e95f0005.js ... done!
https://unpkg.com/react@next/umd/react.development.js
 > vendor/react.3a8927d9.js ... done!
https://unpkg.com/react-dom@next/umd/react-dom.development.js
 > vendor/react-dom.e5cfcfc9.js ... done!
https://unpkg.com/react-dom@next/umd/react-dom-server.development.js
 > vendor/react-dom-server.e84b9fa6.js ... done!
https://unpkg.com/moment@2.18.1/moment.js
 > vendor/moment.7eefd5c2.js ... done!
https://fiddle.azurewebsites.net/tinymce/nightly/tinymce.js
 > vendor/tinymce.c847083c.js ... done!
https://fiddle.azurewebsites.net/tinymce/nightly/plugins/lists/plugin.js
 > vendor/tinymce-plugin-lists.557da104.js ... done!

function print_production_defines() {
global $plugin_version;

echo "define( 'GUTENBERG_VERSION', '$plugin_version' );\n";
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you explain why we want these defines? They don't appear to be used? Is it just because we need to replace the development mode constant so we might as well? Wondering if it'd help simplify this at all if we just moved to stripping out the define section during packaging.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it just because we need to replace the development mode constant so we might as well?

There are a couple of other reasons as well:

@aduth
Copy link
Member

aduth commented Jun 9, 2017

Observing persistent 500 error after trying to activate plugin generated by this branch on a test site. Will follow-up when I learn more.

Edit: Appears to be because the plugin is trying to require newly-introduced blocks/library/latest-posts/index.php (#870) and is not finding it. Also an issue on master.

while IFS='|' read -u 3 url filename; do
echo "$url"
echo -n " > vendor/$filename ... "
curl -s "$url" -o "vendor/_download.tmp.js"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we need to follow redirects here (-L).

Finding I have vendor dependencies files written with the content like the following:

<p>You are being redirected to <a href="/react-dom@16.0.0-alpha.13/umd/react-dom.development.js">/react-dom@16.0.0-alpha.13/umd/react-dom.development.js</a>

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Indeed, I am not sure how I missed this 😬 Fixed in latest push.

@nylen
Copy link
Member Author

nylen commented Jun 9, 2017

Should I expect that this downloads both minified and unminified versions of dependencies?

Yes, so that we respect (at least partly) the SCRIPT_DEBUG setting in users' WP installations - see #1026.

@nylen nylen force-pushed the add/vendor-js-to-plugin-zip branch from 791d72a to 171f9c5 Compare June 9, 2017 21:03
@nylen
Copy link
Member Author

nylen commented Jun 9, 2017

Observing persistent 500 error after trying to activate plugin generated by this branch on a test site.

Fixed in #1095 and master; rebased this PR afterwards. This branch is testing out well for me again now.

Copy link
Member

@aduth aduth left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Working well now 👍

@mtias mtias added [Type] Build Tooling Issues or PRs related to build tooling Gutenberg Plugin Issues or PRs related to Gutenberg Plugin management related efforts labels Jun 12, 2017
@mtias
Copy link
Member

mtias commented Jun 12, 2017

Thanks for working on this one.

@nylen nylen merged commit 16680dd into master Jun 12, 2017
@nylen nylen deleted the add/vendor-js-to-plugin-zip branch June 12, 2017 19:35
@nylen
Copy link
Member Author

nylen commented Jun 12, 2017

Thanks for reviewing. I've been thinking about how to simplify this, and probably (when we decide we're ready to stabilize dependency versions a bit) we can do something like this:

  • Move the code to download vendor dependencies out of PHP and into npm run download-deps or similar. Make the PHP code simply reference these downloaded files.
  • Make other npm run commands (dev, build, package-plugin) depend on this step.

We would lose the current behavior where we automatically download new versions of these dependencies every day, so we probably shouldn't do this until we are reasonably confident in our versions of React and TinyMCE in particular.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Gutenberg Plugin Issues or PRs related to Gutenberg Plugin management related efforts [Type] Build Tooling Issues or PRs related to build tooling
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants