Extend towards full dev environment #104
Comments
|
Yes, yes, yes! I love it @schlessera ! What exists today is a CodeMirror integration with an in-browser rollup, babel, and React Fast Refresh: Code.editor.HMR.demo.mp4In a perfect world, there would be a configurable in-browser Visual Studio Code instance with full support for WP-CLI, composer, npm, WordPress plugins/themes/patterns directory, and the ability to feed custom instructions. Much like Stackblitz. It could even be configurable as a Gutenberg block. During the hackaton we could try to bridge as much of the gap between these two as possible. Specifically, we could explore the following results:
The less backend is required, the easier it will be to host it on WP.org. I'll find out if I can still attend on behalf of Automattic and let you know – I assume it'd fine if I propose a project? Or would you like to? |
|
Awesome! Yes, please go ahead and propose a project. Also make sure to get in touch with @danielbachhuber internally, and Angela Jin who is in touch with the organizers already. I'm one of the two project mentors of this year's hackathon, so I'm happy to give feedback or hop on a call to discuss as you see fit - just ping me in Slack if you want to discuss in any form. |
@adamziel Oh! I thought you were going on sabbatical and unable to attend. If this is not the case, I'd love to work with you on the project. I already submitted this particular project idea but happy to transfer it to you or work on it as co-leads.
What is the technical feasibility of these elements, as well as |
|
There's an article titled, Postgres WASM by Snaplet and Supabase (Oct 2022).
It describes an experimental project where they used Buildroot to create a minimal Linux build with Postgres, then ran it on a virtual machine in the browser using v86.
For network access, they adapted a fork of Websockproxy which "allows the emulator to talk to the internet by converting data sent over a websocket port into TCP packets". (See also v86 networking.) They've open-sourced the work in packages:
Although they're likely not usable as-is, it seems they share some similar building blocks required (if not the code then the concepts) to host WordPress, PHP, and SQLite running on Linux in a virtual x86 machine, all in the browser. Since it implements the whole stack down to the machine, this approach could theoretically support any software that runs in a standard Linux environment, including: Git, WP-CLI, Composer, Node.js and NPM, regular UNIX commands, interactive shell and terminal. That seems ideal for educational and practical purposes. A disadvantage is the weight of the WASM bundle, which in their case was about 30Mb (then optimized to 12 by mounting a compressed file system using 9P). And the complexity of hosting, since it's no longer just a static site but with a corresponding server to enable network access and maybe persistence - features where browser support is currently lacking. |
@danielbachhuber I will be on sabbatical (unless I move it), but I'm excited about this project and will gladly show up anyway :-) We'll figure out the logistics
@eliot-akira covered the WASM part well so I will focus on the transport. Composer, npm, WP.org directories, and git can all run over HTTPS which the browser already speaks. A regular Github.com, however, does not support cross-origin How about supporting git via a browser extension? Extensions can request arbitrary HTTP endpoints without the same restrictions as web apps. Chrome extensions even used to support Relatedly, |
adamziel
added
[Type] Enhancement
|
I wonder how easy would it be to integrate https://busybox.net/live_bbox/live_bbox.html |
|
@adamziel I don't think that is relevant. This is not POSIX software compiled to run in WASM, it is a PC emulator written in JS. This emulates hardware in JS, so the software is just the regular compilation of the x86 stuff. You'd basically start from scratch and would need to implement a bridge layer that connects the internals of that emulator (network stack, etc...) to the WASM/WASI layer you have now. |
|
I also was trying to use the |
… WebApp Redesign (#1731) ## Description Implements a large part of the [website redesign](#1561):  High-level changes shipped in this PR: * Multiple Playgrounds. Every temporary Playground can be saved either in the browser storage (OPFS) or in a local directory (Chrome desktop only for now). * New Playground settings options: Name name, language, multisite * URL as the source of truth for the application state * State management via Redux This work is a convergence of 18+ months of effort and discussions. The new UI opens relieves the users from juggling ephemeral Playgrounds and losing their work. It opens up space for long-lived site configurations and additional integrations. We could bring over all the [PR previewers and demos](https://playground.wordpress.net/demos/) right into the Playground app. Here's just a few features unblocked by this PR: * #1438 – no more losing your work by accident 🎉 * #797 – with multiple sites we can progressively build features we'll eventually propose for WordPress core: * A Playground export and import feature, pioneering the standard export format for WordPress sites. * A "Clone this Playground" feature, pioneering the [Site Transfer Protocol](https://core.trac.wordpress.org/ticket/60375). * A "Sync two Playgrounds" feature, pioneering the Site Sync Protocol * #1445 – better git support is in top 5 most highly requested features. With multiple Playgrounds, we can save your work and get rid of the "save your work before connecting GitHub or you'll lose it" and cumbersome "repo setup" forms on every interaction. Instead, we can make git operations like Pull, Commit, etc. very easy and even enable auto-syncing with a git repository. * #1025 – as we bring in more PHP plumbing into this repository, we'll replace the TypeScript parts with PHP parts to create a WordPress core-first Blueprints engine * #1056 – Site transfer protocol will unlocks seamlessly passing Playgrounds between the browser and a local development environment * #1558 – we'll integrate [the Blueprints directory] and offer single-click Playground setups, e.g. an Ecommerce store or a Slide deck editor. #718. * #539 – the recorded Blueprints would be directly editable in Playground and perhaps saved as a new Playground template * #696 – the new interaction model creates space for additional integrations. * #707 – you could create a "GitHub–synchronized" Playground * #760 – we can bootstrap one inside Playground using a Blueprint and benefit the users immediately, and then gradually work towards enabling it on WordPress.org * #768 – the new UI has space for a "new in Playground" section, similar to what Chrome Devtools do * #629 * #32 * #104 * #497 * #562 * #580 ### Remaining work - [ ] Write a release note for https://make.wordpress.org/playground/ - [x] Make sure GitHub integration is working. Looks like OAuth connection leads to 404. - [x] Fix temp site "Edit Settings" functionality to actually edit settings (forking a temp site can come in a follow-up PR) - [x] Fix style issue with overlapping site name label with narrow site info views - [x] Fix style issue with bottom "Open Site" and "WP Admin" buttons missing for mobile viewports - [x] Make sure there is a path for existing OPFS sites to continue to load - [x] Adjust E2E tests. - [x] Reflect OPFS write error in UI when saving temp site fails - [x] Find a path forward for [try-wordpress](https://github.com/WordPress/try-wordpress) to continue working after this PR - [x] Figure out why does the browser get so choppy during OPFS save. It looks as if there was a lot of synchronous work going on. Shouldn't all the effort be done by a worker a non-blocking way? - [x] Test with Safari and Firefox. Might require a local production setup as FF won't work with the Playground dev server. - [x] Fix Safari error: `Unhandled Promise Rejection: UnknownError: Invalid platform file handle` when saving a temporary Playground to OPFS. - [x] Fix to allow deleting site that fails to boot. This is possible when saving a temp site fails partway through. - [x] Fix this crash: ```ts /** * @todo: Fix OPFS site storage write timeout that happens alongside 2000 * "Cannot read properties of undefined (reading 'apply')" errors here: * I suspect the postMessage call we do to the safari worker causes it to * respond with another message and these unexpected exchange throws off * Comlink. We should make Comlink ignore those. */ // redirectTo(PlaygroundRoute.site(selectSiteBySlug(state, siteSlug))); ``` - [x] Test different scenarios manually, in particular those involving Blueprints passed via hash - [x] Ensure we have all the aria, `name=""` etc. accessibility attributes we need, see AXE tools for Chrome. - [x] Update developer documentation on the `storage` query arg (it's removed in this PR) - [x] Go through all the `TODOs` added in this PR and decide whether to solve or punt them - [x] Handle errors like "site not found in OPFS", "files missing from a local directory" - [x] Disable any `Local Filesystem` UI in browsers that don't support them. Don't just hide them, though. Provide a help text to explain why are they disabled. - [x] Reduce the naming confusion, e.g. `updateSite` in redux-store.ts vs `updateSite` in `site-storage.ts`. What would an unambiguous code pattern look like? - [x] Find a reliable and intuitive way of updating these deeply nested redux state properties. Right now we do an ad-hoc recursive merge that's slightly different for sites and clients. Which patterns used in other apps would make it intuitive? - [x] Have a single entrypoint for each logical action such as "Create a new site", "Update site", "Select site" etc. that will take care of updating the redux store, updating OPFS, and updating the URL. My ideal scenario is calling something like `updateSite(slug, newConfig)` in a React Component and being done without thinking "ughh I still need to update OPFS" or "I also have to adjust that .json file over there" - [x] Fix all the tiny design imperfections, e.g. cut-off labels in the site settings form. ### Follow up work - [ ] Mark all the related blocked issues as unblocked on the project board, e.g. #1703, #1731, and more – [see the All Tasks view](https://github.com/orgs/WordPress/projects/180/views/2?query=sort%3Aupdated-desc+is%3Aopen&filterQuery=status%3A%22Up+next%22%2C%22In+progress%22%2C%22Needs+review%22%2C%22Reviewed%22%2C%22Done%22%2CBlocked) - [ ] Update WordPress/Learn#1583 with info that the redesign is now in and we're good to record a video tutorial. - [ ] #1746 - [ ] Write a note in [What's new for developers? (October 2024)](WordPress/developer-blog-content#309) - [ ] Document the new site saving flow in `packages/docs/site/docs/main/about/build.md` cc @juanmaguitar - [ ] Update all the screenshots in the documentation cc @juanmaguitar - [ ] When the site fails to load via `.list()`, still return that site's info but make note of the error. Not showing that site on a list could greatly confuse the user ("Hey, where did my site go?"). Let's be explicit about problems. - [ ] Introduce notifications system to provide feedback about outcomes of various user actions. - [ ] Add non-minified WordPress versions to the "New site" modal. - [ ] Fix `console.js:288 TypeError: Cannot read properties of undefined (reading 'apply') at comlink.ts:314:51 at Array.reduce (<anonymous>) at callback (comlink.ts:314:29)` – it seems to happen at trunk, too. - [ ] Attribute log messages to the site that triggered them. - [ ] Take note of any interactions that we find frustrating or confusing. We can perhaps adjust them in a follow-up PR, but let's make sure we notice and document them here. - [ ] Solidify the functional tooling for transforming between `URL`, `runtimeConfiguration`, `Blueprint`, and `site settings form state` for both OPFS sites and in-memory sites. Let's see if we can make it reusable in Playground CLI. - [ ] Speed up OPFS interactions, saving a site can take quite a while. - [ ] A mobile-friendly modal architecture that doesn't stack modals, allows dismissing, and understands some modals (e.g. fatal error report) might have priority over other modals (e.g. connect to GitHub). Discuss whether modals should be declared at the top level, like here, or contextual to where the "Show modal" button is rendered. - [ ] Discuss the need to support strong, masked passwords over a simple password that's just `"password"`. - [ ] Duplicate site feature implemented as "Export site + import site" with the new core-first PHP tools from adamziel/wxr-normalize#1 and https://github.com/adamziel/site-transfer-protocol - [x] Retain temporary sites between site changes. Don't just trash their iframe and state when the user switches to another site. Closes #1719 cc @brandonpayton --------- Co-authored-by: Brandon Payton <brandon@happycode.net> Co-authored-by: Bero <berislav.grgicak@gmail.com> Co-authored-by: Bart Kalisz <bartlomiej.kalisz@gmail.com>
Prototypes a `wp_rewrite_urls()` URL rewriter for block markup to migrate the content from, say, `<a href="https://adamadam.blog">` to `<a href="https://adamziel.com/blog">`. * URL rewriting works to perhaps the greatest extent it ever did in WordPress migrations. * The URL parser requires PHP 8.1. This is fine for some Playground applications, but we'll need PHP 7.2+ compatibility to get it into WordPress core. * This PR features `WP_HTML_Tag_Processor` and `WP_HTML_Processor` to enable usage outside of WordPress core. ### Details This PR consists of a code ported from https://github.com/adamziel/site-transfer-protocol. It uses a cascade of parsers to pierce through the structured data in a WordPress post and replace the URLs matching the requested domain. The data flow is as follows: Parse HTML -> Parse block comments -> Parse attributes JSON -> Parse URLs On a high level, this parsing cascade is handled by the `WP_Block_Markup_Url_Processor` class: ```php $p = new WP_Block_Markup_Url_Processor( $block_markup, $base_url ); while ( $p->next_url() ) { $parsed_matched_url = $p->get_parsed_url(); // .. do processing $p->set_raw_url($new_raw_url); } ``` Getting more into details, the `WP_Block_Markup_Url_Processor` extends the `WP_HTML_Tag_Processor` class and walks the block markup token by token. It then drills down into: * Text nodes – where matches URLs using regexps. This part can be improved to avoid regular expressions. * Block comments – where it parses the block attributes and iterates through them, looking for ones that contain valid URLs * HTML tag attributes – where it looks for ones that are reserved for URLs (such as `<a href="">`, looking for ones that contain valid URLs The `next_url()` method moves through the stream of tokens, looking for the next match in one of the above contexts, and the `set_raw_url()` knows how to update each node type, e.g. block attributes updates are `json_encode()`-d. ### Processing tricky inputs When this code is fed into the migrator: ```html <!-- wp:paragraph --> <p> <!-- Inline URLs are migrated --> 🚀-science.com/science has the best scientific articles on the internet! We're also available via the punycode URL: <!-- No problem handling HTML-encoded punycode URLs with urlencoded characters in the path --> https://xn---science-7f85g.com/%73%63ience/. <!-- Correctly ignores similar–but–different URLs --> This isn't migrated: https://🚀-science.comcast/science <br> Or this: super-🚀-science.com/science </p> <!-- /wp:paragraph --> <!-- Block attributes are migrated without any issue --> <!-- wp:image {"src": "https:\/\/\ud83d\ude80-\u0073\u0063ience.com/%73%63ience/wp-content/image.png"} --> <!-- As are URI HTML attributes --> <img src="https://xn---science-7f85g.com/science/wp-content/image.png"> <!-- /wp:image --> <!-- Classes are not migrated. --> <span class="https://🚀-science.com/science"></span> ``` This actual output is produced: ```html <!-- wp:paragraph --> <p> <!-- Inline URLs are migrated --> science.wordpress.com has the best scientific articles on the internet! We're also available via the punycode URL: <!-- No problem handling HTML-encoded punycode URLs with urlencoded characters in the path --> https://science.wordpress.com/. <!-- Correctly ignores similar–but–different URLs --> This isn't migrated: https://🚀-science.comcast/science <br> Or this: super-🚀-science.com/science </p> <!-- /wp:paragraph --> <!-- Block attributes are migrated without any issue --> <!-- wp:image {"src":"https:\/\/science.wordpress.com\/wp-content\/image.png"} --> <!-- As are URI HTML attributes --> <img src="https://science.wordpress.com/wp-content/image.png"> <!-- /wp:image --> <!-- Classes are not migrated. --> <span class="https://🚀-science.com/science"></span> ``` ## Remaining work - [x] Add PHPCBF - [x] Get to zero CBF errors - [x] Get the unit tests to run in CI (e.g. run `composer install`) - [x] Add relevant unit tests coverage ## Follow-up work - [x] Patch `WP_HTML_Tag_Processor` in WordPress core, see WordPress/wordpress-develop#7007 (comment) - [ ] Package our copy of `WP_HTML_Tag_Processor` as a "WordPress polyfill" for standalone usage. - [ ] Make it compatible with PHP 7.2+ ## Testing Instructions (or ideally a Blueprint) CI runs the PHP unit tests. To run this on your local machine, do this: ```sh cd packages/playground/data-liberation composer install cd ../../../ nx test:watch playground-data-liberation ```
I love these explorations that show what is currently possible on the web platform!
Thinking about the even bigger picture here, it would be awesome to have a full development platform in the browser, including a terminal you can run tooling like WP-CLI. This could make use of Browsix to power a POSIX-compatible terminal.
This would be a great learning tool that could be directly integrated into workshops on learn.wordpress.org and similar. So you'd head to learn.wordpress.org, read through a tutorial to learn something new, and then click on the
Try it yourselfbutton to launch a full environment in your browser that has a running WordPress site and all the dev tooling you need.I had been discussing this idea with @danielbachhuber to maybe introduce it for this year's Cloudfest Hackathon. @adamziel Is that something you might potentially be interested in?
The text was updated successfully, but these errors were encountered: