Skip to content

Commit 6ebd956

Browse files
adamzieladamziel
authored
[Data Liberation] "Fetch from a different URL" button for failed media downloads, Interactivity API support (#2040)
## Description Ships user-driven import error handling and makes the import UI more useful by automatically refreshing the progress details. ### User-driven error handling When a remote asset cannot be downloaded, most importers either stop or ignore the error. This PR adds a user interaction to make an explicit decision about what should happen next – do we ignore the missing asset? Do we use another file instead? https://github.com/user-attachments/assets/cea48258-b644-434c-9fb2-1b890c4d86d7 ### Auto-Refreshing Import Status This PR also re-expresses the entire data liberation wp-admin page using the interactivity API, and auto-refreshes the progress: https://github.com/user-attachments/assets/e093268b-5deb-4bc2-a1d2-e2bb1148e153 A part of #1894 ## Technical overview ### User-driven error handling During the frontloading stage, the `WP_Stream_Importer` exposes all the frontloaded entities to the API consumer. The consumer then creates a post of type `frontloading_placeholder` with an initial status `awaiting_download` for each asset, and updates it with progress information and status (success, failure, skipped) as the import progresses. The frontloading stage is not finished until all the frontloaded assets have been processed with a non-error outcome. There's a few ways to recover from errors: * Retry the download – `WP_Stream_Importer` now retries the failed assets URL (via `WP_Retry_Frontloading_Iterator`) before moving on to entities provided by the usual entity source such as a WXR file. * Changing the downloaded URL – done by the user on the wp-admin page * Choosing to skip the download – done by the user on the wp-admin page Sometimes we don't want to require user interactions, e.g. when running the `importWxr` Blueprint step. In those scenarios, we could choose a default error outcome, e.g. "skip failed downloads". ### Auto-refreshing admin page Two `fetch()` requests running in an infinite loop are: * Updating the JavaScript interactivity store with the latest import state from the server * Running the next import step ### Other changes * Adds `php_userstreamop_read` to the Asyncify list – it crashed the importer in `@wp-playground/cli` running in bun. ## Follow-up work * Pretty UI transitions. Right now it's all sudden and jerky. We need progress bars, smooth animations, clear visual causality. * Prevent running the same import in two concurrent requests. This is a serial importer not designed for parallelization. * Run each import step in a transaction – either it all worked and we can commit the changes and an updated cursor, or it didn't work and we roll back the last step. Ideally we'll never see a scenario where an entity was processed, but a crash happened before storing the updated cursor and the next run reprocesses the same entity. ## Testing instructions * Go to the data liberation admin page * Upload a WXR export file * Confirm the import processes automatically and doesn't error out
1 parent 955be03 commit 6ebd956

20 files changed

+1772
-765
lines changed

‎CHANGELOG.md‎

Lines changed: 6 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -4,38 +4,36 @@ All notable changes to this project are documented in this file by a CI job
44
that runs on every NPM release. The file follows the [Keep a Changelog](https://keepachangelog.com/en/1.1.0/)
55
format.
66

7-
## [v1.0.14] (2024-12-02)
7+
## [v1.0.14] (2024-12-02)
88

99
### Blueprints
1010

11-
- Resolve the latest WordPress version from the API instead of assuming it's the same as the last minified build. ([#2027](https://github.com/WordPress/wordpress-playground/pull/2027))
11+
- Resolve the latest WordPress version from the API instead of assuming it's the same as the last minified build. ([#2027](https://github.com/WordPress/wordpress-playground/pull/2027))
1212

1313
### Tools
1414

15-
1615
#### Blueprints Builder
1716

18-
- Add installPlugin support for single plugin files. ([#2033](https://github.com/WordPress/wordpress-playground/pull/2033))
17+
- Add installPlugin support for single plugin files. ([#2033](https://github.com/WordPress/wordpress-playground/pull/2033))
1918

2019
### PHP WebAssembly
2120

22-
- Networking: Preserve the content-type header when fetch()-ing. ([#2028](https://github.com/WordPress/wordpress-playground/pull/2028))
21+
- Networking: Preserve the content-type header when fetch()-ing. ([#2028](https://github.com/WordPress/wordpress-playground/pull/2028))
2322

2423
### Website
2524

26-
- [Web] Re-enable wp-cron. ([#2039](https://github.com/WordPress/wordpress-playground/pull/2039))
25+
- [Web] Re-enable wp-cron. ([#2039](https://github.com/WordPress/wordpress-playground/pull/2039))
2726

2827
### Various
2928

30-
- [Data Liberation] WP_Stream_Importer: User-driven incremental import. ([#2013](https://github.com/WordPress/wordpress-playground/pull/2013))
29+
- [Data Liberation] WP_Stream_Importer: User-driven incremental import. ([#2013](https://github.com/WordPress/wordpress-playground/pull/2013))
3130

3231
### Contributors
3332

3433
The following contributors merged PRs in this release:
3534

3635
@adamziel @brandonpayton
3736

38-
3937
## [v1.0.13] (2024-11-25)
4038

4139
### Enhancements

‎packages/docs/site/docs/main/changelog.md‎

Lines changed: 6 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -9,38 +9,36 @@ All notable changes to this project are documented in this file by a CI job
99
that runs on every NPM release. The file follows the [Keep a Changelog](https://keepachangelog.com/en/1.1.0/)
1010
format.
1111

12-
## [v1.0.14] (2024-12-02)
12+
## [v1.0.14] (2024-12-02)
1313

1414
### Blueprints
1515

16-
- Resolve the latest WordPress version from the API instead of assuming it's the same as the last minified build. ([#2027](https://github.com/WordPress/wordpress-playground/pull/2027))
16+
- Resolve the latest WordPress version from the API instead of assuming it's the same as the last minified build. ([#2027](https://github.com/WordPress/wordpress-playground/pull/2027))
1717

1818
### Tools
1919

20-
2120
#### Blueprints Builder
2221

23-
- Add installPlugin support for single plugin files. ([#2033](https://github.com/WordPress/wordpress-playground/pull/2033))
22+
- Add installPlugin support for single plugin files. ([#2033](https://github.com/WordPress/wordpress-playground/pull/2033))
2423

2524
### PHP WebAssembly
2625

27-
- Networking: Preserve the content-type header when fetch()-ing. ([#2028](https://github.com/WordPress/wordpress-playground/pull/2028))
26+
- Networking: Preserve the content-type header when fetch()-ing. ([#2028](https://github.com/WordPress/wordpress-playground/pull/2028))
2827

2928
### Website
3029

31-
- [Web] Re-enable wp-cron. ([#2039](https://github.com/WordPress/wordpress-playground/pull/2039))
30+
- [Web] Re-enable wp-cron. ([#2039](https://github.com/WordPress/wordpress-playground/pull/2039))
3231

3332
### Various
3433

35-
- [Data Liberation] WP_Stream_Importer: User-driven incremental import. ([#2013](https://github.com/WordPress/wordpress-playground/pull/2013))
34+
- [Data Liberation] WP_Stream_Importer: User-driven incremental import. ([#2013](https://github.com/WordPress/wordpress-playground/pull/2013))
3635

3736
### Contributors
3837

3938
The following contributors merged PRs in this release:
4039

4140
@adamziel @brandonpayton
4241

43-
4442
## [v1.0.13] (2024-11-25)
4543

4644
### Enhancements

‎packages/php-wasm/compile/php/Dockerfile‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -569,6 +569,7 @@ export ASYNCIFY_ONLY=$'"rc_dtor_func",\
569569
"_php_stream_fill_read_buffer",\
570570
"_php_stream_read",\
571571
"php_stream_read_to_str",\
572+
"php_userstreamop_read",\
572573
"zif_fread",\
573574
"wasm_read",\
574575
"php_stdiop_read",\

‎packages/php-wasm/node/asyncify/8_3_0/php_8_3.wasm‎

2.11 KB
Binary file not shown.

‎packages/php-wasm/node/asyncify/php_8_3.js‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
const dependencyFilename = __dirname + '/8_3_0/php_8_3.wasm';
22
export { dependencyFilename };
3-
export const dependenciesTotalSize = 15083501;
3+
export const dependenciesTotalSize = 15085661;
44
export function init(RuntimeName, PHPLoader) {
55
/**
66
* Overrides Emscripten's default ExitStatus object which gets

‎packages/playground/data-liberation/bootstrap.php‎

Lines changed: 39 additions & 37 deletions
Original file line numberDiff line numberDiff line change
@@ -17,25 +17,25 @@
1717
require_once __DIR__ . '/src/byte-readers/WP_Remote_File_Reader.php';
1818
require_once __DIR__ . '/src/byte-readers/WP_Remote_File_Ranged_Reader.php';
1919

20-
if(!class_exists('WP_HTML_Tag_Processor')) {
21-
require_once __DIR__ . "/src/wordpress-core-html-api/class-wp-html-token.php";
22-
require_once __DIR__ . "/src/wordpress-core-html-api/class-wp-html-span.php";
23-
require_once __DIR__ . "/src/wordpress-core-html-api/class-wp-html-text-replacement.php";
24-
require_once __DIR__ . "/src/wordpress-core-html-api/class-wp-html-decoder.php";
25-
require_once __DIR__ . "/src/wordpress-core-html-api/class-wp-html-attribute-token.php";
26-
27-
require_once __DIR__ . "/src/wordpress-core-html-api/class-wp-html-decoder.php";
28-
require_once __DIR__ . "/src/wordpress-core-html-api/class-wp-html-tag-processor.php";
29-
require_once __DIR__ . "/src/wordpress-core-html-api/class-wp-html-open-elements.php";
30-
require_once __DIR__ . "/src/wordpress-core-html-api/class-wp-token-map.php";
31-
require_once __DIR__ . "/src/wordpress-core-html-api/html5-named-character-references.php";
32-
require_once __DIR__ . "/src/wordpress-core-html-api/class-wp-html-active-formatting-elements.php";
33-
require_once __DIR__ . "/src/wordpress-core-html-api/class-wp-html-processor-state.php";
34-
require_once __DIR__ . "/src/wordpress-core-html-api/class-wp-html-unsupported-exception.php";
35-
require_once __DIR__ . "/src/wordpress-core-html-api/class-wp-html-processor.php";
20+
if ( ! class_exists( 'WP_HTML_Tag_Processor' ) ) {
21+
require_once __DIR__ . '/src/wordpress-core-html-api/class-wp-html-token.php';
22+
require_once __DIR__ . '/src/wordpress-core-html-api/class-wp-html-span.php';
23+
require_once __DIR__ . '/src/wordpress-core-html-api/class-wp-html-text-replacement.php';
24+
require_once __DIR__ . '/src/wordpress-core-html-api/class-wp-html-decoder.php';
25+
require_once __DIR__ . '/src/wordpress-core-html-api/class-wp-html-attribute-token.php';
26+
27+
require_once __DIR__ . '/src/wordpress-core-html-api/class-wp-html-decoder.php';
28+
require_once __DIR__ . '/src/wordpress-core-html-api/class-wp-html-tag-processor.php';
29+
require_once __DIR__ . '/src/wordpress-core-html-api/class-wp-html-open-elements.php';
30+
require_once __DIR__ . '/src/wordpress-core-html-api/class-wp-token-map.php';
31+
require_once __DIR__ . '/src/wordpress-core-html-api/html5-named-character-references.php';
32+
require_once __DIR__ . '/src/wordpress-core-html-api/class-wp-html-active-formatting-elements.php';
33+
require_once __DIR__ . '/src/wordpress-core-html-api/class-wp-html-processor-state.php';
34+
require_once __DIR__ . '/src/wordpress-core-html-api/class-wp-html-unsupported-exception.php';
35+
require_once __DIR__ . '/src/wordpress-core-html-api/class-wp-html-processor.php';
3636
}
37-
if (!isset($html5_named_character_references)) {
38-
require_once __DIR__ . "/src/wordpress-core-html-api/html5-named-character-references.php";
37+
if ( ! isset( $html5_named_character_references ) ) {
38+
require_once __DIR__ . '/src/wordpress-core-html-api/html5-named-character-references.php';
3939
}
4040

4141
require_once __DIR__ . '/src/block-markup/WP_Block_Markup_Processor.php';
@@ -56,6 +56,8 @@
5656
require_once __DIR__ . '/src/import/WP_Import_Session.php';
5757
require_once __DIR__ . '/src/import/WP_Stream_Importer.php';
5858
require_once __DIR__ . '/src/import/WP_Markdown_Importer.php';
59+
require_once __DIR__ . '/src/import/WP_Entity_Iterator_Chain.php';
60+
require_once __DIR__ . '/src/import/WP_Retry_Frontloading_Iterator.php';
5961

6062
require_once __DIR__ . '/src/utf8_decoder.php';
6163

@@ -65,50 +67,50 @@
6567
require_once __DIR__ . '/vendor/autoload.php';
6668

6769
// Polyfill WordPress core functions
68-
if (!function_exists('_doing_it_wrong')) {
69-
$GLOBALS['_doing_it_wrong_messages'] = [];
70-
function _doing_it_wrong($method, $message, $version) {
70+
if ( ! function_exists( '_doing_it_wrong' ) ) {
71+
$GLOBALS['_doing_it_wrong_messages'] = array();
72+
function _doing_it_wrong( $method, $message, $version ) {
7173
$GLOBALS['_doing_it_wrong_messages'][] = $message;
7274
}
7375
}
7476

75-
if(!function_exists('wp_kses_uri_attributes')) {
77+
if ( ! function_exists( 'wp_kses_uri_attributes' ) ) {
7678
function wp_kses_uri_attributes() {
77-
return [];
79+
return array();
7880
}
7981
}
8082

81-
if (!function_exists('__')) {
82-
function __($input) {
83+
if ( ! function_exists( '__' ) ) {
84+
function __( $input ) {
8385
return $input;
8486
}
8587
}
8688

87-
if (!function_exists('esc_attr')) {
88-
function esc_attr($input) {
89-
return htmlspecialchars($input);
89+
if ( ! function_exists( 'esc_attr' ) ) {
90+
function esc_attr( $input ) {
91+
return htmlspecialchars( $input );
9092
}
9193
}
9294

93-
if (!function_exists('esc_html')) {
94-
function esc_html($input) {
95-
return htmlspecialchars($input);
95+
if ( ! function_exists( 'esc_html' ) ) {
96+
function esc_html( $input ) {
97+
return htmlspecialchars( $input );
9698
}
9799
}
98100

99-
if (!function_exists('esc_url')) {
100-
function esc_url($url) {
101-
return htmlspecialchars($url);
101+
if ( ! function_exists( 'esc_url' ) ) {
102+
function esc_url( $url ) {
103+
return htmlspecialchars( $url );
102104
}
103105
}
104106

105-
if (!function_exists('wp_kses_uri_attributes')) {
107+
if ( ! function_exists( 'wp_kses_uri_attributes' ) ) {
106108
function wp_kses_uri_attributes() {
107109
return array();
108110
}
109111
}
110112

111-
if (!function_exists('mbstring_binary_safe_encoding')) {
113+
if ( ! function_exists( 'mbstring_binary_safe_encoding' ) ) {
112114
function mbstring_binary_safe_encoding( $reset = false ) {
113115
static $encodings = array();
114116
static $overloaded = null;
@@ -140,7 +142,7 @@ function mbstring_binary_safe_encoding( $reset = false ) {
140142
}
141143
}
142144

143-
if (!function_exists('reset_mbstring_encoding')) {
145+
if ( ! function_exists( 'reset_mbstring_encoding' ) ) {
144146
function reset_mbstring_encoding() {
145147
mbstring_binary_safe_encoding( true );
146148
}

0 commit comments

Comments
 (0)