dart2wasm fails with WebAssembly.compileStreaming: Requested length for array.new_fixed too large #55396

IchordeDionysos · 2024-04-07T13:10:24Z

When compiling our real application and them trying to load the WASM file main.dart.wasm it fails with

Uncaught (in promise) CompileError: WebAssembly.compileStreaming(): Requested length 10890 for array.new_fixed too large, maximum is 10000 @+4396589

flutter doctor -v

[✓] Flutter (Channel main, 3.22.0-5.0.pre.54, on macOS 14.3 23D56 darwin-arm64, locale en-DE)
    • Flutter version 3.22.0-5.0.pre.54 on channel main at /Users/dennis/develop/bin/flutter
    • Upstream repository https://github.com/flutter/flutter.git
    • Framework revision 98d23f709f (15 hours ago), 2024-04-06 14:51:23 -0700
    • Engine revision 605b3f35fa
    • Dart version 3.5.0 (build 3.5.0-18.0.dev)
    • DevTools version 2.34.1

[!] Android toolchain - develop for Android devices (Android SDK version 33.0.1)
    • Android SDK at /Users/dennis/Library/Android/sdk
    ✗ cmdline-tools component is missing
      Run `path/to/sdkmanager --install "cmdline-tools;latest"`
      See https://developer.android.com/studio/command-line for more details.
    ✗ Android license status unknown.
      Run `flutter doctor --android-licenses` to accept the SDK licenses.
      See https://flutter.dev/docs/get-started/install/macos#android-setup for more details.

[✓] Xcode - develop for iOS and macOS (Xcode 15.3)
    • Xcode at /Applications/Xcode.app/Contents/Developer
    • Build 15E204a
    • CocoaPods version 1.14.3

[✓] Chrome - develop for the web
    • Chrome at /Applications/Google Chrome.app/Contents/MacOS/Google Chrome

[✓] Android Studio (version 2023.2)
    • Android Studio at /Users/dennis/Applications/Android Studio.app/Contents
    • Flutter plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/9212-flutter
    • Dart plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/6351-dart
    • Java version OpenJDK Runtime Environment (build 17.0.9+0-17.0.9b1087.7-11185874)

[✓] IntelliJ IDEA Ultimate Edition (version 2023.2.1)
    • IntelliJ at /Users/dennis/Applications/IntelliJ IDEA Ultimate.app
    • Flutter plugin version 76.3.4
    • Dart plugin version 232.9559.10

[✓] VS Code (version 1.85.1)
    • VS Code at /Applications/Visual Studio Code.app/Contents
    • Flutter extension can be installed from:
      🔨 https://marketplace.visualstudio.com/items?itemName=Dart-Code.flutter

[✓] Connected device (3 available)
    • macOS (desktop)                 • macos                 • darwin-arm64   • macOS 14.3 23D56 darwin-arm64
    • Mac Designed for iPad (desktop) • mac-designed-for-ipad • darwin         • macOS 14.3 23D56 darwin-arm64
    • Chrome (web)                    • chrome                • web-javascript • Google Chrome 123.0.6312.107

[✓] Network resources
    • All expected network resources are available.

! Doctor found issues in 1 category.

dart info

If providing this information as part of reporting a bug, please review the information
below to ensure it only contains things you're comfortable posting publicly.

#### General info

- Dart 3.5.0-18.0.dev (dev) (Wed Apr 3 13:07:15 2024 -0700) on "macos_arm64"
- on macos / Version 14.3 (Build 23D56)
- locale is en-DE

#### Process info

| Memory |  CPU | Elapsed time | Command line                                                                               |
| -----: | ---: | -----------: | ------------------------------------------------------------------------------------------ |
|  27 MB | 0.0% |        10:03 | dart language-server --client-id=IntelliJ-IDEA --client-version=IU-232.9559.62 --protocol=analyzer |
|  45 MB | 0.0% |        05:20 | dart language-server --client-id=IntelliJ-IDEA --client-version=IU-232.9559.62 --protocol=analyzer |
|  35 MB | 0.0% |        17:08 | flutter_tools.snapshot daemon                                                              |
|  39 MB | 0.0% |        05:21 | flutter_tools.snapshot daemon                                                              |

Tested on Chrome 123.0.6312.107

I'd like to submit a reproducible example, though the issue comes up when building our whole Flutter Web App (with some adjustments like removing packages depending on ffi, adjusting some yield* usages).

Don't really know where to start building a reproducible example ...
If there is anything that I can do to get to the bottom of this let me know 😌

The text was updated successfully, but these errors were encountered:

IchordeDionysos · 2024-04-07T13:12:43Z

These issues may be related?!
WebAssembly/gc#335
WebAssembly/gc#260

IchordeDionysos · 2024-04-07T13:22:18Z

This is actually easy to reproduce :D

Steps to reproduce:

Start with the sample Flutter application
Change web/index.html to

<!DOCTYPE HTML>
<html>
<head>
  <meta charset="UTF-8">
  <title>Flutter web app</title>
  <script src="flutter.js"></script>
</head>
<body>
<script>
  {{flutter_build_config}}
  _flutter.loader.load();
</script>
</body>
</html>

Build app with flutter build web --wasm
Run app with cd build/web && dhttpd '--headers=Cross-Origin-Embedder-Policy=credentialless;Cross-Origin-Opener-Policy=same-origin'

IchordeDionysos · 2024-04-08T06:41:14Z

Hmm fuck, I can no longer reproduce it with the example app but still with our actual application ...

mraleph · 2024-04-08T07:08:53Z

cc @osa1

osa1 · 2024-04-08T09:57:01Z

@IchordeDionysos Could you provide the .wasm? I don't need the source code, just the Wasm should be enough to find where this array.new_fixed is coming from, and I can debug it from there.

IchordeDionysos · 2024-04-08T11:26:32Z

@osa1 here you go, couldn't upload to GitHub ...:
https://drive.google.com/file/d/1VhVETC8Jh3UK0vvIgWLqcQM-G3wPRb2E/view?usp=sharing

osa1 · 2024-04-08T12:02:18Z

@IchordeDionysos I don't have access to the file.

IchordeDionysos · 2024-04-08T12:04:52Z

@osa1 ahh sorry should now work

osa1 · 2024-04-08T12:52:44Z

@IchordeDionysos Thanks, I can access the file now but it doesn't have any symbols.. Would it be possible for you to build with flutter build web --wasm --no-strip-wasm -O1?

(I don't know why Flutter doesn't allow -O0, hopefully -O1 is enough..)

IchordeDionysos · 2024-04-08T13:34:41Z

@osa1 this command flutter build web --wasm --no-strip-wasm -O1 did not work for me ...
However, this worked flutter build web --wasm --no-strip-wasm (taken from the Flutter WASM docs)

I've uploaded a new file version in the file/link above

It now has those things at the end of the file:

osa1 · 2024-04-08T14:03:42Z

Thanks. It looks like the large arrays are for _getTypeNames, _getTypeRulesSubstitutions, _getTypeRulesSupers intrinsics. We should update the Wasm array literal compiler to allocate the array lazily when it's too large:

sdk/pkg/dart2wasm/lib/constants.dart

Lines 435 to 454 in b574dab

    
           ConstantInfo? _makeWasmArrayLiteral(InstanceConstant constant) { 
        
             w.ArrayType arrayType = 
        
                 translator.arrayTypeForDartType(constant.typeArguments.single); 
        
             w.ValueType elementType = arrayType.elementType.type.unpacked; 
        
             List<Constant> elements = 
        
                 (constant.fieldValues.values.single as ListConstant).entries; 
        
             bool lazy = false; 
        
             for (Constant element in elements) { 
        
               lazy |= ensureConstant(element)?.isLazy ?? false; 
        
             } 
        
             return createConstant(constant, w.RefType.def(arrayType, nullable: false), 
        
                 lazy: lazy, (function, b) { 
        
               for (Constant element in elements) { 
        
                 constants.instantiateConstant(function, b, element, elementType); 
        
               } 
        
               b.array_new_fixed(arrayType, elements.length); 
        
             }); 
        
           }

I'll implement this.

Currently `array.new_fixed` has the upper size limit of 10,000. Larger arrays need to be initialized with `array.new` or `array.new_default`. We handle this correctly in constant list and strings, but we didn't handle it in `WasmArray.literal` consturctor. This CL fixes it. Fixes dart-lang#55396.

osa1 · 2024-04-09T07:57:22Z

Fix in https://dart-review.googlesource.com/c/sdk/+/361780.

Currently `array.new_fixed` has the upper size limit of 10,000. Larger arrays need to be initialized with `array.new` or `array.new_default`. We handle this correctly in constant list and strings, but we didn't handle it in `WasmArray.literal` consturctor. This CL fixes it. Bug: #55396 Bug: #55872 Cherry-pick: https://dart-review.googlesource.com/c/sdk/+/361780 Cherry-pick-request: #55873 Change-Id: I97c8dc104cb4231ba0508d9cd87690cf1bbe95a9 Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/368644 Reviewed-by: Slava Egorov <vegorov@google.com> Commit-Queue: Martin Kustermann <kustermann@google.com> Reviewed-by: Ömer Ağacan <omersa@google.com>

vsmenon added the area-dart2wasm Issues for the dart2wasm compiler. label Apr 8, 2024

osa1 self-assigned this Apr 8, 2024

osa1 mentioned this issue Apr 9, 2024

[dart2wasm] Constant initializer names include the whole constant expression, causing large names #55409

Open

copybara-service bot closed this as completed in c7834b6 Apr 10, 2024

mkustermann mentioned this issue May 30, 2024

[CP] [dart2wasm] Fix array.new_fixed calls with too large size #55873

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

dart2wasm fails with WebAssembly.compileStreaming: Requested length for array.new_fixed too large #55396

dart2wasm fails with WebAssembly.compileStreaming: Requested length for array.new_fixed too large #55396

IchordeDionysos commented Apr 7, 2024

IchordeDionysos commented Apr 7, 2024

IchordeDionysos commented Apr 7, 2024

IchordeDionysos commented Apr 8, 2024

mraleph commented Apr 8, 2024

osa1 commented Apr 8, 2024

IchordeDionysos commented Apr 8, 2024

osa1 commented Apr 8, 2024

IchordeDionysos commented Apr 8, 2024

osa1 commented Apr 8, 2024

IchordeDionysos commented Apr 8, 2024 •

edited

Loading

osa1 commented Apr 8, 2024

osa1 commented Apr 9, 2024

dart2wasm fails with WebAssembly.compileStreaming: Requested length for array.new_fixed too large #55396

dart2wasm fails with WebAssembly.compileStreaming: Requested length for array.new_fixed too large #55396

Comments

IchordeDionysos commented Apr 7, 2024

IchordeDionysos commented Apr 7, 2024

IchordeDionysos commented Apr 7, 2024

IchordeDionysos commented Apr 8, 2024

mraleph commented Apr 8, 2024

osa1 commented Apr 8, 2024

IchordeDionysos commented Apr 8, 2024

osa1 commented Apr 8, 2024

IchordeDionysos commented Apr 8, 2024

osa1 commented Apr 8, 2024

IchordeDionysos commented Apr 8, 2024 • edited Loading

osa1 commented Apr 8, 2024

osa1 commented Apr 9, 2024

IchordeDionysos commented Apr 8, 2024 •

edited

Loading