Migrate Hive v2 to support Flutter web WASM #1291

Rexios80 · 2024-05-29T23:24:03Z

Migrates Hive v2 to support Flutter web WASM since v3 and v4 are major changes (and not stable)

This should not be merged into the main branch, but there isn't a better option right now. This should probalby be merged into a branch created off of the tag v2.2.3 since that is the commit these changes are based on.

IF YOU WANT TO USE THIS NOW

I have released the following packages to replace Hive in my projects. The intent is to keep these up to date with modern Dart/Flutter standards.

hive_ce
hive_ce_flutter
hive_ce_generator

If you do not want to use Hive Community Edition, you can use this PR directly:

dependency_overrides:
  hive:
    git:
      url: https://github.com/Rexios80/hive
      ref: 4eb0dbc7807e7210831a2c71f056ac15c9e32e52
      path: hive
  hive_generator:
    git:
      url: https://github.com/Rexios80/hive
      ref: 4eb0dbc7807e7210831a2c71f056ac15c9e32e52
      path: hive_generator

Make sure to run the generator after updating

Related issues

#1287
isar/isar#1617
isar/isar#1616

theniceboy · 2024-05-31T00:12:38Z

Thank you for your effort! I've been trying to migrate Hive v2 to package:web and have been stuck. I'll be trying your branch very soon and giving you feedback.

theniceboy · 2024-05-31T19:20:57Z

I'm experience a runtime error. Here's the related info

Unsupported operation: Infinity or NaN toInt
main.dart.mjs:56     at Error._throwWithCurrentStackTrace (http://localhost:59048/main.dart.wasm:wasm-function[1606]:0x43ec60)
    at _BoxedDouble.toInt (http://localhost:59048/main.dart.wasm:wasm-function[1910]:0x4434a5)
    at JSNumberToNumber|get#toDartInt (http://localhost:59048/main.dart.wasm:wasm-function[1908]:0x443446)
    at StorageBackendJs.getKeys closure at file:///Users/david/.pub-cache/git/hive-456dab3bd156f7fa421aeb48884785ee2fa50471/hive/lib/src/backend/js/native/storage_backend_js.dart:126:45 (http://localhost:59048/main.dart.wasm:wasm-function[84040]:0xba5142)
    at closure wrapper at file:///Users/david/.pub-cache/git/hive-456dab3bd156f7fa421aeb48884785ee2fa50471/hive/lib/src/backend/js/native/storage_backend_js.dart:126:45 trampoline (http://localhost:59048/main.dart.wasm:wasm-function[84044]:0xba543b)
    at MappedListIterable.elementAt (http://localhost:59048/main.dart.wasm:wasm-function[2386]:0x44d35b)
    at ListIterator.moveNext (http://localhost:59048/main.dart.wasm:wasm-function[2380]:0x44d1eb)
    at new _GrowableList._ofEfficientLengthIterable (http://localhost:59048/main.dart.wasm:wasm-function[2234]:0x44a0a9)

Ran with flutter run -d chrome --debug --wasm using 3.23.0-13.0.pre.54

Rexios80 · 2024-05-31T20:04:20Z

The --debug flag doesn't work right now. See flutter/flutter#148850

But that's probably not the issue here. You re-ran the generator right?

Rexios80 · 2024-05-31T20:16:39Z

That error seems to imply that the issue is with this line:

https://github.com/Rexios80/hive/blob/456dab3bd156f7fa421aeb48884785ee2fa50471/hive/lib/src/backend/js/native/storage_backend_js.dart#L128

But that would mean you somehow have a box key that's infinity or NaN which doesn't make any sense. I suppose we could just call toDartDouble there instead but that seems weird. Any thoughts on how the heck one of your keys is infinity or NaN?

theniceboy · 2024-05-31T22:21:34Z

I did forget to re-ran the generator. The error still shows up. The same code runs well when compiled to main.dart.js without error. Other than using the master channel, I'm not how can I get a useful stack trace.

Another exception I'm having is this:

Unsupported operation: Platform._operatingSystem
main.dart.mjs:56     at Error._throwWithCurrentStackTrace (http://localhost:58619/main.dart.wasm:wasm-function[1608]:0x43f39d)
    at _Platform._operatingSystem (http://localhost:58619/main.dart.wasm:wasm-function[4942]:0x47e16e)
    at _Platform.operatingSystem (http://localhost:58619/main.dart.wasm:wasm-function[4941]:0x47e160)
    at Platform.operatingSystem (http://localhost:58619/main.dart.wasm:wasm-function[4940]:0x47e153)
    at IOPlatform.operatingSystem (http://localhost:58619/main.dart.wasm:wasm-function[15067]:0x55706b)
    at Platform.isAndroid (http://localhost:58619/main.dart.wasm:wasm-function[15066]:0x557042)
    at _defaultSpotlightUrl (http://localhost:58619/main.dart.wasm:wasm-function[15065]:0x557026)
    at new Spotlight (initializer) (http://localhost:58619/main.dart.wasm:wasm-function[15063]:0x557011)

Which may or may not be related. I'm not sure which is the error that's preventing the app to run.

I made the change you mentioned, and now I'm getting this error:

JavaScriptError
main.dart.mjs:56     at StorageBackendJs.initialize inner (http://localhost:64486/main.dart.wasm:wasm-function[85445]:0xbd0841)
    at _awaitHelper closure at org-dartlang-sdk:///dart-sdk/lib/_internal/wasm/lib/async_patch.dart:83:16 (http://localhost:64486/main.dart.wasm:wasm-function[86161]:0xbe1a7b)
    at closure wrapper at org-dartlang-sdk:///dart-sdk/lib/_internal/wasm/lib/async_patch.dart:83:16 trampoline (http://localhost:64486/main.dart.wasm:wasm-function[86166]:0xbe1b3f)
    at _RootZone.runUnary (http://localhost:64486/main.dart.wasm:wasm-function[3417]:0x46e595)
    at _FutureListener.handleValue (http://localhost:64486/main.dart.wasm:wasm-function[3412]:0x46e388)
    at _Future._propagateToListeners closure handleValueCallback at org-dartlang-sdk:///dart-sdk/lib/async/future_impl.dart:859:33 (http://localhost:64486/main.dart.wasm:wasm-function[3390]:0x46dd29)
    at _Future._propagateToListeners (http://localhost:64486/main.dart.wasm:wasm-function[3387]:0x46d950)
    at _Future._completeWithValue (http://localhost:64486/main.dart.wasm:wasm-function[3442]:0x46ea5e)

Hope it helps a little

Rexios80 · 2024-05-31T23:02:09Z

On Flutter stable you need to run flutter build web --wasm --no-strip-wasm or else the traces will be minified

Rexios80 · 2024-05-31T23:26:24Z

The hive code has no call to Platform.isAndroid so that first exception is either an issue in your code or another dependency

I'm not sure what the second exception is about. Probably just another exception caused by the key that's infinity or NaN. Maybe try this:

/// Not part of public API
@visibleForTesting
Future<List<Object?>> getKeys({bool cursor = false}) async {
  var store = getStore(false);

  if (store.has('getAllKeys') && !cursor) {
    final result = await getStore(false).getAllKeys(null).asFuture();
    final keys = <Object?>[];
    for (final key in (result as JSArray).toDart) {
      if (key is JSNumber) {
        final keyDart = key.toDartDouble;
        if (keyDart.isFinite) {
          keys.add(keyDart.toInt());
        } else {
          print('Box key is not finite: $key');
        }
      } else if (key is JSString) {
        keys.add(key.toDart);
      }
    }
    return keys;
  } else {
    final cursors = await store.getCursors();
    return cursors.map((e) => e.key).toList();
  }
}

What I'm afraid of is that you will be missing data due to keys not getting read correctly or something. Let me know how many times that print statement gets called.

theniceboy · 2024-05-31T23:56:25Z

Thanks! I tried the flutter build web --wasm --no-strip-wasm command on stable. The Platform._operatingSystem exception is gone, and the Infinity or NaN toInt exception is also gone. Now the only exception I got was this:

JavaScriptError
main.dart.mjs:56     at FirebaseCoreWeb.initializeApp inner (http://localhost:5066/main.dart.wasm:wasm-function[43670]:0x7d6183)
    at _awaitHelperWithTypeCheck closure at org-dartlang-sdk:///dart-sdk/lib/_internal/wasm/lib/async_patch.dart:97:16 (http://localhost:5066/main.dart.wasm:wasm-function[2425]:0x37ecac)
    at closure wrapper at org-dartlang-sdk:///dart-sdk/lib/_internal/wasm/lib/async_patch.dart:97:16 trampoline (http://localhost:5066/main.dart.wasm:wasm-function[2431]:0x37edad)
    at _RootZone.runUnary (http://localhost:5066/main.dart.wasm:wasm-function[2467]:0x37f8f3)
    at _Future._propagateToListeners (http://localhost:5066/main.dart.wasm:wasm-function[2448]:0x37f4e1)
    at _Future._completeWithValue (http://localhost:5066/main.dart.wasm:wasm-function[2482]:0x37fb93)
    at _Future._asyncCompleteWithValue closure at org-dartlang-sdk:///dart-sdk/lib/async/future_impl.dart:721:29 (http://localhost:5066/main.dart.wasm:wasm-function[2809]:0x384e68)
    at closure wrapper at org-dartlang-sdk:///dart-sdk/lib/async/future_impl.dart:721:29 trampoline (http://localhost:5066/main.dart.wasm:wasm-function[2811]:0x384e7f)

Seemed like there's a bug in the Dart compiler: firebase/flutterfire#12623

It's fixed in the master channel, however I get this error again when I switch to the master channel:

Unsupported operation: Platform._operatingSystem
main.dart.mjs:56     at Error._throwWithCurrentStackTrace (http://localhost:5066/main.dart.wasm:wasm-function[1420]:0x2f5162)
    at Platform.operatingSystem (http://localhost:5066/main.dart.wasm:wasm-function[3200]:0x31f85a)
    at Platform.isWindows (http://localhost:5066/main.dart.wasm:wasm-function[4264]:0x347016)
    at Spotlight (http://localhost:5066/main.dart.wasm:wasm-function[8152]:0x3abef6)
    at SentryFlutter.init inner (http://localhost:5066/main.dart.wasm:wasm-function[43777]:0x7680b7)
    at main inner.1 (http://localhost:5066/main.dart.wasm:wasm-function[9200]:0x3cce3b)
    at _awaitHelperWithTypeCheck closure at org-dartlang-sdk:///dart-sdk/lib/_internal/wasm/lib/async_patch.dart:97:16 (http://localhost:5066/main.dart.wasm:wasm-function[2428]:0x31003e)
    at closure wrapper at org-dartlang-sdk:///dart-sdk/lib/_internal/wasm/lib/async_patch.dart:97:16 trampoline (http://localhost:5066/main.dart.wasm:wasm-function[2434]:0x31013f)

This time it's sentry. I don't think either is related to Hive, but at the moment I'm unable to test out this branch with my app unfortunately.

theniceboy · 2024-06-01T00:00:48Z

Filed this issue: getsentry/sentry-dart#2082

(Not that it's related to Hive. Just for tracking sake)

Rexios80 · 2024-06-01T00:35:06Z

Can you take out sentry for now and see if the app runs?

Also did you have to edit the getKeys method?

theniceboy · 2024-06-01T00:40:59Z

Yes, I updated the getKeys method with the code you provided.

I got rid of Sentry. Now this is the error I'm getting:

JavaScriptError
main.dart.mjs:56     at StorageBackendJs.initialize inner (http://localhost:5066/main.dart.wasm:wasm-function[42752]:0x73513c)
    at _awaitHelperWithTypeCheck closure at org-dartlang-sdk:///dart-sdk/lib/_internal/wasm/lib/async_patch.dart:97:16 (http://localhost:5066/main.dart.wasm:wasm-function[2415]:0x300643)
    at closure wrapper at org-dartlang-sdk:///dart-sdk/lib/_internal/wasm/lib/async_patch.dart:97:16 trampoline (http://localhost:5066/main.dart.wasm:wasm-function[2421]:0x300736)
    at _RootZone.runUnary (http://localhost:5066/main.dart.wasm:wasm-function[2457]:0x301294)
    at _Future._propagateToListeners (http://localhost:5066/main.dart.wasm:wasm-function[2438]:0x300e66)
    at _Future._completeWithValue (http://localhost:5066/main.dart.wasm:wasm-function[2472]:0x301533)
    at _Future._asyncCompleteWithValue closure at org-dartlang-sdk:///dart-sdk/lib/async/future_impl.dart:735:29 (http://localhost:5066/main.dart.wasm:wasm-function[2784]:0x30642d)
    at closure wrapper at org-dartlang-sdk:///dart-sdk/lib/async/future_impl.dart:735:29 trampoline (http://localhost:5066/main.dart.wasm:wasm-function[2786]:0x306444)

Rexios80 · 2024-06-01T01:05:18Z

Can you try/catch your openBox calls and print the error/stacktrace?

theniceboy · 2024-06-01T02:10:47Z

They're already in a try-catch block. Are you sure the error's from the openBox calls?

Rexios80 · 2024-06-02T23:23:52Z

@1l0 I wrote tests for BoxCollection and they pass so the original code you showed me should work now

Rexios80 · 2024-06-03T01:20:03Z

I did a bunch of cleanup. I also refactored the code that required dart 3.4.0, but the web package requires dart 3.3.0 anyways so not really much of an accomplishment.

1l0 · 2024-06-03T02:44:00Z

I have confirmed that it just works ™️. Errors are gone. I don't know why but with build web --wasm Chrome would chose js as a fallback instead of wasm though. flutter/flutter#142822 might be related.

theniceboy · 2024-06-03T03:34:37Z

Just tried, it worked!

Rexios80 · 2024-06-03T11:11:51Z

@1l0 https://docs.flutter.dev/platform-integration/web/wasm#serve-the-output-with-an-http-server

1l0 · 2024-06-03T13:51:11Z

@1l0 https://docs.flutter.dev/platform-integration/web/wasm#serve-the-output-with-an-http-server

Thanks, I overlooked that information. The problem seems to be resolved.

Rexios80 · 2024-06-18T20:44:57Z

@theniceboy @1l0

I have released the following packages to replace Hive in my projects. The intent is to keep these up to date with modern Dart/Flutter standards. I can't promise fixing any issues with Hive itself unless you create PRs.

hive_ce
hive_ce_flutter
hive_ce_generator

Please feel free to create any issues with those packages in their repo: https://github.com/IO-Design-Team/hive_ce

theniceboy · 2024-06-18T20:52:07Z

@Rexios80 You are awesome! starred

Rexios80 added 14 commits May 29, 2024 11:21

Convert backend_manager.dart

4eb0dbc

Refactor backend_manager.dart

80e08a1

Migrate storage_backend_js.dart

8c06b3a

Migrate box_collection_indexed_db.dart

9000568

Migrate tests

c85611b

Update conditional imports

1f9dd04

Fix function definitions

60028ec

Fixing type issues

bb357d5

Fix test

498b83a

Fixing type issues

24adc6a

Fixing things

523ae80

More fixes

7914d19

More fixes

b8e8092

Update generator

456dab3

Rexios80 changed the title ~~Support Flutter web WASM in Hive v2~~ Migrate Hive v2 to support Flutter web WASM May 29, 2024

This was referenced May 31, 2024

Is Hive being abandoned? No more updates yet? #1290

Open

A new chapter: Hive v4 #1225

Open

Rexios80 added 5 commits June 2, 2024 17:01

Refactor asFuture() usages to be typed

d33391c

Use window.self.indexedDB everywhere

9ea3255

Regenerate

5db58f5

Update action to test both web compilers

0ef4d98

Writing BoxCollection tests

5f347b4

Rexios80 added 4 commits June 2, 2024 19:58

Fix cursor issues

2a9944e

Cleanup

ebdef6c

Refactor cursor iteration back into a stream

d98a9a5

Fixing CI

8df4953

Rexios80 added 12 commits June 10, 2024 08:39

CI should pass

339c347

Maybe this time

2797358

Update sdk constraint in all projects

690d174

Fix mixin issues

41be86c

Update all actions dependencies

bc99e26

Fixing more actions issues

6380501

...

f4334da

...

d169f5c

Require Dart 3.4.0

a424eb4

...

b257e9d

...

0e8751e

...

b3fda79

LukasMirbt mentioned this pull request Aug 16, 2024

feat(hydrated_bloc): support wasm felangel/bloc#4230

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Migrate Hive v2 to support Flutter web WASM #1291

Migrate Hive v2 to support Flutter web WASM #1291

Rexios80 commented May 29, 2024 •

edited

Loading

theniceboy commented May 31, 2024

theniceboy commented May 31, 2024

Rexios80 commented May 31, 2024

Rexios80 commented May 31, 2024 •

edited

Loading

theniceboy commented May 31, 2024

Rexios80 commented May 31, 2024

Rexios80 commented May 31, 2024

theniceboy commented May 31, 2024 •

edited

Loading

theniceboy commented Jun 1, 2024 •

edited

Loading

Rexios80 commented Jun 1, 2024 •

edited

Loading

theniceboy commented Jun 1, 2024

Rexios80 commented Jun 1, 2024

theniceboy commented Jun 1, 2024

Rexios80 commented Jun 2, 2024

Rexios80 commented Jun 3, 2024

1l0 commented Jun 3, 2024

theniceboy commented Jun 3, 2024

Rexios80 commented Jun 3, 2024

1l0 commented Jun 3, 2024

Rexios80 commented Jun 18, 2024

theniceboy commented Jun 18, 2024

Migrate Hive v2 to support Flutter web WASM #1291

Are you sure you want to change the base?

Migrate Hive v2 to support Flutter web WASM #1291

Conversation

Rexios80 commented May 29, 2024 • edited Loading

IF YOU WANT TO USE THIS NOW

Related issues

theniceboy commented May 31, 2024

theniceboy commented May 31, 2024

Rexios80 commented May 31, 2024

Rexios80 commented May 31, 2024 • edited Loading

theniceboy commented May 31, 2024

Rexios80 commented May 31, 2024

Rexios80 commented May 31, 2024

theniceboy commented May 31, 2024 • edited Loading

theniceboy commented Jun 1, 2024 • edited Loading

Rexios80 commented Jun 1, 2024 • edited Loading

theniceboy commented Jun 1, 2024

Rexios80 commented Jun 1, 2024

theniceboy commented Jun 1, 2024

Rexios80 commented Jun 2, 2024

Rexios80 commented Jun 3, 2024

1l0 commented Jun 3, 2024

theniceboy commented Jun 3, 2024

Rexios80 commented Jun 3, 2024

1l0 commented Jun 3, 2024

Rexios80 commented Jun 18, 2024

theniceboy commented Jun 18, 2024

Rexios80 commented May 29, 2024 •

edited

Loading

Rexios80 commented May 31, 2024 •

edited

Loading

theniceboy commented May 31, 2024 •

edited

Loading

theniceboy commented Jun 1, 2024 •

edited

Loading

Rexios80 commented Jun 1, 2024 •

edited

Loading