Unable to use isolates with EncryptedMoor

[cadaniel]

So I've tried two attempts

With what's given on the example docs

Future<MoorIsolate> _createMoorIsolate(String dbFile, String password) async {
  // this method is called from the main isolate. Since we can't use
  // getApplicationDocumentsDirectory on a background isolate, we calculate
  // the database path in the foreground isolate and then inform the
  // background isolate about the path.
  final receivePort = ReceivePort();

  await Isolate.spawn(
    _startBackground,
    _IsolateStartRequest(receivePort.sendPort, dbFile, password),
  );

  // _startBackground will send the MoorIsolate to this ReceivePort
  return (await receivePort.first as MoorIsolate);
}

void _startBackground(_IsolateStartRequest request) {
  // this is the entrypoint from the background isolate! Let's create
  // the database from the path we received
  final executor = VmDatabase(File(request.targetPath));
  //final executor = EncryptedExecutor(path: request.targetPath, password: request.password, logStatements: true);
  // we're using MoorIsolate.inCurrent here as this method already runs on a
  // background isolate. If we used MoorIsolate.spawn, a third isolate would be
  // started which is not what we want!
  final moorIsolate = MoorIsolate.inCurrent(
        () => DatabaseConnection.fromExecutor(executor),
  );
  // inform the starting isolate about this, so that it can call .connect()
  request.sendMoorIsolate.send(moorIsolate);
}

Future<void> _initBillingSpecialties(Map<String, dynamic> jsonData) async {
  var dbConnection = await jsonData["isolate"].connect();
  var db = BillingDatabase.connect(dbConnection);
  var json = jsonDecode(jsonData["specialites"]) as Map<String, dynamic>;
  var jsonSpecialties = json["specialties"] as List<dynamic>;
  var specialities = jsonSpecialties.map((s) =>
      DbSpecialtiesCompanion(name: Value(s["specialty_name"]),
          mohNumber: Value(s["moh_specialty"]))).toList();
  return db.specialtyDao.saveAllSpecialties(specialities);
}

// used to bundle the SendPort and the target path, since isolate entrypoint
// functions can only take one parameter.
class _IsolateStartRequest {
  final SendPort sendMoorIsolate;
  final String targetPath;
  final String password;

  _IsolateStartRequest(this.sendMoorIsolate, this.targetPath, this.password);
}

@Singleton(dependsOn: [DatabaseData])
class InitDbService {
  final DatabaseData databaseData;

  InitDbService(this.databaseData);

  Future<void> initDatabase(String dbFile, String password) async {
    var moorIsolate = await _createMoorIsolate(dbFile, password);
    var specialties = await rootBundle.loadString("lib/assets/billing_specialties.json");
    compute(_initBillingSpecialties, {"isolate": moorIsolate, "specialites":specialties});
  }

} 

gives me the error

[VERBOSE-2:dart_isolate.cc(915)] Unhandled exception:
Bad state: No element
#0      Iterable.first (dart:core/iterable.dart:520:7)
#1      Database.userVersion (package:moor_ffi/src/impl/database.dart:150:19)
#2      _VmVersionDelegate.schemaVersion (package:moor_ffi/src/vm_database.dart:102:58)
#3      DelegatedDatabase._runMigrations (package:moor/src/runtime/executor/helpers/engines.dart:266:42)
#4      DelegatedDatabase.ensureOpen.<anonymous closure> (package:moor/src/runtime/executor/helpers/engines.dart:247:13)
<asynchronous suspension>
#5      BasicLock.synchronized (package:synchronized/src/basic_lock.dart:32:26)
#6      DelegatedDatabase.ensureOpen (package:moor/src/runtime/executor/helpers/engines.dart:238:25)
#7      _MoorServer._runEnsureOpen (package:moor/src/runtime/isolate/server.dart:93:34)
#8      _MoorServer._handleRequest (package:moor/src/runtime/isolate/server.dart:58:18)
#9      new _MoorServer.<anonymous closure>.<anonymous closure> (package:moor/src/runtime/isolate/server.dart<…> 

When I use the other executor final executor = EncryptedExecutor(path: request.targetPath, password: request.password, logStatements: true);

gives me the error

[VERBOSE-2:dart_isolate.cc(915)] Unhandled exception:
ServicesBinding.defaultBinaryMessenger was accessed before the binding was initialized.
If you're running an application and need to access the binary messenger before `runApp()` has been called (for example, during plugin initialization), then you need to explicitly call the `WidgetsFlutterBinding.ensureInitialized()` first.
If you're running a test, you can call the `TestWidgetsFlutterBinding.ensureInitialized()` as the first line in your test's `main()` method to initialize the binding.
#0      defaultBinaryMessenger.<anonymous closure> (package:flutter/src/services/binary_messenger.dart:76:7)
#1      defaultBinaryMessenger (package:flutter/src/services/binary_messenger.dart:89:4)
#2      MethodChannel.binaryMessenger (package:flutter/src/services/platform_channel.dart:140:62)
#3      MethodChannel.invokeMethod (package:flutter/src/services/platform_channel.dart:314:35) 

If I use an unencrypted database the first executor works just fine.

[simolus3]

The EncryptedExecutor uses platform channels to work, which in my understanding aren't available on background isolates yet (flutter/flutter#13937).

Just to be sure, the database you tried to connect to via a VmDatabase was encrypted in the first snippet? It's not possible to open encrypted databases with moor_ffi, which just wraps the regular sqlite3 library. Maybe we could have a version of moor_ffi that binds to precompiled sqlcipher binaries.

[cadaniel]

Yes, the errors in both cases were created using EncryptedExecutor( password: _password, path: _dbFile, logStatements: true)

I'm working on medical-based applications so I do need encryption. If you're able to point me in the right direction I'm happy to help contribue.

[davidmartos96]

@cadaniel I am using SQLCipher in my app also, and to use moor_ffi this is what I have done.

Some things to note: This has been tested on Flutter desktop, haven't tried on Android/iOS. It should work, but you need to provide the SQLCipher libraries yourself.
Also this is without using isolate. I don't know if the overrideForAll would work in a moor isolate setup.

In your main.dart

import 'package:moor_ffi/open_helper.dart';
void main(){
    open.overrideForAll(sqlcipherOpen);
}

DynamicLibrary sqlcipherOpen() {
  if (Platform.isLinux || Platform.isAndroid) {
    return DynamicLibrary.open('libsqlcipher.so');
  }
  if (Platform.isMacOS || Platform.isIOS) {
    return DynamicLibrary.open('/usr/lib/libsqlcipher.dylib');
  }
  if (Platform.isWindows) {
    DynamicLibrary.open('libsqlcipher.dll');
  }

  throw UnsupportedError(
      'moor_ffi does not support ${Platform.operatingSystem} yet');
}

class MyDatabase extends _$MyDatabase {
  MyDatabase(Directory directory)
      : super(
          VmDatabaseEncrypted(
            File(dbPath),
            password: "password",
            logStatements: true,
          ),
        );
}

// sqlcipher_moor_ffi.dart
import 'dart:async';

import 'dart:io';

import 'package:moor/backends.dart';
import 'package:moor/moor.dart';
import 'package:moor_ffi/moor_ffi.dart';

class VmDatabaseEncrypted extends DelegatedDatabase {
  /// Creates a database that will store its result in the [file], creating it
  /// if it doesn't exist.
  factory VmDatabaseEncrypted(
    File file, {
    String password = '',
    bool logStatements = false,
  }) {
    final vmDatabase = VmDatabase(file, logStatements: logStatements);
    return VmDatabaseEncrypted._(vmDatabase, password);
  }

  factory VmDatabaseEncrypted.memory({
    String password = '',
    bool logStatements = false,
  }) {
    final vmDatabase = VmDatabase.memory(logStatements: logStatements);
    return VmDatabaseEncrypted._(vmDatabase, password);
  }

  VmDatabaseEncrypted._(
    VmDatabase vmDatabase,
    this.password,
  ) : super(
          _VmEncryptedDelegate(vmDatabase.delegate, password),
          logStatements: vmDatabase.logStatements,
          isSequential: vmDatabase.isSequential,
        );

  final String password;
}

class _VmEncryptedDelegate extends DatabaseDelegate {
  final String password;
  final DatabaseDelegate delegate;

  _VmEncryptedDelegate(
    this.delegate,
    this.password,
  );

  @override
  Future<void> open([GeneratedDatabase db]) async {
    await delegate.open(db);
    await delegate.runCustom('PRAGMA KEY = "$password"', <dynamic>[]);
    return Future.value();
  }

  @override
  FutureOr<bool> get isOpen => delegate.isOpen;

  @override
  Future<void> runBatched(List<BatchedStatement> statements) {
    return delegate.runBatched(statements);
  }

  @override
  Future<void> runCustom(String statement, List args) {
    return delegate.runCustom(statement, args);
  }

  @override
  Future<int> runInsert(String statement, List args) {
    return delegate.runInsert(statement, args);
  }

  @override
  Future<QueryResult> runSelect(String statement, List args) {
    return delegate.runSelect(statement, args);
  }

  @override
  Future<int> runUpdate(String statement, List args) {
    return delegate.runUpdate(statement, args);
  }

  @override
  TransactionDelegate get transactionDelegate => delegate.transactionDelegate;

  @override
  DbVersionDelegate get versionDelegate => delegate.versionDelegate;
}

[cadaniel]

Thanks @davidmartos96

I have a fork at https://github.com/cadaniel/moor that I'm starting to take a look at this.

I took your solution however I'm getting "/usr/lib/libsqlcipher.dylib" doesn't exist. Running on both iOS simulator and on a physical device.

Seems to work perfectly on Android though.

[davidmartos96]

@cadaniel As I said, you would need to bundle the SQLCipher libraries in each platform yourself. I don't have much experience on how to do that on iOS.

[vanlooverenkoen]

Thanks @davidmartos96

I have a fork at https://github.com/cadaniel/moor that I'm starting to take a look at this.

I took your solution however I'm getting "/usr/lib/libsqlcipher.dylib" doesn't exist. Running on both iOS simulator and on a physical device.

Seems to work perfectly on Android though.

How did you fix the iOS implementation? Can't figure out how to encrypt my iOS database

[iampopal]

@vanlooverenkoen How did you fix iOS issue, in my emulator and physical device the libsqlcipher also not exists

[vanlooverenkoen]

@iampopal we did this:
#1810 (comment)

And fully read this: #1810

We only had the issue in release mode