Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

RFC: Support Datastore for Web and Desktop #3180

Open
abdallahshaban557 opened this issue Jun 12, 2023 · 55 comments
Open

RFC: Support Datastore for Web and Desktop #3180

abdallahshaban557 opened this issue Jun 12, 2023 · 55 comments
Labels
datastore Issues related to the DataStore Category feature-request A request for a new feature or an enhancement to an existing API or category. pending-maintainer-response Pending response from a maintainer of this repository

Comments

@abdallahshaban557
Copy link
Contributor

abdallahshaban557 commented Jun 12, 2023

We are creating this issue to get insights and feedback into our plans for supporting offline first and caching experiences with our API offerings when targeting Web and Desktop. We want to use this issue to collect feedback from you on the use cases you want to use Offline first and caching for, so that we can validate the path forward for us as we explore it further.

Please help us with providing us with the exact use cases you want to use Datastore for on Web and Desktop, and how it would benefit your users. For example:

  • Cache data retrieved from API calls.
  • Create a faster first-time loading experience by grabbing the latest retrieved data to users.
  • Allow users to browse the app even if their network disconnects by using the latest retrieved data.
  • Automatically re-connect and retrieve data updates when the internet connection is re-established.
@abdallahshaban557 abdallahshaban557 added feature-request A request for a new feature or an enhancement to an existing API or category. datastore Issues related to the DataStore Category labels Jun 12, 2023
@jamontesg
Copy link

Are you kidding, are you just going to start the project?

@abdallahshaban557
Copy link
Contributor Author

Hi @jamontesg - We have created this feature request to make sure we capture all feedback on Datastore specifically, since our other categories have shipped for Web and Desktop support.

@dnys1 dnys1 pinned this issue Jun 13, 2023
@dorontal
Copy link

Why we think Datastore + Web would be great for our (music collaboration) app? Here are a couple of more points to add to the bullet points at the top of this issue (which are also all benefits that we see):

  1. Offline first apps are accessible by more people on earth: we want a musicians living in a remote village with no Internet to be able to walk into an Internet cafe, download some tracks, then go back to their village and record along with those tracks with no Internet, until they are ready to upload the recorded result back, say a month later when they return to the Internet cafe. Datastore can make it much easier to create an offline-first app due to its DB caching.

  2. Apps running in the browser bypass the judgment / vetting of Apple or Google's app stores. When your app runs in the browser, there is less risk that one of those two companies would decline your app. There is less overhead from not having to wait for the approval process.

  3. We hope Datastore will become fully optimized (e.g, DB updates that send partial data with only the fields that have changed rather than the entire model instance), saving us the time of having to do those optimizations ourselves.

  4. We hope Datastore can help our client apps make fewer network requests, in general, by use of extensive caching - this will make it less costly to have any app up and running on AWS - it lowers the barrier to entry for any app publisher because it lowers the overhead costs of doing business because the app's users consume less cloud resources.

@abdallahshaban557
Copy link
Contributor Author

Thank you @dorontal for this feedback!

@dkliss
Copy link

dkliss commented Jun 27, 2023

On device storage is important for all above use cases mentioned. In context of a cross-platform application, if all platforms cannot support a unified offline storage system it will become a challenge to remotely sync data to AWS from desktop or web devices. Offline first also ensures that in areas with limited bandwidth, data sync is scheduled at off-peak or user defined times to ensure user experience stays positive.

@loint
Copy link

loint commented Jul 8, 2023

@abdallahshaban557
Chat is the core feature for almost enterprise web and apps so we can have a better experiences and consistency (multi-platform app) in single implementation - our user is able to read message even their connections are interrupted.
We're waiting for this support for a long time because hard to maintain both web and app with same chat data synchronization.
Local storage is also benefit us in case we want to store all in local first and sync to server later to have a better experiences without depends direct on a real time connection.

@dgagnon
Copy link

dgagnon commented Jul 8, 2023

My notes:

  1. Make sure to include a fully working example of datastore + authenticator UI + cognito auth.
  2. The datastore documentation needs to be explicit about which feature from appsync/graphQL are not supported, i.e. multiple owners. At the moment, the datastore doc only contains part of the information needed to manage the graphql models, so users are directly to the API(graphql) doc which very often will not work with the datastore. Alternatively, keeping the datastore to feature parity with appsync would help greatly.
  3. Please make sure in include performance tests in your integration suite: above 20 models 10s of thousands of records, everything because really slow, when it even works. (think an online store with different kind of speciality products).
  4. The datastore starts on configure, which creates exception when not logged in ( since you need to configure amplify to do the authentication. This creates confusions for developers. Unless a public API category is configured. datastore should not start until auth happened. Adding a flag to delay starting datastore would work as well.

As for the uses cases: it is, and always has been, offline support. This is the only reason we are using AWS. We built three version of our app so far and have had to completely removed datastore, which breaks the only advantage amplify has over the competition.

I will update my message as new ideas comes in from the team.

@jamilsaadeh97
Copy link
Contributor

I think that offline first capabilities is best when it comes to querying and syncing data to the cloud. I'm using DataStore extensively and I must say, it's great but, at the same time, lacking in some areas like starting time, ability to handle a lot of models and data effectively, the differences with GraphQL API like multi owner....

My use case is a reservations application and offline first is the way to go since it offers a better user experience while everything is syncing to the cloud in the background. Sadly I can't have my app in the Web since DataStore is not implemented there.

For the past week I've been testing the possibility to migrate everything to GraphQL API and most probably I will, but I saw that DataStore makes everything easier, especially observeQuery() so that's why I like it but in my opinion it needs to be better in order to be production-ready for applications that offers services.

If the support of Web and Desktop means that DataStore will be rewritten in Dart, and, as result, will become more effective in iOS and Android, then Amplify Flutter would be the best out there.

@YUzushio
Copy link

YUzushio commented Aug 4, 2023

I have high expectations for DataStore to operate on the Web. The characteristic of building apps with Flutter is its operation across all platforms.

While it's possible to implement with the GraphQL API, the usability of DataStore, which can be used on iOS and Android, is exceptionally excellent.

I am considering creating an app with Amplify Flutter, but the fact that it doesn't support the Web is causing significant issues for my business plans.

I can't propose whether to go online first or not, but having it implemented and executable before any discussion of cost and search efficiency is the most significant benefit for me. It would also be good news for many others who have given up on implementing Amplify Flutter.

By supporting the web, DataStore will undoubtedly make AmplifyFlutter the best solution in the world.

@Heilum
Copy link

Heilum commented Aug 8, 2023

We need amplify_datastore on macOS/Windows very much. please implement them officially. thanks in advance!

@CaptainDario
Copy link

@abdallahshaban557, is this being worked on?
I am planning an offline first sync feature and would like to know if this is just a proposal or if it is actively being worked on.

@jamontesg
Copy link

@abdallahshaban557 any dates ?

@thomasklaush
Copy link

Would be good, to get an approximate timeline for the feature.
I think a mix of Native App and Web for the Backend is a common topic.
Since Datastore and API don't work in parallel due to the Conflict Detection, this feature would be awesome for web as well!
I will go online soon with the web portal and looking forward to using it.

@abdallahshaban557
Copy link
Contributor Author

abdallahshaban557 commented Oct 6, 2023

Hi @CaptainDario - to be fully transparent, it is at a stage or two beyond a proposal right now, and are experimenting on how to consistently achieve the required feature-set across all the platforms we support.

@jamontesg and @thomasklaush We do not have an exact timelines yet, since we have not aligned on the best implementation path yet. We will continue to provide updates here when we have a clearer path forward.

@alterhuman
Copy link

Any update on this?

@nickordoodle
Copy link

nickordoodle commented Nov 27, 2023

From a Flutter perspective, the specific features are less relevant and the actual support as soon as possible for Web and Desktop is way more important. Whatever gets this to production fastest I think is the answer.

If DataStore becomes supported for remaining platforms, Flutter developers can confidently build cross-platform solely relying on Amplify's APIs.

I have faced roadblock after roadblock due to lack of platform support although it seems web support overall has increased drastically since inception.

I would also like to hear an update.

ex-Amazonian

@fabriquetonvoyage
Copy link

fabriquetonvoyage commented Nov 30, 2023

Hi @abdallahshaban557,

👍👍👍 Allow users to browse the app even if their network disconnects by using the latest retrieved data.
👍👍 Automatically re-connect and retrieve data updates when the internet connection is re-established.

It would be great for my "Travel Roadbook App" to have datastore support for web. Indeed, when the connection is lost (often during a trip), the users need to get all information about there travel and day by day schedule.

So, do you plan any release of this feature soon or do we have to forget it and build a custom code for web support ? ... It's so frustrating !

Thanks.

PS : More and more users don't want to install one more native app...

@abdallahshaban557
Copy link
Contributor Author

Hi @fabriquetonvoyage - I am no longer part of the Amplify Flutter team, but @haverchuck can help provide more insights into the future of the library!

@thomasklaush
Copy link

@haverchuck please some information.
We go online soon.
Would be nice, to know the way and if I have to go with the problematic workaround in production.

@jamontesg
Copy link

Hi @haverchuck , any future for this project?

@BrandowBuenos
Copy link

BrandowBuenos commented Dec 14, 2023

Hello, everyone. I have an app in which Amplify has been working extremely well. However, I need to develop the web version and I just wanted to port the existing app to the web, but the datastore is not supported yet.

Is there any way to use it nonetheless? If not, is there a timeline for adding this support to the datastore library?

app: www.budgi.it

@thomasklaush
Copy link

@BrandowBuenos I used for web the API library.
But now I have two solutions and no data stored in the web version.
But I can use the same GraphQl interface and Data.
Only the conflict resolution makes issues, because Datastore uses Versioning.
If data is changed from Web, this makes issues.

@BrandowBuenos
Copy link

@BrandowBuenos

final request = ModelQueries.list(
        EquipmentEntry.classType,
        where: queryPredicate,
      );
      final response = await Amplify.API.query(request: request).response;

Don't forget to go through all pages of the result, this was an issue when I startet.

PaginatedResult<EquipmentEntry>? result = response.data;
bool hasNextResult = result!.hasNextResult;

while (hasNextResult == true) {
          logger.w("Next");
          final nextRequest = result.requestForNextResult;
          final secondResult = await Amplify.API.query(request: nextRequest!).response;
.
.
.
}

Thanks @thomasklaush!

@ZoroLH
Copy link

ZoroLH commented Jan 26, 2024

Our app is running on the boat with weak internet connectiong. Offline ability is necessary for us. Our data is on cloud firestore now and we are planning to integrate aws iot. Datastore can make our framework simple.

We are using web app for product, Macos for development would help.

@thomasklaush
Copy link

Still no timeline?

An Easy way would be, to make the field version and deleted accessible for APIGraphQL.
This would make it possible to work with both.
@haverchuck any possibility for that?

Currently, I have this workflow:
I update the backend by amplify push.
Since I use Datastore, I have to enable Conflict Resolution.

This breaks GraphQL API, since I cannot read and write version field and cannot read deleted field.
I have to manually deactivate Conflict Resolution in the console for 10 functions.

If I want to use a custom Conflict Resolution via Lambda, amplify push reports an error on the backend update, so this unfortunately does not work.

Additionally, I take the risk for Data corruption.
If I get one wrong Entry, Datastore will not load any of the Table.

I will get in Production in April, and hopefully I can find a robust solution until this date.

@jamontesg
Copy link

This request is abandoned, the person who started it no longer works on the team and the person in charge shows no signs of life.
I have started to implement the web datastore, for my own.
Does anyone have an approach to the topic of how to work without deactivating conflict resolution and being able to remain in sync with the mobile part ?

Regards

@fabriquetonvoyage
Copy link

fabriquetonvoyage commented Feb 18, 2024

It's just crazy that AWS give up this feature in Amplify ...
How can we (the community of users) raise this point to the Amplify Team ? Maybe when they do their "Amplify Office Hours" meetings on discord ?

@BrandowBuenos
Copy link

It would be a great help to provide support without the offline caching part.

Perhaps even using secure storage for simplified storage.

@dgagnon
Copy link

dgagnon commented Feb 18, 2024

It would be a great help to provide support without the offline caching part.

Perhaps even using secure storage for simplified storage.

datastore without offline data storage is not datastore, it's appsync. I am pretty sure you can use appsync already on web.

@thomasklaush
Copy link

@dgagnon you can do it, but you cannot use Datastore on Mobile and AppSync on Web combined.
Main issue is, that with Amplify generated fields for Datastore, you cannot access via AppSync the fields Version and Deleted.
This leads to a situation, where you cannot update and delete elements from web.

@fabriquetonvoyage
Copy link

Hi @haverchuck , any news or insights about this feature ? Thanks.

@BrandowBuenos
Copy link

Hey everyone,

I wanted to provide an update on recent changes made to our app and web platform. Specifically, I've completely removed the datastore and implemented a custom synchronization system from scratch.

Previously, we relied on the datastore for data storage and synchronization, but it often proved to be unstable, failing to sync data reliably. In response to this issue, I've developed a new approach that offers more robust synchronization capabilities and compatibility with web.

Here's a breakdown of what I've implemented:

Datastore Removal: The existing datastore functionality has been entirely removed from both the app and web platforms.
Custom Sync System: I've developed a custom synchronization system to replace the datastore. This system operates as follows:

  • Data is saved locally using FlutterSecureStorage, ensuring secure storage on the device and web compatibility.
  • Whenever an item is edited, it is added to a synchronization queue.
  • Synchronization occurs immediately upon editing. If successful, the changes are propagated immediately. If not, the system retries synchronization every 30 seconds.

@fabriquetonvoyage
Copy link

Really nice Brandow ... It could be great if Amplify team could add this feature to the framework...
Do you publish the code in github or gitlab ? Thank you so much

@jamontesg
Copy link

Thanks for the news @BrandowBuenos , we are very excited about this functionality.
We are ready for help you with testing

Regards

@thomasklaush
Copy link

I assume since Datastore will not be available in Gen2, this feature will be deprecated.
When is the migration tool available?
There is no concrete info
https://docs.amplify.aws/flutter/start/migrate-to-gen2/

@ZoroLH
Copy link

ZoroLH commented Jun 28, 2024

Is there anyway we can support offline case with Gen2?

@jamontesg
Copy link

Hi @BrandowBuenos , any news for web platform ?

@BrandowBuenos
Copy link

Hey everyone!

Unfortunately, I had to remove the Amplify Datastore due to all the errors and the lack of support for the web version. So, I created my own implementation. I'm now using SecureStorage to store data locally.

For synchronization, I compare the _version fields of each item and always keep the one with the latest version. Additionally, I set up a listener that checks every 15 seconds for any unsynchronized items and syncs them automatically.

@fabriquetonvoyage
Copy link

You right in amplify docs that "you'll offer a migration guide soon" for Datastore.

Could someone from Amplify dev team respond us ? @abdallahshaban557 @haverchuck

@jamontesg
Copy link

Hi @fabriquetonvoyage , any idea about to get some web functionality without wait aws-flutter team for ever ?

@fabriquetonvoyage
Copy link

Hi @jamontesg, I think we need to abandon the Datastore and switch to GraphQL API ... That's why I'm waiting for migration guide ...

@thomasklaush
Copy link

@Jordan-Nelson can you give any insight, if there is a timeline for this migration guide?

@BrandowBuenos
Copy link

BrandowBuenos commented Jul 31, 2024

Really nice Brandow ... It could be great if Amplify team could add this feature to the framework... Do you publish the code in github or gitlab ? Thank you so much

Hey @fabriquetonvoyage! Sorry for the delay. I was improving and adding support for IndexedDB on the web for handling large volumes of data. I wanted to share the solution I developed as an alternative to DataStore. You can find it at the following link: https://gist.github.com/BrandowBuenos/934db9e4be683561d9285af7803c4863

@jamontesg
Copy link

Hi @BrandowBuenos your code don't works for me. You can adjust this code for seems more to datastore , for use 'models/ModelProvider.dart' and How to use model1Controller.model1List ??

Sorry ,sorry. and Kind Regards

@stMerlHin
Copy link

Is there anyway we can support offline case with Gen2?

+1

@github-actions github-actions bot added the pending-maintainer-response Pending response from a maintainer of this repository label Nov 28, 2024
@thomasklaush
Copy link

I made my own solution with sembast.
But is some work for all the synchronization

@jamontesg
Copy link

Hi @thomasklaush, could you share with us your approach ?
Kind Regards

@thomasklaush
Copy link

Hi @jamontesg sure can I, but it is a quite simple solution with own classes and files for each type.
And I implemented sync just from the cloud currently, because the other way is not important for me.
I will make it a little bit more beautiful and share it with you

@thomasklaush
Copy link

Basiscally I Seperated my Repos in two different ones.
The Repository is the DB and the Remote Repository is the AWS Cloud.

I added a last changed timestamp which I use in the Remote Repo, to fetch only new events.
The last synced timestamp is stored in the DB and updated after a sucessfull sync.

I Use Bloc, so the Logic is done in the Cubit.

Would be really cool to make it generic and more dense, but currently I have no time for it.

DB Setup:

import 'dart:io';

import 'package:amplify_flutter/amplify_flutter.dart';
import 'package:injectable/injectable.dart';
import 'package:logger/logger.dart';
import 'package:path/path.dart';
import 'package:path_provider/path_provider.dart';
import 'package:sembast/sembast_io.dart';

Logger logger = Logger();

@lazySingleton
class AppDatabase {
  Database? _database;

  Future<Database> get database async {
    if (_database == null) {
      await _openDatabase();
    }
    return _database!;
  }

  // Configure sembast as Singleton
  Future _openDatabase() async {
    // get the application documents directory
    final dir = await getApplicationDocumentsDirectory();
    // make sure it exists
    await dir.create(recursive: true);
    final dbPath = join(dir.path, 'database_app.db');
    // open the database
    _database = await databaseFactoryIo.openDatabase(dbPath);
  }

  Future<bool> isUserLoggedIn() async {
    try {
      // Fetch the current session
      final session = await Amplify.Auth.fetchAuthSession();

      // Check if the user is signed in
      return session.isSignedIn;
    } catch (e) {
      //logger.i('Error checking if user is signed in: $e');
      return false; // If there's an error, treat as not signed in
    }
  }

  Future<void> clearDatabaseOnLogout() async {
    // Close the current database (make sure it's not in use)
    await _database?.close();

    // Get the path to the database file
    final dir = await getApplicationDocumentsDirectory();
    String dbPath = '${dir.path}/kubikos_app.db';

    // Delete the database file
    File dbFile = File(dbPath);
    if (await dbFile.exists()) {
      await dbFile.delete();
      logger.w('Database deleted successfully.');
    } else {
      logger.e('Database file not found.');
    }
  }
}

Example Data Class:

import 'package:injectable/injectable.dart';
import 'package:kubikos/core_lib/model/exercise/exercise.dart';
import 'package:kubikos/core_lib/model/exercise/i_exercise_data_remote_repository.dart';
import 'package:kubikos/core_lib/model/exercise/i_exercise_data_repository.dart';
import 'package:kubikos/features/database/app_database.dart';
import 'package:logger/logger.dart';
import 'package:sembast/sembast.dart';

Logger logger = Logger();

@LazySingleton(as: IExerciseDataRepository)
class ExerciseDatabase implements IExerciseDataRepository {
  ExerciseDatabase(
    this.db,
    this.exerciseDataRepository,
  ) {
    // init the database
    initDB();
  }

  bool initDone = false;

  final AppDatabase db;
  final IExerciseDataRemoteRepository exerciseDataRepository;
  final exerciseStore = stringMapStoreFactory.store('exercise');

  @override
  Future<bool> initDB() async {
    //await clearDB();
    // check if user is logged in and wait for it
    while (!(await db.isUserLoggedIn())) {
      await Future.delayed(const Duration(seconds: 3));
    }
    // check if the DB is empty
    try {
      final records = await exerciseStore.find(await db.database);
      if (records.isEmpty) {
        logger.i('ActivityDB is empty, sync activities from the cloud');
        await restoreDBfromCloud();
      }
    } catch (e) {
      logger.e('Error fetching exercises: $e');
      await clearDB();
      await restoreDBfromCloud();
      return false;
    }
    //removeDuplicates();
    initDone = true;
    return true;
  }

  Future<List<Exercise>> restoreDBfromCloud() async {
    try {
      final List<Exercise> exercises = await exerciseDataRepository.getAvailableExercises();
      final List<Exercise> preprocessedExercises = preprocesseExercises (exercises);
      preprocessedExercises .forEach((exercise) async {
        // Add the activity to the DB if it is not already in there with the same activityId
        exerciseStore.record(exercise.exerciseId).put(await db.database, exercise.toJson(), merge: true);
      });
      return exercises;
    } catch (e) {
      logger.e('Error fetching activities: $e');
    }
    return [];
  }

  Future<bool> clearDB() async {
    await exerciseStore.delete(await db.database);
    return false;
  }

  /// Get an exercise from the local DB
  ///
  /// Returns an exercise
  /// Returns an empty exercise if an error occurred
  ///
  /// The function will wait for the DB to be initialized before it fetches the exercise
  /// If the exercise is not found, it will return an empty exercise
  @override
  Future<Exercise> getExerciseData(String exerciseId) async {
    // Wait for the DB to be initialized
    while (!initDone) {
      await Future.delayed(const Duration(seconds: 1));
    }
    final result = await exerciseStore.record(exerciseId).getSnapshot(await db.database);
    if (result != null) {
      // Log the result
      // logger.i('ExerciseDB from DB ${result.value}');
      final exercise = Exercise.fromJson(result.value);
      return exercise;
    } else {
      //logger.i('ExerciseDB $exerciseId not found in DB');
      return const Exercise.empty();
    }
  }

  /// Get all exercises from the local DB
  /// Returns a list of exercises
  /// Returns an empty list if an error occurred
  @override
  Future<List<Exercise>> getAvailableExercises() async {
    // Wait for the DB to be initialized
    while (!initDone) {
      await Future.delayed(const Duration(seconds: 1));
    }
    logger.d('Get available exercises');
    try {
      final records = await exerciseStore.find(await db.database);
      if (records.isNotEmpty) {
        logger.d('Get available exercises1');
        return Future.wait(records.map((record) async {
          final exercise = Exercise.fromJson(record.value);
          return exercise;
        }).toList());
      } else {
        return restoreDBfromCloud();
      }
    } catch (e) {
      logger.e('Error fetching activities: $e');
      return [];
    }
  }

  /// Emit a stream of exercises from the local DB
  /// Returns a stream of exercises
  ///
  /// The function will emit a stream of activities from the local DB
  /// and update the stream whenever the DB changes
  /// Returns a stream of exercises
  /// Returns an empty stream if an error occurred
  Stream<List<Exercise>> getAvailableExercisesStream() async* {
    yield* exerciseStore.query().onSnapshots(await db.database).asyncMap((event) async {
      return await Future.wait(event.map((snapshot) async {
        final exercise = Exercise.fromJson(snapshot.value);
        return exercise;
      }).toList());
    });
  }

  /// Sync the activities from the API to the local DB
  /// Returns true if new activities were found
  /// Returns false if no new activities were found
  /// Returns false if an error occurred
  ///
  /// lastSyncTime: the last time the activities were synced
  ///
  /// The function will fetch all activities from the API that were created or updated after lastSyncTime
  /// and add them to the local DB
  @override
  Future<bool> syncExercises(int lastSyncTime) async {
    //logger.d("DateTime ${DateTime.fromMillisecondsSinceEpoch(lastSyncTime)}");
    try {
      List<Exercise> exercises = await exerciseDataRepository.getAvailableExercises(lastSyncTime: lastSyncTime);
      if (exercises.isEmpty) {
        logger.i('No new Exercise found, no Sync needed');
        return false;
      }
      exercises = preprocesseExercises(exercises);
      exercises.forEach((exercise) async {
        // Check each activity if it is already in the DB by id and only add it if it is not
        // if it's already in the DB, we need to update it
        exerciseStore.record(exercise.exerciseId).put(await db.database, exercise.toJson(), merge: true);
      });
      logger.i('Synced ${exercises.length} Exercises');
    } catch (e) {
      logger.e('Error fetching Exercises: $e');
    }
    return true;
  }

  List<Exercise> someDataprocessing(List<Exercise> exercises) {
    exercises = exercises.map((exercise) {
     ....}
    return exercises;
  }
}

Sync Manager Class:

import 'package:injectable/injectable.dart';
import 'package:kubikos/core_lib/model/equipment/i_equipment_data_repository.dart';
import 'package:kubikos/core_lib/model/exercise/i_exercise_data_repository.dart';
import 'package:kubikos/features/activity_management/domain/i_activity_repository.dart';
import 'package:kubikos/features/database/app_database.dart';
import 'package:kubikos/features/force_test/model/i_force_test_repository.dart';
import 'package:kubikos/features/machine_management/model/i_machine_management_repository.dart';
import 'package:kubikos/features/status_data/domain/i_status_data_repository.dart';
import 'package:logger/logger.dart';
import 'package:sembast/sembast.dart';

Logger logger = Logger();

@injectable
class SyncManager {
  SyncManager(
    this._appDatabase,
    this._activityDatabase,
    this._profileDataDatabase,
    this._exerciseDatabase,
    this._userEquipmentDatabase,
    this._equipmentDatabase,
    this._forceTestRepository,
  );

  final AppDatabase _appDatabase;
  final IActivityRepository _activityDatabase;
  final IStatusDataRepository _profileDataDatabase;
  final IExerciseDataRepository _exerciseDatabase;
  final IMachineManagementRepository _userEquipmentDatabase;
  final IEquipmentDataRepository _equipmentDatabase;
  final IForceTestRepository _forceTestRepository;

  // Sync the database
  Future<void> syncDatabase() async {
    logger.d("Syncing database");
    var store = StoreRef.main();
    final lastSyncTime = await store.record('lastSyncTime').get(await _appDatabase.database) as int?;
    // Save the last sync time in unix time, has to be done prior to the call to syncAllDatabases
    await store.record('lastSyncTime').put(await _appDatabase.database, DateTime.now().millisecondsSinceEpoch);
    // log the last sync time
    logger.d("Last Sync Time: ${lastSyncTime == null ? 'null' : DateTime.fromMillisecondsSinceEpoch(lastSyncTime)}");
    // Sync the database
    /* final val = */ syncAllDatabases(lastSyncTime ?? 0);
    return;
  }

  /* Stream<Future<bool>> syncDatabasePeriodically(int intervalSeconds) async* {
    yield* Stream<Future<bool>>.periodic(Duration(seconds: intervalSeconds), (_) async {
      logger.d("Syncing database");
      var store = StoreRef.main();
      final lastSyncTime = await store.record('lastSyncTime').get(await _appDatabase.database) as int?;
      // Save the last sync time in unix time, has to be done prior to the call to syncAllDatabases
      await store.record('lastSyncTime').put(await _appDatabase.database, DateTime.now().millisecondsSinceEpoch);
      // log the last sync time
      logger.d("Last Sync Time: ${lastSyncTime == null ? 'null' : DateTime.fromMillisecondsSinceEpoch(lastSyncTime)}");
      // Sync the database
      final val = syncAllDatabases(lastSyncTime ?? 0);
      return val;
    });
  } */

  /// Sync all databases.
  /// Calls the sync function of all databases asynchronously
  Future<bool> syncAllDatabases(int lastSyncTime) async {
    // Sync some databases before others
    await _profileDataDatabase.syncProfileData(lastSyncTime);
    // Sync the exercise database
    await _exerciseDatabase.syncExercises(lastSyncTime);

    // Sync the profile data database
    await _userEquipmentDatabase.syncUserEquipments(lastSyncTime);
    // Sync all other databases simultaneously
    Future.wait([
      // Sync the user equipment database
      _userEquipmentDatabase.syncEquipmentActivities(lastSyncTime),
      // Sync the equipment database
      _equipmentDatabase.syncEquipments(lastSyncTime),
      // Sync the activity database
      _activityDatabase.syncActivities(lastSyncTime),
      // Sync the force test database
      _forceTestRepository.syncForceTests(lastSyncTime),
    ]);
    return false;
  }

  // Clear the database
  Future<void> clearDB() async {
    await _appDatabase.clearDatabaseOnLogout();
  }
}

Cubit:

import 'dart:async';

import 'package:bloc/bloc.dart';
import 'package:freezed_annotation/freezed_annotation.dart';
import 'package:injectable/injectable.dart';
import 'package:kubikos/features/database/sync_manager.dart';

part 'sync_manager_cubit.freezed.dart';
part 'sync_manager_state.dart';

@Singleton()
class SyncManagerCubit extends Cubit<SyncManagerState> {
  SyncManagerCubit(
    this._syncManager,
  ) : super(const SyncManagerState.initial()) {
    syncNow();
  }

  final SyncManager _syncManager;
  StreamSubscription? _syncSubscription;

  void syncNow() {
    logger.d("Syncing database");
    emit(const SyncManagerState.synchronizing());
    _syncManager.syncDatabase().then((value) {
      emit(const SyncManagerState.synced());
    });
  }

  void stopSync() {
    _syncSubscription?.cancel();
  }

  Future<void> deleteDatabase() async {
    await _syncManager.clearDB();
  }
}

@jamontesg
Copy link

jamontesg commented Dec 9, 2024

@thomasklaush my friend thanks

@thomasklaush
Copy link

@jamontesg if you transform it to a generic approach, I would be happy if you can share it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
datastore Issues related to the DataStore Category feature-request A request for a new feature or an enhancement to an existing API or category. pending-maintainer-response Pending response from a maintainer of this repository
Projects
None yet
Development

No branches or pull requests