Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Datastore operations fails after internet fluctuations #13035

Open
3 tasks done
Akash-T2S opened this issue Feb 21, 2024 · 7 comments
Open
3 tasks done

Datastore operations fails after internet fluctuations #13035

Akash-T2S opened this issue Feb 21, 2024 · 7 comments
Assignees
Labels
DataStore Related to DataStore category question General question React Native React Native related issue V5

Comments

@Akash-T2S
Copy link

Before opening, please confirm:

JavaScript Framework

React Native

Amplify APIs

DataStore

Amplify Version

v5

Amplify Categories

No response

Backend

None

Environment information

# Put output below this line

# Put output below this line
System:
    OS: macOS 13.3.1
    CPU: (8) arm64 Apple M1
    Memory: 47.81 MB / 16.00 GB
    Shell: 5.9 - /bin/zsh
  Binaries:
    Node: 14.21.3 - /opt/homebrew/opt/node@14/bin/node
    npm: 6.14.18 - /opt/homebrew/opt/node@14/bin/npm
    Watchman: 2023.08.07.00 - /opt/homebrew/bin/watchman
  Browsers:
    Chrome: 119.0.6045.105
    Safari: 16.4
  npmPackages:
    @aws-amplify/datastore-storage-adapter: 2.0.43 => 2.0.43 
    @azure/core-asynciterator-polyfill: 1.0.2 => 1.0.2 
    @babel/core: 7.12.9 => 7.12.9 (7.23.2)
    @babel/preset-typescript: 7.22.5 => 7.22.5 (7.23.2)
    @babel/runtime: 7.12.5 => 7.12.5 (7.23.2)
    @react-native-async-storage/async-storage: 1.13.2 => 1.13.2 (1.19.4)
    @react-native-community/art: 1.2.0 => 1.2.0 
    @react-native-community/datetimepicker: 6.7.0 => 6.7.0 
    @react-native-community/eslint-config: 2.0.0 => 2.0.0 
    @react-native-community/masked-view: 0.1.11 => 0.1.11 
    @react-native-community/netinfo: 9.3.10 => 9.3.10 
    @react-native-community/progress-view: 1.3.2 => 1.3.2 
    @react-native-community/push-notification-ios: 1.10.1 => 1.10.1 
    @react-native-firebase/analytics: 17.5.0 => 17.5.0 
    @react-native-firebase/app: 17.5.0 => 17.5.0 
    @react-native-firebase/crashlytics: 17.5.0 => 17.5.0 
    @react-native-firebase/firestore: 17.5.0 => 17.5.0 
    @react-native-firebase/messaging: 17.5.0 => 17.5.0 
    @react-native-google-signin/google-signin: 8.2.1 => 8.2.1 
    @react-native-picker/picker: 2.4.8 => 2.4.8 
    @react-navigation/core: 3.7.9 => 3.7.9 
    @react-navigation/native: 3.8.4 => 3.8.4 
    @react-navigation/web: 1.0.0-alpha.9 => 1.0.0-alpha.9 
    @svgr/webpack: 5.5.0 => 5.5.0 
    @tsconfig/react-native: 2.0.3 => 2.0.3 
    @types/jest: 29.2.1 => 29.2.1 
    @types/react: 18.0.0 => 18.0.0 (18.2.37)
    @types/react-test-renderer: 18.0.0 => 18.0.0 
    HelloWorld:  0.0.1 
    ReactWebConfigExample:  1.0.0 
    amazon-cognito-identity-js: 6.3.1 => 6.3.1 
    amazon-cognito-identity-js/internals:  undefined ()
    appcenter: 4.4.5 => 4.4.5 
    appcenter-analytics: 4.4.5 => 4.4.5 
    appcenter-crashes: 4.4.5 => 4.4.5 
    aws-amplify: 5.3.6 => 5.3.6 
    aws-amplify-react-native: 7.0.2 => 7.0.2 
    aws-sdk: 2.1431.0 => 2.1431.0 
    axios: 1.4.0 => 1.4.0 (0.26.0, 0.21.4)
    babel-jest: 26.6.3 => 26.6.3 
    babel-loader: 8.2.2 => 8.2.2 
    babel-plugin-react-native-web: 0.17.1 => 0.17.1 
    base-64: 1.0.0 => 1.0.0 (0.1.0)
    compression-webpack-plugin: 10.0.0 => 10.0.0 
    core-js: 3.32.0 => 3.32.0 (2.6.12, 1.2.7)
    country-from-iso2: 1.0.1 => 1.0.1 
    css-loader: 6.4.0 => 6.4.0 
    dotenv-webpack: 7.0.3 => 7.0.3 
    enzyme: 3.11.0 => 3.11.0 
    enzyme-adapter-react-16: 1.15.5 => 1.15.5 
    eslint: 7.32.0 => 7.32.0 
    eslint-config-prettier: 7.0.0 => 7.0.0 (6.15.0)
    eslint-plugin-jest: 24.1.3 => 24.1.3 (22.4.1)
    eslint-plugin-prettier: 3.2.0 => 3.2.0 (3.1.2)
    eslint-plugin-react: 7.21.5 => 7.21.5 
    eslint-plugin-react-hooks: 4.3.0 => 4.3.0 
    file-loader: 6.2.0 => 6.2.0 
    file-saver: 2.0.5 => 2.0.5 
    firebase: 9.22.0 => 9.22.0 
    firebase/analytics:  undefined ()
    firebase/app:  undefined ()
    firebase/app-check:  undefined ()
    firebase/auth:  undefined ()
    firebase/auth/cordova:  undefined ()
    firebase/auth/react-native:  undefined ()
    firebase/compat:  undefined ()
    firebase/compat/analytics:  undefined ()
    firebase/compat/app:  undefined ()
    firebase/compat/app-check:  undefined ()
    firebase/compat/auth:  undefined ()
    firebase/compat/database:  undefined ()
    firebase/compat/firestore:  undefined ()
    firebase/compat/functions:  undefined ()
    firebase/compat/installations:  undefined ()
    firebase/compat/messaging:  undefined ()
    firebase/compat/performance:  undefined ()
    firebase/compat/remote-config:  undefined ()
    firebase/compat/storage:  undefined ()
    firebase/database:  undefined ()
    firebase/firestore:  undefined ()
    firebase/firestore/lite:  undefined ()
    firebase/functions:  undefined ()
    firebase/installations:  undefined ()
    firebase/messaging:  undefined ()
    firebase/messaging/sw:  undefined ()
    firebase/performance:  undefined ()
    firebase/remote-config:  undefined ()
    firebase/storage:  undefined ()
    geo-point-in-polygon: 1.0.0 => 1.0.0 
    hello-world:  0.0.1 
    hermes-inspector-msggen:  1.0.0 
    html-webpack-plugin: 5.3.2 => 5.3.2 
    husky: 4.3.8 => 4.3.8 
    ignite-cli: 8.4.2 => 8.4.2 
    inline-style-prefixer: 6.0.1 => 6.0.1 
    jest: 26.6.3 => 26.6.3 
    jest-enzyme: 7.1.2 => 7.1.2 
    jetifier: 1.6.8 => 1.6.8 
    libphonenumber-js: 1.10.14 => 1.10.14 
    libphonenumber-js/build:  undefined ()
    libphonenumber-js/core:  undefined ()
    libphonenumber-js/max:  undefined ()
    libphonenumber-js/max/metadata:  undefined ()
    libphonenumber-js/min:  undefined ()
    libphonenumber-js/min/metadata:  undefined ()
    libphonenumber-js/mobile:  undefined ()
    libphonenumber-js/mobile/examples:  undefined ()
    libphonenumber-js/mobile/metadata:  undefined ()
    lint-staged: 10.5.3 => 10.5.3 
    localforage: 1.10.0 => 1.10.0 
    lodash: 4.17.21 => 4.17.21 
    lottie-ios: 3.4.1 => 3.4.1 (3.1.8)
    lottie-react-native: 5.1.4 => 5.1.4 (4.1.3)
    md5: 2.3.0 => 2.3.0 
    metro-react-native-babel-preset: 0.70.3 => 0.70.3 (0.66.2)
    mockdate: 3.0.5 => 3.0.5 
    moment: 2.29.4 => 2.29.4 
    moment-timezone: 0.5.39 => 0.5.39 
    msw: 0.35.0 => 0.35.0 
    pako: 1.0.11 => 1.0.11 (2.0.4)
    patch-package: 6.5.0 => 6.5.0 
    performance-now: 2.1.0 => 2.1.0 
    postman-collection: 4.1.0 => 4.1.0 
    prettier: 2.8.0 => 2.8.0 
    promise.allsettled: 1.0.7 => 1.0.7 
    prop-types: 15.7.2 => 15.7.2 (15.8.1, 15.5.8)
    qs: 6.11.2 => 6.11.2 (6.11.0, 6.5.3)
    query-string: 7.0.1 => 7.0.1 (6.14.1, 6.10.1)
    react: 17.0.2 => 17.0.2 
    react-datetime: 3.2.0 => 3.2.0 
    react-devtools: 4.26.1 => 4.26.1 
    react-dom: 17.0.2 => 17.0.2 
    react-easy-crop: 4.6.2 => 4.6.2 
    react-icomoon: 2.5.4 => 2.5.4 
    react-localization: 1.0.19 => 1.0.19 
    react-native: 0.67.5 => 0.67.5 
    react-native-calendars: 1.1291.1 => 1.1291.1 
    react-native-camera: 4.2.1 => 4.2.1 
    react-native-circular-progress: 1.3.7 => 1.3.7 
    react-native-code-push: 7.0.5 => 7.0.5 
    react-native-config: 1.4.11 => 1.4.11 
    react-native-contacts: 7.0.5 => 7.0.5 
    react-native-country-picker-modal: 2.0.0 => 2.0.0 
    react-native-device-info: 10.3.0 => 10.3.0 
    react-native-document-picker: 8.1.3 => 8.1.3 
    react-native-draggable-flatlist: 2.6.2 => 2.6.2 
    react-native-element-dropdown: 2.4.0 => 2.4.0 
    react-native-exit-app: 1.1.0 => 1.1.0 
    react-native-external-display: 0.5.18 => 0.5.18 
    react-native-fast-image: 8.6.3 => 8.6.3 
    react-native-fs: 2.20.0 => 2.20.0 
    react-native-gesture-handler: 1.10.3 => 1.10.3 
    react-native-get-random-values: 1.9.0 => 1.9.0 
    react-native-image-crop-picker: 0.38.1 => 0.38.1 
    react-native-image-picker: 4.10.1 => 4.10.1 
    react-native-keyboard-aware-scroll-view: 0.9.5 => 0.9.5 
    react-native-largelist: 3.1.0-rc.2 => 3.1.0-rc.2 
    react-native-linear-gradient: 2.6.2 => 2.6.2 
    react-native-loader: 1.3.1 => 1.3.1 
    react-native-maps: 1.2.0 => 1.2.0 
    react-native-modal: 13.0.1 => 13.0.1 
    react-native-modal-datetime-picker: 14.0.0 => 14.0.0 
    react-native-permissions: 3.6.1 => 3.6.1 (2.2.2)
    react-native-phone-input: 1.3.5 => 1.3.5 
    react-native-popup-menu: 0.16.1 => 0.16.1 
    react-native-print: 0.10.0 => 0.10.0 
    react-native-progress: 5.0.0 => 5.0.0 
    react-native-push-notification: 8.1.1 => 8.1.1 
    react-native-qrcode-scanner: 1.5.5 => 1.5.5 
    react-native-qrcode-svg: 6.1.2 => undefined (6.1.2, )
    react-native-reanimated: 1.13.4 => 1.13.4 
    react-native-render-html: 6.3.4 => 6.3.4 
    react-native-restart: 0.0.24 => 0.0.24 
    react-native-safe-area-context: 3.3.2 => 3.3.2 
    react-native-screens: 2.16.1 => 2.16.1 
    react-native-select-contact: 1.6.3 => 1.6.3 
    react-native-send-intent: 1.3.0 => 1.3.0 
    react-native-sha1: 1.2.3 => 1.2.3 
    react-native-simple-download-manager: 1.4.1 => 1.4.1 
    react-native-sound: 0.11.2 => 0.11.2 
    react-native-splash-screen: 3.3.0 => 3.3.0 
    react-native-spring-scrollview: 3.0.1-rc.5 => 3.0.1-rc.5 
    react-native-sqlite-storage: 6.0.1 => 6.0.1 
    react-native-svg: 13.6.0 => 13.6.0 
    react-native-swipe-list-view: 3.2.5 => 3.2.5 
    react-native-swiper: 1.6.0 => 1.6.0 
    react-native-switch-selector: 2.2.1 => 2.2.1 
    react-native-tab-view: 2.15.2 => 2.15.2 
    react-native-vector-icons: 9.2.0 => 9.2.0 
    react-native-version-number: 0.3.6 => 0.3.6 
    react-native-walkthrough-tooltip: 1.4.0 => 1.4.0 
    react-native-web: 0.17.1 => 0.17.1 
    react-native-web-linear-gradient: 1.1.2 => 1.1.2 
    react-native-web-lottie: 1.4.4 => 1.4.4 
    react-native-web-maps: 0.3.0 => 0.3.0 
    react-native-web-swiper: 2.2.4 => 2.2.4 
    react-native-web-webview: 1.0.2 => 1.0.2 
    react-native-webview: 11.24.0 => 11.24.0 
    react-native-zohosalesiq-mobilisten: 6.0.0 => 6.0.0 
    react-navigation: 4.4.4 => 4.4.4 
    react-navigation-drawer: 2.7.1 => 2.7.1 
    react-navigation-hooks: 1.1.0 => 1.1.0 
    react-navigation-stack: 1.8.1 => 1.8.1 
    react-navigation-tabs: 2.7.0 => 2.7.0 
    react-redux: 7.2.8 => 7.2.8 
    react-test-renderer: 18.0.0 => 18.0.0 (16.14.0)
    reactotron-react-native: 5.0.3 => 5.0.3 
    reactotron-redux: 3.1.3 => 3.1.3 
    redux: 4.2.0 => 4.2.0 
    redux-persist: 6.0.0 => 6.0.0 
    redux-persist-filesystem-storage: 3.0.0 => 3.0.0 
    redux-persist/integration/react:  undefined ()
    redux-saga: 1.2.2 => 1.2.2 
    redux-saga-tester: 1.0.874 => 1.0.874 
    redux-saga/effects:  undefined ()
    rn-fetch-blob: 0.12.0 => 0.12.0 
    rn-placeholder: 3.0.3 => 3.0.3 
    style-loader: 3.3.0 => 3.3.0 
    ts-loader: 9.4.4 => 9.4.4 
    typescript: 4.9.5 => 4.9.5 
    universal-cookie: 4.0.4 => 4.0.4 
    url-loader: 4.1.1 => 4.1.1 
    use-file-picker: 1.5.1 => 1.5.1 
    uuid: 9.0.0 => 9.0.0 (3.4.0, 8.3.2, 8.0.0, 7.0.3)
  npmGlobalPackages:
    corepack: 0.15.1
    n: 9.1.0
    npm: 6.14.18



Describe the bug

I have been using Amplify Datastore for the past 2 years and after the update to version 5.x.x I've encountered an issue with intermittent internet connectivity. Whenever there are fluctuations in the network, the networkStatus of the HubListener first reports false and then true. After a full or delta sync, any Datastore action fails and throws errors, rendering Datastore unusable until I refresh the app.

The issue is depend on the next action after the internet fluctuations.

Issue: DataStoreStateError: Tried to execute DataStore.query()/DataStore.Start() while DataStore was "Stopping".
This can only be done while DataStore is "Started" or "Stopped". To remedy:
Ensure all calls to stop() and clear() have completed first.
If this is not possible, retry the operation until it succeeds

Once internet is back usual delta sync too fails at times. This can be reproduced in both Android & iOS devices

Expected behavior

Datastore operations should perform even after internet fluctuations and should not affect the usage of it.

Reproduction steps

  1. Sync models
  2. Once AppSync event is ready
  3. Pull out the internet cable if Wifi is connected and put back
  4. Repeat step 3 twice or thrice
  5. Turn off the Wifi and Turn on back
  6. Connect back the internet
  7. Wait for delta sync to happen
  8. Delta sync fails and other operations will also fail

Code Snippet

// Put your code below this line.
DataStore.configure({
                authProviders: {
                    functionAuthProvider: async () => {
                        return {
                            token: licenseKey
                        };
                    }
                },
                syncExpressions: getSyncExpression(storeId, settingsTimeZone),
                syncPageSize: 1000,
                maxRecordsToSync: 100000,
                fullSyncInterval: FULL_SYNC_INTERVAL,
                storageAdapter: SQLiteAdapter,
                errorHandler: (error) => {
                  
                }
            });

Log output

// Put your logs below this line


aws-exports.js

No response

Manual configuration

No response

Additional configuration

No response

Mobile Device

No response

Mobile Operating System

No response

Mobile Browser

No response

Mobile Browser Version

No response

Additional information and screenshots

No response

@Akash-T2S Akash-T2S added the pending-triage Issue is pending triage label Feb 21, 2024
@chrisbonifacio chrisbonifacio added the DataStore Related to DataStore category label Feb 21, 2024
@david-mcafee
Copy link
Contributor

Hi @Akash-T2S! I have a few follow-up questions to help us better understand the issue:

  1. Are you upgrading from V4 to V5? If so, have you already taken a look at the migration steps in the README?
  2. It looks like you're using Amplify version 5.3.6 - have you attempted to upgrade to the latest version of V5 (5.3.16)?
  3. Have you attempted to upgrade to the latest version of @aws-amplify/datastore-storage-adapter (2.1.17)?
  4. Are you calling any of the DataStore lifecycle functions (i.e. start, stop, or clear)? If so, could you provide a code snippet demonstrating how, and where, you are calling them?

Thank you!

@david-mcafee david-mcafee self-assigned this Feb 23, 2024
@Akash-T2S
Copy link
Author

Akash-T2S commented Feb 23, 2024

@david-mcafee Thanks for these steps,

Step 1 - has been followed thoroughly and implemented in the same way.
Step 2 & 3 - I shall update to the mentioned version and get back.
Step 4 - DataStore lifecycle functions are triggered as below

DS Configure:

const configureDataStore = () => {
    DataStore.configure({
        authProviders: {
            functionAuthProvider: async () => {
                return {
                    token: licenseKey
                };
            }
        },
        syncExpressions: getSyncExpression(storeId, settingsTimeZone),
        syncPageSize: 1000,
        maxRecordsToSync: 100000,
        fullSyncInterval: FULL_SYNC_INTERVAL,
        storageAdapter: SQLiteAdapter,
        errorHandler: (error) => {}
    });
}

DS Life cycle:

const clearDataStore = async () => {
    await DataStore.clear();
};
const stopDataStore = async () => {
    await DataStore.stop();
};
const startDataStore = async () => {
    await DataStore.start();
};

DS Initialisation:

useEffect(() => {
    configureDataStore();
    startDataStore();
    if (!hubListener) {
        hubListener = Hub.listen('datastore', async (hubData) => {
            if (event === 'ready') {
                const order = await DataStore.query(Order, (c) =>
                  c.and((c) => [
                      c.storeId.eq(storeId),
                      c.deliveryAt.ge(getGraterThanDatesForSyncExpression(settingsTimeZone))
                  ])
                );
            }

        }
    }
    return () => {
        removeOrderSubscription();
        removePrinterObjSubscription();
        if (isValidElement(hubListener)) {
            hubListener();
            hubListener = null;
        }
        if (isValidElement(pubsubListner)) {
            pubsubListner();
            pubsubListner = null;
        }
    };
}, []);

@nadetastic nadetastic added the investigating This issue is being investigated label Feb 23, 2024
@nadetastic nadetastic added V5 React Native React Native related issue labels Mar 7, 2024
@cwomack
Copy link
Member

cwomack commented Mar 7, 2024

@Akash-T2S, can you share any relevant code where you're calling DataStore.clear() and DataStore.stop (or based on your code snippets for the DS Lifecycles, clearDataStore and stopDataStore respectively?

@cwomack cwomack added pending-response question General question and removed investigating This issue is being investigated pending-triage Issue is pending triage labels Mar 7, 2024
@Akash-T2S
Copy link
Author

Akash-T2S commented Mar 12, 2024

@cwomack clearDataStore will be called only on logout. Whereas stopDataStore will be called when there is any KeepAliveTimeOut happens (KA acknowledgement is missed) through 'api' channel.

Hub.listen('api', async (data) => {
    const { payload } = data;
    if (payload?.event === CONNECTION_STATE_CHANGE) {
        if (payload?.data?.connectionState === 'ConnectedPendingKeepAlive') {
            await DataStore.stop();
            await DataStore.start();
        }
    }
}

@david-mcafee
Copy link
Contributor

@Akash-T2S - it looks like the failure is occurring when you attempt to perform a DataStore operation (e.g. DataStore.query) while DataStore is still "stopping". To avoid this, you can wait for DataStore to be in a "ready" state (see this section of our docs) before attempting to perform an operation.

Additionally, since DataStore works offline, there shouldn't be a need to stop / start DataStore on connection state changes. Can you help us understand your usecase for stopping / starting DataStore on connection state changes?

Lastly, have you attempted to upgrade to the versions listed in this comment?

@Akash-T2S
Copy link
Author

@david-mcafee I agree with your point, I make query’s and other operations only when AppSync ready event trigger(after all model synced). Also, I'm trying to stop and start only when the pending keep alive is missed and not when the transition of network status. Even when I don't have that block of code the same issue happens. This reproducible steps are tricky but tbh it occurs more often in the latest version.

Regarding the version update, No luck on that.

@david-mcafee
Copy link
Contributor

@Akash-T2S - thank you for your response! To confirm, are you saying that the issue is still present even when you completely disable all calls to DataStore.stop() in your app? If you could provide us with more detailed reproduction steps and/or context with regards to how and where you are calling DataStore operations and lifecycle methods, that would help us better understand the issue!

Lastly, you mentioned that you had no luck on updating the Amplify version - can you help us understand the issue so we can assist with unblocking you? Thank you!!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
DataStore Related to DataStore category question General question React Native React Native related issue V5
Projects
None yet
Development

No branches or pull requests

5 participants