-
Notifications
You must be signed in to change notification settings - Fork 199
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Old data not visible for some users #3738
Comments
Please see attached verbose log file - https://drive.google.com/file/d/12c5dvjpmoHZIYtNEkdGePfmvpd5SDwV4/view?usp=sharing |
From the attached amplify-issue.zip, I can see the sync expression being used let syncExpressions = [
DataStoreSyncExpression.syncExpression(UserProfile2.schema, where: {
UserProfile2.keys.user_id.eq(userId)
}),
DataStoreSyncExpression.syncExpression(ExerciseSession2.schema, where: {
ExerciseSession2.keys.user_id.eq(userId)
}),
DataStoreSyncExpression.syncExpression(TimedData.schema, where: {
if let dateLimit = dateLimit {
TimedData.keys.user_id.eq(userId).and(TimedData.keys.time_stamp.gt(dateLimit.temporalDateTime))
} else {
TimedData.keys.user_id.eq(userId)
}
}),
DataStoreSyncExpression.syncExpression(MenstrualLog.schema, where: {
if let dateLimit = dateLimit {
MenstrualLog.keys.user_id.eq(userId).and(MenstrualLog.keys.time_stamp.gt(dateLimit.temporalDateTime))
} else {
MenstrualLog.keys.user_id.eq(userId)
}
}),
DataStoreSyncExpression.syncExpression(MobileToken2.schema, where: {
MobileToken2.keys.user_id.eq(userId)
}),
DataStoreSyncExpression.syncExpression(SleepSession2.schema, where: {
if let dateLimit = dateLimit {
SleepSession2.keys.user_id.eq(userId).and(SleepSession2.keys.sleep_window_end.gt(dateLimit.temporalDateTime))
} else {
SleepSession2.keys.user_id.eq(userId)
}
}),
DataStoreSyncExpression.syncExpression(SleepStage.schema, where: {
if let dateLimit = dateLimit {
SleepStage.keys.user_id.eq(userId).and(SleepStage.keys.data_timestamp.gt(dateLimit.temporalDateTime))
} else {
SleepStage.keys.user_id.eq(userId)
}
}),
DataStoreSyncExpression.syncExpression(SpotCheck.schema, where: {
if let dateLimit = dateLimit {
SpotCheck.keys.user_id.eq(userId).and(SpotCheck.keys.data_timestamp.gt(dateLimit.temporalDateTime))
} else {
SpotCheck.keys.user_id.eq(userId)
}
}),
DataStoreSyncExpression.syncExpression(UserDailyMedians2.schema, where: {
if let dateLimit = dateLimit {
UserDailyMedians2.keys.user_id.eq(userId).and(UserDailyMedians2.keys.time_stamp.gt(dateLimit.temporalDateTime))
} else {
UserDailyMedians2.keys.user_id.eq(userId)
}
}),
DataStoreSyncExpression.syncExpression(UserGoals.schema, where: {
UserGoals.keys.user_id.eq(userId)
}),
DataStoreSyncExpression.syncExpression(UserPreference.schema, where: {
UserPreference.keys.user_id.eq(userId)
}),
DataStoreSyncExpression.syncExpression(DeviceTelemetry.schema, where: {
if dateLimit != nil {
DeviceTelemetry.keys.user_id.eq(userId).and(DeviceTelemetry.keys.time_stamp.ge((Date() - .hours(2)).temporalDateTime))
} else {
DeviceTelemetry.keys.user_id.eq(userId)
}
})
]
try Amplify.add(plugin: AWSDataStorePlugin(
modelRegistration: DataStoreModels(),
configuration: .custom(errorHandler: onError, syncMaxRecords: maxLimit, syncExpressions: syncExpressions)
)) The data to be synchronized is filtered down to the userId or when DataStore performs a The synchronization is going to be drastically improved if we applied the syncExpression with the sync query call on the server side. This is pending proiritization of this PR #3550 . Some challenges we faced with getting the server side filter released is due to supporting the edge case of supporting customers that do not have AppSync backends that support server side filter (as that feature was launched after the client side filter implementation was put in place). The problems you see may be solved if your AppSync backend is provisioned with There could be a bug in the client side evaluation such that given a dateLimit and the response model, it is incorrectly discarding it. The verbose file is over 100M, could you narrow down the logs to just the output from the result of calling Also could you share code snippet of how and what the |
can you give us the full code of the extensions used by this line inside .now.advanced(by: -7, byAdding: .day).startOfDay |
@lawmicha this is the code used by extension - public func advanced( |
@lawmicha for datelimit, we had set it while login, you can check the sessionmanager class attached in zip file, inside signin function you will get a call to startAndPopulateDataStore with the dateLimit |
The verbose log you provided is too extensive for us to investigate. Could you please narrow it down to the specific issue where fetching 1 week data is successful, but fetching single day data is not? Additionally, in the verbose log, there should be generated GraphQL queries. Could you verify whether the single-day query, which is supposed to retrieve the data, is actually fetching the data in the AWS AppSync console? |
Hi @5d |
Hi @amruth-movano, I checked the code and I misspoke earlier, it is passing the syncExpression as the filter input for sync query. It is applying the syncExpression client side only on the subscription events. As @5d mentioned, can you add Can you provide us with these logs that show the GraphQL queries, and you should also be able to replay them in the AppSync console to see the response which will be reconciled to the local database |
HI @lawmicha |
The queries we're looking for can be seen in
The query contains the filter with the correct condition. For the other file,
How are you making this call? and what else can you get from the error object returned, such as the underlying error? |
@lawmicha Got the issue, for users data have not been uploaded, it was in their local cache of data store but not got uploaded. And when user logged out, we first try to synchronise with data store and then data gets cleared. So for this user data has not been uploaded for one table & its uploaded for remaining tables. So can you help me how can we upload data safely & fastly to dynamodb. I am having lots of data on daily basis to upload to db. Currently we are doing it below way. But sometimes its taking 2-3 days to get data saved in db -
|
Clearing the local database with If you clearing because there's a different user signing into the app, then you could clear not on sign out, but when sign in is signed in with a user different from the previous. |
Thanks for the review on our code @lawmicha Could also help us in a better approach on how can we upload multiple rows to datastore. Currently we are adding it singly. But in our app it can happen that for 1 table in a day for single user there can be maximum 11520 rows, depend on the user use case. Can you guide me how can we implement best way to upload data to dyanamodb which will ensure that data will get uploaded as soon as possible. |
Hi @amruth-movano, AppSync conflict resolution enabled backends requires DataStore to perform the synchronization logic. Conflicts are handled per model which is why we cannot perform batch uploads. However, DataStore does optimizes how many model instances have to be synced, so for example if you save then update the model (two calls to If you're producing 11520 rows of data, make sure to update What does your data model look like and can you store the information in a different way to reduce the number of rows created? |
@lawmicha So overall conclusion is - We won't be able to do the batch operations. We have do it one by one only, right? Also below is config from our code -
|
If you are provisioning your AppSync service with |
@lawmicha |
Feel free to open another issue if you have more questions |
This issue is now closed. Comments on closed issues are hard for our team to see. |
Describe the bug
Hi Team,
some of our users are not able to see their old data which is older than a week.
For some users if i fetch weekly data i.e. if i fetch data for whole week at one time, it's giving me result but when I fetch data for single day in past, it's not giving me any result.
Below is my schema for one table. I have also attached my logic files that how are we configuring amplify. Can you please check and let us know why amplify is behaving like this way?
amplify-issue.zip
Steps To Reproduce
Expected behavior
Amplify datastore should give us the data, if it present in dynamodb
Amplify Framework Version
2.33.6
Amplify Categories
DataStore
Dependency manager
Swift PM
Swift version
5.9.2
CLI version
12.11.1
Xcode version
15.0
Relevant log output
Is this a regression?
Yes
Regression additional context
No response
Platforms
iOS
OS Version
iOS 17.0
Device
iPhone 13
Specific to simulators
No response
Additional context
No response
The text was updated successfully, but these errors were encountered: