Datastore not sync when add ownderField and/or groupsField in schema #1566
Comments
Jordan-Nelson
added
datastore
|
Hi @surisakc can you paste the generated Dart class files for |
@HuiSF here they are |
|
@HuiSF FYI, here is the configuration that works i.e. the datastore is sync'ed up at the app's startup and the datastore has the correct Building2 records when login with different accounts based on the type Building2 @model @auth(rules: [{allow: owner, identityClaim: "sub::username", ownerField: "users"}, {allow: private, provider: iam}]) { |
HuiSF
added
the
to-be-reproduced
|
Hi @surisakc based on what you've written, it seems like you saved these models before changing your schema.graphql? If that is correct, could you try clearing your database, and deleting your app locally (to clear local store), rebuilding your schema.graphql and then saving/syncing your models again? |
Hi @fjnoyp, from what I did test in Android last Friday with the latest schema I mentioned above, if I deleted the datastore locally and ran the app, it would sync up the datastore with correct data for an account. If I logged-out and logged-in with different account, it didn't sync up the datastore with correct data for this account (still saw the dataset belonged to the previous account). Again, if I deleted the datastore after logged-out every time, then next login it will sync up datastore correctly. |
|
Could it be the issue with
|
|
Hello @surisakc sorry for the delayed response. I did some digging, looking at your schema: type Building2 @model @auth(rules: [{allow: owner, ownerField: "users"}]) {
id: ID!
name: String!
status: GenericStatus!
data: String
users: [String]
}There is a known limitation with AppSync, that realtime subscription is not supported with dynamic group authorization. Hence Amplify DataStore doesn't support this type of authorization as well, but only static group authorization. Please see the documentation. |
HuiSF
added
pending-community-response
|
Hi @HuiSF, thank you for your response. I saw in the document about the known limitation as well but from my test the iOS worked if I set We need this dynamic group authorization in our app. If we use the static group authorization, it means the app has to dynamically create and delete a group and add users (userIds) in the group if I understand correctly and I may have to use the AWS Cognito API library directly if Amplify doesn't provide the API to do so, correct? Thanks! |
|
Hi @surisakc , for this auth rule Your thought is correct using low level Cognito API to create groups. But I'm not sure if it's a good idea, as Cognito quota has a limit on creating user groups. Do you mind to described your use case design around the groups? I'll see if we can find any alternatives to resolve this. |
|
@HuiSF I agreed with you on using the low level Cognito API to create groups at runtime may not be a good solution. Using the groups as static should be ideal. From what I was testing, we wanted to share some models in datastore which means;
|
|
@HuiSF A quick question about using a lambda function to override authorization rule in schema.
|
|
Hi @surisakc
I doubt this will work as when you rely on owner based auth, it will check db owner field value and the username attached int he JWT token. For this case I think it breaks. But have you tried something like this?
# quick example, I haven't verified this is working
type Building2 @model @auth(rules: [{allow: owner, ownerField: "familyId", identityClaim: "familyId"}]) {
id: ID!
name: String!
status: GenericStatus!
data: String
familyId: String
}
|
Hi @HuiSF Thanks for the reply. For the solution you suggested. I have some questions.
Here is how I tested it and if it was correct steps then it didn't work. Let me know if I missed any step here.
Thanks! |
|
The process you described makes sense to me.
Yes I believe so, setting With this use case, I think you need to poll userAttributes of currently signed in user often (or other equivalent), to ensure the user is still assigned with that particular |
But from testing, this solution doesn't work (the datastore doesn't have the I saw something similar to your suggestion from here https://docs.amplify.aws/cli/graphql/authorization-rules/#configure-custom-identity-and-group-claims which I did try like schema below but it didn't work either. Does it have to be I set By setting |
|
Hi @surisakc looking at the resolver, it listed fields in the model schema as Which indicates your model should have |
I think it uses access token by default, might need to set idToken to the HTTP request Authorization header, though amplify-flutter doesn't support custom header yet. And using id token may not be the best practice for security... |
Hi @HuiSF, sorry for not posting the latest schema which has |
Hi @HuiSF Thank you for your response. So, for the example that using Another question is when the app starts up first time, how does Amplify sync datastore? i.e. which AppSync API function that Amplify will use to sync datastore at this stage and which resolver.
Thanks! |
Jordan-Nelson
added
the
pending-triage
|
Hello @surisakc Sorry for the delay -
This example is for using 3rd-party auth provider (OIDC in the doc) with which you override the identity claim to use the fields listed in the JWT token generated by this 3rd-party, not an attribute managed by Cognito. Based on the findings and your testing, I think using custom user attributes along with idToken unfortunately is not doable now due to
It making requests using all the sync queries (one sync query per model). Were you looking at the generated lambda function files? You should be able to get a better view in AppSync console ("functions " in left navigation menu), where you can view the functions and their implementations associated with sync queries. e.g.
Additional thoughts: In the doc you linked in the latest comment, it mentioned about "Pre Token Generation Lambda Trigger", it has potentials to modify the token before the service return the token to client. This example shows a way to override user group. |
Hi @HuiSF I was guessing and checking the generated lambda functions in AppSync console which generated from the resolver templates generated by the amplify CLI on my machine under the folder
|
That's correct.
Not I'm aware of, but this is like a multi-tenants use case there may be similar requests in AppSync but I don't have a list in handy.
Need to search around.
If API plugin APIs work with this custom authorizer, including subscription, then DataStore should work with it too, as DataStore is fully depends on API plugin.
It should work for DataStore as well as the answer to the previous question. |
Regarding this point, I'm syncing up with amplify-ios, amplify-android and amplify-cli maintainers to determine the correct behavior - whether the amplify libraries should resolve the custom claim from idToken when nothing can't be resolved from access token. |
Hi @HuiSF Thank you for checking! I did more testing and it seems if amplify-android library for flutter could somehow makes the library to work like amplify-ios (for flutter) then it may fix the issue i.e. if amplify-android can support Test 3 like amplify-ios does. Test 2 and Test 4 partially work in Android because the Here are my testsFlutter: v2.10.5 Schema using a list of owners which has known limitation (not support realtime subscription)
GraphQLResponseException{message=Subscription error for Building2: [GraphQLResponse.Error{message='Validation error of type UnknownArgument: Unknown field argument users @ 'onDeleteBuilding2'', locations='null', path='null', extensions='null'}], errors=[GraphQLResponse.Error{message='Validation error of type UnknownArgument: Unknown field argument users @ 'onDeleteBuilding2'', locations='null', path='null', extensions='null'}], recoverySuggestion=See attached list of GraphQLResponse.Error objects.}
I/amplify:aws-datastore( 3827): Setting currentState to STOPPED
I/amplify:flutter:datastore( 3827): Successfully stopped datastore remote synchronization
I/amplify:aws-datastore( 3827): Orchestrator lock released.
E/amplify:aws-datastore( 3827): Failure encountered while attempting to start API sync.
E/amplify:aws-datastore( 3827): DataStoreException{message=Error during subscription., cause=ApiAuthException{message=Attempted to subscribe to a model with owner-based authorization without sub::username which was specified (or defaulted to) as the identity claim., cause=null, recoverySuggestion=If you did not specify a custom identityClaim in your schema, make sure you are logged in. If you did, check that the value you specified in your schema is present in the access key.}, recoverySuggestion=Evaluate details.}
E/amplify:aws-datastore( 3827): at com.amplifyframework.datastore.appsync.AppSyncClient.lambda$subscription$3(AppSyncClient.java:331)
E/amplify:aws-datastore( 3827): at com.amplifyframework.datastore.appsync.-$$Lambda$AppSyncClient$797ziDK0io-qXODzROLOA77stS8.accept(Unknown Source:4)
W/amplify:aws-datastore( 3827): DataStoreException{message=Error during subscription., cause=ApiAuthException{message=Attempted to subscribe to a model with owner-based authorization without sub::username which was specified (or defaulted to) as the identity claim., cause=null, recoverySuggestion=If you did not specify a custom identityClaim in your schema, make sure you are logged in. If you did, check that the value you specified in your schema is present in the access key.}, recoverySuggestion=Evaluate details.}
W/amplify:aws-datastore( 3827): at com.amplifyframework.datastore.appsync.AppSyncClient.lambda$subscription$3(AppSyncClient.java:331)
W/amplify:aws-datastore( 3827): at com.amplifyframework.datastore.appsync.-$$Lambda$AppSyncClient$797ziDK0io-qXODzROLOA77stS8.accept(Unknown Source:4)
Original Resolver Custom Resolver Test 4 Error GraphQLResponseException{message=Subscription error for Building2: [GraphQLResponse.Error{message='Validation error of type UnknownArgument: Unknown field argument users @ 'onDeleteBuilding2'', locations='null', path='null', extensions='null'}], errors=[GraphQLResponse.Error{message='Validation error of type UnknownArgument: Unknown field argument users @ 'onDeleteBuilding2'', locations='null', path='null', extensions='null'}], recoverySuggestion=See attached list of GraphQLResponse.Error objects.}
z
|
|
Hi @HuiSF I think this solution works in both iOS and Android to share models among users. I didn't test by removing the three custom Subscription resolvers as I thought they may be needed but may be not. Schema
3 Custom Subscription Resolvers Update the Generated Models
Error in Android
|
For Android, it doesn't work every time when start the app and login i.e. the datastore isn't sync'ed every time (the Amplify event |
This error causes subscription to fail, which stops sync engine so you would see the unsuccessful sync. This error is due to the subscription selection set contains field is not defined in GraphQL schema (located in |
Here is my schema. The error points to the unknown field argument Here is one of the three subscription resolvers. |
Hi @HuiSF I have fixed this issue but not sure if this is the right way and if it will have any other issues later on. Basically I have to go to AppSync Console and search in Schema for those Subscription functions that set in the schema to use Original The fix that works There is other issue in the Datastore library (Amplify_Flutter v0.5.1) of iOS and Android that doesn't work with the complex query anymore (let me know if you want me to open an issue for this) Original Datastore query works in both iOS and Android in Amplify_Flutter v0.2.10 To make the query to work in iOS, I have to separate the where clause into two steps: To make the query to work in Android, I have to update the two query fields (see |
HuiSF
removed
the
pending-community-response
|
Hi @surisakc Thanks for providing the details about the workaround.
So far, I'm unsure if Amplify CLI and GraphQL transformer do have the support of your use case... But looking at your schema and resolver changes it makes sense to me. As long as your custom resolvers are able to ensure correct auth scope for your data, I think it's fine to use it. I will find some time to do some testing with your workaround as well.
We do have integration tests for ensure the compound query to be working. And I just ran the tests based on version Could you sharing your implementation of |
Hi @HuiSF Thank you for your response. The issue I had in Amplify Android that was when it called Here is the Also, I use this auth setting Amplify.Datastore.observe() in the app, correct? I did test and it didn't, just want to confirm. Or it has to do with the auth rules in some Subscription resolvers that I need to customize them for this to work?Will it also doesn't trigger if I use |
|
Hi @surisakc for the nested predicate, Amplify libraries don't have a perfect support at this moment. amplify-ios and amplify-android are working differently on this as well, hence, amplify-flutter can only take a lowest common support as for Flutter consumers both iOS and Android should behave the same. Our current recommendation is to make separate queries to retrieve models when necessary. Better support of nested query predicates are on the roadmap (tracked in this issue: #1449)
That's correct, DataStore's sync engine uses API plugin, all the limitation on the API plugin are applicable to DataStore API sync as well. |
|
From testing a fix implemented by amplify-android for these two issues (aws-amplify/amplify-android#2035, #2632) I can see that when multiple table join is needed by the query predicate, it can generate correct SQL command with proper table namespace in the where clause. I'm optimistically closing this issue, please feel free to reach out if anything. |
|
Hi, I can reproduce the original issue by using the exemple from the documentation at this link: https://docs.amplify.aws/cli/graphql/authorization-rules/#multi-user-data-access Versions:
type Task @model @auth(rules: [{ allow: owner, ownerField: "authors" }]) {
content: String
authors: [String]
}Relevant Logs: Generated code: input ModelSubscriptionTaskFilterInput {
content: ModelSubscriptionStringInput
and: [ModelSubscriptionTaskFilterInput]
or: [ModelSubscriptionTaskFilterInput]
_deleted: ModelBooleanInput
}
type Subscription {
onCreateTask(filter: ModelSubscriptionTaskFilterInput): Task @aws_subscribe(mutations: ["createTask"]) @aws_iam @aws_cognito_user_pools
onUpdateTask(filter: ModelSubscriptionTaskFilterInput): Task @aws_subscribe(mutations: ["updateTask"]) @aws_iam @aws_cognito_user_pools
onDeleteTask(filter: ModelSubscriptionTaskFilterInput): Task @aws_subscribe(mutations: ["deleteTask"]) @aws_iam @aws_cognito_user_pools
}Code to replicate: Future<void> configureAmplify() async {
// Add any Amplify plugins you want to use
final authPlugin = AmplifyAuthCognito();
// await Amplify.addPlugin(authPlugin);
// Add the following lines to your app initialization to add the DataStore plugin
final datastorePlugin = AmplifyDataStore(
modelProvider: ModelProvider.instance,
// Be sure to add the authModeStrategy
authModeStrategy: AuthModeStrategy.multiAuth,
);
// await Amplify.addPlugin(datastorePlugin);
final api = AmplifyAPI();
// You can use addPlugins if you are going to be adding multiple plugins
await Amplify.addPlugins([authPlugin, api, datastorePlugin]);
// await Amplify.addPlugins([authPlugin]);
// Once Plugins are added, configure Amplify
// Note: Amplify can only be configured once.
try {
await Amplify.configure(amplifyconfig);
} on AmplifyAlreadyConfiguredException {
safePrint("Tried to reconfigure Amplify; this can occur when your app restarts on Android.");
}
} |
Description
Using the schema below that has
ownerFieldandgroupsFieldboth point to a list of string. When run the app in iOS, the datastore isn't sync'ed. In Android, the app crashed with error messages.schema with two fields that cause issue
also tried this schema which has issue like above
schema without two fields that has no issue
Categories
Steps to Reproduce
ownerFieldandgroupsFieldboth point to a list of string (userIds)modelsyncedand no error messages.modelsyncedwith error messages.Screenshots
No response
Platforms
Android Device/Emulator API Level
No response
Environment
[✓] Flutter (Channel stable, 2.10.5, on macOS 12.3.1 21E258 darwin-arm, locale en-CA) • Flutter version 2.10.5 at /Users/<user>/fvm/versions/2.10.5 • Upstream repository https://github.com/flutter/flutter.git • Framework revision 5464c5bac7 (2 weeks ago), 2022-04-18 09:55:37 -0700 • Engine revision 57d3bac3dd • Dart version 2.16.2 • DevTools version 2.9.2 [✓] Android toolchain - develop for Android devices (Android SDK version 31.0.0) • Android SDK at /Users/<user>/Library/Android/sdk • Platform android-31, build-tools 31.0.0 • Java binary at: /Applications/Android Studio.app/Contents/jre/Contents/Home/bin/java • Java version OpenJDK Runtime Environment (build 11.0.10+0-b96-7249189) • All Android licenses accepted. [✓] Xcode - develop for iOS and macOS (Xcode 13.3.1) • Xcode at /Applications/XCode.app/Contents/Developer • CocoaPods version 1.11.3 [✓] Android Studio (version 2020.3) • Android Studio at /Applications/Android Studio.app/Contents • Flutter plugin can be installed from: 🔨 https://plugins.jetbrains.com/plugin/9212-flutter • Dart plugin can be installed from: 🔨 https://plugins.jetbrains.com/plugin/6351-dart • Java version OpenJDK Runtime Environment (build 11.0.10+0-b96-7249189) [✓] VS Code (version 1.66.2) • VS Code at /Applications/Visual Studio Code.app/Contents • Flutter extension version 3.38.1 [✓] Connected device (3 available) • sdk gphone64 arm64 (mobile) • emulator-5554 • android-arm64 • Android 12 (API 31) (emulator) • iPhone 13 Pro (mobile) • 1CA31312-509F-4F0C-9A9D-DABC5FE54518 • ios • com.apple.CoreSimulator.SimRuntime.iOS-15-4 (simulator) • iPhone 13 Pro Max (mobile) • 273BEB6B-5F10-4914-B1EB-15E19F906CD0 • ios • com.apple.CoreSimulator.SimRuntime.iOS-15-4 (simulator) [✓] HTTP Host Availability • All required HTTP hosts are available • No issues found!Dependencies
Device
Android Emulator Pixel 3a
OS
Android 12.0 (API 31) arm64
CLI Version
8.1.0
Additional Context
No response
The text was updated successfully, but these errors were encountered: