many to many Relations

[moweex]

** Which Category is your question related to? **
Datastore
** What AWS Services are you utilizing? **
AWS AppSync,
** Provide additional details e.g. code snippets **
We are implementing an RN application using AWS Amplify, we have a schema with a many to many relationships, as suggested we used a bridge model to link the tow models

Example Schema:

type Task
@model
{
id:ID!
name: String!
icon: String
records: [RecordTask] @connection(name: "TaskRecords")
}

type TimeTrackingRecord
@model
{
id: ID!
hours: Int!
minutes: Int!
time: AWSDateTime!
building: Building @connection(name: "BuildingTimeTrackingRecords")
tasks: [RecordTask] @connection(name: "RecordTasks")
}

type RecordTask
@model
{
id: ID!
record: TimeTrackingRecord @connection(name:"RecordTasks")
task: Task @connection(name: "TaskRecords")
}

Here RecordTask is used to link both Task & TimeTrackingRecord, we used amplify codegen models to generate models for us.

To complete our logic we are doing the following using datastore

1- Creating a set of Tasks
2- Creating a TimeTrackingRecord
3- loop over tasks and creating RecordTask

Then we tried two solutions:-
1- Updating already created TimeTrackingRecord to link it to the created RecordTask => Error on the update mutation by Appsync (no reason provided)

2- Relay that Datastore will connect the Tasks and TimeTrackingRecord, but unfortunately when we try to access TimeTrackingRecord tasks it is always empty however the RecordTask already created.

Any advice how can improve our M2M model to work fine with DataStore.

[manueliglesias]

Hi @moweex

Any advice how can improve our M2M model to work fine with DataStore.

Your schema is fine, it works as expected, we tested with this sample app and your schema (minus the Building type) provisioned to a back-end.

Note the order of the saves in the addRecordTask function:

1. Task
2. TimeTrackingRecord
3. RecordTask

Schema

type Task @model {
  id: ID!
  name: String!
  icon: String
  records: [RecordTask] @connection(name: "TaskRecords")
}

type TimeTrackingRecord @model {
  id: ID!
  hours: Int!
  minutes: Int!
  time: AWSDateTime!
  # building: Building @connection(name: "BuildingTimeTrackingRecords")
  tasks: [RecordTask] @connection(name: "RecordTasks")
}

type RecordTask @model {
  id: ID!
  record: TimeTrackingRecord @connection(name: "RecordTasks")
  task: Task @connection(name: "TaskRecords")
}

App (.tsx)

import React, { useEffect, useState } from "react";
import "./App.css";

import { DataStore } from "@aws-amplify/datastore";
import { Task, TimeTrackingRecord, RecordTask } from "./models";

function App() {
  const [recordTasks, setRecordTasks] = useState([] as RecordTask[]);

  useEffect(() => {
    queryRecordTasks();

    return () => {};
  }, []);

  async function queryRecordTasks() {
    const recordTasks = await DataStore.query(RecordTask);

    setRecordTasks(recordTasks);
  }

  async function addRecordTask() {
    const tasks = [];

    for (let i = 0; i < 5; i++) {
      const task = new Task({
        name: `the task ${i + 1}`
      });

      await DataStore.save(task);

      tasks.push(task);
    }

    const record = new TimeTrackingRecord({
      hours: 2,
      minutes: 30,
      time: new Date().toISOString()
    });

    await DataStore.save(record);

    for (const task of tasks) {
      const recordTask = new RecordTask({
        task,
        record
      });

      await DataStore.save(recordTask);
    }

    await queryRecordTasks();
  }

  return (
    <div className="App">
      <header className="App-header">
        <button onClick={addRecordTask}>Add</button>
        <pre style={{ textAlign: "left" }}>
          {JSON.stringify(recordTasks, null, 2)}
        </pre>
      </header>
    </div>
  );
}

export default App;

I hope this helps, let us know how it goes.

[moweex]

Hi @manueliglesias ,

thank you very much for your response and demo application, maybe my question was not clear enough.

Savings are working fine with our schema, we are using it the exact way you mentioned in your code. however, our issue is after saving a recordTask, if you query TimeTrackingRecord, you will find that the tasks property is always empty.

const records = await DataStore.query(TimeTrackingRecord);

for (const record of records) {
console.log(record.tasks);
}

so it seems the save function not updating the other end of the relationship, even after a sync call with appsync, tasks still empty, to get it we are doing it manually by querying RecordTask with filter.

[ashika01]

@moweex : Hi, thank you for bring this up. Currently, we only support loading from the one part of 1:M relationship. For eq, if you have Post to Comment as 1:M relationship. You can query for the post from comment but not the other way around.

But, we are tracking this internally to eager/lazy load all many side from the one side(comments from post). We will mark this issue as feature request to track updates.

[ghost]

Please note, that similar problem is reported also in amplify-cli (however, it wasn't directly related to DataStore):
aws-amplify/amplify-cli#3438

[smithad15]

Any update on this? I was quite excited about the prospects of DataStore, it's API, and how it works, however without this ability I cannot use it at all within my app. Being able to lazy-load relations would obviously be the preferred method to match the capabilities of GraphQL, but I would even just settle for the ability to manually pull records by filtering on IDs at this point. It feels like that fix primarily involves adjusting the generated models to include the foreign ID so that the predicate filter can be used with DataStore.query. I've got a mix of M:M and 1:M relations in my app and none of the models are outputting the foreignID field as described in the docs for querying relations.

[ryanto]

I just ran into this issue as well.

I'm building an application for tracking books. I've got an index page that shows all of the books and their authors. Books have a M:M relationship to authors through a BookAuthor model and I'm unsure how to correctly query these relationships.

Here's what I'm doing, my index pages queries books and then I loop over each of those books and query their BookAuthors. The code looks something like this...

books = await DataStore.query(Book)
bookAuthorQueries = books.map(book => async (
 (await DataStore.query(BookAuthor)).filter(ba => ba.book.id === book.id)
))

Does this seem like the best approach for querying has many relationships?

Also, will this approach stop working once I reach a certain number of records? Are there any limits I should be aware of?

This is the first app I've written using Dynamo so sorry if this is a silly question.

[vrebo]

Any update about this?

[cheunjm]

Is there any rough timeline for this "lazy-loading" feature?

[samwightt]

Do we have a date for the release of the lazy loading feature? Very disappointing that such a basic feature has not been implemented and has not received attention.

[micstepper]

Would be nice to get at least a short update about your plans....

[PTaylour]

@ryanto we're using the same approach as you (as a stop-gap?). The default page size limit on Datastore.query is 100, so you'll need to up that if you have more than 100 authors.

• Datastore docs on pagination

// TEMP FIX(?)
DataStore.query(BookAuthor, Predicates.ALL, {
  page: 0,
  limit: 1_000_000
});

[kai-ten]

Agreed on hearing of an update, I can't help but think about the serious performance hit I take when querying the join table to get data from both sides of the relationship..

@smithad15 's comment: #5054 (comment) seems to best capture my thoughts on a solution as well. Seems like there would have to be another index table generated in DDB for each of the two Objects being joined in the M-M relation. Each Object's model would then include the FK for when the object is queried (this is a M-M anyways, expect a bit higher of a performance hit when querying an index for the FK)

Rather than getting both objects returned in the BookAuthor join table (see @ryanto 's #5054 (comment)), we would then be able to query only against the Author object without having to duplicate the effort by querying both Book and Author.

tl;dr
Make an index table for both @model objects in the M-M relationship. Avoid having to query against the JoinTable to minimize duplication, save money on query time, and improve performance.

Thoughts?

[sanyashvets]

guys, updates?

[padunk]

Hi,
I also having an issued with M:M relationships like this one.
Any new updates yet?

[arabold]

Querying for relations seem to work well when using predicates:

const author = await DataStore.query(Author, authorID);
const books = await DataStore.query(Book, book => book.authorID("eq", authorID));

This, at least to me, seems preferable over the use of filter. I would assume it's also much less resource-intense than filtering over the final array as suggested in the documentation. Why isn't that mentioned anywhere?

[BrianMagus]

Has anyone found a workaround for this? It's sort of laughable that you'd have to query Parent A to get the ID and use that to query related Child B. Doesn't that sort of defeat the point of relations in the first place? Unless you can query related records, relations are basically just honorary.

[ThomasRooney]

My workaround is this: https://gist.github.com/ThomasRooney/89faeed810d1d18dfa16d0dd16eef0b2

It is a react hook that will resolve the connection before it returns the full structure.

It will also only follow 1 layer of connections, but when you have a deep structure you can layer them next to each other, similiar to @arabold's answer.

Interface for returning many items looks like (for a N:M connection, sorted):

  const { loaded, items: tests } = useDataStore<TestModel>(TestSchema, {
    runs: {
      model: RunSchema,
      criteria: (test) => (run) => run.testId('eq', test.id),
      paginationProducer: {
        sort: (condition) => condition.updatedAt('DESCENDING'),
      },
    },
  });

[Vazgen7788]

🤬

[salimdriai]

Still no updates on this ?

[amitchaudhary140]

It sucks. Just to get child connections, I need to create VMs and manually fetch parent id and then fetch child records based on paren id. There should be an option of whether user wants to load data eagerly or lazily.

[undefobj]

It sucks. Just to get child connections, I need to create VMs and manually fetch parent id and then fetch child records based on paren id. There should be an option of whether user wants to load data eagerly or lazily.

Lazy Loading was released late last year, does this help? https://aws.amazon.com/blogs/mobile/new-lazy-loading-nested-query-predicates-for-aws-amplify-datastore/

[amitchaudhary140]

@undefobj Unfortunately no.

I have models somewhat like these:

enum MembershipLevel {
FREE
PREMIUM
}

type Membership @model @auth(rules: [{allow: public}]) {
id: ID!
name: String
details: String
price: Float
value: MembershipLevel
}

type EGTopic @model @auth(rules: [{allow: public}]) {
id: ID!
color: String
membership: Membership @connection
questions: [EGQuestion] @connection(keyName: "byEGTopic", fields: ["id"])
}

1. When I fetch EGTopic, I don't get Membership (which is 1:1) and questions (which is 1:M). This is what everyone using DataStore wants. The control to load child data lazily or eagerly should be in the hands of developers and not by default. Lazy loading content is fine when there is lot of child data. But that's not always the case with everyone.
2. How to fetch user "membership" based on EGTopicId even through lazy loading? Because, for example:

return await DataStore.query(Membership, m => m.)

What should I type after m. ? There is no topic id which I can use to compare. I don't see any documentation for 1:1 relationship.

3. Of course, I can solve all this using GrapqQL API instead of DataStore. But I want to serve my users offline as well so I want to go with DataStore.

I would appreciate your quick response.

[svidgen]

@amitchaudhary140 The @connection directive in your model suggests you're using pretty old versions of the Amplify CLI and JS libraries. Per the article @undefobj linked, the lazy loading and nested predicates features are recent features. That means you'll need to be on recent versions of the CLI and JS libs.

You can follow the migration guide to update your CLI version + schema to take advantage of the V5 library. Once you migrate your schema and update your libs+app accordingly, your IDE's autocompletion/intellisense should help a great deal in completing that query for you. It should guide you to write something like this:

await DataStore.query(Membership, m => m.id.eq(topic.membershipId))

Or simply this:

await topic.membership;

To try this out, your next steps would be:

1. Update your CLI version (run amplify upgrade)
2. Follow the migration guide
3. Update your amplify-js version (npm install aws-amplify@latest)
4. Update your app code according to the breaking changes guidance

And then, of course, play around with lazy loading and nested predicates and let us know how it goes!

[amitchaudhary140]

@svidgen Still no luck. FYI, I copied the models from AWS Studio. So how does the schema change take place over there?

[svidgen]

The steps from my previous comment are intended to be run locally. Use the amplify pull command if you don't already have your backend/schema pulled down to your local development environment.

And, if it's not clear from the migration guide, you'll want to perform an amplify push after you've upgraded the schema.

[amitchaudhary140]

I performed below steps:

1. amplify upgrade
2. npm install aws-amplify@latest
3. Deleted old models from src directory completely and schema.graphql as well.
4. amplify pull (It generated models again)

I believe I don't have to go and do manual change in schemas (as mentioned in migration guide) as I had deleted them and they were rebuilt when pulled. But I can't still access membership from topic. Not sure what I am missing.

Should I delete whole backend? Below is what it looks like after performing all steps.

[svidgen]

The step that looks like it's missing from your list is amplify migrate api. I believe that's the command we designated to auto-migrate your schemas. After you run that, I believe you'll need to push again.

After that, you can upgrade your amplify-js dep to v5 and update your app code according to the aforementioned breaking changes guide.

[amitchaudhary140]

Thanks @svidgen . It worked till now. I have one more question that if we have two models generated (Eager and Lazy) then why can't we load data eagerly?

DataStore should provide some way to load data eagerly.