Skip to content

Commit 31516c9

Browse files
Adding examples for one-to-one relations
1 parent 36962f4 commit 31516c9

File tree

1 file changed

+87
-9
lines changed

1 file changed

+87
-9
lines changed

content/integrate/redis-data-integration/data-pipelines/data-denormalization.md

Lines changed: 87 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -25,13 +25,89 @@ at the expense of speed.
2525
A Redis cache, on the other hand, is focused on making *read* queries fast, so RDI provides data
2626
*denormalization* to help with this.
2727

28-
## Nest strategy
28+
## Joining one-to-one relationships
2929

30-
*Nesting* is the strategy RDI uses to denormalize many-to-one relationships in the source database.
31-
It does this by representing the
32-
parent object (the "one") as a JSON document with the children (the "many") nested inside a JSON map
33-
attribute in the parent. The diagram belows shows a nesting with the child objects in a map
34-
called `InvoiceLineItems`:
30+
You can join one-to-one relationships by making more than one job to write to the same Redis key.
31+
32+
First, you must configure the parent entity to use `merge` as the `on_update`.
33+
34+
```yaml
35+
# jobs/customers.yaml
36+
source:
37+
table: customers
38+
39+
output:
40+
- uses: redis.write
41+
with:
42+
data_type: json
43+
on_update: merge
44+
```
45+
46+
Then, you can configure the child entity to write to the same Redis key as the parent entity. You can do this by using the `key` attribute in the `with` block of the job, as shown in this example:
47+
48+
```yaml
49+
# jobs/addresses.yaml
50+
source:
51+
table: addresses
52+
53+
transform:
54+
- uses: add_field
55+
with:
56+
field: customer_address
57+
language: jmespath
58+
# You can use the following JMESPath expression to create a JSON object and combine the address fields into a single object.
59+
expression: |
60+
{
61+
"street": street,
62+
"city": city,
63+
"state": state,
64+
"zip": zip
65+
}
66+
67+
output:
68+
- uses: redis.write
69+
with:
70+
data_type: json
71+
# We specify the key to write to the same key as the parent entity.
72+
key:
73+
expression: concat(['customers:id:', customer_id])
74+
language: jmespath
75+
on_update: merge
76+
mapping:
77+
# You can specify one or more fields to write to the parent entity.
78+
- customer_address: customer_address
79+
```
80+
81+
The joined data will look like this in Redis:
82+
83+
```json
84+
{
85+
"id": "1",
86+
"first_name": "John",
87+
"last_name": "Doe",
88+
"customer_address": {
89+
"street": "123 Main St",
90+
"city": "Anytown",
91+
"state": "CA",
92+
"zip": "12345"
93+
}
94+
}
95+
```
96+
97+
{{< note >}}
98+
Not setting `merge` as the `on_update` strategy for all jobs targeting the same key, will cause the entire parent record in Redis to be overwritten whenever any related record in the source database is updated, resulting in the loss of values written by other jobs.
99+
{{< /note >}}
100+
101+
When using this approach, you must ensure that the `key` expression in the child job matches the key expression in the parent job. If you use a different key expression, the child data will not be written to the same Redis key as the parent data.
102+
103+
In the example above, the `addresses` jobs uses the default key pattern to write to the same Redis key as the `customers` job. You can find more information about the default key pattern [here]({{< relref "/integrate/redis-data-integration/data-pipelines/transform-examples/redis-set-key-name" >}}).
104+
105+
You can also use custom keys for the parent entity, as long as you use the same key for all jobs that write to the same Redis key.
106+
107+
## Joining one-to-many relationships
108+
109+
To join one-to-many relationships you can use the *Nesting* strategy.
110+
With it, the parent object (the "one") is represented as a JSON document with the children (the "many") nested inside a JSON map attribute in the parent. The diagram below shows a nesting with the child objects in a map called `InvoiceLineItems`:
35111

36112
{{< image filename="/images/rdi/ingest/nest-flow.webp" width="500px" >}}
37113

@@ -57,7 +133,9 @@ output:
57133
58134
```
59135

60-
When you have configured the parent model, you must also configure the child entities. To do this, use the `nest` block, as shown in this example:
136+
Once you have configured the parent entity, you can then configure the child entities to be nested under the parent entity based on their relation type.
137+
138+
After you have configured the parent model, you must also configure the child entities. To do this, use the `nest` block, as shown in this example:
61139

62140
```yaml
63141
# jobs/invoice_line.yaml
@@ -91,7 +169,7 @@ The job must include the following attributes in the `nest` block:
91169
`schema` attributes. Note that this attribute refers to a Redis *key* that will be added to the target
92170
database, not to a table you can access from the pipeline. See [Using nesting](#using-nesting) below
93171
for the format of the key that is generated.
94-
- `nesting_key`: The unique key of each child entry in the json map that will be created under the path.
172+
- `nesting_key`: The unique key of each child entry in the JSON map that will be created under the path.
95173
- `parent_key`: The field in the parent entity that stores the unique ID (foreign key) of the parent entity. Can not be composite key.
96174
- `child_key`: The field in the child entity that stores the unique ID (foreign key) to the parent entity. You only need to add this attribute if the name of the child's foreign key field is different from the parent's. Can not be composite key.
97175
- `path`: The [JSONPath](https://goessner.net/articles/JsonPath/)
@@ -100,7 +178,7 @@ The job must include the following attributes in the `nest` block:
100178
- `structure`: (Optional) The type of JSON nesting structure for the child entities. Currently, only a JSON map
101179
is supported so if you supply this attribute then the value must be `map`.
102180

103-
## Using nesting
181+
### Using nesting
104182

105183
There are several important things to note when you use nesting:
106184

0 commit comments

Comments
 (0)