Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(foreign keys): add foreign key models #3275

Merged

Conversation

gabe-lyons
Copy link
Contributor

Adding the modeling for foreign keys + schema fields

Checklist

  • The PR conforms to DataHub's Contributing Guideline (particularly Commit Message Format)
  • Links to related issues (if applicable)
  • Tests for the changes have been added/updated (if applicable)
  • Docs related to the changes have been added/updated (if applicable)

import com.linkedin.metadata.key.SchemaFieldKey

/**
* A union of all supported metadata aspects for a Dataset
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fix comment

import com.linkedin.common.Urn

/**
* Key for a SchemaField
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we call these models DatasetSchemaField?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hmm.. actually updating this to be container rather than dataset so we have the flexibility to add schema fields to other things

}
dataset: Urn

/**
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Field path spec is going to get much more important :)

"entityTypes": [ "schemaField" ]
}
}
foreignFields: array[Urn]
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What's a use case with multiple foreign key fields?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(a,b,c) in Table 1 are compound keys pointing to (d,e,f) in Table 2

"name": "ForeignKeyToDataset",
"entityTypes": [ "dataset" ]
}
foreignDataset: Urn
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Consider placing this at the top of this model

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it makes sense where it is- this is just for sake of easy reference.

Copy link
Collaborator

@jjoyce0510 jjoyce0510 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like how you've kept things fairly minimal

name: string

/**
* Fields the constraint maps to on the foreign dataset
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maybe a note that these are expected to be schema field urns... and what that structure looks like?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

that is already annotated in the entityTypes property below :)

Copy link
Contributor

@shirshanka shirshanka left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall LGTM with minor documentation comments. Can we remove the new snapshot models... since we are trying to move to less monolithic models.

@gabe-lyons gabe-lyons force-pushed the AddingForeignKeyPrimaryKeyModels branch from 4abbbee to b4acd57 Compare September 21, 2021 17:58
@Searchable = {
"fieldType": "URN",
}
container: Urn
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

container / parent?

Copy link
Contributor

@shirshanka shirshanka left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@shirshanka shirshanka merged commit ff527f4 into datahub-project:master Sep 22, 2021
xdl pushed a commit to G-Research/datahub that referenced this pull request Oct 4, 2021
xdl added a commit to G-Research/datahub that referenced this pull request Oct 5, 2021
jgiannuzzi pushed a commit to G-Research/datahub that referenced this pull request Oct 6, 2021
swaroopjagadish pushed a commit to swaroopjagadish/datahub that referenced this pull request Oct 7, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants