Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Pydantic V2 fails to parse ingestion source #12623

Open
remisalmon opened this issue Feb 13, 2025 · 0 comments
Open

Pydantic V2 fails to parse ingestion source #12623

remisalmon opened this issue Feb 13, 2025 · 0 comments
Labels
bug Bug report

Comments

@remisalmon
Copy link
Contributor

Describe the bug

Pydantic V2 that is installed as a dependency of the acryl-datahub pip package fails to a file base lineage ingestion source.

To Reproduce
Steps to reproduce the behavior:

  1. Install acryl-datahub==0.15.0.1 in a new python venv
  2. Create a datahub-lineage-file ingestion source: https://datahubproject.io/docs/generated/ingestion/sources/file-based-lineage/
  3. Run datahub ingest --config <ingestion source above>.yml
  4. Ingestion fails with ERROR {datahub.ingestion.run.pipeline:537} - Ingestion pipeline threw an uncaught exception: 757 validation errors for LineageConfig

(the 757 is an example)

Expected behavior

The ingestion does not fail.

A workaround is to force pydantic>=1,<2 in the pip requirements.

@remisalmon remisalmon added the bug Bug report label Feb 13, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Bug report
Projects
None yet
Development

No branches or pull requests

1 participant