feature: Dataset dependencies in separate fields

[arturowczarek]

• When flag columnLineage.deprecatedMechanismEnabled, then the dataset dependencies are extracted from property field into separate field (dataset)

Problem

The dataset dependencies are mixed with field dependencies

Relates to: #3084
This is a continuation of: #3098
The next change is #3100

Solution

We should have the dataset dependencies separate

Note: All schema changes require discussion. Please link the issue for context.

• Your change modifies the core OpenLineage model
• Your change modifies one or more OpenLineage facets

If you're contributing a new integration, please specify the scope of the integration and how/where it has been tested (e.g., Apache Spark integration supports S3 and GCS filesystem operations, tested with AWS EMR).

One-line summary:

Checklist

• You've signed-off your work
• Your pull request title follows our guidelines
• Your changes are accompanied by tests (if relevant)
• Your change contains a small diff and is self-contained
• You've updated any relevant documentation (if relevant)
• Your comment includes a one-liner for the changelog about the specific purpose of the change (not required for changes to tests, docs, or CI config)
• You've versioned the core OpenLineage model or facets according to SchemaVer (if relevant)
• You've added a header to source files (if relevant)

---

SPDX-License-Identifier: Apache-2.0
Copyright 2018-2024 contributors to the OpenLineage project

[codecov-commenter]

⚠️ Please install the to ensure uploads and comments are reliably processed by Codecov.

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 100.00%. Comparing base (a97dae9) to head (4b3d6b8).

❗ Your organization needs to install the Codecov GitHub app to enable full functionality.

Additional details and impacted files

@@             Coverage Diff              @@
##             main     #3097       +/-   ##
============================================
+ Coverage   85.70%   100.00%   +14.29%     
============================================
  Files          54         5       -49     
  Lines        3112       173     -2939     
============================================
- Hits         2667       173     -2494     
+ Misses        445         0      -445     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[pawel-big-lebowski]

I think the original idea behind PR was: let's copy existing tests, bcz there will be a lot of tests changes when implementing dataset lineage feature. The problem with this reasoning is that it turned out only 4 existing tests are affected by the new feature, but going this way requires copying 900 lines of code.

Is my understanding correct?