v0.13.0

DataHub v0.13.0 Release Notes Summary

User Experience

• NEW - Asset Documentation Forms & UI-Editable Properties: Define specific documentation requirements via a Form, and empower your asset owners to capture their valuable knowledge via UI-Editable Properties. Watch the demo here!
• NEW - DataHub Incidents: Create, communicate, and data quality and observability incidents when they inevitably arise. Watch the demo here!
  UI Improvements: Editing secrets, handling forms, and rendering token pages and lineage diagrams have been improved for a smoother user interface experience.
• UI Improvements: Editing secrets, handling forms, and rendering token pages and lineage diagrams have been improved for a smoother user interface experience.

Developer Experience

• Security Upgrades: Core dependencies like shiro-core and FastAPI have been upgraded to fix vulnerabilities, ensuring a safer development environment.
• GraphQL/OpenAPI Enhancements: New GraphQL endpoints and better OpenAPI documentation provide more powerful tools for API interaction, making developers' jobs easier.
• Performance Tuning: Backend improvements for search operations and ingestion processes make the platform faster and more reliable.

Metadata Ingestion

• Platform Integrations: Enhanced support for dbt, Metabase, BigQuery, AWS Glue, Oracle, and Redshift allows for more comprehensive metadata capture, making integration with these platforms smoother.
• Ingestion Framework: The reliability of ingestion has been improved, with new capabilities like support for tags from Tableau datasources and compatibility with Airflow 2.5.0, facilitating a broader range of data synchronization tasks.
• Connector Improvements: Ingestion connectors for external data tools have been streamlined, ensuring easier integration and data synchronization.

Other Improvements and Fixes

• Enhanced internal testing frameworks with Cypress and pytest-random-order for ingestion tests.
• Simplified developer workflows with configurable Docker Compose project names in CLI.
• Addressed various ingestion-related bugs for platforms like Feast and Snowflake.
• Enhanced the UI codebase with TypeScript compilation linting and updated styles.
• Streamlined CI processes for pull requests and linting conditions.
• Version Upgrades: Upgraded pytest-docker, Pegasus, and SQLglot, among others, to improve stability and performance. Security vulnerabilities addressed by upgrading FastAPI, gitdb, and follow-redirects.

Notable Breaking Changes

• Updates to MySQL version for quickstarts and migration to Neo4j 5.x may impact existing setups.
• JDK17 build requirement and Docker Compose > 2.20 needed for building DataHub.
• Python 3.8+ requirement for the acryl-datahub CLI.
• Changes in Unity Catalog ingestion source configs and Redshift lineage generation.
• Deprecation of Spark 2.x and associated JDK8 build requirements.

For full details on breaking changes, please visit DataHub's update guide.

Acknowledgements

A huge thank you to all our contributors for making this release possible. Your hard work and dedication are greatly appreciated.

First-Time Contributors

7onn, Adityamalik123, atjones0011, BlueHorn07, diegoreico, dim-ops, fer-marino, Gerrit-K, gp1105739, ilpianista, ingthorb, KaYunKIM, Kunal-kankriya, muzzacode, nnnkkk7, pankajmahato-visa, rubiojr, ryaminal, scalvanese452, sleeperdeep, stevenayers.

Repeat Contributors

allizex, arunvasudevan, cburroughs, feldjay, gaurav2733, iprentic, KulykDmytro, kushagra-apptware, mayurinehate, nmbryant, noggi, purnimagarg1, rinzool, sgomezvillamor, shubhamjagtap639, siddiquebagwan-gslab, siladitya2, skrydal, sumitappt, TonyOuyangGit, wngus606, yangjiandan, Salman-Apptware.

DataHub Maintainers

anshbansal, asikowitz, chriscollins3456, darnaut, david-leifker, eboneil, ethan-cartwright, gabe-lyons, hsheth2, jjoyce0510, maggiehays, pedro93, RyanHolstien, shirshanka, sid-acryl, treff7es, yoonhyejin.

What's Changed

• fix(ingest/transformer): correct registration by @anshbansal in #9418
• docs(ingest/sql-queries): Rearrange sections by @asikowitz in #9426
• fix: Adjusting the view of the Column Stats by @Salman-Apptware in #9430
• feat(patch): support fine grained lineage patches by @RyanHolstien in #9408
• fix(CVE-2023-6378): update logback classic by @RyanHolstien in #9438
• feat: allow the sidebar size to be draggable by @Salman-Apptware in #9401
• fix(json-schema): do not send invalid URLs by @anshbansal in #9417
• fix(ingest/profiling) Fixing profile eligibility check by @treff7es in #9446
• fix(ingest): avoid git dependency in dbt by @hsheth2 in #9447
• feat(ingest): add retries for tableau by @hsheth2 in #9437
• docs(updating-datahub): update docs for v0.12.1 by @david-leifker in #9441
• feat: Allow specifying Data Product URN via UI by @Salman-Apptware in #9386
• Add button to copy urn of an Ownership Type by @Salman-Apptware in #9452
• docs(ingest/tableau): add token to sink config in sample recipe by @KaYunKIM in #9411
• feat(glossary): add ability to clone glossary term(name and documentation) from term profile menu by @allizex in #9445
• feat(ingestion): Add typeUrn handling to ownership transformers by @skrydal in #9370
• fix(ingest): reduce GraphQL Logs to warning for circuit breaker by @arunvasudevan in #9436
• fix: support Apollo caching for settings / Policies by @Salman-Apptware in #9442
• refactor | PRD-785 | datahub oss: migrate use of useGetAuthenticatedU… by @sumitappt in #9456
• refactor(ui): Minor improvements & refactoring by @jjoyce0510 in #9420
• feat(ingest): add ingest --no-progress option by @BlueHorn07 in #9300
• fix(powerbi): add access token refresh by @anshbansal in #9405
• fix | PRD-463 | Stop trying to ping the track endpoint on login home … by @sumitappt in #9462
• feat(ingest/unity): enable hive metastore ingestion by @mayurinehate in #9416
• feat(ingestion/transformer): create tag if not exist by @siddiquebagwan-gslab in #9076
• fix(ingest): make user_urn and group_urn generation consider user and… by @shirshanka in #9026
• feat(ingestion): Add test_connection methods for important sources by @shubhamjagtap639 in #9334
• docs: fix sample command for container logs by @nnnkkk7 in #9427
• fix(ingest): bump source configs json schema version by @hsheth2 in #9424
• fix(ingest/profiling): Add option to enable external table profiling by @treff7es in #9463
• fix(operations): fix get index sizes integer wrap by @RyanHolstien in #9450
• feat(build): gradle 8, jdk17, neo4j 5 by @david-leifker in #9458
• feat(ingest/redshift): drop repeated operations by @hsheth2 in #9440
• feat: Deprecation 'Note' changed to Markdown Renderable by @kushagra-apptware in #9396
• feat : markdown support for group description by @gaurav2733 in #9455
• feat(ingest): enable CLL for dbt by default by @hsheth2 in #9466
• fix(ingest/snowflake) - Fixing snowflake url with default region by @treff7es in #9443
• Fix downstream CI issue by @noggi in #9479
• feat(ingest): pydantic v2 compatibility by @hsheth2 in #9434
• feat(gms): Add support for platform-based browse by @jjoyce0510 in #9376
• Searching for users on Users page shows incorrect roles: Issue resolved by @kushagra-apptware in #9474
• fix(ingest/redshift: Fixing operation query to not return duplicate operations by @treff7es in #9481
• Fade recipe section to transparent on Ingestion Run Details by @purnimagarg1 in #9404
• Allow message_name field for protobuf ingestion by @eboneil in #9480
• feat(docker-compose): consolidate docker-compose profiles by @david-leifker in #9478
• Adds urnBasedPagination option to datahub-upgrade RestoreIndices by @nmbryant in #9232
• fix(quickstart): force strings for mysql version by @david-leifker in #9485
• fix(docker): fix frontend dev docker path by @david-leifker in #9488
• fix: Tab doesn't represent the page you are on for non-data asset pages by @Salman-Apptware in #9468
• Do not sync demo in downstream repos by @noggi in #9493
• fix(ingest): update ingest_stats event with transformer types by @hsheth2 in #9487
• feat | PRD-874 | Keep the same tab selected when browsing Glossary tr… by @sumitappt in #9469
• style | PRD-893 | Tag overflow is missing padding by @sumitappt in #9497
• feat(analytics): change MAU chart to be until last month by @anshbansal in #9499
• fix(kafka): fix infinite deserialization logging by @RyanHolstien in #9494
• fix(ingest/fivetran): only materialize upstream lineage by @hsheth2 in #9490
• feat(ingest): handle multiline string coercion by @hsheth2 in #9484
• fix(ingest/databricks): Pinning databricks sdk to not fail on mypy issues by @treff7es in #9500
• feat(ingest): remove librdkafka hacks by @hsheth2 in #9507
• feat(ingest): rename custom package path from models to metadata by @hsheth2 in #9502
• Prd 886 edit links by @kushagra-apptware in #9498
• feat(ingest): support CLL for redshift materialized views with auto refresh by @hsheth2 in #9508
• feat: Input field added for the Custom cron, and issue on search sour… by @kushagra-apptware in #9510
• fix(ingest): update dbt type inference by @hsheth2 in #9512
• feat(ingest/redshift): merge CLL instead of overwriting by @hsheth2 in #9513
• Add debounce on search on ingestion listing page by @gaurav2733 in #9516
• fix: correct the color of edit links by @kushagra-apptware in #9517
• fix(frontend): Add JSON list oidc group extraction logic by @ethan-cartwright in #9495
• feat(frontend): align frontend sso code with refactors by @RyanHolstien in #9506
• fix(ingest/glue): glue table partition (Athena v3) profiling parameters retrieval error by @KulykDmytro in #9521
• fix(ingest/fivetran): handle missing user_id for a connection by @ryaminal in #9460
• feat(ui): Allow copying assertion urn from the UI by @sumitappt in #9523
• feat(ingest/openapi): support proxies and alternate auth schemes by @fer-marino in #9492
• build(ingest/feast): upgrade to latest feast version by @cburroughs in #9439
• build: enable gradle caching by @hsheth2 in #9525
• chore(build): update base-requirements + add script for regeneration by @hsheth2 in #9524
• add automatic reset password test by @gaurav2733 in #9515
• feat(ingestion/bigquery): Use sqlglot_lineage for usage and add more perf timers by @shubhamjagtap639 in #9247
• fix(ingest/mongodb): support disabling schemaSamplingSize by @diegoreico in #9295
• fix(ingest): Fix mongodb ingestion when platform_instance is missing from recipe by @TonyOuyangGit in #9486
• fix(ingest/snowflake): explicit set schema if public schema is absent by @mayurinehate in #9526
• fix | PRD-901 | Border is too thick for sidebar by @sumitappt in #9528
• Humanise duration shown on ingestion page by @gaurav2733 in #9530
• fix(cli): fix issue with upsert for data product external url by @anshbansal in #9534
• fix posts are failing to be created as Admin user by @gaurav2733 in #9533
• while creating secrets via UI validate validate character~ by @gaurav2733 in #9548
• feat(ui): add databricks logo by @mayurinehate in #9473
• feat(databricks): add hive metastore analyze profiling by @mayurinehate in #9511
• Fix/PRD-906 make setting manage policy test by @Kunal-kankriya in #9547
• search user functionality added by @kushagra-apptware in #9532
• fix(ci): make test flexible to allow sha-based cli versions by @shirshanka in #9551
• feat/PRD-895-Navigation-test by @Kunal-kankriya in #9545
• docs(acryl cloud): release notes for 0.2.14.1 by @anshbansal in #9554
• ci(doc): tweak build rule to avoid docker build for docs by @anshbansal in #9555
• fix(ingest): improve kafka-connect test stability by @hsheth2 in #9519
• fix(ingest/looker): add user stats to report by @hsheth2 in #9505
• perf(lineage): Rewrite lineage query for Elastic graph store by @shirshanka in #9552
• feat(ingest): improve config loading helpers by @hsheth2 in #9477
• feat(patch): add dashboardInfo and chartInfo support for patch by @RyanHolstien in #9536
• feat(docker): docker compose profiles updates by @david-leifker in #9514
• feat(ui): switch to vite and vitest by @hsheth2 in #9451
• fix(graphql): BatchGetEntitiesResolver respects order by @shirshanka in #9557
• ci(label): datahub-community-champion label based off usernames by @anshbansal in #9558
• feat(quickstart): add actions to debug quickstart by @david-leifker in #9559
• refactor: PRD-900 | Review glossary related cypress tests and extract… by @sumitappt in #9549
• fix(compose-profiles): hostname kafka-broker -> broker by @david-leifker in #9563
• build(spark-smoke): dump container logs for spark smoke test by @david-leifker in #9565
• fix(ingest/looker): limit dashboard object lifetime to thread by @hsheth2 in #9564
• feat(ingest): track thread count in ingestion report by @hsheth2 in #9566
• build(deps): bump @adobe/css-tools from 4.3.1 to 4.3.2 in /datahub-web-react by @dependabot in #9561
• Lower count for CSV download query from 500 to 200 to alleviate timeouts by @eboneil in #9568
• fix(ci): ignore binaries from test results upload by @hsheth2 in #9560
• fix(ci): fix label logic by @anshbansal in #9571
• bump(docker-compose): migrate to v2 docker compose cli by @david-leifker in #9569
• feat(ingestion/transformer): Add dataset dataproduct transformer by @shubhamjagtap639 in #9491
• Fix - Removed AWS S3 Platform by @Kunal-kankriya in #9581
• feat(entity-registry): entity registry plugins by @david-leifker in #9538
• Fix/glossary test by @Kunal-kankriya in #9584
• fix(owner): last modified on adding owner by @anshbansal in #9553
• fix(frontend): restrict redirect uri domain by @RyanHolstien in #9592
• docs(slack): add note about private Slack channels by @hsheth2 in #9589
• fix(ingest/snowflake): set platform instance for foreign keys by @hsheth2 in #9577
• fix(ingest/redshift): include table type in custom properties by @hsheth2 in #9576
• fix(graphql): fix npe in access mapper by @RyanHolstien in #9593
• fix(ingest/bigquery): support google-cloud-bigquery 3.15.0 by @hsheth2 in #9595
• Fix - updated verification steps by @Kunal-kankriya in #9598
• fix(ingestion/starburst-trino): Fix validation errors in TrinoJoinedAccessEvent by @shubhamjagtap639 in #9580
• feat(ingest/snowflake): improve accuracy of computed sample-based pro… by @mayurinehate in #9600
• feat(ingestion/fivetran): Add fivetran bigquery destination support by @shubhamjagtap639 in #9531
• feat: Auto-focus on entity profile action modals by @Salman-Apptware in #9444
• fix(ingest): support database_pattern in clickhouse and clickhouse-usage by @Gerrit-K in #9482
• feat(ingest/tableau): include platform_instance in browse v1 path by @hsheth2 in #9607
• Fix/nested domain test by @Kunal-kankriya in #9609
• feat(ingest/tableau): add retries on OSErrors by @hsheth2 in #9596
• feat(spring): update spring to version 6 and spring boot to version 3 by @RyanHolstien in #9567
• feat(ingest): key-partitioning for rest emitter by @hsheth2 in #9613
• fix(ingest/nifi): make s3 dataset urn consistent with s3 source by @mayurinehate in #9599
• Fix - Changed popup text visibility and removed unnecessary code by @Kunal-kankriya in #9611
• feat(ingest/looker): add retries to API calls by @hsheth2 in #9606
• fix(graphql): Add jsonProps to SchemaField type by @pedro93 in #9622
• feat(ingest/looker): add backpressure-aware executor by @hsheth2 in #9615
• fix(ingest/mongodb): Fix downsampling the collection schema output undetermined by @TonyOuyangGit in #9612
• Unskip "type" filter tests by @Kunal-kankriya in #9630
• feat(ingest/dbt): speed up dbt CLL by @hsheth2 in #9617
• Added Homepage Post Steps by @Kunal-kankriya in #9620
• feat(graphql): generate ToString method by @anshbansal in #9634
• fix(internal schema): changes to make internal schema registry more compatible with other schema registries by @darnaut in #9636
• fix(autocomplete): Filter default entity types based on those present in types map by @jjoyce0510 in #9625
• feat: inital champions page by @yoonhyejin in #9429
• docs: add intro video to the docs main page by @yoonhyejin in #9597
• feat(search): add support for empty aggregations through searchable annotation by @RyanHolstien in #9624
• docs: add events link by @hsheth2 in #9638
• feat(ingest/dbt-cloud): Add way to customize dbt cloud access endpoint by @treff7es in #9633
• build(ingest/feast): upgrade to latest feast version (0.35.x) by @cburroughs in #9627
• feat(classifier): Add support for excluding list of exact column names by @ethan-cartwright in #9472
• feat(cli): add threads to sink report by @anshbansal in #9642
• fix(extension) Fixes routing for the embed pages for chrome extension by @chriscollins3456 in #9648
• feat: add an endpoint to edit posts by @muzzacode in #9629
• feat(monitoring): add metrics for kafka consumers by @anshbansal in #9643
• feat(web): update aws logos by @rinzool in #9660
• fix(docs-website): update copy around the video on index by @hsheth2 in #9661
• fix(protobuf): fix reseved field error in fields in nested messages by @wngus606 in #9318
• feat(ingest/databricks): ingest hive metastore by default, more docs by @mayurinehate in #9601
• fix(docker):The datahub-frontend service failed to start when executing dev.sh #7616 by @yangjiandan in #7618
• feat(openapi): Implement getIndexSizes function from rest.li in openapi by @iprentic in #8730
• feat(ingest/sql-parsing): Support file backed dict in SqlParsingBuilder for lineage by @asikowitz in #9654
• feat(posts): add edit support for posts by @gaurav2733 in #9666
• feat(roles): Add support for roles in groups in GMS by @pedro93 in #9659
• fix(frontend): Add fallback for image load error on Avatar by @ingthorb in #9501
• fix(user-removal): resolve user removal list update issue by @gaurav2733 in #9671
• feat(ui) Add standardized GQL error handling function to FE by @chriscollins3456 in #9470
• feat(kafka): add health indicator for kafka by @RyanHolstien in #9662
• perf(entity-service): batch exists calls by @david-leifker in #9663
• feat(ingest/airflow): support airflow 2.8 dataset listeners by @hsheth2 in #9664
• fix(docs): Correct Kafka Connect sink documentation by @pedro93 in #9672
• docs(observability): add databricks as supported source by @mayurinehate in #9619
• feat(build): conditional ci by @david-leifker in #9673
• build(deps-dev): bump vite from 4.5.0 to 4.5.2 in /datahub-web-react by @dependabot in #9676
• feat(superset): add some custom properties for dashboards by @rinzool in #9670
• fix(superset): handle comma in dataset table name by @rinzool in #9656
• feat(openapi): allow swagger 3.x by @dim-ops in #9646
• feat(ingest/superset): add domains and platform_instance support by @rinzool in #9647
• fix(workflow): workflow tweaks by @david-leifker in #9678
• feat(ingest/databricks): view upstream lineage for hive metastore by @mayurinehate in #9657
• feat(ingest/dynamodb): add domain arg by @dim-ops in #9658
• feat(backend): structured properties and forms by @david-leifker in #9626
• feat(ingest/lookml): support complex lookml manifests by @hsheth2 in #9688
• docs: Add Case Studies and DataHub Basics to Resources tab by @maggiehays in #9687
• fix(ingestion/mssql): use platform_instance for mssql dataflow/datajob by @sleeperdeep in #9527
• Added to Verify Edit Post Steps by @Kunal-kankriya in #9683
• feat(ingest): improve sql tests + dbt reporting by @hsheth2 in #9649
• feat(ingest/sql-parser): test redshift temp tables by @hsheth2 in #9653
• feat(policies): Allow policies to be applied to resources based on tags by @pedro93 in #9684
• feat(role-assignment): add role assignment dropdown in the UI for groups by @gaurav2733 in #9689
• feat(ingest/looker): add browse paths for charts by @hsheth2 in #9639
• fix: Fix error message when attempting to delete home page posts with… by @Salman-Apptware in #9667
• feat(ui) Support auto-render aspects on remaining entity types by @chriscollins3456 in #9696
• fix(consumer): wait for internal schema registry by @darnaut in #9699
• feat(build): support jdk8 spark lineage by @david-leifker in #9697
• feat(ingest): add ODBC library and tools to base image by @darnaut in #9701
• feat (resolver): Add new endpoint for editing secrets by @muzzacode in #9665
• feat(ingest): handling for const in json schema by @anshbansal in #9694
• refactor(ingest): simplify adding aspects to MCEs in transformers by @hsheth2 in #9686
• docs(openapi): improve description fields by @dim-ops in #9690
• feat(docs): Add documentation on Incident Change Event by @pedro93 in #9709
• feat(ingest/dbt): support aws config without region by @hsheth2 in #9650
• fix(test): improve cypress tests by @david-leifker in #9711
• feat(ingest/tableau): map trino_jdbc platform type by @hsheth2 in #9708
• fix(oidc settings): effective JWS algorithm setting by @darnaut in #9712
• feat(ingest/metabase): Use new sql parser; reduce error reporting levels by @asikowitz in #9714
• fix(ingestion/bigquery): Table-view-snapshot Lineage Bug fix by @shubhamjagtap639 in #9579
• feat(ui) Add structured properties support in the UI by @chriscollins3456 in #9695
• docs(): Updating docs for assertions to correct databricks assertions support by @jjoyce0510 in #9713
• test(spark-lineage): minor tweaks by @david-leifker in #9717
• feat(search): support filtering on count type searchable fields for equality by @RyanHolstien in #9700
• feat(ui): Supporting rendering custom assertion descriptions by @jjoyce0510 in #9722
• infra(ui): Add a react context provider allowing sub-components to update theme conf by @jjoyce0510 in #9674
• fix(ingestion/metabase): Fetch Dashboards through Collections by @7onn in #9631
• fix(ingest/glue): Profiling breaks for non-partitioned tables due to absent Table.PartitionKeys by @KulykDmytro in #9591
• fix(search): fix filters for hasX and numValues fields by @RyanHolstien in #9729
• fix(ingest/airflow): fix plugin support for airflow 2.5.0 by @hsheth2 in #9719
• fix(cli): fix example data contract yaml + update airflow codecov by @hsheth2 in #9707
• fix(ingest/metabase): add missing sql parser dep by @hsheth2 in #9725
• feat(glossary-term-dropdown): include parent term groups in dropdown by @gaurav2733 in #9715
• fix(ingest/s3): Not sorting schema fields to keep original order by @treff7es in #9349
• fix(ingest/test): Fixing breaking change in moto 5.0 library by @treff7es in #9736
• build(graphql): simplify gradle graphql codegen task by @shirshanka in #9734
• feat(edit-secret): frontend support to edit secrets by @gaurav2733 in #9737
• chore(cli): drop support for python 3.7 by @hsheth2 in #9731
• docs: Add slack survey page by @yoonhyejin in #9590
• feat(platform): add support for via nodes by @shirshanka in #9733
• feat(ingestion/tableau) Ingest tags from datasource by @eboneil in #9730
• feat(ingest/okta): option to only ingest users from filtered groups by @anshbansal in #9738
• feat(cli): add sibling helper CLI command by @anshbansal in #9744
• chore(gms): allow adjusting the log level for gms debug log file by @darnaut in #9743
• build(ingestion-smoke): fix smoke dockerfile by @david-leifker in #9724
• feat(cli): option to init via username password, add lint for smoke-test by @anshbansal in #9675
• feat(forms) Update form model to include searchable on actor assignment by @chriscollins3456 in #9747
• feat(ingestion/dbt): multiple node owner separated by comma by @sid-acryl in #9740
• fix(mceConsumer): prevent health endpoint authentication failure by @RyanHolstien in #9748
• fix(ingest/transformer): generate ownership aspect from handle_end_of_stream by @sid-acryl in #9720
• fix(aspects): fix default aspect generation for non-restli locations by @david-leifker in #9746
• chore(compose): mysql compose healthcheck by @david-leifker in #9754
• chore(gradle): datahub-frontend docker dependency avoid yarnInstall by @david-leifker in #9758
• fix(search): fix default entities for aggregation filters by @david-leifker in #9761
• feat(ingestion/redshift): collapse lineage to permanent table by @sid-acryl in #9704
• fix(ingest): upgrade pytest-docker by @hsheth2 in #9765
• fix(patch): patch builder import TagUrn by @david-leifker in #9764
• feat(forms) Add support for metadata forms on entities in the UI by @chriscollins3456 in #9718
• chore(ingest/feast): drop redundant typeguard constraint by @cburroughs in #9770
• feat(ingestion/feast): loosen feast dependency by @cburroughs in #9769
• feat(ci): check for packages, change location of lint by @anshbansal in #9759
• feat(tag-filter): Add tag as filer in policy creation page by @gaurav2733 in #9756
• feat(ingest/metabase): Allow to configure a different url for links by @ilpianista in #9727
• docs(elastricsearch): add description fiels by @dim-ops in #9693
• fix(ingest/snowflake): stop pinning transitive spacy dependency by @cburroughs in #9768
• feat(cli): Add support for custom ownership types to dataproduct CLI by @skrydal in #9762
• docs: update champion bio by @yoonhyejin in #9773
• fix(ingest/fivetran): Fixing issue when connector user is empty by @treff7es in #9774
• feat(ingest/glossary): support pydantic v2 in business-glossary source by @hsheth2 in #9763
• fix(datahub-flow-diagram): add missing image for onboarding tour by @gaurav2733 in #9783
• Update aws.md by @scalvanese452 in #9726
• fix(gms/search):Fix missing mapping of DOUBLE searchable type by @siladitya2 in #9651
• chore(lint): apply spotless by @david-leifker in #9789
• docs(observe): add upsert assertion monitor graphql examples by @mayurinehate in #9766
• fix(docs): fix the subtitle on the slack page by @yoonhyejin in #9781
• feat(airflow) Override datajob external_url by @gp1105739 in #9681
• fix(browsev2): align browse and aggregate queries by @RyanHolstien in #9790
• misc: datahub-upgrade improvements, aspect key & default aspects fixes by @david-leifker in #9796
• chore(pegasus): upgrade pegasus, remove gradle 8 hacks for pegasus plugin by @david-leifker in #9798
• feat(graphql): get raw aspects for assertions, allow aspectNames filter by @mayurinehate in #9792
• feat(ingest/slack): source to get user info from slack by @anshbansal in #9776
• Added mutiple tests scenarios by @Kunal-kankriya in #9750
• feat(token): helper to debug owner of raw token by @anshbansal in #9793
• fix(forms) Update styles of submit/confirm button and fix num input bug by @chriscollins3456 in #9797
• feat(structured-props) Extend structured props support to other entity types by @chriscollins3456 in #9780
• feat(recommendations): Correct searchable entities for top platform by @pedro93 in #9800
• feat(ingest/dbt): speed up test result only ingestion by @hsheth2 in #9791
• feat(access-token-list): admin being able to view all org access tokens by @gaurav2733 in #9803
• feat(graphql): add configuration for limiting graphql query complexity by @RyanHolstien in #9802
• feat(docs): embed luma calendar widget by @yoonhyejin in #9784
• docs: fix links to metadata modeling by @rubiojr in #9685
• feat(ingest): sql parsing aggregator by @hsheth2 in #9786
• feat(ingest/glossary): handle ownership type in business glossary connector by @sgomezvillamor in #9805
• test(ingest): use pytest-random-order for unit tests by @hsheth2 in #9753
• feat(cli): make docker compose project name configurable by @hsheth2 in #9808
• feat(ingest/oracle): support changing data dictionary (ALL_ or DBA_) by @sleeperdeep in #8873
• feat(quickstart): remove container_name from compose files by @hsheth2 in #9809
• fix(cli): fix quickstart command with pydantic v2 by @hsheth2 in #9806
• docs: update docs referencing jdk8 by @hsheth2 in #9807
• feat(openapi): allow Bearer token by @dim-ops in #9669
• Upgrade shiro-core to 1.13 by @gp1105739 in #9818
• fix(mysql-setup): specify charset and collation when creating table by @darnaut in #9817
• feat(build): upgrade fastapi and gitdb due to vulnerablities by @gp1105739 in #9819
• fix(ingest/profiling): ignore limit/offset when counting table row count in profiling by @treff7es in #9635
• build(deps): bump follow-redirects from 1.15.2 to 1.15.4 in /docs-website by @dependabot in #9603
• fix(forms) Set form prompts default not required by @chriscollins3456 in #9820
• refactor(graphql): consolidate cache for stats by @david-leifker in #9826
• feat(forms) Extend forms support to other entity types by @chriscollins3456 in #9801
• doc(policies): consistent descriptions for root user policies by @anshbansal in #9823
• doc(acryl cloud): release notes for 0.2.15.1 by @anshbansal in #9837
• fix(forms) Prevent schema field prompts from being required by @chriscollins3456 in #9831
• fix(ui/token): token page failing to render by @anshbansal in #9827
• fix(ui/lineage): Do not crash on query entity by @asikowitz in #9836
• feat(): Adding supports for data incidents for datasets, dashboards, charts, data jobs, data flows by @jjoyce0510 in #9710
• feat(ingest): reporting logging tweaks by @hsheth2 in #9835
• feat(operations): add explain endpoint for search by @RyanHolstien in #9832
• fix(search): fix inequality filters for count fields by @RyanHolstien in #9858
• feat(ingest): clean up reporting methods by @hsheth2 in #9848
• feat(graphql): Miscellaneous query additions by @asikowitz in #9855
• refactor(ui): Update eslint and rearrange imports by @asikowitz in #9854
• feat(models,ui): Support query nodes in lineage by @asikowitz in #9850
• feat(ui): Add and update platform logos by @asikowitz in #9849
• feat(ingest/dynamoDB): flatten struct fields by @TonyOuyangGit in #9852
• fix(ingest/looker): update for Looker query API breaking change by @feldjay in #9865
• chore(graphql): bump graphql engine version by @david-leifker in #9864
• fix(ingest/redash): Limit size of RedashSourceReport by @atjones0011 in #9873
• feat(forms) Add batchRemoveForm graphql endpoint by @chriscollins3456 in #9840
• fix(lineage): logging reduction and fixes by @RyanHolstien in #9878
• feat(ingest): add escape hatch methods to SqlParsingAggregator by @hsheth2 in #9860
• fix(ingest): better logging line attribution by @hsheth2 in #9876
• fix(ingest): bump sqlglot + add some debug info to tests by @hsheth2 in #9867
• docs(ingest): document remote for ingest deploy by @gabe-lyons in #9897
• build(workflow): test enable workflow for docs change by @david-leifker in #9901
• feat(ingest/mongodb): improve sorting when downsampling collection schema by @TonyOuyangGit in #9856
• feat(ingest/snowflake): speed up snowflake usage query by @hsheth2 in #9899
• feat(structured-properties): soft delete by @david-leifker in #9812
• feat(ingest): add timeout to sqlglot parser by @hsheth2 in #9868
• feat(models): Add subtypes aspect to dataFlow by @asikowitz in #9866
• fix(cli): add-sibling preserve existing siblings by @stevenayers in #9907
• feat(ingest/redshift): redshift lineage v2 by @hsheth2 in #9904
• build(frontend): Add tsc to lint by @asikowitz in #9911
• fix(posts): add padding to post link card by @gaurav2733 in #9883
• feat(ingestion/trino): Add sibling support in ingestion by @shubhamjagtap639 in #9853
• PRD-1069 | Changed secret name by @Kunal-kankriya in #9917
• feat(policies): Make policies searchable by privilege, type, status or editable fields by @pedro93 in #9877
• feat(ci): exempt more names from being marked as community by @anshbansal in #9918
• feat(ingest/qlik): Qlik cloud connector integration by @shubhamjagtap639 in #9682
• feat(ingest): use uv for python package installs by @hsheth2 in #9885
• feat(ingest/tableau): add non-embedded custom sql to project container by @hsheth2 in #9875
• fix(ingest/patch): Adding proper json patch path quoting by @treff7es in #9872
• feat(token-status-filter): add filter for expired and all tokens by @gaurav2733 in #9930
• feat(ingest/slack): add option to ingest slack channels by @anshbansal in #9884
• feat(ingest/csv-enricher): add support for ownership type urn by @anshbansal in #9935
• feat(patch): refactor cll patch by @RyanHolstien in #9922
• fix(pr): make prs only pull changes from pr branch instead of head merge by @RyanHolstien in #9937
• feat(ingest): loosen sentry requirements by @hsheth2 in #9936
• fix(react): include flow diagram in docs build by @hsheth2 in #9785
• fix(ingest/mysql): Adding support for bit type by @treff7es in #9950
• feat(ingest/patch): add helper for auto-quoting by @hsheth2 in #9938
• feat(ingest): Support for JSONL in s3 source with max_rows support by @Adityamalik123 in #9921
• feat(ingest): fix bugs in SqlParsingAggregator by @hsheth2 in #9926
• fix(ui/schema): error handling add by @anshbansal in #9952
• PRD-1067 | added owner in gen_datahub_groups and condition for maximu… by @Kunal-kankriya in #9888
• feat(ci): skip smoke-test lint if there are no changes by @hsheth2 in #9945
• feat(search): search access controls by @david-leifker in #9892
• feat(ingest/sql-parser): add alias for mariadb by @hsheth2 in #9956
• docs(ingest/lookml): update known discrepancy list by @hsheth2 in #9941
• chore(vulnerability): Bumped up versions for vulnerability fix by @pankajmahato-visa in #9929
• Revert "chore(vulnerability): Bumped up versions for vulnerability fi… by @RyanHolstien in #9961

New Contributors

• @KaYunKIM made their first contribution in #9411
• @BlueHorn07 made their first contribution in #9300
• @nnnkkk7 made their first contribution in #9427
• @ryaminal made their first contribution in #9460
• @fer-marino made their first contribution in #9492
• @diegoreico made their first contribution in #9295
• @Kunal-kankriya made their first contribution in #9547
• @Gerrit-K made their first contribution in #9482
• @muzzacode made their first contribution in #9629
• @ingthorb made their first contribution in #9501
• @dim-ops made their first contribution in #9646
• @sleeperdeep made their first contribution in #9527
• @7onn made their first contribution in #9631
• @sid-acryl made their first contribution in #9740
• @ilpianista made their first contribution in #9727
• @scalvanese452 made their first contribution in #9726
• @gp1105739 made their first contribution in #9681
• @rubiojr made their first contribution in #9685
• @atjones0011 made their first contribution in #9873
• @stevenayers made their first contribution in #9907
• @Adityamalik123 made their first contribution in #9921
• @pankajmahato-visa made their first contribution in #9929

Full Changelog: v0.12.1...v0.13.0