DataHub v0.8.9

Release Highlights

• Support for nested structs, union types and key-value schemas in Kafka
• Support for JDBC Connector based sources in Kafka Connect
• Support for Okta as a source for User and Group metadata
• Support for using AWS Glue schema registry

Breaking Changes

• [#3079] : Introduces a change to fieldPath encoding in schema metadata. Note: This is a backwards compatible change for the storage layer. Old fieldPaths will still be rendered correctly. At read time, fieldPaths in the new encoding will be translated to the old encoding to discover tags written before this change. Tags and Descriptions applied to fields earlier (which were being stored in the old format) will be migrated on applying new tags or editing descriptions.

Important Bug Fixes

• [#3070] Charts and Dataset lineage was broken in release 0.8.8. This has been fixed via [gma-125]

ChangeLog

• #3093 @gabe-lyons fix: fixing key-value after adding version
• #3088 @dexter-mh-lee fix(mysql-setup): Change default charset to utf8mb4
• #3091 @gabe-lyons feat: Adding clarity around qualified unions and removing extra lines for structs
• #3090 @dexter-mh-lee feat(workflow): Add mysql/postgres setup workflow
• #3083 @dexter-mh-lee feat: add support for AWS glue schema registry
• #3043 @jjoyce0510 feat(ingest): Adding an Okta Integration to extract Users, Groups, Group Membership
• #3080 @gabe-lyons fix(react): bolding field name if single token
• #3081 @shirshanka docs: Update Aug town-hall dates and previous town-hall details
• #3079 @rslanka feat: Adding support for nested schemas in ingestion and visualization
• #3078 @kevinhu fix(ingest): sqlalchemy-snowflake add constraints to make sure we don't pull in 1.2.5
• #2987 @jjoyce0510 docs(deploy): Adding confluent cloud doc
• #3076 @chinmay-bhat feat(ingest): add support for jdbc connector to kafka-connect source
• #3071 @kevinhu feat(docs): link to SQL profiling docs from each SQL source
• #3074 @jjoyce0510 refactor(build): Remove unnecessary ext modules.
• #3073 @dexter-mh-lee fix(docker): remove unnecessary components from docker-compose
• #3072 @dexter-mh-lee fix(docker): upgrade base image version
• #3068 @kevinhu feat(docs): add overrides for sidebar labels and S3 guide in sources dropdown
• #3069 @kevinhu fix(ingest): remove tags from bootstrap_mce since that is deprecated
• #3070 @gabe-lyons chore: upgrading gma to 0.2.80
• #2990 @rahulbsw feat(ingest): Added support for "add dataset ownership by regex match"
• #3067 @kevinhu fix(ingest): apply case insensitive regex matching by default
• #3066 @aseembansal-gogo docs(ingest): fix typos
• #3041 @kevinhu feat(docs): reorder and restyle navbar
• #3062 @gabe-lyons feat(cli): datahub init & docs for it
• #3064 @dexter-mh-lee fix(quickstart): remove mem_limit for datahub containers
• #3059 @kevinhu feat(docs): link to ingestion quickstart under ingestion section
• #3058 @kevinhu docs(ingest): link to docs from recipes
• #3014 @jjoyce0510 chore(frontend): Remove unused files 1/4
• #2986 @aseembansal-gogo feat(ingest): add transformers to clear dataset ownership, mark status, add browse paths
• #3056 @dexter-mh-lee fix(ingest): stop looker source from unnecessarily filling out owners
• #3055 @dexter-mh-lee fix(ingest): add default configurable timeout for rest emitter
• #3039 @kevinhu fix(docs): update metadata ingestion dev guide
• #3031 @kevinhu feat(docs): refactor source and sink ingestion docs
• #3054 @gabe-lyons fix(frontend): fixing homepage jitter
• #3051 @gabe-lyons feat(quickstart): linking to slack from quickstart
• #3053 @jjoyce0510 docs: Add Exact match search CURL example.
• #3033 @kevinhu feat(ingest): replace and warn against relative imports
• #3035 @aseembansal-gogo feat(ingest): add underlying platform for glue
• #3052 @jjoyce0510 docs: Add tags GMS API documentation
• #2973 @saxo-lalrishav fix(Business Glossary): updated glossary term search strategy
• #2996 @jensenity feat(postgres): add postgres setup docker image
• #3044 @gabe-lyons fix(frontend): fixing external url logic in charts and dashboard mapper
• #3045 @gabe-lyons fix(frontend): hide dashboard date when null
• #3022 @jsotelo fix(frontend): add support for SASL_KERBEROS_SERVICE_NAME & SASL_PLAINTEXT
• #3036 @jjoyce0510 fix(frontend): Fix exception casting in EntityClient
• #3037 @kevinhu fix(ingest): detect malformed Glue S3 script paths
• #3038 @kevinhu fix(ingest): replace backticks for lookml
• #3040 @kevinhu fix(ingest): add bigquery type mappings
• #3042 @gabe-lyons fix(frontend): fixing lineage tokenization
• #3034 @jjoyce0510 fix(gms): Validate unrecognized model fields.