Enable Change Data Feed

[TioLuiso]

We are developing an application for Azure Databricks. We are using DBR 8.3, which has the experimental feature of Change Data Feed.
For testing purposes, we have a dockerized local environment with Spark 3.1.2 and spark-delta 1.0.0. When we want to create a CDF enabled table, we use:

DeltaTable.createIfNotExists(spark).tableName("TableName").addColumns(schema).location(path).property("delta.enableChangeDataFeed", "true").execute()

When we try to execute that, we receive the error:

Configuration delta.enableChangeDataFeed cannot be set. Change data feed from Delta is not yet available.

Is there anyway to enable this for local testing purposes?

[tdas]

CDF is only available in Delta on Databricks right now. So using Delta 1.0 locally will not allow you to use CDF. I suggest contacting your Databricks contact person to discuss this further.

[TioLuiso]

Thanks a lot for your response
Ok, I see.
Is it expected that eventually CDF could be used out of Databricks? Is there something in the works there?
Anyway, thanks a lot

[dennyglee]

Hi @TioLuiso - a great way to chime in is to vote in the Delta OSS 2021H2 Roadmap - here's the link specifically to CDF. HTH!

[scottsand-db]

@TioLuiso - see #1105.

[igvog]

Looking forward to adding CDF to Delta OSS! @dennyglee

[dennyglee]

Definitely! We're marching towards Delta Lake 2.0 (the RC became available for testing in late June). Test it out now if you can and provide us feedback, eh?! :)

[zsxwing]

2.0.0 has been released. Closing this as a duplicate of #1105