[Umbrella] Support Streaming Lakehouse for Lance table format

[wuchong]

Search before asking

• I searched in the issues and found nothing similar.

Motivation

It's possible to use Fluss in multimodal AI use cases as a real-time pipeline, supporting ingestion of multimodal data, and integrating with Lance table format in the Streaming Lakehouse architecture.

Solution

No response

Anything else?

No response

Willingness to contribute

• I'm willing to submit a PR!

[beryllw]

I'm interested in this Lance format integration issue. Could you please assign it to me?
I'd like to know the expected timeline for completion. I'll prepare a design draft document promptly.

[xx789633]

@beryllw Hi, this integration is almost finished. You can check my branch here: https://github.com/cwang9208/fluss/tree/lance

[beryllw]

Hi, this integration is almost finished.

Well done, looking forward to your PR.

[xx789633]

• [Lance connector] implement LanceLakeCatalog #1235

[wuchong]

• [Lance connector] implement LanceLakeCatalog #1235

Hi @cwang9208 , you can continue to edit this comment to include more issues. We can help to organize them into sub-issues.

[naivedogger]

[connector] Pass custom properties to lake writer. #1283