arrow-rs contains a copy of the "format" directory

[andygrove]

Describe the bug
This repo contains a copy of the "format" directory, containing flatbuffer definitions. We should instead reference the format directory from the arrow monorepo.

[nevi-me]

What would be a good approach to keep a copy of the *fbs files? I wonder if it could make sense to create a repository called arrow-format, so that this can be shared by various repos.

Should I bring this up on the ML?

[jorgecarleitao]

What would be a good approach to keep a copy of the *fbs files? I wonder if it could make sense to create a repository called arrow-format, so that this can be shared by various repos.

Should I bring this up on the ML?

Ahahah, was about to ping you for help here. :)

AFAIK arrow-flight/build.rs checks for changes on the definitions. So, even as a repo, it requires everyone to clone it, or us to submodule it.

An alternative is to simply remove the build.rs and perform the update manually; the definition does not change so often, anyways.

I agree that a lightweight repo with the specification alone would help.

[nevi-me]

I agree that a lightweight repo with the specification alone would help.

I've been out of the loop on the discussions, but from what i've been able to follow, it looks like Julia might also get its own repo, to move development to the apache org.

So I think if the format repo gets treated like the parquet-format as an arrow-format, it could work out well as a lightweight repo.

[nevi-me]

I've sent a message to the ML, to ask about this

[elferherrera]

Is it possible to link this "format" folder to the arrow "format" folder?

[nevi-me]

@andygrove we'll maintain a copy, and sync with the folder in the arrow repo when there are changes.

Can we close this?

[andygrove]

Thanks for the update @nevi-me. Closing this now.