create new datafusion-execution crate, start splitting code out
#5432
Conversation
more disk_manager move registry
alamb
changed the title
datafusion-execution crate, start splitting code out
| pub mod runtime_env; | ||
|
|
||
| // backwards compatibility | ||
| pub use datafusion_execution::disk_manager; |
There was a problem hiding this comment.
I moved three modules in this first PR -- runtime_env will take a little more work as will context so I plan to do them as smaller follow on PRs
| @@ -310,6 +310,7 @@ dot -Tsvg dev/release/crate-deps.dot > dev/release/crate-deps.svg | |||
| (cd datafusion/row && cargo publish) | |||
| (cd datafusion/physical-expr && cargo publish) | |||
| (cd datafusion/optimizer && cargo publish) | |||
| (cd datafusion/execution && cargo publish) | |||
There was a problem hiding this comment.
I think this is the biggest change (a new crate to publish) -- cc @andygrove
|
Benchmark runs are scheduled for baseline = f9f40bf and contender = 17b2f11. 17b2f11 is a master commit associated with this PR. Results will be available as each benchmark for each run completes. |
Which issue does this PR close?
Part of splitting out physical plan: #1754
First part of #5405
Rationale for this change
the
datafusioncrate is quite large as it has all the physical plan code as well as the physical optimizerI would like to split out physical_plan and physical_optimizer (to mimic datafusion_expr and datafusion_optimizer).
However in order to do so I need to move parts of datafusion that they depend on to a new module
What changes are included in this PR?
Changes:
datafusion-executioncratedatafusion/core/src/executionto that new crateItems remaining (planned as follow on PRs):
I plan to leave
SessionContextin the core datafusion crate as it pulls everything together (relies on everything)Are these changes tested?
Covered by existing tests
Are there any user-facing changes?
There will be a new crate, but otherwise no