Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Minor: Update quarterly_roadmap.md #8044

Closed
wants to merge 3 commits into from
Closed

Minor: Update quarterly_roadmap.md #8044

wants to merge 3 commits into from

Conversation

alamb
Copy link
Contributor

@alamb alamb commented Nov 3, 2023

Which issue does this PR close?

N/A

Rationale for this change

In general I think it would be good for us to communicate more broadly what we are working on, for at least two reasons:

  1. So people who may be interested / impacted have a chance to weigh in on designs
  2. To help find people to help with projects that may need additional assistance

I would love additional ideas about how to do this -- maybe weekly summaries of "this week in DataFusion" 🤔 Any takers for doing so?

What changes are included in this PR?

Update quarterly roadmap. See rendered version here: https://github.com/apache/arrow-datafusion/blob/alamb-patch-1/docs/source/contributor-guide/quarterly_roadmap.md

Are these changes tested?

Are there any user-facing changes?

@alamb alamb changed the title Update: Update quarterly_roadmap.md Minor: Update quarterly_roadmap.md Nov 3, 2023
@alamb alamb marked this pull request as ready for review November 3, 2023 17:00
- Improve data output (`COPY`, `INSERT` and DataFrame) output capability [#6569](https://github.com/apache/arrow-datafusion/issues/6569)
- Implementation of `ARRAY` types and related functions [#6980](https://github.com/apache/arrow-datafusion/issues/6980)
- Write an industrial paper about DataFusion for SIGMOD [#6782](https://github.com/apache/arrow-datafusion/issues/6782)
- Faster Merging with Parallel Cascaded Merge: [#7181](https://github.com/apache/arrow-datafusion/issues/7181)
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

FYI @wiedld

@alamb alamb requested a review from andygrove November 3, 2023 17:03
Copy link
Contributor Author

@alamb alamb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There are several potential non trivial changes underway that I think would benefit significantly from additional help - both design feedback, as well as code review, and assistance communicating, etc.

I would really love to help them all but I am running out of time, and I want to be as clear as possible that I am unlikely to be able to help with all of this.

My ask for other maintainers to offer to help guide contributors through some of the following efforts, ideally focused on areas that are aligned with their interests


- Improved planning speed, especially for schemas with large numbers of columns [#5637](https://github.com/apache/arrow-datafusion/issues/5637) / [#7698](https://github.com/apache/arrow-datafusion/issues/7698)
- Make it easier to customize DataFusion, with split out function packages, etc. - [#7977](https://github.com/apache/arrow-datafusion/issues/7977)
- User Defined Types [#7923](https://github.com/apache/arrow-datafusion/issues/7923)
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fyi @yukkit

- Improved planning speed, especially for schemas with large numbers of columns [#5637](https://github.com/apache/arrow-datafusion/issues/5637) / [#7698](https://github.com/apache/arrow-datafusion/issues/7698)
- Make it easier to customize DataFusion, with split out function packages, etc. - [#7977](https://github.com/apache/arrow-datafusion/issues/7977)
- User Defined Types [#7923](https://github.com/apache/arrow-datafusion/issues/7923)
- ORC Support [#4707](https://github.com/apache/arrow-datafusion/issues/4707)
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The following work items are still in the planning / proposal stage.

- Improved planning speed, especially for schemas with large numbers of columns [#5637](https://github.com/apache/arrow-datafusion/issues/5637) / [#7698](https://github.com/apache/arrow-datafusion/issues/7698)
- Make it easier to customize DataFusion, with split out function packages, etc. - [#7977](https://github.com/apache/arrow-datafusion/issues/7977)
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@alamb
Copy link
Contributor Author

alamb commented Dec 12, 2023

This seems not to have garnered much attention, so let's just shut it down

@alamb alamb closed this Dec 12, 2023
@alamb alamb deleted the alamb-patch-1 branch January 17, 2024 14:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants