Skip to content

Commit a2f84f3

Browse files
alambalamb
authored
Docs: Consolidate feature proposal content into roadmap (apache#17156)
* Docs: Consolidate feature proposal content into roadmap * demote to proper headings
1 parent 75eb80c commit a2f84f3

File tree

2 files changed

+73
-70
lines changed

2 files changed

+73
-70
lines changed

‎docs/source/contributor-guide/index.md‎

Lines changed: 3 additions & 68 deletions
Original file line numberDiff line numberDiff line change
@@ -37,14 +37,15 @@ You can find how to setup build and testing environment [here](https://datafusio
3737
## Finding and Creating Issues to Work On
3838

3939
You can find a curated [good-first-issue] list to help you get started.
40+
You can read about how we plan larger projects in the [Roadmap and Improvement Proposals](roadmap.md) section.
4041

4142
[good-first-issue]: https://github.com/apache/datafusion/issues?q=is%3Aissue+is%3Aopen+label%3A%22good+first+issue%22
4243

4344
### Open Contribution and Assigning tickets
4445

4546
DataFusion is an open contribution project, and thus there is no particular
46-
project imposed deadline for completing any issue or any restriction on who can
47-
work on an issue, nor how many people can work on an issue at the same time.
47+
project imposed deadline for completing issues or restrictions on who can
48+
work on an issue, nor limits to how many people can work on an issue at the same time.
4849

4950
Contributors drive the project forward based on their own priorities and
5051
interests and thus you are free to work on any issue that interests you.
@@ -62,72 +63,6 @@ unable to make progress you should unassign the issue by using the `unassign me`
6263
link at the top of the issue page (and ask for help if are stuck) so that
6364
someone else can get involved in the work.
6465

65-
### Discussing New Features
66-
67-
If you plan to work on a new feature that doesn't have an existing ticket, it is
68-
a good idea to open a ticket to discuss the feature. Advanced discussion often
69-
helps avoid wasted effort by determining early if the feature is a good fit for
70-
DataFusion before too much time is invested. Discussion on a ticket can help
71-
gather feedback from the community and is likely easier to discuss than a 1000
72-
line PR.
73-
74-
If you open a ticket and it doesn't get any response, you can try `@`-mentioning
75-
recently active community members in the ticket to get their attention.
76-
77-
### What Contributions are Good Fits?
78-
79-
DataFusion is designed to be highly extensible, and many features can be
80-
implemented as extensions without changes or additions to the core. Support for
81-
new functions, data formats, and similar functionality can be added using those
82-
extension APIs, and there are already many existing community supported
83-
extensions listed in the [extensions list].
84-
85-
Query engines are complex pieces of software to develop and maintain. Given our
86-
limited maintenance bandwidth, we try to keep the DataFusion core as simple and
87-
focused as possible, while still satisfying the [design goal] of an easy to
88-
start initial experience.
89-
90-
With that in mind, contributions that meet the following criteria are more likely
91-
to be accepted:
92-
93-
1. Bug fixes for existing features
94-
2. Test coverage for existing features
95-
3. Documentation improvements / examples
96-
4. Performance improvements to existing features (with benchmarks)
97-
5. "Small" functional improvements to existing features (if they don't change existing behavior)
98-
6. Additional APIs for extending DataFusion's capabilities
99-
7. CI improvements
100-
101-
Contributions that will likely involve more discussion (see Discussing New
102-
Features above) prior to acceptance include:
103-
104-
1. Major new functionality (even if it is part of the "standard SQL")
105-
2. New functions, especially if they aren't part of "standard SQL"
106-
3. New data sources (e.g. support for Apache ORC)
107-
108-
[extensions list]: ../library-user-guide/extensions.md
109-
[design goal]: https://docs.rs/datafusion/latest/datafusion/index.html#design-goals
110-
111-
### Design Build vs. Big Up Front Design
112-
113-
Typically, the DataFusion community attacks large problems by solving them bit
114-
by bit and refining a solution iteratively on the `main` branch as a series of
115-
Pull Requests. This is different from projects which front-load the effort
116-
with a more comprehensive design process.
117-
118-
By "advancing the front" the community always makes tangible progress, and the strategy is
119-
especially effective in a project that relies on individual contributors who may
120-
not have the time or resources to invest in a large upfront design effort.
121-
However, this "bit by bit approach" doesn't always succeed, and sometimes we get
122-
stuck or go down the wrong path and then change directions.
123-
124-
Our process necessarily results in imperfect solutions being the "state of the
125-
code" in some cases, and larger visions are not yet fully realized. However, the
126-
community is good at driving things to completion in the long run. If you see
127-
something that needs improvement or an area that is not yet fully realized,
128-
please consider submitting an issue or PR to improve it. We are always looking
129-
for more contributions.
130-
13166
# Developer's guide
13267

13368
## Pull Request Overview

‎docs/source/contributor-guide/roadmap.md‎

Lines changed: 70 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -17,7 +17,7 @@ specific language governing permissions and limitations
1717
under the License.
1818
-->
1919

20-
# Roadmap
20+
# Roadmap and Improvement Proposals
2121

2222
The [project introduction](../user-guide/introduction) explains the
2323
overview and goals of DataFusion, and our development efforts largely
@@ -44,7 +44,7 @@ make review efficient and avoid surprises.
4444

4545
[The current list of `EPIC`s can be found here](https://github.com/apache/datafusion/issues?q=is%3Aissue+is%3Aopen+epic).
4646

47-
# Quarterly Roadmap
47+
## Quarterly Roadmap
4848

4949
The DataFusion roadmap is driven by the priorities of contributors rather than
5050
any single organization or coordinating committee. We typically discuss our
@@ -56,3 +56,71 @@ For more information:
5656
1. [Search for issues labeled `roadmap`](https://github.com/apache/datafusion/issues?q=is%3Aissue%20%20%20roadmap)
5757
2. [DataFusion Road Map: Q3-Q4 2025](https://github.com/apache/datafusion/issues/15878)
5858
3. [2024 Q4 / 2025 Q1 Roadmap](https://github.com/apache/datafusion/issues/13274)
59+
60+
## Improvement Proposals
61+
62+
### Discussing New Features
63+
64+
If you plan to work on a new feature that doesn't have an existing ticket, it is
65+
a good idea to open a ticket to discuss the feature. Advanced discussion often
66+
helps avoid wasted effort by determining early if the feature is a good fit for
67+
DataFusion before too much time is invested. Discussion on a ticket can help
68+
gather feedback from the community and is likely easier to discuss than a 1000
69+
line PR.
70+
71+
If you open a ticket and it doesn't get any response, you can try `@`-mentioning
72+
recently active community members in the ticket to get their attention.
73+
74+
### What Contributions are Good Fits?
75+
76+
DataFusion is designed to be highly extensible, and many features can be
77+
implemented as extensions without changes or additions to the core. Support for
78+
new functions, data formats, and similar functionality can be added using those
79+
extension APIs, and there are already many existing community supported
80+
extensions listed in the [extensions list].
81+
82+
Query engines are complex pieces of software to develop and maintain. Given our
83+
limited maintenance bandwidth, we try to keep the DataFusion core as simple and
84+
focused as possible, while still satisfying the [design goal] of an easy to
85+
start initial experience.
86+
87+
With that in mind, contributions that meet the following criteria are more likely
88+
to be accepted:
89+
90+
1. Bug fixes for existing features
91+
2. Test coverage for existing features
92+
3. Documentation improvements / examples
93+
4. Performance improvements to existing features (with benchmarks)
94+
5. "Small" functional improvements to existing features (if they don't change existing behavior)
95+
6. Additional APIs for extending DataFusion's capabilities
96+
7. CI improvements
97+
98+
Contributions that will likely involve more discussion (see Discussing New
99+
Features above) prior to acceptance include:
100+
101+
1. Major new functionality (even if it is part of the "standard SQL")
102+
2. New functions, especially if they aren't part of "standard SQL"
103+
3. New data sources (e.g. support for Apache ORC)
104+
105+
[extensions list]: ../library-user-guide/extensions.md
106+
[design goal]: https://docs.rs/datafusion/latest/datafusion/index.html#design-goals
107+
108+
### Design Build vs. Big Up Front Design
109+
110+
Typically, the DataFusion community attacks large problems by solving them bit
111+
by bit and refining a solution iteratively on the `main` branch as a series of
112+
Pull Requests. This is different from projects which front-load the effort
113+
with a more comprehensive design process.
114+
115+
By "advancing the front" the community always makes tangible progress, and the strategy is
116+
especially effective in a project that relies on individual contributors who may
117+
not have the time or resources to invest in a large upfront design effort.
118+
However, this "bit by bit approach" doesn't always succeed, and sometimes we get
119+
stuck or go down the wrong path and then change directions.
120+
121+
Our process necessarily results in imperfect solutions being the "state of the
122+
code" in some cases, and larger visions are not yet fully realized. However, the
123+
community is good at driving things to completion in the long run. If you see
124+
something that needs improvement or an area that is not yet fully realized,
125+
please consider submitting an issue or PR to improve it. We are always looking
126+
for more contributions.

0 commit comments

Comments
 (0)