Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

SymPEP Purpose and Process and SymPEP Template #2

Merged
merged 16 commits into from
Sep 18, 2023
222 changes: 222 additions & 0 deletions SymPEP-0001.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,222 @@
# SymPEP 1 --- SymPEP Purpose and Process

**Author** Aaron Meurer
**Status** Draft
**Type** Process
**Created** 2021-01-27
**Resolution**
moorepants marked this conversation as resolved.
Show resolved Hide resolved

## Abstract

SymPEPs are a formal process whereby important changes are proposed and
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
SymPEPs are a formal process whereby important changes are proposed and
SymPEPs are a formal process whereby important controversial changes are proposed and

I just read this "Matplotlib Enhancement Proposals (MEP), inspired by cpython's PEP's but less formal, are design documents for large or controversial changes to Matplotilb." and liked the word "controversial". This may help people defer to a normal PR or mailing list discussion before coming to make a SYMPEP.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this is the most significant point that's missing here, and it really deserves more text than just a single word. It isn't clear just what sorts of changes require a SymPEP and which changes can just be an issue or PR. I really think the default should be to not have a SymPEP, at least for codebase changes (process/governance changes are different). This isn't CPython. We don't need a proposal just to add or change anything.

Actually I'm not sure "controversial" is the right word to use here. A very big change may be uncontroversial, at least in principle, but still require a SymPEP because it requires a lot of coordination, or because we need to get everyone on the same page with the details.

Of course, controversial things should be discussed as well, but I'm a little unclear what that means. If something is controversial, shouldn't it just be rejected, because that's by definition a lack of community consensus.

discussed. Corresponding to each SymPEP is a document that outlines the
proposed change(s), the discussion around those changes, and motivation for
that change. This document outlines the details of the SymPEP process and
corresponding documents.

## What is a SymPEP?

SymPEP stands for "SymPy Enhancement Proposal". SymPEPs represent a formal
process whereby important changes are proposed and discussed in the SymPy
community. Corresponding to each SymPEP is a SymPEP document which outlines
the proposal and corresponding discussion. SymPEPs are based on
[PEP](https://www.python.org/dev/peps/) process used by the Python language
community to propose changes to the Python language. It also takes motivations
from other similar processes in communities adjacent to SymPy, such as the
[NEP](https://numpy.org/neps/nep-0000.html) process for NumPy and the
[MEP](https://matplotlib.org/devel/MEP/index.html) process for Matplotlib.
However, the SymPEP process differs from these processes in many ways, so
those who may already be familiar with similar processes from other
communities should read this document to understand how it works for SymPEPs.

In particular, SymPEPs are significantly less formal than Python PEPs.
moorepants marked this conversation as resolved.
Show resolved Hide resolved

## Types

There are three kinds of SymPEPs:
moorepants marked this conversation as resolved.
Show resolved Hide resolved

A **Standards Track** SymPEP describes a new feature or important change for
moorepants marked this conversation as resolved.
Show resolved Hide resolved
SymPy.

An **Informational** SymPEP describes a SymPy design issue, or provides general
guidelines or information to the Python community, but does not propose a new
moorepants marked this conversation as resolved.
Show resolved Hide resolved
feature. Informational SymPEPs do not necessarily represent a SymPy community
consensus or recommendation, so users and implementers are free to ignore
Informational SymPEPs or follow their advice.

A **Process** SymPEP describes a process surrounding SymPy, or proposes a
change to (or an event in) a process. Process SymPEPs are like Standards Track
SymPEPs but apply to areas other than the SymPy library itself. They may
propose an implementation, but not to SymPy’s codebase; they require community
consensus. Examples include procedures, guidelines, changes to the
decision-making process, and changes to the tools or environment used in SymPy
development. Any meta-SymPEP is also considered a Process SymPEP.

## Purpose

The SymPEP process and corresponding documents serve several purposes:

- To decide, as a community, whether a proposed change should be made.
- To decide on the process by which a change will be implemented, for example,
whether the change should be implemented in several stages, whether
intermediate releases are required for things like deprecations, and so on.
- To document the above discussions and decisions for future reference.
- To document the motivations for a change from the perspective of end users
who may be affected by it. Here "end users" means both users of the SymPy
library, as well as developers working on parts of SymPy itself which may be
affected by the change.
moorepants marked this conversation as resolved.
Show resolved Hide resolved

Importantly, **a SymPEP is not documentation** for the proposed change. End
user documentation should be included with the implementation of the feature
in the corresponding SymPy documentation. This also means that other
documentation should not cross-reference a SymPEP as if it were documentation
for a feature. Even technical discussion of a feature should be documented
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree with the sentiment here that the SymPEP should not be considered as a replacement for up to date documentation of features etc. I think it is still useful to have cross references to SymPEPs though just like it is useful to note changes in the documentation like new in 1.13 etc.

separately from a SymPEP. The reason is that SymPEPs will necessarily include
details that are irrelevant to the final implementation, such as discussions
of alternate implementations which were rejected and discussions of
implementation of the change.
moorepants marked this conversation as resolved.
Show resolved Hide resolved

## SymPEP Workflow

Discussions around a proposed change may begin informally on the mailing list
or SymPy issue tracker. However, once it is decided that the formal SymPEP
process is desired for a change, the discussion should move to the
[SymPEP repository](https://github.com/sympy/SymPEPs) on GitHub.

Every SymPEP must have a champion. This is a person who is responsible for
writing the SymPEP, leads the discussion around it, and tries to create
consensus around it.
moorepants marked this conversation as resolved.
Show resolved Hide resolved

The author of a SymPEP should fork the repository and create a pull request
with a new SymPEP document based on the [SymPEP template](SymPEP-template).
The SymPEP document may be named `SymPEP-XXXX.md` until a number is assigned.
One of the core SymPy developers will then assign a number to the SymPEP, in
which case `XXXX` should be replaced with the number with leading 0s. The
person who assigns the number should also update the
[README](https://github.com/sympy/SymPEPs/blob/main/README.md) of the main
[SymPEP repository](https://github.com/sympy/SymPEPs) to list that number.
Numbers should be assigned to SymPEPs as soon as it is determined that they
are a legitimate proposal. If a SymPEP ends up being rejected or postponed, it
moorepants marked this conversation as resolved.
Show resolved Hide resolved
keeps its number, as rejection or postponement status is still a discussion
that should be documented in the SymPEP. SymPEP numbers should generally be
assigned in increasing numeric order.

Discussion on the SymPEP should then continue on the SymPEP pull request.
Discussions may also take place in other places, such as [GitHub
discussions](https://github.com/sympy/SymPEPs/discussions) or the [mailing
list](http://groups.google.com/group/sympy). All discussions should be
cross-referenced in the "Discussions" section of the SymPEP document.
moorepants marked this conversation as resolved.
Show resolved Hide resolved

For each SymPEP, the community should decide whether a draft implementation is
needed before acceptance or not. For instance, if a SymPEP concerns the
details of how a feature is implemented, it should be accepted before that
happens. On the other hand, the community may decide that it cannot come to a
consensus about a SymPEP until a draft implementation is proposed.

Once the community reaches a consensus about a SymPEP, the status of a SymPEP
should be updated (see below). This consensus may be to accept or to reject
the SymPEP, or to defer it. Here the "community" refers to the broader
community that has a stake in the SymPEP, not just the core SymPy developers.
The purpose of the SymPEP process is not to create a cabal of decision makers,
but rather to enhance the involvement of the broader SymPy community in the
decision making process.
Comment on lines +131 to +137
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here "consensus" is a very tricky concept. Inherently the idea of working by strict consensus does not scale well up to large groups of people because it implies that any single person has a "veto". I often find this tricky on pull requests because often tangential comments might seem to give a negative opinion but it is not clear how strongly that negative opinion is felt.

Also consensus typically refers only to those who participate visibly in a discussion and often there will be others who do not understand the details of something but might still be affected by it.

I can see that the wording here is careful not to define the decision making process formally and perhaps it is better to keep it that way. I suggest softening the word consensus somehow to allow that it does not strictly require unanimous agreement. Perhaps "broad consensus" or "loose consensus" is better?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've used "tacit consensus" in the past. But I think we actually do operate on a proper full consensus, but that the lead developer (Ondrej, then Aaron, and then now possibly you) could overrule a dissenting vote if full consensus can't actually be achieved.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually, I think it is best to leave it ambiguous. We have a culture of our method of consensus that is publicly visible. If someone attempts to use/push a different way of consensus that is different than we already do, then people will point to the prior approaches to show how things are done. Worst case is that the lead dev takes control and makes a decision outside of consensus.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see the SymPEP process as basically a replacement for the BDFL style decision making we've had in the past (which hasn't really been necessary for a long time anyway). And if we really did want to formalize some sort of actual governance mechanism, that should be done itself through a SymPEP. But that should not be part of this SymPEP, which is just about the template and process for SymPEPs themselves.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure I see how this is a replacement. Given that we have not defined consensus in an explicit way, this can coexist with BDFL style (which is consensus with fallback on dictator if all else fails). I don't see merging this and accepting it as removing anything about the informal, yet existing, decision making process we already have.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reading through this paragraph again, I agree with Jason. I intentionally left this ambiguous, because the alternative is to outline some rigorous decision making process, which implies some sort of governance which doesn't exist (or at least hasn't been decided on).

In fact, the main point I was trying to make with this paragraph is that a SymPEP should involve every relevant stakeholder, not just the core dev team (whatever that means). That also includes non-SymPy developers such as users of SymPy and downstream dependents.

I would use the term "clear consensus". If something isn't clearly agreed upon, then at best that means the default is to not accept, at least without some further decision making process which isn't defined here.


### Status

The **status** section at the top of the SymPEP document (see the
[template](SymPEP-template)) should be updated according to the current status
of the SymPEP.

All SymPEPs should be created with the **Draft** status. **Draft** status
SymPEPs generally live in a pull request.

Eventually, after discussion, there may be a consensus that the SymPEP should
be accepted–see the next section for details. At this point the status
becomes **Accepted**.

Once a SymPEP has been **Accepted**, the reference implementation must be
completed. When the reference implementation is complete and incorporated into
the main source code repository, the status will be changed to **Final**.

A SymPEP can also be assigned status **Deferred**. The SymPEP author or a core
developer can assign the SymPEP this status when no progress is being made on
the SymPEP.

A SymPEP can also be **Rejected**. Perhaps after all is said and done it was
not a good idea. It is still important to have a record of this fact. The
**Withdrawn** status is similar—it means that the SymPEP author themselves has
decided that the SymPEP is actually a bad idea, or has accepted that a
competing proposal is a better alternative.

SymPEPs can also be **Superseded** by a different SymPEP, rendering the
original obsolete.

Process SymPEPs may also have a status of **Active** if they are never meant
to be completed, e.g. SymPEP 1 (this SymPEP).

### Merging the SymPEP Document Pull Request

Whenever a SymPEP moves from the **Draft** status to one of the other above
statuses, the header should be updated, and the corresponding pull request
should be merged. This way the document lives inside of the SymPEP repository
proper. SymPEPs that are merged are not set in stone, and may be updated by
future pull requests (although SymPEPs that are **Accepted** should generally
not be significantly modified once they have reached that status). The purpose
of merging is simply to make the SymPEP visible in the repository, even if it
isn't yet accepted or rejected. SymPEP discussions that have stalled should
also be merged, so that the SymPEP becomes visible in the SymPEP repository
proper—again, discussion may be picked up again with a new pull request.
moorepants marked this conversation as resolved.
Show resolved Hide resolved

## Accepting a SymPEP
moorepants marked this conversation as resolved.
Show resolved Hide resolved

Once a SymPEP is Accepted by consensus of all interested contributors, an
moorepants marked this conversation as resolved.
Show resolved Hide resolved
email should be sent to the [SymPy mailing
list](http://groups.google.com/group/sympy) with a subject like:

Proposal to accept SymPEP #<number>: <title>

In the body of your email, you should:

- link to the latest version of the SymPEP,

- briefly describe any major points of contention and how they were resolved,

- include a sentence like: “If there are no substantive objections within 7
days from this email, then the SymPEP will be accepted; see SymPEP 1 for
more details.”

After you send the email, add the the email thread to the Discussion section
of the SymPEP, so that people can find it later.

Generally the SymPEP author will be the one to send this email, but anyone can
do it – the important thing is to make sure that everyone knows when a SymPEP
is on the verge of acceptance, and give them a final chance to respond. If
moorepants marked this conversation as resolved.
Show resolved Hide resolved
there’s some special reason to extend this final comment period beyond 7 days,
then that’s fine, just say so in the email. It shouldn’t be less than 7 days,
because sometimes people are traveling or similar and need some time to
respond.
moorepants marked this conversation as resolved.
Show resolved Hide resolved

In general, the goal is to make sure that the community has consensus, not
provide a rigid policy for people to try to game. When in doubt, err on the
side of asking for more feedback and looking for opportunities to compromise.

If the final comment period passes without any substantive objections, then
the SymPEP can officially be marked **Accepted**. A followup email should then
be sent notifying the list. Update the SymPEP by setting its **Status** to
**Accepted**, and its **Resolution** header to a link to your followup email.
The SymPEP pull request should be merged at this time, if it hasn't been
already, so that it is visible in the SymPEP repository proper.

If there are substantive objections, then the SymPEP remains in Draft state,
discussion continues as normal, and it can be proposed for acceptance again
later once the objections are resolved.

## Discussion

- [Initial mailing list post proposing
SymPEPs](https://groups.google.com/forum/#!msg/sympy/5RVMiWuCjoA/lr64dS1BBAAJ)
- [sympy/SymPEPs#2](https://github.com/sympy/SymPEPs/pull/2)

## Copyright

This document has been placed in the public domain.
136 changes: 136 additions & 0 deletions SymPEP-template.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,136 @@
SymPEP X — Template and Instructions
====================================

**Author** list of authors' real names and optionally, email addresses
**Status** Draft | Active | Accepted | Deferred | Rejected | Withdrawn | Final | Superseded
**Type** Standards Track | Informational | Process
**Created** date created on, in yyyy-mm-dd format
**Resolution** url to discussion (required for Accepted | Rejected | Withdrawn)

## Abstract

The abstract should be a short description of what the SymPEP will achieve.

Note that the — in the title is an em dash, not -.

## Motivation and Scope

This section describes the need for the proposed change. It should describe
the existing problem, who it affects, what it is trying to solve, and why.
This section should explicitly address the scope of and key requirements for
the proposed change.

## Usage and Impact

This section describes how users of SymPy will use features described in this
SymPEP. It should be comprised mainly of code examples that wouldn't be possible
without acceptance and implementation of this SymPEP, as well as the impact the
proposed changes would have on the ecosystem. This section should be written
from the perspective of the users of SymPy, and the benefits it will provide
them; and as such, it should include implementation details only if
necessary to explain the functionality.

Note that this section should *not* be treated as documentation of the
proposed functionality. Documentation should be included in the implementation
itself. The purpose of this section is to motivate why the change should be
made in the first place. See [SymPEP 1](SymPEP-0001) for more information.

## Backwards compatibility

This section describes the ways in which the SymPEP breaks backward compatibility.

The mailing list post will contain the SymPEP up to and including this section.
Its purpose is to provide a high-level summary to users who are not interested
in detailed technical discussion, but may have opinions around, e.g., usage and
impact.

## Detailed description

This section should provide a detailed description of the proposed change.
It should include examples of how the new functionality would be used,
intended use-cases and pseudocode illustrating its use.

## Related Work

This section should list relevant and/or similar technologies, possibly in other
libraries. It does not need to be comprehensive, just list the major examples of
prior and relevant art.

## Implementation

This section lists the major steps required to implement the SymPEP. Where
possible, it should be noted where one step is dependent on another, and which
steps may be optionally omitted. Where it makes sense, each step should
include a link to related pull requests as the implementation progresses.

Any pull requests or development branches containing work on this SymPEP should
be linked to from here. (A SymPEP does not need to be implemented in a single
pull request if it makes sense to implement it in discrete phases).

## Alternatives

If there were any alternative solutions to solving the same problem, they should
be discussed here, along with a justification for the chosen approach.

## Discussion

This section may just be a bullet list including links to any discussions
regarding the SymPEP:

- This includes links to mailing list threads or relevant GitHub issues.

## References

Any references should be listed here. Note this does not include links to
discussions about the SymPEP, which are listed in the previous section. For
online resources, freely accessible and stable online resources such as
Wikipedia, Wolfram MathWorld, and the NIST Digital Library of Mathematical
Functions (DLMF), which are unlikely to suffer from hyperlink rot, should be
preferred. Non-online references such as textbook or literature references may
also be used here.

## Copyright

Each SymPEP must be explicitly labeled as placed in the public domain, using
the below sentence (the below sentence also applies to this template).

This document has been placed in the public domain.

## Extra Notes About SymPEPs

NOTE: This section is not part of the template. It should not be included in
any SymPEPs. It is a list of notes of things that apply to all parts of the
above template.

Some extra notes about writing a SymPEP:

- All SymPEPs live in the [SymPEPs
repository](https://github.com/sympy/SymPEPs/) on GitHub. All proposed
SymPEPs should be made as pull requests to that repo (see [SymPEP
1](SymPEP-0001)).
- The number corresponding to a SymPEP will be assigned when it is first
proposed to the community. You may use `XXXX` as a placeholder number until
this is done.
- The file for a SymPEPs should be named `SymPEP-XXXX.md` where `XXXX` is a
four digit number, preceded with leading 0s. It should be placed at the root
of the [SymPEPs repository](https://github.com/sympy/SymPEPs/).
- All SymPEP documents should be written in Markdown, following this template.
The Markdown should be compatible with GitHub Flavored Markdown, so that the
SymPEPs are viewable on GitHub. This means that Markdown features
not supported by GitHub, such as footnotes, should be avoided.
<!-- XXX: Perhaps we should abandon this and only require them to be
readable in some rendered format. That would allow us to use MathJAX, which
could be useful. -->
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Math is supported in GitHub markdown now, so this can be removed. In fact, it might be worth pointing that out somewhere.

- Any references to other SymPEPs should be written as internal links, e.g.,
[SymPEP 1](SymPEP-0001).
- Images or diagrams may be included in the SymPEP if they improve the
understanding of the discussion. Code examples should always be included as
plain text. Any image or diagram should be accompanied by corresponding text
describing what is in it. Images should be stored in an `images` directory
at the root of the SymPEPs repository. Images should be titled like
`XXXX-image-name.svg` where `XXXX` is the four digit number of the SymPEP
and `image-name` is the name of the image. For example,
`images/0001-process-diagram.svg`. Vector images are preferred when
possible. Note that images should only be included when they significantly
improve the discussion in the SymPEP. The vast majority of SymPEPs should
not include images.