diff --git a/.gitignore b/.gitignore index e69de29..e43b0f9 100644 --- a/.gitignore +++ b/.gitignore @@ -0,0 +1 @@ +.DS_Store diff --git a/ZEP/README.md b/ZEP/README.md new file mode 100644 index 0000000..9a718ce --- /dev/null +++ b/ZEP/README.md @@ -0,0 +1,4 @@ +# ZEP +Zarr Enhancement Proposal + +This folder contains instructions, template and ZEP 0 files. diff --git a/ZEP/assets/flowchart.png b/ZEP/assets/flowchart.png new file mode 100644 index 0000000..8d00ea6 Binary files /dev/null and b/ZEP/assets/flowchart.png differ diff --git a/ZEP/instructions/zep0000.md b/ZEP/instructions/zep0000.md new file mode 100644 index 0000000..4446563 --- /dev/null +++ b/ZEP/instructions/zep0000.md @@ -0,0 +1,409 @@ +# Zarr Community Feedback Process - Zarr Enhancement Proposal (ZEP) + +--- + +## ZEP 0 — Purpose and process + +Author: Sanket Verma + +Status: Active + +Type: Process + +Created: 2022-14-03 + +## What is ZEP? + +ZEP stands for Zarr Enhancement Proposal. A ZEP is a design document providing +information to the Zarr community, describing a modification or enhancement of +the Zarr specification, a new feature for its processes or environment. +The ZEP should provide specific proposed changes to the Zarr specification and +a narrative rationale for the specification changes. + +We intend ZEPs to be the primary mechanism for evolving the spec, collecting +community input on major issues and documenting the design decision that has +gone into Zarr. In addition, the ZEP author is responsible for building +consensus within the community and documenting dissenting opinions. + +Because the ZEPs are maintained as text files in a versioned repository, their +revision history is the historical record of the feature proposal. + +WHERE: + +- Developers refer to contributors and maintainers of the project +- User(s) refers to an individual or group of individuals or the broader +community using the project in any way. + +## ZEP audience + +The typical primary audience for ZEPs is the developers working on Zarr and its +various implementations, the Zarr steering council as well as the Zarr +community. + +The broader Zarr community may also choose to use the process to document +expected feature additions and to manage complex design coordination problems +that require collaboration across multiple projects. + +## ZEP types + +> Specification ZEP + +Specification ZEPs deal with the changes related to [Zarr Specifications]. +The SPEC ZEP (abbreviation of Specification ZEP) for now introduces changes in +the core specification of Zarr. The changes related to core specification follow a +strict and thorough review process and should be adopted by everyone in the +Zarr community. + +- Core protocol ZEP + +Describes a ZEP which involves changes in the core specification of Zarr. Core +protocol ZEPs (commonly known as Core ZEPs) are a part of SPEC ZEPs and apply to +[Zarr Specifications] and its various implementations under the +[zarr_implementations] GitHub repo. The core protocol ZEPs should be adopted by +every implementation of Zarr and, in general, the overall Zarr community. Core +ZEPs must go through a thorough review process, including involvement, +discussion, and voting from the author of the proposed ZEP, Zarr Steering +Council, author(s) of various Zarr implementations, and open-source +projects/research groups using Zarr and the general Zarr community. The general +advice is that everyone should be made aware of changes introduced by core +ZEPs. Core protocol ZEPs require community consensus, and developers or users +are typically not free to ignore them. + +Note: Currently, Specification ZEPs only deal with the core specification of Zarr. +The scope of SPEC ZEPs will be extended in the future. + +> Informational ZEP + +Describes a ZEP design issue or provides general guidelines or information to +the Zarr community but does not propose a new feature. Informational ZEPs do +not necessarily represent Zarr community consensus or recommendation, so users +and implementers are free to ignore informational ZEPS or follow their advice. + +> Process ZEP + +Describes a new process around Zarr and its implementations or proposes a +change to (or an event in) a process. They may propose an implementation, but +not to the Zarr’s specification or its implementations; they require community +consensus; unlike informational ZEPs, they are more than recommendations, and +developers or users are typically not free to ignore them. Examples include +procedures, guidelines, changes to the decision-making process, and changes to +the tools or environment used in Zarr’s specification development. Any meta-ZEP +is also considered a Process ZEP. + +## ZEP Workflow + +The ZEP process begins with a new idea for Zarr. It is highly recommended that +a single ZEP contain a single key proposal or new idea. Small enhancements or +patches often don’t need a ZEP and can be injected into the Zarr development +workflow with a pull request to the Zarr ZEP repo. The more focused the ZEP, +the more successful it tends to be. If in doubt, split your ZEP into several +well-focused ones. + +Each ZEP must have a champion -- someone who writes the ZEP using the style and +format described below, shepherds the discussions in the appropriate forums, +and attempts to build community consensus around the idea. The ZEP champion +(a.k.a. Author) should first attempt to ascertain whether the idea is suitable +for a ZEP. Posting to [Gitter] or creating an issue on the [community] repository +or posting on the organization [discussions] are the best ways to go about doing this. + +The ZEP champion is the lead author of the ZEP. A ZEP should have a lead +author and can have multiple co-author(s). + +Vetting an idea publicly before going as far as writing a ZEP is meant to save +the potential author time. Asking the Zarr community first if an idea is +original helps prevent too much time being spent on something that is +guaranteed to be rejected based on prior discussion. It also helps to make sure +the idea applies to the entire community and not just the author. Just because +an idea sounds good to the author does not mean it will work for most people. + +Once the champion has asked the Zarr community whether an idea has any chance +of acceptance, a draft ZEP should be presented to the appropriate venue +mentioned below. This allows the author to flesh out the draft ZEP to make it +properly formatted, of high quality, and address initial concerns about the +proposal. + +After the PR for the ZEP is in place, a post should be made on the GitHub +organisation [discussions], containing the sections up to “Backward +compatibility” to limit discussion there to usage and impact. Discussion on the +pull request will have a broader scope, including details of implementation. + +Spec ZEPs consist of two parts, a PR to the [zarr-specs] repository containing +changes to the spec and a narrative document explaining the need, importance and +use-case for the change. It is also highly recommended that the specification +ZEP be accompanied by an implementation PR in at least one of the repositories +represented in the [zarr_implementations]. + +### Submitting a ZEP + +For Spec ZEPs, the proposal should be submitted as a draft ZEP via two GitHub +pull requests, one to the zarr-developers/zep repo and the other to the +[zarr-specs] repository. + +The first PR should contain the narrative text of the ZEP and should be +submitted in the zep repository with the name `zep-.rst` where `` is an +appropriately assigned four-digit number. The draft ZEP must use the [ZEP X - +Template and Instructions][template.md] file. + +The second PR should contain actual changes and should be submitted in the +[zarr-specs] repository. That PR repository should mention the assigned +four-digit ZEP number from the zarr-developers/zep repository. + +For ZEPs other than spec, the proposal should be submitted as a draft ZEP via a +GitHub pull request to the zarr-developers/zep repository with the name +`zep-.rst` where `` is an appropriately assigned four-digit number (e.g., +zep-0000.rst). The draft ZEP must use the +[ZEP X - Template and Instructions][template.md] file. + +A few points to consider while submitting your ZEP: + +- It should sound complete. The ideas must make technical sense. +- The title should accurately describe the content. +- The ZEP’s language (spelling, grammar, sentence structure etc.) and code style should be correct and conformant. + +The [Zarr Steering Council] +and the Zarr Implementations Council will not unreasonably deny publication of +a ZEP. Reasons for denying ZEP include duplication of effort, being technically +unsound, not providing proper motivation or addressing backwards compatibility, +or not taking care of Zarr [CODE OF CONDUCT]. + +### Discussing a ZEP + +As soon as the draft ZEP is committed to the ZEP repository, the author(s) +should create a discussion thread for the ZEP to provide a central place to +discuss and review its contents. The ZEP author(s) may select the GitHub +discussion feature in the organisation [discussions] or the ZEP repository and +similarly use the discussion feature in the [zarr-specs] repository for +specification ZEPs. Alternatively, the author(s) may open a new issue in the +Zarr [community] repository or start the discussion in the Zarr [Gitter] channel. +The discussion regarding the ZEP should follow Zarr’s [CODE OF CONDUCT] at all +times. + +ZEP authors are responsible for collecting community feedback on a ZEP. +However, to avoid long-winded and open-ended discussions, strategies such as +soliciting private or more narrowly-tailored feedback in the early design +phase, collaborating with other community members with expertise in the ZEP’s +subject matter, and picking appropriately-specialised discussion for the ZEP’s +topic should be considered. ZEP authors should use their discretion here. + +Once the ZEP is committed to the ZEP repository, substantive issues should +generally be discussed on the canonical public thread, as opposed to private +channels or unrelated venues. This ensures everyone can follow and contribute, +avoids fragmenting the discussion, and makes sure it is fully considered as +part of the ZEP review process. Comments, support, concerns and other feedback +on this designated thread are a critical part when reviewing the ZEP. + +### Review and Resolution + +The possible paths of the status of ZEPs are as follows: + +![Flowchart](/ZEP/assets/flowchart.png) + +All ZEPs should be created with the `Draft` status. + +Eventually, after the discussion, there may be a consensus that the ZEP should +be accepted - see the next section for details. At this point, the status +becomes `Accepted`. + +Once a Specification ZEP has been Accepted, the second PR which was submitted in +the [zarr-specs] repository must be merged. After that, the status will be +changed to `Final`. + +To allow the gathering of additional design and interface feedback before +committing to long term stability for specification change or standard library +API, ZEP may also be marked as `“Provisional”`. This is short for +“Provisionally Accepted” and indicates that the proposal has been accepted for +inclusion in the reference implementation or storage specification, but +additional user feedback is needed before the full design can be considered +“Final”. Unlike regular accepted ZEPs, provisionally accepted ZEPs may still be +Rejected or Withdrawn even after the related changes have been included in a +Zarr release. + +Wherever possible, it is considered preferable to reduce the scope of a +proposal to avoid the need to rely on the `“Provisional”` status (e.g. by +deferring some features to later ZEPs), as this status can lead to version +compatibility challenges in the wider Zarr ecosystem. + +A ZEP can also be assigned status `Deferred`. The ZEP author or [Zarr Steering +Council] or Zarr Implementations Council can assign the ZEP this status when no +progress is being made on the ZEP. + +A ZEP can also be `Rejected`. Perhaps, after all, is said and done it was not a +good idea. It is still important to have a record of this fact. The `Withdrawn` +status is similar—it means that the ZEP author themselves has decided that the +ZEP is a bad idea, or has accepted that a competing proposal is a better +alternative. + +When a ZEP is `Accepted`, `Rejected`, or `Withdrawn`, the ZEP should be updated +accordingly. In addition to updating the status field, at the very least the +`Resolution` header should be added with a link to the relevant link of the +discussion. + +ZEPs can also be `Superseded` by a different ZEP, rendering the original +obsolete. The `Replaced-By` and `Replaces` headers should be added to the +original and new ZEPs respectively. Process ZEPs may also have a status of +`Active` if they are never meant to be completed, e.g. ZEP 0 (this ZEP). + +### How does a ZEP become accepted? + +A ZEP is `Accepted` by the consensus of all interested contributors. We need a +concrete way to tell whether the agreement has been reached. + +> For Core ZEPs: + +We believe Core ZEPs are of utmost importance and should follow a thorough +review before acceptance. Core ZEPs introduce changes in the core +specification, which are necessary for every other implementation and everyone +in the general community to follow. The [Zarr Steering Council] and +Implementations Council closely review the Core ZEPs, while the author(s) should +simultaneously engage the community in their ZEP at several discussion forums +mentioned previously. The ZSC and ZIC would take community consensus into account +while taking the final decision on the Core ZEPs. Author(s) should ensure that +the involvement and discussion form the consensus on the ZEP from the author of +various Zarr implementations, open-source projects/research groups using Zarr, +the general Zarr community, and anyone else they, ZSC and ZIC think should be +included in the discussions. The Core ZEPs are accepted by: + +- Unanimous approval of the Zarr Steering Council + +- Majority approval of the Zarr Implementations Council + +> *Approval indicates that implementation plans to implement the new spec +> features. Not all implementations must implement all features, but a majority is +> required.* + +- No vetos from the Zarr Implementations Council + +> *Each implementation has the right to veto a ZEP if it would cause severe +> problems for that implementation.* + +Read more about [Zarr Implementation Council]. + + +> For other ZEPs + +For ZEPs, other than specifications, the author(s) must form a consensus around +their proposal. They may refer to [Discussing a ZEP] section to pick an avenue +for discussion and engaging the community. + +When you think ZEP is ready to accept, create a new discussion in [zarr-specs] +for Specifications ZEPs using ‘General’ and in zarr-developers/zep for other +ZEPs and as the category with a subject like: + +> `Proposal to accept ZEP : ` + +In the body of your discussion, you should: + +- Link to the latest version of ZEP, +- Briefly describe any major points of contention and how they were resolved, +- Include a sentence like: “If there are no substantive objections within 7 days + from this post, then the ZEP will be accepted; + +After you create the discussion, you should make sure to link the newly created +thread in the `Discussion` section of the ZEP, so that people can find it later. + +Generally, the ZEP author will be the one to create this post, but anyone can +do it – the important thing is to make sure that everyone knows when a ZEP is +on the verge of acceptance, and give them a final chance to respond. If +there’s some special reason to extend this final comment period beyond +`7 days`, then that’s fine, just say so in the post. You shouldn’t do less +than `7 days`, because sometimes people are travelling or similar and need +some time to respond. + +There may be a case that a ZEP didn’t attract needed attention towards it, the +engagement from the community is low, and `7 days` pass by. In this case, +the author(s) of the ZEP must make necessary efforts to spread the word about +the ZEP through [Gitter] or using the organisation [discussions] feature of +GitHub or, if needed, creating an additional PR in the [community] repository. +In addition, the author(s) should get in touch with the [Zarr Steering Council] +to prevent the case that the ZEP was accepted due to less participation from the +community. + +If all the above options are exhausted by the author(s), then it’s the +responsibility of the [Zarr Steering Council] to take the final decision on the +ZEP. + +In general, the goal is to make sure that the community has consensus, not +provide a rigid policy for people to try to game. When in doubt, err on the +side of asking for more feedback and looking for opportunities to compromise. + +If the final comment period passes without any substantive objections, then the +ZEP can officially be marked `Accepted`. You should create a follow-up +discussion thread in zarr-developers/zep repository notifying everyone +(celebratory emoji optional but encouraged 🎉✨), and then update the ZEP by +setting its `:Status:` to `Accepted`, and it's `:Resolution:` header to a link +to your follow-up discussion thread. + +If there are substantive objections, then the ZEP remains in `Draft` state, +discussion continues as normal, and it can be proposed for acceptance again +later once the objections are resolved. + +In unusual cases, the [Zarr Steering Council] may be asked to decide whether a +controversial ZEP is `Accepted`. + +### Maintenance + +In general, SPEC ZEPs are no longer modified after they have reached the Final +state. The changes made to the Zarr’s core specification in the +[zarr-specs] repository are considered the ultimate reference. However, +finalised SPEC ZEPS may be updated with minor changes as needed. + +Process ZEPs may be updated over time to reflect changes to development +practices and other details. The precise process followed in these cases will +depend on the nature and purpose of the ZEP being updated. + +## ZEP Format + +ZEPs are UTF-8 encoded text files using the +[reStructureText](http://docutils.sourceforge.net/rst.html) format. Please see +the [ZEP X - Template and Instructions][template.md] file and the +[reStructuredTextPrimer](https://www.sphinx-doc.org/en/stable/rest.html) for +more information. + +### Header Preamble + +``` +:Author: <list of authors’ real names and email addresses> +:Status: < Draft | Active | Accepted | Deferred | Rejected | Withdrawn | Final | Superseded > +:Type: <Specification | Process | Informational> +:Created: <date created on, in dd-mmm-yyyy format> +:Require: <Previous ZEP number> +:Zarr-Version: <version number> +:Replaces: <ZEP number> +:Replaced-By: <ZEP number> +:Resolution: <Link to discussion thread> +``` + +The Author header lists the names and the email addresses of all the authors of +the ZEP. The format of the Author header value must be: + +``` +Random J. User <address@dom.ain> +``` + +## Discussion + +https://github.com/orgs/zarr-developers/discussions + +## Copyright + +This document has been placed in the public domain. + + +<!-- External links --> +[Gitter]: https://gitter.im/zarr-developers/community +[Zarr Specifications]: https://zarr-specs.readthedocs.io/ + +<!-- governance specific links --> +[CODE OF CONDUCT]: https://github.com/zarr-developers/.github/blob/main/CODE_OF_CONDUCT.md +[Zarr Steering Council]: https://github.com/zarr-developers/governance/blob/master/GOVERNANCE.md#steering-council + +<!-- github links: capitalization matches that of the repo --> +[community]: https://github.com/zarr-developers/community +[discussions]: https://github.com/orgs/zarr-developers/discussions +[zarr_implementations]: https://github.com/zarr-developers/zarr_implementations +[zarr-specs]: https://github.com/zarr-developers/zarr-specs + +<!-- TODO: these need updating --> +[Zarr Implementation Council]: https://github.com/zarr-developers/governance/pull/17 +[Discussing a ZEP]: https://github.com/MSanKeys963/governance/blob/master/ZEP/instructions/zep0000.md#discussing-a-zep +[template.md]: https://github.com/MSanKeys963/governance/blob/master/ZEP/template/template.md \ No newline at end of file diff --git a/ZEP/template/template.md b/ZEP/template/template.md new file mode 100644 index 0000000..461b9ed --- /dev/null +++ b/ZEP/template/template.md @@ -0,0 +1,74 @@ +# ZEP — Template and Instructions + +--- + +``` +Author: <list of authors’ real name and email addresses> + +Status: < Draft | Active | Accepted | Deferred | Rejected | Withdrawn | Final | Superseded > + +Type: <Specification | Informational | Process> + +Created: <date created on, in dd-mmm-yyyy format> + +Resolution: <url> (required for Accepted | Rejected | Withdrawn) +``` + +## Abstract + +The abstract should be a short description of what the ZEP will achieve. + +## Motivation and Scope + +This section describes the need for the proposed change. It should describe the existing problem, who it affects, what it is trying to solve, and why. +This section should explicitly address the scope of and key requirements for the proposed change. + +## Usage and Impact + +This section describes how users of Zarr will use the new features, spec changes or a new process described in this ZEP. It should be comprised mainly of code examples that wouldn’t be possible +without acceptance and implementation of this ZEP, as well as the impact the proposed changes would have on the ecosystem. This section should be written from +the perspective of the users of Zarr, and the benefit it will provide them; as such, it should include implementation details only if necessary to explain the +functionality. + +## Backward Compatibility + +This section describes how the ZEP breaks backward compatibility. + +Its purpose is to provide a high-level summary to users who are not interested in detailed technical discussion, but may have opinions around, e.g., usage and +impact. + +## Detailed description + +This section should provide a detailed description of the proposed change. It should include examples of how the new functionality would be used, intended +use-cases and pseudo-code illustrating its use. + +## Related Work + +This section should list relevant and/or similar technologies, possibly in other libraries. It does not need to be comprehensive, just list the major examples +of prior and relevant art. + +## Implementation + +This section lists the major steps required to implement the ZEP. Where possible, it should be noted where one step is dependent on another, and which steps may +be optionally omitted. Where it makes sense, each step should include a link to related pull requests as the implementation progresses. + +Any pull requests or development branches containing work on this ZEP be linked to from here. (A ZEP does not need to be implemented in a single pull request if +it makes sense to implement it in discrete phases). + +## Alternatives + +If there were any alternative solutions to solving the same problem, they should be discussed here, along with a justification for the chosen approach. + +## Discussion + +This section should have links related to any discussion regarding the ZEP. It could be GitHub issues and/or discussions. (The links to discussions in past +if any, goes in this section.) + +## References and Footnotes + +Each ZEP must either be explicitly labelled as placed in the public domain (see this ZEP as an example) or licensed under the +[Open Publication License](https://www.opencontent.org/openpub/). + +## Copyright + +This document has been placed in the public domain.