Third draft for ZEP(Zarr Enhancement Proposal), a community feedback process for Zarr

.gitignore

@@ -0,0 +1 @@
+.DS_Store

ZEP/README.md

@@ -0,0 +1,4 @@
+# ZEP
+Zarr Enhancement Proposal
+
+This folder contains instructions, template and ZEP 0 files.

ZEP/instructions/zep0000.md

@@ -0,0 +1,216 @@
+# Zarr Community Feedback Process - Zarr Enhancement Proposal (ZEP)
+
+--- 
+
+## ZEP 0 — Purpose and process
+
+Author: Sanket Verma <svsanketverma5@gmail.com>
+
+Status: Active
+
+Type: Process
+
+Created: 2022-14-03
+
+## What is ZEP?
+
+ZEP stands for Zarr Enhancement Proposal. A ZEP is a design document providing information to the Zarr community, describing a modification or enhancement of the Zarr storage specification, a new feature for its processes or environment. The ZEP should provide specific proposed changes to the Zarr specification and a narrative rationale for the specification changes.
+
+We intend ZEPs to be the primary mechanisms for evolving the spec, collecting community input on major issues and documenting the design decision that has gone into Zarr. In addition, the ZEP author is responsible for building consensus within the community and documenting dissenting opinions.
+
+Because the ZEPs are maintained as text files in a versioned repository, their revision history is the historical record of the feature proposal.
+
+## ZEP audience
+
+The typical primary audience for ZEPs is the developers working on Zarr and its various implementations, the Zarr steering council as well as the Zarr community.
+
+The broader Zarr community may also choose to use the process to document expected feature additions and to manage complex design coordination problems that
+require collaboration across multiple projects.
+
+## ZEP types
+
+> Specification ZEP
+
+Specification ZEPs deal with the changes related to [Zarr storage specification](https://zarr.readthedocs.io/en/stable/spec.html). The SPEC ZEP (abbreviation of Specification ZEP) introduces changes in the core specification of Zarr. The changes related to core specification follow a strict and thorough review process and should be adopted by everyone in the Zarr community.
+
+- Core protocol ZEP
+
+Describes a ZEP which involves changes in the core specification of Zarr. Core protocol ZEPs (commonly known as Core ZEPs) are a part of SPEC ZEPs and apply to [Zarr storage specification](https://github.com/zarr-developers/zarr-specs) and its various implementations under the [Zarr Implementations](https://github.com/zarr-developers/zarr_implementations) GitHub repo. The core protocol ZEPs should be adopted by every implementation of Zarr and, in general, the overall Zarr community. Core ZEPs must go through a thorough review process, including involvement, discussion, and voting from the author of the proposed ZEP, Zarr Steering Council, author(s) of various Zarr implementations, and open-source projects/research groups using Zarr and the general Zarr community. The general advice is that everyone should be made aware of the change by introducing core ZEP. Core protocol ZEP require community consensus, and users are typically not free to ignore them.
+
+Note: Currently, Specification ZEPs only deal with the core storage specification of Zarr. The scope of SPEC ZEPs will be extended in the future.
+
+> Informational ZEP
+
+Describes a ZEP design issue or provides general guidelines or information to the Zarr community but does not propose a new feature. Informational ZEPs do not necessarily represent Zarr community consensus or recommendation, so users and implementers are free to ignore informational ZEPS or follow their advice.
+
+> Process ZEP
+
+Describes a new process around Zarr and its implementations or proposes a change to(or an event in) a process. They may propose an implementation, but not to Zarr’s codebase; they require community consensus; unlike informational ZEPs, they are more than recommendations, and users are typically not free to ignore them. Examples include procedures, guidelines, changes to the decision-making process, and changes to the tools or environment used in Zarr’s specification development. Any meta-ZEP is also considered a Process ZEP.
+
+## ZEP Workflow
+
+The ZEP process begins with a new idea for Zarr. It is highly recommended that a single ZEP contain a single key proposal or new idea. Small enhancements or 
+patches often don’t need a ZEP and can be injected into the Zarr development workflow with a pull request to the Zarr ZEP repo. The more focused the ZEP, 
+the more successful it tends to be. If in doubt, split your ZEP into several well-focused ones.
+
+Each ZEP must have a champion—someone who writes the ZEP using the style and format described below, shepherds the discussions in the appropriate forums, 
+and attempts to build community consensus around the idea. The ZEP champion (a.k.a. Author) should first attempt to ascertain whether the idea is suitable for a ZEP. 
+Posting to the [Gitter](https://gitter.im/zarr-developers/community) is the best way to go about doing this.
+
+Vetting an idea publicly before going as far as writing a ZEP is meant to save the potential author time. Asking the Zarr community first if an idea is original 
+helps prevent too much time being spent on something that is guaranteed to be rejected based on prior discussion. It also helps to make sure the idea applies to 
+the entire community and not just the author. Just because an idea sounds good to the author does not mean it will work for most people.
+
+Once the champion has asked the Zarr community whether an idea has any chance of acceptance, a draft ZEP should be presented to the appropriate venue mentioned below. This allows the author to flesh out the draft ZEP to make it properly formatted, of high quality, and address initial concerns about the proposal.
+
+After the PR for the ZEP is in place, a post should be made on the GitHub organisation discussion-[here](https://github.com/orgs/zarr-developers/discussions) containing the sections up to “Backward compatibility” to limit discussion there to usage and impact. Discussion on the pull request will have a broader scope, including details of implementation.
+
+Spec ZEPs consist of two parts, a PR to the [spec repo](https://github.com/zarr-developers/zarr-specs) containing changes to the spec and a narrative document explaining the need, importance and use-case for the change. It is also highly recommended that the specification ZEP be accompanied by an implementation PR in at least one the [Zarr implementations](https://github.com/zarr-developers/zarr_implementations).
+
+### Submitting a ZEP
+
+For Spec ZEPs, the proposal should be submitted as a draft ZEP via two GitHub pull requests, one to the zarr-developers/zep repo and the other to the [zarr-developers/zarr-specs](https://github.com/zarr-developers/zarr-specs) repo. 
+
+The first PR should contain the narrative text of the ZEP and should be submitted in the zep repository with the name `zep-<n>.rst` where `<n>` is an appropriately assigned four-digit number. The draft ZEP must use the [ZEP X - Template and Instructions](https://github.com/MSanKeys963/governance/blob/master/ZEP/template/template.md) file.
+
+The second PR should contain actual changes and should be submitted in the [zarr-developers/zarr-specs](https://github.com/zarr-developers/zarr-specs) repository. The PR in the [zarr-specs](https://github.com/zarr-developers/zarr-specs) repository should mention the assigned four-digit ZEP number from the zarr-developers/zep repository.
+
+For ZEPs other than spec, the proposal should be submitted as a draft ZEP via a GitHub pull request to the zarr-developers/zep repository with the name `zep-<n>.rst` where `<n>` is an appropriately assigned four-digit number (e.g., zep-0000.rst). The draft ZEP must use the [ZEP X - Template and Instructions](https://github.com/MSanKeys963/governance/blob/master/ZEP/template/template.md) file.
+
+A few points to consider while submitting your ZEP:
+
+- It should sound complete. The ideas must make technical sense.
+- The title should accurately describe the content.
+- The ZEP’s language (spelling, grammar, sentence structure etc.) and code style should be correct and conformant.
+
+The [Zarr Steering Council](https://github.com/zarr-developers/governance/blob/master/GOVERNANCE.md#steering-council) will not unreasonably deny publication of a ZEP. Reasons for denying ZEP include duplication of effort, being technically unsound, not providing proper 
+motivation or addressing backwards compatibility, or not taking care of Zarr [CODE OF CONDUCT](https://github.com/zarr-developers/.github/blob/main/CODE_OF_CONDUCT.md).
+
+### Discussing a ZEP
+
+As soon as the draft ZEP is committed to the ZEP repository, the author(s) should create a discussion thread for the ZEP to provide a central place to discuss and review its contents. The ZEP author(s) may select the GitHub discussion feature in the organisation [here](https://github.com/orgs/zarr-developers/discussions) or the ZEP repository and similarly use the discussion feature in the [specs](https://github.com/zarr-developers/zarr-specs) repository for specification ZEPs. Alternatively, the author(s) may open a new issue in the [Zarr community repository](https://github.com/zarr-developers/community) or start the discussion in the [Zarr Gitter channel](https://gitter.im/zarr-developers/community). The discussion regarding the ZEP should follow Zarr’s [CODE OF CONDUCT](https://github.com/zarr-developers/.github/blob/main/CODE_OF_CONDUCT.md) at all times.
+
+ZEP authors are responsible for collecting community feedback on a ZEP. However, to avoid long-winded and open-ended discussions, strategies such as soliciting 
+private or more narrowly-tailored feedback in the early design phase, collaborating with other community members with expertise in the ZEP’s subject matter, 
+and picking appropriately-specialised discussion for the ZEP’s topic should be considered. ZEP authors should use their discretion here.
+
+Once the ZEP is committed to the ZEP repository, substantive issues should generally be discussed on the canonical public thread, as opposed to private 
+channels or unrelated venues. This ensures everyone can follow and contribute, avoids fragmenting the discussion, and makes sure it is fully considered as part 
+of the ZEP review process. Comments, support, concerns and other feedback on this designated thread are a critical part when reviewing the ZEP.
+
+### Review and Resolution
+
+The possible paths of the status of ZEPs are as follows:
+
+![Flowchart](/ZEP/assets/flowchart.png)
+
+All ZEPs should be created with the `Draft` status.
+
+Eventually, after the discussion, there may be a consensus that the ZEP should be accepted - see the next section for details. At this point, the status becomes 
+`Accepted`.
+
+Once a Specification ZEP has been Accepted, the second PR which was submitted in the [zarr-specs](https://github.com/zarr-developers/zarr-specs) repository must be merged. After that, the status will be changed to `Final`.
+
+To allow the gathering of additional design and interface feedback before committing to long term stability for specification change or standard library API, ZEP may also be marked as `“Provisional”`. This is short for “Provisionally Accepted” and indicates that the proposal has been accepted for inclusion in the reference implementation or storage specification, but additional user feedback is needed before the full design can be considered “Final”. Unlike regular accepted ZEPs, provisionally accepted ZEPs may still be Rejected or Withdrawn even after the related changes have been included in a Zarr release.
+
+Wherever possible, it is considered preferable to reduce the scope of a proposal to avoid the need to rely on the `“Provisional”` status (e.g. by deferring some features to later ZEPs), as this status can lead to version compatibility challenges in the wider Zarr ecosystem.
+
+A ZEP can also be assigned status `Deferred`. The ZEP author or [Zarr Steering Council](https://github.com/zarr-developers/governance/blob/master/GOVERNANCE.md#steering-council) can assign the ZEP this status when no progress is being made on the ZEP.
+
+A ZEP can also be `Rejected`. Perhaps, after all, is said and done it was not a good idea. It is still important to have a record of this fact. The `Withdrawn` 
+status is similar—it means that the ZEP author themselves has decided that the ZEP is a bad idea, or has accepted that a competing proposal is a better alternative.
+
+When a ZEP is `Accepted`, `Rejected`, or `Withdrawn`, the ZEP should be updated accordingly. In addition to updating the status field, at the very least the 
+`Resolution` header should be added with a link to the relevant link of the discussion.
+
+ZEPs can also be `Superseded` by a different ZEP, rendering the original obsolete. The `Replaced-By` and `Replaces` headers should be added to the original and 
+new ZEPs respectively.
+Process ZEPs may also have a status of `Active` if they are never meant to be completed, e.g. ZEP 0 (this ZEP).
+
+### How does a ZEP becomes accepted?
+
+A ZEP is `Accepted` by the consensus of all interested contributors. We need a concrete way to tell whether the agreement has been reached. 
+
+> For Core ZEPs:
+
+We believe Core ZEPs are of utmost importance and should follow a thorough review before acceptance. Core ZEPs introduce changes in the storage specification, which are necessary for every other implementation and everyone in the general community to follow. The [Zarr Steering Council](https://github.com/zarr-developers/governance/blob/master/GOVERNANCE.md#steering-council) and Implementations Council closely review the Core ZEPs, while the author(s) should simultaneously engage the community in their ZEP at several discussion forums mentioned previously. The ZSC and IC would take community consensus into account while taking the final decision on the Core ZEPs. Author(s) should ensure that the involvement and discussion form the consensus on the ZEP from the author of various Zarr implementations, open-source projects/research groups using Zarr, the general Zarr community, and anyone else they, ZSC and IC think should be included in the discussions. The Core ZEPs are accepted by:
+
+- Unanimous approval of the Zarr Steering Council
+
+- Majority approval of the Zarr Implementations Council
+>*Approval indicates that implementation plans to implement the new spec features. Not all implementations must implement all features, but a majority is required.*
+
+- No vetos from the Zarr Implementations Council
+>*Each implementation has the right to veto a ZEP if it would cause severe problems for that implementation.*
+
+Read more about Implementations Council [here](https://github.com/zarr-developers/governance/pull/17).
+
+
+> For other ZEPs
+
+For ZEPs, other than specifications, the author(s) must form a consensus around their proposal. They may refer to [Discussing a ZEP](https://github.com/MSanKeys963/governance/blob/master/ZEP/instructions/zep0000.md#discussing-a-zep) section to pick an avenue for discussion and engaging the community.
+
+When you think ZEP is ready to accept, create a new discussion in [zarr-specs](https://github.com/zarr-developers/zarr-specs) for Specifications ZEPs using ‘General’ and in zarr-developers/zep for other ZEPs and as the category with a subject like:
+
+> `Proposal to accept ZEP <number>: <title>`
+
+In the body of your discussion, you should:
+
+- Link to the latest version of ZEP,
+- Briefly describe any major points of contention and how they were resolved,
+- Include a sentence like: “If there are no substantive objections within 7 days from this post, then the ZEP will be accepted;
+
+After you create the discussion, you should make sure to link the newly created thread in the `Discussion` section of the ZEP, so that people can find it later.
+
+Generally, the ZEP author will be the one to create this post, but anyone can do it – the important thing is to make sure that everyone knows when a ZEP is on the verge of acceptance, and give them a final chance to respond. If there’s some special reason to extend this final comment period beyond `7 days`, then that’s fine, just say so in the post. You shouldn’t do less than `7 days`, because sometimes people are travelling or similar and need some time to respond.
+
+There may be a case that a ZEP didn’t attract needed attention towards it, the engagement from the community is low, and `seven days` pass by. In this case, the author(s) of the ZEP must make necessary efforts to spread the word about the ZEP through [Gitter](https://gitter.im/zarr-developers/community) or using the organisation discussion feature of GitHub [here](https://github.com/orgs/zarr-developers/discussions) or, if needed, creating an additional PR in the [community](https://github.com/zarr-developers/community) repository. In addition, the author(s) should get in touch with the [Zarr Steering Council](https://github.com/zarr-developers/governance/blob/master/GOVERNANCE.md#steering-council) to prevent the case that the ZEP was accepted due to less participation from the community.
+
+If all the above options are exhausted by the author(s), then it’s the responsibility of the [Zarr Steering Council](https://github.com/zarr-developers/governance/blob/master/GOVERNANCE.md#steering-council) to take the final decision on the ZEP.
+
+In general, the goal is to make sure that the community has consensus, not provide a rigid policy for people to try to game. When in doubt, err on the side of asking for more feedback and looking for opportunities to compromise.
+
+If the final comment period passes without any substantive objections, then the ZEP can officially be marked `Accepted`. You should create a follow-up discussion thread in zarr-developers/zep repository notifying everyone (celebratory emoji optional but encouraged 🎉✨), and then update the ZEP by setting its `:Status:` to `Accepted`, and it's `:Resolution:` header to a link to your follow-up discussion thread.
+
+If there are substantive objections, then the ZEP remains in `Draft` state, discussion continues as normal, and it can be proposed for acceptance again later once the objections are resolved.
+
+In unusual cases, the [Zarr Steering Council](https://github.com/zarr-developers/governance/blob/master/GOVERNANCE.md#steering-council) may be asked to decide whether a controversial ZEP is `Accepted`.
+
+### Maintenance
+
+In general, SPEC ZEPs are no longer modified after they have reached the Final state. The changes made to the Zarr’s core storage specification in the [zarr-developers/spec](https://github.com/zarr-developers/zarr-specs) repository are considered the ultimate reference. However, finalised SPEC ZEPS may be updated with minor changes as needed.
+
+Process ZEPs may be updated over time to reflect changes to development practices and other details. The precise process followed in these cases will 
+depend on the nature and purpose of the ZEP being updated.
+
+## ZEP Format
+
+ZEPs are UTF-8 encoded text files using the [reStructureText](http://docutils.sourceforge.net/rst.html) format. Please see the [ZEP X - Template and Instructions](https://github.com/MSanKeys963/governance/blob/master/ZEP/template/template.md) file and the [reStructuredTextPrimer](https://www.sphinx-doc.org/en/stable/rest.html) for more information.
+
+### Header Preamble
+
+```
+:Author: <list of authors’ real names and email addresses>
+:Status: < Draft | Active | Accepted | Deferred | Rejected | Withdrawn | Final | Superseded >
+:Type: <Specification | Process | Informational>
+:Created: <date created on, in dd-mmm-yyyy format>
+:Require: <Previous ZEP number>
+:Zarr-Version: <version number>
+:Replaces: <ZEP number>
+:Replaced-By: <ZEP number>
+:Resolution:  <Link to discussion thread>
+```
+
+The Author header lists the names and the email addresses of all the authors of the ZEP. The format of the Author header value must be:
+
+```
+Random J. User <address@dom.ain>
+```
+
+## Discussion
+
+https://github.com/orgs/zarr-developers/discussions
+
+## Copyright
+
+This document has been placed in the public domain.