From 229dc6585d40de889cec8a772edf9b88d5398b10 Mon Sep 17 00:00:00 2001 From: Alan Zimmerman Date: Sat, 10 Dec 2016 19:54:48 +0200 Subject: [PATCH 1/6] Clarify the path to a proposal being "finished" By stating that it is currently unknown --- README.rst | 15 +++++---------- 1 file changed, 5 insertions(+), 10 deletions(-) diff --git a/README.rst b/README.rst index 8c6a02e..6341f8e 100644 --- a/README.rst +++ b/README.rst @@ -125,13 +125,8 @@ process, here is a short summary, 4. Discussion will proceed on the pull request; it is very likely that multiple iterations will be necessary before the proposal stabilizes. -*Next two steps need to be discussed/agreed* - -5. When discussion has died down notify the (yet to be formed) Ecosystem Committee via - email. The committee will review the proposal, the feedback collected on - the pull request, and general community sentiment and decide whether the - proposal will be accepted. - -6. When your proposal is accepted your pull request will be merged. At this - point you or someone else may choose to implement your proposal. - +5. To be decided (as a community): what it means for a proposal to be accepted, + and what reasonable expectations are if this happens. + + In particular, changes require resources, and the haskell ecosystem has become + a big ship which has a very wide turning circle. From f2979a0d2722c0d19e31e6f793f0065d1493543a Mon Sep 17 00:00:00 2001 From: gbaz Date: Fri, 26 Jan 2018 15:46:09 -0500 Subject: [PATCH 2/6] Create 0000-uncurated-layer.rst --- proposals/0000-uncurated-layer.rst | 53 ++++++++++++++++++++++++++++++ 1 file changed, 53 insertions(+) create mode 100644 proposals/0000-uncurated-layer.rst diff --git a/proposals/0000-uncurated-layer.rst b/proposals/0000-uncurated-layer.rst new file mode 100644 index 0000000..63b0962 --- /dev/null +++ b/proposals/0000-uncurated-layer.rst @@ -0,0 +1,53 @@ +.. proposal-number:: Leave blank. This will be filled in when the proposal is + accepted. + +.. highlight:: haskell + +Uncurated Hackage Layer +============== + +Motivation +---------- + +There is a tension between two purposes of Hackage -- first as a central repository of Haskell code, and second as a curated store that has artifacts that are intended to be correctly built and depended upon in a self-contained fashion. (I.e. which contain all information necessary to correctly build them). The way the latter is accomplished is by asking packages follow the package version policy (https://pvp.haskell.org/) that is used to inform clients of a package of changes to that package that might affect them, and to provide a way for clients to specify a particular range of versions of a dependency that they are compatible with. + +The aim of this proposal is to separate these two purposes, by allowing authors to distinguish if they wish to opt-out of following the pvp and the attendant curation process that helps to maintain correct dependency information. In so doing, they recognize that their discoverability on Hackage may be reduced, since the UI will give precedence to those packages which, in conjunction with potential cabal revisions, are self-contained with regards to the information necessary to build them. + +Proposed Change +--------------- + +This is a phased proposal with gradual rollout possible, in dependency order of the steps. The first phase is immediate. Other phases may be rolled out at any pace, but necessarily in the order specified (including the collapse of two steps into a joint step). + +The desired end-state will have the following properties: + +1) Packages will have an additional flag set in the Hackage package database, that indicates if they are curated or not. This flag is set *per version*. +2) Package authors will set this flag *on upload*, by setting the "x-uncurated" property of the cabal file of a package to "true". If no "x-uncurated" property is set, this will be considered "false". +3) Hackage will provide two package repository roots -- http://hackage.haskell.org and http://hackage.haskell.org/uncurated These roots will provide index-01.tar.gz files that contain the information, respectively, for curated packages, or for all packages. The uncurated root will contain no revision information. +4) Curated packages cannot depend on uncurated packages, and the hackage server will detect this as an error at upload time. +5) Uncurated packages may be "adopted" into the curated ecosystem by trustees. Metadata revisions necessarily remove the x-uncurated property from the revised cabal metadata. + +The first phase of this rollout is simply social. It has the following properties: + ++ Hackage trustees will recognize and respect the uncurated flag, and not contact those who set it with any issues. They *will* retain the ability to make metadata revisions, bearing in mind that they must remove the x-uncurated property from revised metadata. + +The second phase is a technical change as soon as possible to enforce the semantics of x-uncurated: + ++ Hackage will ensure that no revision has x-uncurated set to true. + +The third phase is implementation of UI: + ++ The uncurated flag will be detected and displayed on a package's page, as part of the general data provided about a package. It will also be provided in search and browse results. Ideally, search and browse results will be extended in general with the ability to preform in-page filtering on flags and fields, such as "library", curated status, deprecation status, etc. + +The fourth phase is indices: + ++ The uncurated package repo root will be built and provided. + +Fifth, Hackage can now begin to enforce the policy regarding curated packages not depending on uncurated packages. + ++ Curated package uploads will be checked on upload to ensure they don't have dependencies on uncurated packages. Further, the curated index will only provide information on curated packages. + +In the end state, the intent is that uncurated (as well as deprecated) packages will be hidden by default in the filtering settings of search and browse interfaces. These UI settings will be able to be persisted, so that users may change their preferences in this regard. + +Future Plans +--------------- +In the future, full support for collections may be implemented in Hackage. At such point, the current "curated" layer may simply become one of a number of collections (though likely the largest). The UI surrounding display and discovery of curated and uncurated packages, etc. would likely be transformed in such a setting (and indeed the two package repos might be merged again, with support for distinguishing curated and uncurated instead coming through some future collection specification). This is just speculation at the moment, but worth bearing in mind. The end-state given by this proposal is not necessarily the end-state of hackage -- just something that accomplishes the narrow goals set forward in the motivation without reliance on other potential work that is not yet fully specified. From 7bbc492362f59599bf1fddaa75f6b6676369c7d7 Mon Sep 17 00:00:00 2001 From: tomjaguarpaw Date: Sat, 27 Jan 2018 12:23:02 +0000 Subject: [PATCH 3/6] Minor edits --- proposals/0000-uncurated-layer.rst | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/proposals/0000-uncurated-layer.rst b/proposals/0000-uncurated-layer.rst index 63b0962..5db3722 100644 --- a/proposals/0000-uncurated-layer.rst +++ b/proposals/0000-uncurated-layer.rst @@ -9,9 +9,9 @@ Uncurated Hackage Layer Motivation ---------- -There is a tension between two purposes of Hackage -- first as a central repository of Haskell code, and second as a curated store that has artifacts that are intended to be correctly built and depended upon in a self-contained fashion. (I.e. which contain all information necessary to correctly build them). The way the latter is accomplished is by asking packages follow the package version policy (https://pvp.haskell.org/) that is used to inform clients of a package of changes to that package that might affect them, and to provide a way for clients to specify a particular range of versions of a dependency that they are compatible with. +There is a tension between two purposes of Hackage -- first as a central repository of Haskell code, and second as a curated store that has artifacts that are intended to be correctly built and depended upon in a self-contained fashion (i.e. which contain all information necessary to correctly build them). The way the latter is accomplished is by asking packages follow the Package Versioning Policy (https://pvp.haskell.org/) that is used to inform clients of a package of changes to that package that might affect them, and to provide a way for clients to specify a particular range of versions of a dependency that they are compatible with. -The aim of this proposal is to separate these two purposes, by allowing authors to distinguish if they wish to opt-out of following the pvp and the attendant curation process that helps to maintain correct dependency information. In so doing, they recognize that their discoverability on Hackage may be reduced, since the UI will give precedence to those packages which, in conjunction with potential cabal revisions, are self-contained with regards to the information necessary to build them. +The aim of this proposal is to separate these two purposes, by allowing authors to distinguish if they wish to opt-out of following the PVP and the attendant curation process that helps to maintain correct dependency information. In so doing, they recognize that their discoverability on Hackage may be reduced, since the UI will give precedence to those packages which, in conjunction with potential cabal revisions, are self-contained with regards to the information necessary to build them. Proposed Change --------------- From ac3d50288242be45143cabfc00597d6e6c9dceb8 Mon Sep 17 00:00:00 2001 From: gbaz Date: Tue, 30 Jan 2018 00:27:15 -0500 Subject: [PATCH 4/6] Update 0000-uncurated-layer.rst --- proposals/0000-uncurated-layer.rst | 20 ++++++++++++-------- 1 file changed, 12 insertions(+), 8 deletions(-) diff --git a/proposals/0000-uncurated-layer.rst b/proposals/0000-uncurated-layer.rst index 63b0962..08cb241 100644 --- a/proposals/0000-uncurated-layer.rst +++ b/proposals/0000-uncurated-layer.rst @@ -21,32 +21,36 @@ This is a phased proposal with gradual rollout possible, in dependency order of The desired end-state will have the following properties: 1) Packages will have an additional flag set in the Hackage package database, that indicates if they are curated or not. This flag is set *per version*. -2) Package authors will set this flag *on upload*, by setting the "x-uncurated" property of the cabal file of a package to "true". If no "x-uncurated" property is set, this will be considered "false". -3) Hackage will provide two package repository roots -- http://hackage.haskell.org and http://hackage.haskell.org/uncurated These roots will provide index-01.tar.gz files that contain the information, respectively, for curated packages, or for all packages. The uncurated root will contain no revision information. +2) Package authors will set this flag *on upload*, by setting the "x-curation" property of the cabal file of a package. If no "x-uncurated" property is set, this will be considered "curated". Along with "curated" and "uncurated," other sub-settings will be made available. In particular: "uncurated-no-trustee-contact" and "uncurated-seeking-adoption". +3) Hackage will provide two package repository roots -- http://hackage.haskell.org and http://uncurated.hackage.haskell.org These roots will provide index-01.tar.gz files that contain the information, respectively, for curated packages, or for all packages. The uncurated root will contain no revision information. 4) Curated packages cannot depend on uncurated packages, and the hackage server will detect this as an error at upload time. 5) Uncurated packages may be "adopted" into the curated ecosystem by trustees. Metadata revisions necessarily remove the x-uncurated property from the revised cabal metadata. The first phase of this rollout is simply social. It has the following properties: -+ Hackage trustees will recognize and respect the uncurated flag, and not contact those who set it with any issues. They *will* retain the ability to make metadata revisions, bearing in mind that they must remove the x-uncurated property from revised metadata. ++ Hackage trustees will recognize and respect the uncurated flag. When it is set to "uncurated-no-trustee-contact" they will not contact authors with any issues. They *will* retain the ability to make metadata revisions, bearing in mind that they must remove the x-uncurated property from revised metadata. -The second phase is a technical change as soon as possible to enforce the semantics of x-uncurated: +The second phase is a technical change as soon as possible to enforce the semantics of x-curation: -+ Hackage will ensure that no revision has x-uncurated set to true. ++ Hackage will ensure that no revision has x-curation set to any variant of "uncurated". The third phase is implementation of UI: -+ The uncurated flag will be detected and displayed on a package's page, as part of the general data provided about a package. It will also be provided in search and browse results. Ideally, search and browse results will be extended in general with the ability to preform in-page filtering on flags and fields, such as "library", curated status, deprecation status, etc. ++ The curation flag will be detected and displayed on a package's page, as part of the general data provided about a package. It will also be provided in search and browse results. Ideally, search and browse results will be extended in general with the ability to perform in-page filtering on flags and fields, such as "library", curated status, deprecation status, perhaps presence in distributions or compatibility with ghc versions, etc. (note: integration with tags may be a mechanism for this). The fourth phase is indices: + The uncurated package repo root will be built and provided. -Fifth, Hackage can now begin to enforce the policy regarding curated packages not depending on uncurated packages. +Fifth, Hackage can now begin to gauge the costs of enforcing the policy regarding curated packages not depending on uncurated packages (or, more precisely, having an install-plan that can operate purely out of the curated index). + ++ Warnings will be given in such situations, and statistics will be collected as to the frequency of these warnings and the blocking packages in need of adoption. + +Sixth, when there is confidence among the trustees that the impact will not be too significant, the policy can be enforced. + Curated package uploads will be checked on upload to ensure they don't have dependencies on uncurated packages. Further, the curated index will only provide information on curated packages. -In the end state, the intent is that uncurated (as well as deprecated) packages will be hidden by default in the filtering settings of search and browse interfaces. These UI settings will be able to be persisted, so that users may change their preferences in this regard. +At this point we can proceed to a discussion on default filtering settings, bearing in mind that settings can always be saved and persisted per-user. Future Plans --------------- From 040fa7b408376ce0510af6acbfc6f8054369105a Mon Sep 17 00:00:00 2001 From: gbaz Date: Tue, 30 Jan 2018 00:35:39 -0500 Subject: [PATCH 5/6] Update 0000-uncurated-layer.rst --- proposals/0000-uncurated-layer.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/proposals/0000-uncurated-layer.rst b/proposals/0000-uncurated-layer.rst index e9e771b..3c77816 100644 --- a/proposals/0000-uncurated-layer.rst +++ b/proposals/0000-uncurated-layer.rst @@ -22,7 +22,7 @@ The desired end-state will have the following properties: 1) Packages will have an additional flag set in the Hackage package database, that indicates if they are curated or not. This flag is set *per version*. 2) Package authors will set this flag *on upload*, by setting the "x-curation" property of the cabal file of a package. If no "x-uncurated" property is set, this will be considered "curated". Along with "curated" and "uncurated," other sub-settings will be made available. In particular: "uncurated-no-trustee-contact" and "uncurated-seeking-adoption". -3) Hackage will provide two package repository roots -- http://hackage.haskell.org and http://uncurated.hackage.haskell.org These roots will provide index-01.tar.gz files that contain the information, respectively, for curated packages, or for all packages. The uncurated root will contain no revision information. +3) Hackage will provide two package repository roots -- http://hackage.haskell.org and http://uncurated.hackage.haskell.org These roots will provide index-01.tar.gz files that contain the information, respectively, for curated packages, or for all packages. The uncurated root will contain no revision information for uncurated packages (even those that have been adopted), but will contain revision information for curated packages. 4) Curated packages cannot depend on uncurated packages, and the hackage server will detect this as an error at upload time. 5) Uncurated packages may be "adopted" into the curated ecosystem by trustees. Metadata revisions necessarily remove the x-uncurated property from the revised cabal metadata. From 6d14ad88ba8a686b71bc9af37200ee4d5570ad5f Mon Sep 17 00:00:00 2001 From: gbaz Date: Tue, 20 Feb 2018 00:41:18 -0500 Subject: [PATCH 6/6] Update 0000-uncurated-layer.rst x-uncurated -> x-curation --- proposals/0000-uncurated-layer.rst | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/proposals/0000-uncurated-layer.rst b/proposals/0000-uncurated-layer.rst index 3c77816..f0c0077 100644 --- a/proposals/0000-uncurated-layer.rst +++ b/proposals/0000-uncurated-layer.rst @@ -21,14 +21,14 @@ This is a phased proposal with gradual rollout possible, in dependency order of The desired end-state will have the following properties: 1) Packages will have an additional flag set in the Hackage package database, that indicates if they are curated or not. This flag is set *per version*. -2) Package authors will set this flag *on upload*, by setting the "x-curation" property of the cabal file of a package. If no "x-uncurated" property is set, this will be considered "curated". Along with "curated" and "uncurated," other sub-settings will be made available. In particular: "uncurated-no-trustee-contact" and "uncurated-seeking-adoption". +2) Package authors will set this flag *on upload*, by setting the "x-curation" property of the cabal file of a package. If no "x-curation" property is set, this will be considered "curated". Along with "curated" and "uncurated," other sub-settings will be made available. In particular: "uncurated-no-trustee-contact" and "uncurated-seeking-adoption". 3) Hackage will provide two package repository roots -- http://hackage.haskell.org and http://uncurated.hackage.haskell.org These roots will provide index-01.tar.gz files that contain the information, respectively, for curated packages, or for all packages. The uncurated root will contain no revision information for uncurated packages (even those that have been adopted), but will contain revision information for curated packages. 4) Curated packages cannot depend on uncurated packages, and the hackage server will detect this as an error at upload time. -5) Uncurated packages may be "adopted" into the curated ecosystem by trustees. Metadata revisions necessarily remove the x-uncurated property from the revised cabal metadata. +5) Uncurated packages may be "adopted" into the curated ecosystem by trustees. Metadata revisions necessarily remove the x-curation property from the revised cabal metadata. The first phase of this rollout is simply social. It has the following properties: -+ Hackage trustees will recognize and respect the uncurated flag. When it is set to "uncurated-no-trustee-contact" they will not contact authors with any issues. They *will* retain the ability to make metadata revisions, bearing in mind that they must remove the x-uncurated property from revised metadata. ++ Hackage trustees will recognize and respect the uncurated flag. When it is set to "uncurated-no-trustee-contact" they will not contact authors with any issues. They *will* retain the ability to make metadata revisions, bearing in mind that they must remove the x-curation property from revised metadata. The second phase is a technical change as soon as possible to enforce the semantics of x-curation: