-
Notifications
You must be signed in to change notification settings - Fork 691
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Create Cabal-syntax for .cabal parsing and types #7620
Conversation
Cabal-syntax/Cabal-syntax.cabal
Outdated
Distribution.Compat.Binary | ||
Distribution.Compat.DList | ||
Distribution.Compat.Exception | ||
Distribution.Compat.Lens | ||
Distribution.Compat.Newtype | ||
Distribution.Compat.NonEmptySet | ||
Distribution.Compat.Prelude | ||
Distribution.Compat.Semigroup |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We may want to split the compat stuff between Cabal and Cabal-syntax, like it's done now with cabal-install and Cabal, but I'm not sure it's worth doing in this case.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'd leave them for now, and just drop pre-GHC-8 support (#7531) before next major. That would cleanup plenty of stuff, leaving mostly "We rather not depend on dlist
package and make our own copy".
The Distribution.Compat
and Distribution.Utils
are hard to separate. Lens
is kind of utility, but otoh it's a compatibility shim because we cannot depend on lens
🤷
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This indeed looks very simple.
Could you make changes I mentioned.
Making CI pass will need some changes there, but I can take a look at that.
Note: GHC devs will need to be aware of this change, their build system will need adjustments too.
Cabal-syntax/Cabal-syntax.cabal
Outdated
Distribution.Version | ||
Language.Haskell.Extension | ||
|
||
other-modules: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Most of these modules should be exposed-modules
. In fact, the different lists could be combiled.
I don't think any module have to be hidden (i.e. other-module
), except maybe some Compat
modules, though I'm not sure about these either.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Updated
-- after checking if the file exists. | ||
-- | ||
-- Argument order is chosen to encourage partial application. | ||
readAndParseFile |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Moving these seems wrong. (probably related that this module have to exposed?)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
So I moved these because "Verbosity" seems like a cabal-install specific thing as opposed to a Cabal-syntax thing, so I left it in Cabal
.
These seem to be mostly for nice error reporting in a context where "Verbosity" makes sense, but I would guess most external use cases would rather handle the errors differently.
Edit: And the only internal use was in Distribution/Simple.hs, so it was easy enough to just pop over there.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Good point about Verbosity
.
Most (maybe even all?)
Looks like very few unit tests are testing |
|
Running into some small compatibility issues, and not sure what the standard practices are here.
|
I'm not sure if moving the checks to cabal-syntax makes sense or not as a whole? I would thing the checks are for the semantics of a well-formed cabal file, not for parsing and manipulating its syntax. I can see arguments either way, though. On the latter, could you use reexported modules? https://cabal.readthedocs.io/en/3.4/cabal-package.html#pkg-field-library-reexported-modules I'm not sure if that fits in the support window for Cabal features, but I think it might? |
@gbaz, consider |
As the comments say:
Hackage-server needs both types of checks, so I imagine it would need stuff that went beyond cabal-syntax regardless... |
So? The package content checking interface is abstracted: -- | A record of operations needed to check the contents of packages.
-- Used by 'checkPackageContent'.
--
data CheckPackageContentOps m = CheckPackageContentOps {
doesFileExist :: FilePath -> m Bool,
doesDirectoryExist :: FilePath -> m Bool,
getDirectoryContents :: FilePath -> m [FilePath],
getFileContents :: FilePath -> m BS.ByteString
}
-- | Sanity check things that requires looking at files in the package.
-- This is a generalised version of 'checkPackageFiles' that can work in any
-- monad for which you can provide 'CheckPackageContentOps' operations.
--
-- The point of this extra generality is to allow doing checks in some virtual
-- file system, for example a tarball in memory.
--
checkPackageContent :: (Monad m, Applicative m)
=> CheckPackageContentOps m
-> PackageDescription
-> m [PackageCheck] Or put it other way around: |
Right, we could put all those checks into cabal syntax. My understanding is those checks do not belong in cabal-syntax, because they're beyond the scope of cabal-syntax, which is, as the name says, cabal syntax, and checks for well-formedness of a tarball don't make sense there. I'm proposing we put no checks in cabal syntax, since most clients will not need them, or those that will, will want all of them, including the ones that don't belong there. As one of the primary people currently maintaining hackage-server, I really don't care if it needs to continue to depend on the whole |
Reexported modules were exactly what I needed, thanks. Looking at the check code again, there wasn't really a compelling reason to move it in the first place. Just a little over-aggressive in trimming the test dependencies. I'll roll that back. |
5e42df5
to
13b3345
Compare
Rolled back the PackageDescription Check change, updated the rest of the modules, and I think CI should work now. |
@@ -13,8 +13,9 @@ library | |||
build-depends: | |||
, base | |||
, bytestring | |||
, Cabal ^>=3.7.0.0 | |||
, QuickCheck ^>=2.13.2 || ^>=2.14 | |||
, Cabal ^>=3.7.0.0 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What Cabal-QuickCheck
and Cabal-described
need from Cabal
?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Cabal-QuickCheck
needs:
Distribution.Simple.Compiler (DebugInfoLevel(..), OptimizationLevel(..), PackageDB(..),ProfDetailLevel(..),knownProfDetailLevels
Distribution.Simple.Flag (Flag(..))
Distribution.Simple.InstallDirs
Distribution.Simple.Setup (HaddockTarget(..), TestShowDetails(..))
Distribution.Utils.NubList
Distribution.Verbosity
Cabal-described
needs Distribution.Verbosity
instance Described Verbosity where
describe _ = REUnion
[ REUnion ["0", "1", "2", "3"]
, REUnion ["silent", "normal", "verbose", "debug", "deafening"]
<> REMunch reEps (RESpaces <> "+" <>
-- markoutput is left out on purpose
REUnion ["callsite", "callstack", "nowrap", "timestamp", "stderr", "stdout" ])
]
The -- | The name of the auto-generated Paths_* module associated with a package
autogenPathsModuleName :: PackageDescription -> ModuleName
autogenPathsModuleName pkg_descr =
ModuleName.fromString $
"Paths_" ++ map fixchar (prettyShow (packageName pkg_descr))
where fixchar '-' = '_'
fixchar c = c can very well be in |
I tried this patch and it Except of |
Updated artifact/bootstrap/meta CI, but not sure how to teach |
I think I've now fixed doctest and the Windows artifact issue, which just leaves the ghc 7.X failures due to reexported-modules not being supported. |
Other minor (?) caveat, reexported modules dont work well with Not sure if cabal can be built with stack, but it should to contribute to universal harmony 😛 |
|
That said. It would be very nice to check what From my code-only scan, it seems it needs only macro generation (i.e. the same as And now I'm starting to think where those belong. :( |
To be clear, I was only reexporting the single module Distribution.Package so that hacakge-security didn't need to be patched for cabal-install to build successfully. I can also update hackage-security if that route is easier.
From a quick look, the two most likely issues I saw were the Edit: A slightly more rigorous search:
|
|
Any suggestions on how to deal with the breaking change for |
Just make a branch of hackage-security with the patch. Since its under the haskell org umbrella, we can make sure the patch is accepted when this is merged. That said, I don't think reexported modules is problematic here. The concern about stack is about backpack, afaik, not reexported modules, which is a much earlier feature, and the version number coupling should not be a concern in our particular case. (i.e. the idea of an update to the syntax lib without an update to the main Cabal lib seems unlikely) |
yeah, i overlooked it, i hit the issue using |
f81c533
to
9da489e
Compare
ad61883
to
8bca501
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
The hackage-security PR is merged, the dummy Cabal-syntax is on Hackage, all seems ready to go. Last moment to have a critical look before merge. This is quite a fabulous PR IMHO, but also very disruptive, so it'd be good not to revert and re-revert it too many times... |
I think this is great overall (and we might or might not move some modules around in the future). However, to avoid reverts, you should make GHC devs aware that this is going to happen soon, cause if they realise this is too late (in GHC release pipeline), there will be too much pressure to revert indeed. |
Than you for the heads up. I've just told the GHC hackers and will liaison with them as needed. |
8bca501
to
c9d5a0f
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Could you add Cabal-syntax
in README.md
, under "This Cabal Git repository contains the following packages:"?
c9d5a0f
to
ce8dc17
Compare
Updated |
I suspect that it shouldn't be too hard to adapt GHC to this change but I'll have to try to know for certain. |
Cabal-syntax/Cabal-syntax.cabal
Outdated
mtl >= 2.1 && < 2.3, | ||
parsec >= 3.1.13.0 && < 3.2, | ||
pretty >= 1.1.1 && < 1.2, | ||
text >= 1.2.3.0 && < 1.3, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This needs to be bumped to support text-2.0
as GHC has already bumped.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Indeed, we bumped this already on master, but mechanical rebasing did not reflect that in a PR a new package. I suppose the bounds need to be resynchronized with Cabal.cabal.
Should we wait? |
I am currently working on attempting the adaptation. So far so good but I will feel better once both build systems have passed the testsuite. I'll let you know shortly. |
Is |
Yes. (EDIT: And ghc-make/hadrian shouldn't rebuild it either). |
I have confirmed that with ghc!7401 and the following patch I can bootstrap GHC: diff --git a/Cabal-syntax/Cabal-syntax.cabal b/Cabal-syntax/Cabal-syntax.cabal
index 7649ac76b..91c04002d 100644
--- a/Cabal-syntax/Cabal-syntax.cabal
+++ b/Cabal-syntax/Cabal-syntax.cabal
@@ -41,7 +41,7 @@ library
mtl >= 2.1 && < 2.3,
parsec >= 3.1.13.0 && < 3.2,
pretty >= 1.1.1 && < 1.2,
- text >= 1.2.3.0 && < 1.3,
+ text (>= 1.2.3.0 && < 1.3) || (>= 2.0 && < 2.1),
time >= 1.4.0.1 && < 1.12,
-- transformers-0.4.0.0 doesn't have record syntax e.g. for Identity
-- See also https://github.com/ekmett/transformers-compat/issues/35
diff --git a/ghc-packages b/ghc-packages
index 8d827f6f9..e9e369c42 100644
--- a/ghc-packages
+++ b/ghc-packages
@@ -1,2 +1,3 @@
Cabal
+Cabal-syntax |
Well spotted re |
ce8dc17
to
c253f7d
Compare
Fixed text and ghc-packages + bumped time to |
I think we are go. :) |
c253f7d
to
a64b9ae
Compare
Please include the following checklist in your PR:
Please also shortly describe how you tested your change. Bonus points for added tests!
This PR splits
Cabal
intoCabal-syntax
andCabal
(re: #7559).The changes were generally conservative to avoid making design decisions. I mostly just pulled PackageDescription.hs and GenericPackageDescription.hs into the new package, and then moved code over as necessary to fix all the compilation errors.
The two code changes I did make were to move a few utility parsing functions so that Distribution.Verbosity stayed in
Cabal
rather thanCabal-syntax
. Otherwise it was just import changes to avoid going through the Distribution.Simple.Utils module.The result is split almost exactly in half (126 modules in
Cabal-syntax
, 121 modules inCabal
)