missing docs on MANY new packages!!!!

[cartazio]

Have we fixed this in hackage server head? when is it getting rolled out and haddocked regenerated?

http://hackage.haskell.org/package/cabal-debian-3.8.3 is a new example of this problem i just found :(

[blitzcode]

I also noticed missing documentation for a lot of packages like OpenGL and GLFW-b, have to rely on the docs of older versions or the ones I have installed locally. This is rather annoying as Hoogle has links to the main Hackage pages, and I then have to manually locate the alternative ones. A fix would be much appreciated!

[balta2ar]

Confirm the problem. Here is another package with missing docs: http://hackage.haskell.org/package/http-streams-0.7.0.2

[istathar]

To @balta2ar's comment, it's not a problem with the package in question; it renders fine locally (as mirrored at http://research.operationaldynamics.com/projects/http-streams/doc/Network-Http-Client.html) and the part that was spliced out of the package, http-common is fine on Hackage — ie http://hackage.haskell.org/package/http-common-0.7.0.1/docs/Network-Http-Types.html is fine.

AfC

[z0w0]

helm-0.5.0 is missing docs too, they build fine locally.

[funrep]

I confirm this problem too, verbalexpressions which I uploaded recently is missing docs too. Thought I first had to build docs locally, but then realized that cabal's sdist doesnt pack them into the package anyway.

[cartazio]

#56 would be a workaround ... we just need to make it discoverable

[istathar]

Why should it need a workaround? Hackage 1 never had any problems with any of these packages.

[cartazio]

well, then write the patch needed, whatever it is. In the mean time, thats a workaround, upload your own docs :)

[dcoutts]

Note that we're now running a new doc builder client which should resolve some of these issues (it's currently going through a big backlog). For the things not resolved, it does at least generate build logs (see /package/foo-1.0/reports/$n/log) so we can see what's wrong. Mostly I expect this will now be missing C libs which we'll need to add to the builder machine.

In particular, as a test case I've deleted the broken docs for http://hackage.haskell.org/package/verbalexpressions-1.0.0.0, so keep an eye on http://hackage.haskell.org/package/verbalexpressions-1.0.0.0/reports/ and http://hackage.haskell.org/package/verbalexpressions-1.0.0.0/reports/1/log

[dcoutts]

Note also, that if you find packages with broken docs, one can delete them and the doc builder should automatically retry. e.g.

curl -X DELETE http://you:passwd@hackage.haskell.org/package/helm-0.5.0

You have to be in the maintainer or trustee group to do this. So this would be a good task for trustees, speaking of which, we ought to recruit a few! http://hackage.haskell.org/packages/trustees/

[funrep]

I tried to upload docs to the verbalexpressions package but I got this error:

$ curl -X PUT -d @dist/doc.tar http://usr:passwd@hackage.haskell.org/package/verbalexpressions-1.0.0.0/docs
Unexpected content-type: For this resource the content-type was expected to be application/x-tar, rather than application/x-www-form-urlencoded

What format should the file I gonna upload be? I just made the dist/doc a tar since I thought that was the accepted format reading the server API.

I also tried the script which was presented in issue #56 but got the following error:

$ sh build-docs.sh 
build-docs.sh: 8: build-docs.sh: Syntax error: "(" unexpected

I changed the parentheses to quotes since I thought it seemed like a string but then I got this error:

$ sh build-docs.sh
http://hackage.haskell.org/package/verbalexpressions-1.0.0.1 404
http://hackage.haskell.org/package/verbalexpressions-1.0.0.1/candidate 404
Found no uploaded version
Resolving dependencies...
Configuring verbalexpressions-1.0.0.1...
build-docs.sh: 72: build-docs.sh: Bad substitution

I really don't know bash so I thought I rather ask here how to upload docs by hand.

[dcoutts]

@klrr to answer the curl question, are indeed uploading the right format (ie a tar file), so all you need to do is tell the server that that is what you're doing. By default curl tells the server that it is sending the results of a html form (application/x-www-form-urlencoded), so you just need to override that and tell it the type you're actually sending (ie tar)

To do that is the -H flag to specify the 'Content-Type header, -H 'Content-Type: application/x-tar':

curl -X PUT -H 'Content-Type: application/x-tar' -d @dist/doc.tar http://usr:passwd@hackage.haskell.org/package/verbalexpressions-1.0.0.0/docs

[dcoutts]

As for using verbalexpressions as a test case, we can now see that the new doc builder is more useful than the old, because it uploads a report and a build log:

• http://hackage.haskell.org/package/verbalexpressions-1.0.0.0/reports/1/
• http://hackage.haskell.org/package/verbalexpressions-1.0.0.0/reports/1/log

So the report says that regex-pcre failed to build, and verbalexpressions depends on that. Looking at the log we can see why: the pcre C lib is missing on the build machine.

So any of the admins with access to the build machine can fix this by installing that C lib, then grepping the reports for affected packages:

$ grep 'DependencyFailed regex-pcre' /srv/hackage/var/build/results/*.report -l
/srv/hackage/var/build/results/aeson-schema-0.2.0.1.report
/srv/hackage/var/build/results/anatomy-0.4.report
/srv/hackage/var/build/results/atomo-0.4.0.2.report
/srv/hackage/var/build/results/hark-0.2.report
/srv/hackage/var/build/results/hellage-0.1.1.report
/srv/hackage/var/build/results/hyakko-0.6.5.report
/srv/hackage/var/build/results/verbalexpressions-1.0.0.0.report

Then to trigger their rebuild, the admin has to remove these packages from the failed file, and then the doc builder will have another go.

[funrep]

@dcoutts Thanks for the quick reply! I then couldn't upload it since it was in wrong format but I changed it to ustar format as suggested in issue #56. Now I get the following error though which I don't really get since doc.tar is not empty at all.

$ curl -X PUT -H 'Content-Type: application/x-tar' -d @dist/doc.tar http://foo:bar@hackage.haskell.org/package/verbalexpressions-1.0.0.0/docs
Warning: Couldn't read data from file "dist/doc.tar", this makes an empty
Warning: POST.
Invalid documentation tarball: truncated tar archive

[dcoutts]

@klrr does dist/doc.tar exist? Is it a valid tarball. The fact that curl cannot read from it suggests it's either missing, or wrong permissions or something.

BTW, you may prefer to just hold off, I've installed the pcre C lib dev files on the builder machine and prodded the builder to try again, and verbalexpressions is on the builder's TODO list again.