JSON Backend

[jfischoff]

It would nice to be able to directly dump out a version the Interfaces that Haddock generates as JSON

This need is being driven by some Hackage redesign work. We would like to distribute the generation of documentation in a safe way (HTML is hard to keep safe) and also have output we can cache forever.

Having an external format in JSON will allow Hackage to cache the Interface information forever, and more easily change the presentation layer. There are other benefits as well, in terms of analysis, and future Haddock features e.g, pass through markup.

I am volunteering to do this work.

[bitemyapp]

Does this have to be serialized? Could haddock be used as a library and return a Haskell data type to avoid this complexity?

[jfischoff]

Probably yes.

On Wed, Nov 26, 2014 at 3:03 PM, Chris Allen notifications@github.com
wrote:

Does this have to be serialized? Could haddock be used as a library and
return a Haskell data type to avoid this complexity?

—
Reply to this email directly or view it on GitHub
#352 (comment).

[cartazio]

im pretty sure that the data rep has breaking changes every ghc version (because its using GHC internal bits), so using haddock as a library for the reading back the serialization layer may not be viable :)

[bitemyapp]

@cartazio I'm saying make an agnostic, Haddock-spun, representation since it has to parse and deal with the documentation-specific content anyway.

We lose much of the benefit of our type system when we resort to serialization for things like this.

If representation varies between GHC versions, then make it a sum type of GHC versions. Do whatever has to be done, but I'd really like a data type.

My preference is that Haddock's binary frontend become a convenience for end-users not necessarily interacting with Hackage, with the doc-builder for Hackage invoking Haddock as a library.

This has maintenance/hassle benefits since intermediate writing to disk can be skipped as well. Less error prone.

[cartazio]

umm, the sum type wouldnt work, Internally haddock is using GHC's AST types. You only have access to one of those at a time.

[bitemyapp]

@cartazio Final encoding?

[cartazio]

how would a final encoding help? and what do you mean ?

[bitemyapp]

Looks like we're fine with Interface/GHC types - understanding that new versions could break things.

So, my only remaining point is that I'd like it if we gave avoiding serialization some thought by invoking Haddock as a library.

I take it that means haddock-library would need augmented?

[Fuuzetsu]

haddock-library are those Haddock bits which don't deal with any GHC API stuff at all so it's unlikely you'll be touching that. haddock-api is the bulk of work; the haddock package now is nothing but a Main module.

The idea of Haddock-specific type has been mentioned by Simon a few times; the idea is to avoid exposing any GHC types but instead use our own which doesn't break every GHC release. This poses the problem of trying to come up with such a type and do translations. Seems easy so far. I think the main problem here is that GHC gives us GHC-specific things while in the interface files we would now have Haddock-specific thing and we'd need to do matching up: currently it's simple because we look up GHC types when given GHC types. Anyway, I think if we're taking this approach then a separate issue is worthwhile. If this is implemented then it is no longer impossible to have a single Haddock version support multiple major GHC versions: currently even if we don't make any changes on our end, we still can only work with one major version.

More on topic, what information would such JSON file have? You don't give any specifics.

I'm happy for you to do the work but I think it'd be even better if this was outside of Haddock core. We'd like to encourage user backends to not ship with Haddock: it is less burden on us to maintain and users are in power. If there are problems in trying to implement a backend outside of the core then we should remedy that instead.

It is quite late so I may be rambling, feel free to ask stuff.

[jfischoff]

I agree, the backend should live outside of Haddock proper. Closing.

[beerendlauwers]

Is there a package that provides this functionality? I'd like to look up the type and definition of a function in the JSON. (and probably whatever else we could cram into it)

[alanz]

I'm pretty sure leksah should be able to provide this

[beerendlauwers]

Thanks for the hint! This seems pretty interesting. I'll see what happens when I have this puke out JSON.