Moved manual to worldengine-data

[esampson]

Moved manual to worldengine-data so images could be included. This is necessary so that PDF and HTMLzip files will contain the images.

[psi29a]

The only problem here is that we won't get any API documentation that is generated for us auto-magically which is the key selling point of Sphinx. I had hopped that this was the last option to choose. :(

@ftomassetti we need to hammer this out with what is important to us.

[ftomassetti]

@psi29a you mean documentation generated from comments in the code? I think that at this moment we are not doing that anyway. For now this is a manual for users, not for developers. Perhaps we could keep the two things separated.

[psi29a]

Yes and no, it generates API documentation based on the source, it fills in the rest (input and return types, explanation of usage) with comments. We use it extensively at work here and we better get there as well if we are trying to provide to the world a library to use.

[psi29a]

I'm of two minds about this, for someone wanting to use the library/cli... they would be developers. I don't expect end-users to use either as they would be focused on using the GUI (if/when someone works on that).

Perhaps I'm being too forward thinking.

[esampson]

@psi29a said:

The only problem here is that we won't get any API documentation that is generated for us auto-magically which is the key selling point of Sphinx. I had hopped that this was the last option to choose. :(

@ftomassetti we need to hammer this out with what is important to us.

As I see it there are 5 possible options.

1. Allowing binary data into the github repository for Worldengine.
2. Not have images in the manual.
3. Not allowing PDF or HTML copies of the manual (or simply living with the fact that they will be missing images)
4. Using a different repository for the manual.
5. Split the manual into 2 separate manuals. One is the technical manual that is autogenerated and based off the code in the worldengine repository. The other is a user's manual based off of the worldengine-data repository.

From my point of view option 2 is not viable. There's simply too many occasions where it will be necessary to show an image. In the biomes section there are 4 images.

• Two of the images could be done away with without too much difficulty (the scatter plot images). Since people can generate their own scatter plots all that is required is a carefully worded description of what they will see. Clumsy, but doable.
• One could be done away with with a fair amount of difficulty (color chart for biomes). ReST does not have very good support for customizing tables. A table could be created but it would be fairly ugly and the column showing the colors would have to be removed (there's no way in ReST to color the background of cells). People would be given the hex code for the color but they would have to go to another program to actually see what they look like. Not really an ideal solution.
• One picture could only be replaced with great difficulty (our implementation of the Holdridge life zones). It would in theory be able to replace that with a table since we are only using two axes. However we would lose the grouping. It would also be much more difficult for people looking at it to understand how our implementation coincides with the general model. Finally, while the table is a technical possibility ReST charts look pretty God awful.
• These are the images currently being used. When we start writing up example scenarios there will probably be situations where it will be really cumbersome to explain what is going on without images. A primary example would be changing --temps and --humids to produce a desert world or an ice world.

I don't really see option 3 as a particularly good idea, either. I'm one of those people whose mind is always going in half a dozen directions. Subsequently I will often have a sudden thought and want to look at a manual when I am no where near my computer. As a result I really like being able to download manuals to my tablet and being able to look them up. Yes, it is also possible for me to use my phone to provide a wireless connection for my tablet, but that is a more cumbersome option.

Option 5 is workable. It has the disadvantage that the technical manual can't support images. This prevents us from showing example outputs but is still probably a manageable limitation (I can't think of any technical manuals that I've seen that include example output images).

[ftomassetti]

I would be fine with 1 and 5, but I think Bret does not like 1 (in the past I was responsible for making the repository very heavy to download and he had to clean it up, removing files also from the history, probably something not very funny to do)

[esampson]

In order of preference I probably like 1, 4, 5, 3, then 2. 1 only narrowly nudges out 4 in my opinion because I would prefer keeping everything together as opposed to splitting it apart. My preference for 4 over 5 is greater because while the manual's data is split into a separate repository at least the manual is still kept together, and again, I prefer keeping things together than splitting them apart.

I still feel that 5 is quite workable, however, even with my preference not to split up the data. I'm simply stating that in my opinion 4 beats out 5 by a more respectable amount. Sort of the difference in 99%, 98%, and 92%.

I should also add that option 4 does not mean we can't do the technical manual that is automagically generated from comments in the code. We simply need to copy the code to the worldengine-data repository. I don't know if this can be setup to automatically happen or if it would be necessary to manually update the data. It does mean we have the same data in both repositories but since the worldengine-data repository is not generally meant for downloading that isn't as much of a problem.

[psi29a]

I don' t understand why we can't use real URLs to images in our documentation? Instead of referencing images 'locally', that we instead reference one from the WWW so we can host our images, scripts, binary data here in worldengine-data.

[psi29a]

.. image:: https://raw.githubusercontent.com/Mindwerks/worldengine-data/master/images/examples/rivers.png

^-- does this not work?

[esampson]

Not really.

It works when you view the document through RTD (https://worldengine.readthedocs.org/en/latest/biomes.html), however it does not work when you download a PDF or an HTMLzip (https://media.readthedocs.org/pdf/worldengine/latest/worldengine.pdf).

It isn't that RTD won't put images into PDFs or HTMLzips (https://media.readthedocs.org/pdf/worldengine-ecs/latest/worldengine-ecs.pdf). It simply won't do it if you are linking to files outside the immediate repository.

[ftomassetti]

Would it make any sense to have a script which download the images and then
run the command for generating the documents?
On 28 Nov 2015 9:50 a.m., "esampson" notifications@github.com wrote:

Not really.

It works when you view the document through RTD (
https://worldengine.readthedocs.org/en/latest/biomes.html), however it
does not work when you download a PDF or an HTMLzip (
https://media.readthedocs.org/pdf/worldengine/latest/worldengine.pdf).

It isn't that RTD won't put images into PDFs or HTMLzips (
https://media.readthedocs.org/pdf/worldengine-ecs/latest/worldengine-ecs.pdf).
It simply won't do it if you are linking to files outside the immediate
repository.

—
Reply to this email directly or view it on GitHub
#18 (comment)
.

[psi29a]

Would git submodule be an option? Kill a few birds with one stone. That way we reference worldengine-data/docs in the worldengine/docs, think of it as the git version of a sym-link.

https://help.github.com/articles/using-submodules-with-pages/ <-- solves our http://world-engine.org problem for example.

[esampson]

I'm not sure that would work. RTD would need to do fetches for the submodules.

[psi29a]

According to this, it works:
readthedocs/readthedocs.org#606
and
readthedocs/readthedocs.org#658

It doesn't hurt to try I think. :)

So here is another possibility:
in worldengine, symlink worldengine-data to worldengine/data, that way we can reference images in both our github pages and in read-the-docs. This would allow us to keep documentation in worldengine.

This also gives us the added benefit of simplifying our travis and appveyor code.

[psi29a]

Problem solved I think... https://git-lfs.github.com/

We'll have to try it out! :)

If this works out, we an merge this repo back into worldengine.