link example models to readthedocs

[wang-boyu]

This PR mainly links examples to Read the Docs, with a few minor fixes.

It is still a draft PR, as I am not sure whether to use videos or gifs to demonstrate our examples.

With videos:

• It's possible to embed YouTube videos inside Read the Docs. But in GitHub markdown files, we can only link an image to YouTube. Hence for each example, there are two slightly different markdown files to maintain: one in docs/examples folder, the other in examples folder.

With gifs:

• The markdown in docs/examples folder can simply link to that in examples folder without modification.
• The downside is we will need to host these gifs inside the repository, although they are not very large, from hundreds KB to 2 MB each.

To compare, the geo_schelling and geo_schelling_points examples are using gifs, while the rest are using YouTube videos. Once we agree on an approach, I will update accordingly.

[codecov]

Codecov Report

Base: 79.50% // Head: 79.54% // Increases project coverage by +0.03% 🎉

Coverage data is based on head (d505be0) compared to base (e50d28b).
Patch has no changes to coverable lines.

❗ Current head d505be0 differs from pull request most recent head 498e29e. Consider uploading reports for the commit 498e29e to get more accurate results

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #101      +/-   ##
==========================================
+ Coverage   79.50%   79.54%   +0.03%     
==========================================
  Files           6        6              
  Lines         610      611       +1     
  Branches       99      109      +10     
==========================================
+ Hits          485      486       +1     
  Misses        108      108              
  Partials       17       17              

Impacted Files	Coverage Δ
mesa_geo/raster_layers.py	73.92% <0.00%> (ø)
mesa_geo/geospace.py	85.39% <0.00%> (+0.08%)	⬆️

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

☔ View full report at Codecov.
📢 Do you have feedback about the report comment? Let us know in this issue.

[rht]

But in GitHub markdown files, we can only link an image to YouTube.

Does the iframe render the YouTube video correctly in RTD? Maybe we should look for a precedent in other projects. Maybe they link to a live Jupyter notebook instead of a video/GIF?

[wang-boyu]

Yes RTD renders both the gifs and the embedded YouTube videos correctly, at least on my laptop.

For agents.jl, the video files are used directly in the Examples: https://juliadynamics.github.io/Agents.jl/stable/examples/schelling/, source code here. But I'm not sure whether they use RTD or not.

[rht]

I see. My vote is to use embedded YouTube via iframe. Those hundreds of KBs of GIFs will bog down the Git repo size (that collaborators will have to download them when they git clone). Even if we need the older versions of the videos for record keeping, storing in YouTube suffices. I don't think we need to go as far as to have the videos accountable in a public Git repo for transparency's sake (so that they couldn't be tampered) or anything. Thoughts @Corvince @tpike3 @jackiekazil ?

I couldn't find any schelling.mp4 file in Agents.jl's repo: https://github.com/JuliaDynamics/Agents.jl/search?q=schelling.mp4.

[wang-boyu]

Looks like they put everything inside the gh-pages branch: https://github.com/JuliaDynamics/Agents.jl/tree/gh-pages/dev/examples

[rht]

I see. I did complain that their repo size is huge: JuliaDynamics/Agents.jl#389 (~263 MB at the time).

[tpike3]

I see. My vote is to use embedded YouTube via iframe. Those hundreds of KBs of GIFs will bog down the Git repo size (that collaborators will have to download them when they git clone). Even if we need the older versions of the videos for record keeping, storing in YouTube suffices. I don't think we need to go as far as to have the videos accountable in a public Git repo for transparency's sake (so that they couldn't be tampered) or anything. Thoughts @Corvince @tpike3 @jackiekazil ?

I couldn't find any schelling.mp4 file in Agents.jl's repo: https://github.com/JuliaDynamics/Agents.jl/search?q=schelling.mp4.

I agree, I think a smaller repo is better. We want a lean clone. This feeds to the #1406 discussion and is also a consideration for the cached results approach as we look further down the road.

[wang-boyu]

I'm more in favor of gifs, as they 1) looks nicer than embedded videos, 2) plays automatically and doesn't require Internet connection, and 3) works with both RTD and GitHub markdown.

But I do admit that they do not scale: 1MB per example gif $\times$ 100 examples $=$ 100 MB which is unnecessarily too large. It would be better if we could host them somewhere else (like the way for YouTube videos) but I couldn't find any reliable dedicated platform for this. Perhaps GitHub gist?

If there's no other objection I'll update this PR to use embedded YouTube videos.

[rht]

2. plays automatically and doesn't require Internet connection

To read the page itself, wouldn't it require an Internet connection?

Perhaps GitHub gist?

It seems to be possible: https://www.elliotblackburn.com/upload-gifs-to-github-gists/. But this also means we could have a projectmesa/mesa-geo-videos repo instead of GH Gist. This sounds like a viable option as well.

[wang-boyu]

To read the page itself, wouldn't it require an Internet connection?

You're right! 😂

It seems to be possible: https://www.elliotblackburn.com/upload-gifs-to-github-gists/. But this also means we could have a projectmesa/mesa-geo-videos repo instead of GH Gist. This sounds like a viable option as well.

In this case a mesa-geo-videos repo sounds better, since we can accept contributions through pull requests. Shall we go for this approach or stick with YouTube videos?

[rht]

I think I prefer mesa-geo-videos repo now. A Git repo is easier to replicate. And yes, enabling PR for contributions is a huge plus.

[wang-boyu]

Alright! I will update this PR to remove all gifs and use YouTube videos first. But I will keep this PR open until our monthly meeting this weekend when we discuss the examples. We can also talk about whether to have a repo to store example videos for mesa-geo.

[jackiekazil]

+1 to the direction this conversation is going

[Corvince]

I agree that we should not include gifs in this repository, but we should still be able to include the videos as gifs! I don't think we need to upload them to YouTube, we can just include the link to the other repo.

[rht]

@wang-boyu what's your take on this comment? JuliaDynamics/Agents.jl#389 (comment). I think the videos in Agents.jl doc are generated automatically via abmvideo.

[wang-boyu]

Well, that makes sense! Thank you @rht @Datseris for your input on this.

Let's say we have a separate repo to store the videos. Then the workflow to add an example would need two PRs - one in mesa-geo-videos and another one in mesa-geo. Additional efforts are also needed to make sure that they are in sync thereafter.

On the other hand, being able to export the models as videos will be very useful (see projectmesa/mesa#885). We can try to integrate it into the doc building procedure if possible.

For this PR I suggest we use YouTube videos first (since it's already done), and switch to generated videos once it's implemented.

We can continue the discussion during our dev meeting.

[Datseris]

I think the videos in Agents.jl doc are generated automatically via abmvideo.

Yes, videos are autogenerated with this function. They are displayed via HTLM on the page. The syntax is native HTML syntax that can be included in markdown documents that Documetner.jl processes into HTML. E.g.,: https://github.com/JuliaDynamics/Agents.jl/blob/main/examples/schelling.jl#L3-L7 (Literate.jl first processes normal .jl files into markdown .md files, and then Documetner.jl processes markdown files into HTML. From our perspective as developers we simply write normal, fully runnable julia scripts which call abmvideo to generate the video. We add as a comment the HTML commands that display the video. That's it. The other Julia packages take care of everything. The workflow is amazing and minimizes development effort. I love it!!!)

Let's say we have a separate repo to store the videos. Then the workflow to add an example would need two PRs - one in mesa-geo-videos and another one in mesa-geo. Additional efforts are also needed to make sure that they are in sync thereafter

That's exactly why I use abmvideo and video production by running an actual julia process that generates the video. Because this is simply too much additional effort and unsustainable in the long run. The downside? The gh-pages branch of the Agents.jl GitHub repo is large. I would choose this option any day of the week though...

On the other hand, being able to export the models as videos will be very useful (see projectmesa/mesa#885).

In Agents.jl any model can be done into a video via the abmvideo function. The same function can instead launch an interactive GUI like the ones you have in mesa, [like this one] https://juliadynamics.github.io/Agents.jl/stable/agents_visualizations/#Interactive-ABM-Applications-1 And the same function provides the documentation videos. E.g., https://github.com/JuliaDynamics/Agents.jl/blob/main/examples/schelling.jl#L198-L215

---

BTW my two cents: don't consider gif at all as a format and use mp4 for everything. It plays practically everywhere and requires much less MB for the same video... In what sort of documentation generation you can't show mp4 videos?

[wang-boyu]

Thank you for sharing your thoughts and those cool features from Agents.jl. They are very helpful. I (almost) want to switch from Python to Julia : )

[rht]

This is tangent, but usually we could have a Julia backend wrapped as a Python package, so that the code is usable to the Python audience.
There are several examples:

• Python library for symbolic regression, PySR
• Alternative implementation of Qiskit features in Julia, qiskit-alt. Qiskit is a Python library for quantum computing

[Datseris]

Thank you for sharing your thoughts and those cool features from Agents.jl. They are very helpful. I (almost) want to switch from Python to Julia : )

Agents.jl would certainly benefit by having more people on the dev team, as currently we are only two people making all of this work...

[wang-boyu]

@rht shall we review and merge this PR first, instead of waiting for the vote at projectmesa/mesa#1411?

I can update this repo later depending on the discussion results.

[rht]

Sounds good.