Documentation generated by GDScript documentation comments disappears

[eh-jogos]

Godot version

4.0 beta16

System information

Manjaro Linux using i3, Foward+

Issue description

Lets's say you have a script using documentation comments, it will have it's own Documentation page generated in the editor.

If you close the editor, then open the editor again, work on another, unrelated script and save it, close the editor and open it again, the documentation page for your first script will be gone.

Searching for it will also give no results:

Then to get the documentation generated again, you have to open the first script, make some change to it and save it. The old trick of commenting/uncommenting any line and saving also works.

Steps to reproduce

• Create any documentation comments for any script
• Open its documentation
• Close the project and open again
• Do some change to any other script and save
• Close the project and open again
• The generated documentation for your first script will be gone, and if your search help for it it will have disappeared.

Minimal reproduction project

gdscript_documentation_comments.zip

[dalexeev]

Probably related:

• Script classes are not regenerated project-wide after first import #72154
• Extensions of custom classes invalid upon deletion of .godot folder #72226

[anvilfolk]

Since both those issues have been resolved, @eh-jogos would you be able to test on Beta 17 and see if the issue is still present?

[eh-jogos]

@anvilfolk just tested the MRP with beta 17 and it's still happening.

[anvilfolk]

Good to know!

For whomever is looking at this: right now the DocData information is generated by the GDScript compiler, and compilation happens in GDScript::reload(). I assume that script compilation happens when the project is loaded, so this might be an issue of the order in DocData is created and made available. Maybe right now it's made available first and only then generated for GDScripts (thus making it not available)?

Maybe someone from the documentation team can shine a little light :)

[limbonaut]

Still happening in 4.0-stable

[anvilfolk]

I am revamping the documentation generation system in #72095 and have some leads on a fix for this one :)

My understanding is that scripts are only loaded on-demand, so if you open a single script, then its documentation, and try to click a link to another script documentation page, that won't have been generated (because that script hasn't been opened, compiled, and docgen'd). I'll be trying to trigger script compilation/docgen when the user clicks a hyperlink in the help screen :)

[Variable-ind]

i can still replicate this issue in Godot 4.2 beta 2

Steps to reproduce (Modified )

• Create any documentation comments for any script
• Open its documentation
• (Go to File --> Close all to close any open scripts) THEN Close the project and open again Project --> Reload current project
• Do some change to any other script and save
• Close the project and open again Project --> Reload current project
• The generated documentation for your first script will be gone, and if your search help for it it will have disappeared.

Api-.3.mp4

Minimal reproduction project

Docs bug.zip

[anvilfolk]

There were some conversations about how to fix this a while back.

Godot already builds a mapping of global class names to the file they are declared in, so that when some script refers to a global class name, it knows what file to load. I suspect the best way to tackle this issue is to duplicate or reuse this functionality in order to register classes into the helper dialog so that they are at least found and shown on the list.

[anvilfolk]

Very slowly thinking my way through a solution for this.

There are some annoyances:

• Documentation is only generated when scripts are fully compiled
• Search Help dialog is populated with classes whose documentation has been generated
• All user scripts would need to be compiled in order for their documentation to show up

I feel like compiling all of a project's scripts on editor startup so that documentation appears is a terrible idea for large projects.

For discussing solutions, here are a couple of terms I'm introducing:

• Doc stub: documentation data structures that are incomplete in some way, e.g. missing class members or type info.
• Complete doc stub: documentation data structures with information about all class members, but no type info.

Possible solutions:

1. Generate documentation stubs for all scripts just from their global class name, which has been cached and is known. This would be really fast. However, these would be incomplete doc stubs, so Search Help wouldn't find, e.g. signals/constructors/functions within user scripts, until they are compiled. So the behavior reported in this issue remains except for finding classes. This inconsistency feels inappropriate to me 😞
2. Provide a ScriptLanguage-level API to generate complete doc stubs (without type information). The parser would still run for all project scripts, though the analyzer & compiler wouldn't. Still likely non-trivial for large projects? However, Search Help would now find everything, and trigger a full script compile + docgen when a docs are opened but a stub is found.
3. The same as above, but with caching all doc stubs so that the entire project's worth of scripts don't need to be parsed at runtime, and can simply be loaded from a cache file.

I'm most tempted by 3, even though I am still unhappy about adding a documentation generation API that loads scripts behind the scenes at the ScriptLanguage level. Right now, scripts are loaded almost exclusively through ResourceLoader::load(), and this would open up a new way to (partially) load scripts for docgen purposes. This does seem the only place that it makes sense for the docgen request to exist though, since ResourceLoader::load() needs to return a fully compiled script and adding a ResourceLoader::load_documentation() feels silly since docs only exist for scripts.

So approach 3 is more code and work because of all the extra caching stuff, but may also be the best compromise to keep the editor loading smoothly. It feels like the best trade-off to me. What do folks think?

[YuriSizov]

@anvilfolk Sounds like you want to have all script documentation cached, and only regenerated if the corresponding class was modified or if there is no cache at all. While this comes with all the drawbacks of having cache, that's the only way to provide all the documentation at startup, without having to generate it at each startup.

[anvilfolk]

It sounds like you're saying it's totally acceptable to cache the entire documentation (instead of just some stubs with relevant information for Search Help) and load it on editor start. I was coming at it from the perspective of that maybe being too much, but perhaps not? If that feels acceptable, then that sounds like the best solution.

It does mean that at some point, we're asking all project scripts to get compiled, but that only happens once :) Does that sound right?

[YuriSizov]

We already cache the entire class reference. We don't cache extensions, but they instead always generate docs on load.

So it seems fine to me to cache documentation from scripts. We may have to do the same for extensions, though detecting changes there may be trickier.

One issue to deal with would be external updates to scripts when the editor is off. But we already handle that somehow? Probably using the last modified date on the file.

[anvilfolk]

Sweet, thanks for the information! I might try to get a crack at it then :)

[tokengamedev]

@anvilfolk, here is my perspective, as I am a heavy user of the code documentation.

Thinking out loud:

• The documentation being generated in my perspective, should be called as documentation data instead of cache. It has to be persistent and loaded in memory for the project during startup.
• When the user changes the documentation in the code at runtime, documentation data will not be updated unless the scripts are fully compiled and files are saved. It means there may be two versions of the same documentation data at runtime.
• If scripts are not fully compiled, then user can only view the old version of the data in the help section.
• When the files are modified offline (when Godot is sleeping 😴 ) , on project start, it will still show the old documentation data, and it will follow the same procedure as above (fully compiled scripts)
• Add an option in menu to reset the whole documentation "Clear cache and restart", which will clean up the documentation and flag to create one during startup. It may take time, but it is expected, and it should be synchronous.

Only thing I am not aware of hooks to start the documentation data creation process, during startup or when script files are saved.

[theraot]

I'm running into this in v4.3.beta1.official [a4f2ea9], I noticed because the tooltips on the properties in the inspector won't show up until I force Godot to parse the script again by doing some change on it.

[AThousandShips]

How about beta3? Just to be sure

[theraot]

Tested. I have the same behaviour in v4.3.beta3.official [82cedc8].

[cranky-corvid]

I ended up here after noticing that my project's auto-generated documentation has recently stopped showing up in Search Help. The issue occurs in both v4.3.rc1.official [e343dbb] and v4.2.2.stable.official [15073af].

[Skaruts]

I just noticed this issue in 4.2.2-stable, but I don't have to reload the project twice for it to happen. I just have to reload once and no need to edit any other files.

Meanwhile, a (crappy) workaround is to edit the affected files. Just add a space somewhere and save, and it will be listed in the docs again (and probably persist, at least until you delete the .godot folder, I think).

[ThePat02]

I just noticed this issue in 4.2.2-stable, but I don't have to reload the project twice for it to happen. I just have to reload once and no need to edit any other files.

Is there any other way around this? This is an absolute nightmare when working on a addon with a fair amount of documented classes. Can I manually force Godot to recompile all scripts?

[anvilfolk]

I've been working on a fix (#95821). I've been dealing with a sick kid so haven't had time to incorporate the latest feedback but hopefully it will not be a problem for much longer!

Super excited to see GDScript documentation become more and more useful, and more and more used!

In the meantime, the PR does work, so you can always download it and compile yourself or use its build artifacts :)

[ThePat02]

@anvilfolk Thank you for your great work. For now the tooltips in VSCode should too! I am also excited for your fix and improvements to this great feature!