Documenting Maiko -- what do the opcodes do?

[lassik]

The SUBR (i.e. VM opcode) dispatch table seems to be in Maiko's src/subr.c. Could we add a specially formatted comment into the C code to document each SUBR, and write a script to extract those comments and generate a nice HTML document from them?

If the description of each SUBR is right next to its implementation, that would be the easiest way to keep the two in sync.

(See also #28)

[lassik]

It might also help to extract each SUBR into its own C function and put the documentation comment on top of the function. Most SUBR implementations are already in their own function (but there is no consistent naming convention) but some are embedded in the big switch.

C compilers have good inliners nowadays, so there shouldn't be a performance penalty from extracting even small code snippets into their own functions.

[masinter]

There are OPCODEs and SUBRs; they're different. OPCODEs can do anything. There is one opcode called SUBRCALL which then has its own dispatch table.

The main problem we have is not "finding" t'he documentation. It's that some documentation is old or (too often) non-existent. I think we need to build documentation and then we can worry about keeping it in sync.

[masinter]

closing the issue was a mistake. We can write our own comment-string-scanner and pull out the documentation we have.
No need to restructure the code or modify it in any way.

[lassik]

Something that works surprisingly well is to use a /// prefix for line comments containing documentation (assuming you can use the C feature of // comments), then concatenate all such lines into one file and strip the /// prefix. If you use e.g. Markdown markup in those comments, you arrive at a finished Markdown document.

In Unix, cat *.c | grep '^///' | sed -e 's@^/// @@' -e 's@^///@@' >documentation.md

There are also existing C documentation tools like Doxygen, but those are somewhat complex.

[lassik]

Documenting the calls and refactoring their implementation can probably be tackled as two different and mostly orthogonal tasks, since the comment markup is agnostic to where in the source code it happens to be attached.

[nbriggs]

I prefer doxygen comments if we’re going to do any documentation.

[orcmid]

assuming you can use the C feature of // comments)

If you compile for C99 with extensions, or can set /std:c11 or equivalent in compiler source-code options, this should all work without complaint.

Unfortunately, the MSFT VC option /doc: seems to presume that the comments use a particular schema of XML. Not the best choice these days. Hmm.

[orcmid]

Unfortunately, the MSFT VC option /doc: seems to presume that the comments use a particular schema of XML. Not the best choice these days. Hmm.

BUMMER ANYHOW:
cl : Command line warning D9032 : '-doc' : XML documentation comments are not supported by the C compiler

And if it was, it only works on a single-source-file input (not counting preprocessor headers).

This makes the @lassik technique preferable, and markdown seems more flexible, although it requires more agreement about conventions. I have no skin in this game, so have no position.

[nbriggs]

C99, no extensions. Using /* */ style.

[masinter]

There are two kinds of interfaces between C and Lisp (there are more, but two are numbered: opcodes and subrs. Well, there's a SUBRCALL opcode that works like a function call.
There are several sources of information about each: the implementation in C, the "UFN" in Lisp (if there is one), the compiler, the opcode test cases. To develop documentation for the virtual machine it might be necessary to look at all of those.

[masinter]

After a few years as a wishlist item without much hope of anyone taking this on... this is a research project with a deliverable without any good way of testing correctness.

There's lots of good discussion though, so ... pushing this to a Discussion