nim doc --backend:js, nim doc --doccmd:-d:foo, nim r --backend:js, --doccmd:skip + other improvements

[timotheecour]

flags introduced here are by design orthogonal so that they work as you'd expect when you combine them.

nim --backend:js|c|cpp|objc (or -b:js etc)

• fix importjs+nim doc,Error:importjs pragma requires JavaScript target #13891 (importjs+nim doc,Error:importjs pragma requires JavaScript target)
• fix Allow specifying/forcing the compilation backend in Nim for nim doc #13129 (Allow specifying/forcing the compilation backend in Nim for `nim doc)
• we now decouple backends from commands. Everything that worked before still works now, but we can now run things like nim doc --backend:js to generate js specific docs. This opens door to showing docs specific to different backends, which would be more accurate than what we have now (often wrong since it assumes c target for nim js in system.nim for example)
• --backend is then passed on to runnableExamples, which run the examples for the same backend backend by default. (--doccmd:--backend:cpp would be able to override it should user want that, see below)
• before this PR, there was no easy way to change the backend in a cmdline invocation (unless you used a separate nimscript file + setCommand); now you can simply do it by appending --backend:js to the command right before the $file argument, eg: nim c -b:cpp $file => will use cpp backend

nim r -b:js (etc)

• likewise, nim r (introduced in new cmd: nim r main [args...] to compile & run, saving binary under $nimcache/main #13382 just for c backend) now works with other backends, eg: nim r -b:js main
• nim r should now be the de facto default to run tests, regardless of backend, since it avoids polluting your sources with generated js or binaries

nim doc --doccmd:cmd

• before PR, there was no way to pass options to runnableExamples command; now there is, eg: nim doc --doccmd:'-d:foo --threads:on --lib:lib' main
  ; this fixes Allow specifying/forcing the compilation backend in Nim for nim doc #13129 (comment)

nim doc --doccmd:skip, nim rst2html --doccmd:skip

• before PR there was no way to skip runnableExamples or (in rst) test snippets ; now you simply pass --doccmd:skip; very useful when you just want to see how the generated html file looks like without waiting for all the runnableExamples to run (or snippets to compile), during development.
  eg:
• nim rst2html --outdir:doc/html doc/manual.rst => 50s
• nim rst2html --outdir:doc/html --doccmd:skip doc/manual.rst => 0.9s

--usenimcache

• refs doc2 outputs in current work dir #6583 ; now you can use --usenimcache and avoid polluting both $pwd and $projectdir
• --usenimcache was implicitly introduced in new cmd: nim r main [args...] to compile & run, saving binary under $nimcache/main #13382 just for nim r; now it's explicitly a flag decoupled from nim r and works in combination with other commands, eg: nim doc --usenimcache => will output html under nimcache
  --usenimcache by design uses whatever nimcache resolves to after each command was processed, eg:
  nim doc --nimcache:foo --usenimcache --nimcache:bar main => will output html uner bar, not foo
• it works with any command, eg: nim c --usenimcache main or nim doc --usenimcache

note: setting -d:js (etc) manually is bad, hence the need for --backend:js

Note that simply setting -d:js is naive and won't work in many cases (eg importjs etc; or may end up with both defined(c) and defined(js) etc), likewise with -d:cpp; in fact future PR should probably issue warning when user code sets -d:js (+other backends) as it'd be error prone.

tests

• tests are added for every feature

potential future directions

• make --usenimcache default for nim doc ; IMO that's the best resolution for doc2 outputs in current work dir #6583; any -o or --outdir would override --usenimcache anyway
• nim r doc --port:1234 main will save the html under nimcache (because -r implies --usenimcache), and setup (on 1st call) a livereload server (see https://chrome.google.com/webstore/detail/livereload/jnihajbhpnppcggbcgedagnkighmdlei?hl=en) so that the html opens in the same tab and auto-refreshes
• nim r -b:js --port:1234 (-d:nimbrowser) main will create a template html file that simply links against the generated js file so you can see it running in a browser, it will also use the livereload server to auto-refresh the same tab; this avoids setting up a server manually just to see how nim js code runs in a browser

[kaushalmodi]

@timotheecour Thank you! for working on this. I started looking into implementing it but didn't have time to dig further on how to add switches that affected "nim doc" and not the regular "nim".

[Araq]

It's excellent work but some tests seem to be red and it's not the usual flaky tests.

[timotheecour]

@Araq PTAL

some tests seem to be red and it's not the usual flaky tests

fixed it, was a 1 liner fix

• --doccmd:skip now allows nim rst2html --outdir:doc/html --doccmd:skip doc/manual.rst to be 50x faster
• I don't understand need for keep binary compatible in TCommands, I'm tempted to just remove the newly deadcode enum members
• --usenimcache should be default for nim doc IMO, which would fix doc2 outputs in current work dir #6583 2nd regression, but that shd be in another PR

CI failures unrelated EDIT: all green now

• misc CI flaky tests timotheecour/Nim#113 (comment) (this i saw 2 days ago from another unrelated PR so unrelated to this PR)
• misc CI flaky tests timotheecour/Nim#113 (comment) (nimx; same root cause as I described here [WIP] fix https://github.com/nitely/nim-regex/issues/60 #13868 (comment))

[alaviss]

Can we have this for nimsuggest too? :D

[timotheecour]

Can we have this for nimsuggest too? :D

IMO it's best as a separate PR, this PR is big enough already; my guess is this should be a simple followup, eg:
nimsuggest uses conf.cmd = cmdIdeTools; in future PR it can learn --backend and it should all just work

• likewise with nim check and other commands; EDIT: these should work reliably as of fix some issues with --backend #14363