support directly markdown files

[pietroppeter]

a normal markdown file (with no need to run nim code) could generate an html directly.

The change should be in nimibook.publish and we could thus support the following

let toc = newToc("Example", "book"):
  entry("Introduction (only markdown)", "index.md")  # this will create a page starting from the plan markdown
  entry("Another chapter (with nim code)", "chapter.nim")

[zetashift]

I wanted to write down my nodes for Godot and Nim and maybe publish them, since they are all mostly text with some code sprinkled in I'd like to try out this, but I'd need to have markdown support directly.

And since this looks fairly simple(dangerous words), if you want I can try to whip out a PR?

[pietroppeter]

yep, thanks, welcome to give it a go! you might want to start from the work done in #11 (should be merged soon enough). it basically mean you will need to add code in the mdOutput proc (likely will be rename mdPublish).

probably something like this could already work inside the proc (given mdfilename) but of course the devil is in the details! 😈

proc ...
  nbInit # will the nimibCustomSwitch be needed or we want to include it directly here?
  nbText mdfilename.readFile
  nbDoc.filename = mdfilename
  nbSave

[zetashift]

I currently have

proc mdPublish*(entry: Entry) =
  nbInit
  nbText entry.path.readFile
  nbDoc.filename = entry.path.extractFilename
  nbSave

this as an implementation but I'm getting the following error:

Hint:  [Link]
Hint: 120706 lines; 2.099s; 146.969MiB peakmem; Release build; proj: /home/rishi/dev/nim/nimibook/genbook.nim; out: /home/rishi/.cache/nim/genbook_r/genbook [
SuccessX]
Hint: /home/rishi/.cache/nim/genbook_r/genbook  [Exec]
Hint: used config file '/home/rishi/.choosenim/toolchains/nim-1.4.6/config/nim.cfg' [Conf]
Hint: used config file '/home/rishi/.choosenim/toolchains/nim-1.4.6/config/config.nims' [Conf]
Hint: used config file '/home/rishi/dev/nim/nimibook/config.nims' [Conf]
Hint: used config file '/home/rishi/dev/nim/nimibook/book/config.nims' [Conf]
......................................................................
[USING CUSTOM NBPOSTINIT]
..
[USING CUSTOM NBPOSTINIT]
/home/rishi/dev/nim/nimibook/book/index.nim(2, 1) template/generic instantiation of `nbInit` from here
/home/rishi/dev/nim/nimibook/src/nimibook/types.nim(26, 3) template/generic instantiation of `nbInit` from here
/home/rishi/dev/nim/nimibook/book/nbPostInit.nim(2, 1) Error: 'import' is only allowed at top level
Error: execution of an external program failed: '/home/rishi/.cache/nim/genbook_r/genbook '
stack trace: (most recent call last)
/tmp/nimblecache-3316067695/nimscriptapi_212352811.nim(187, 16)
/home/rishi/dev/nim/nimibook/nimibook.nimble(18, 11) genbookTask
/home/rishi/.choosenim/toolchains/nim-1.4.6/lib/system/nimscript.nim(280, 7) selfExec
/home/rishi/.choosenim/toolchains/nim-1.4.6/lib/system/nimscript.nim(280, 7) Error: unhandled exception: FAILED: /home/rishi/.choosenim/toolchains/nim-1.4.6/b
in/nim  r -d:release genbook.nim [OSError]
     Error: Exception raised during nimble script execution

Can nbInit be called from a proc?

[HugoGranstrom]

nbInit includes several (optional) include statements and those can't be in procs. :/ For example: https://github.com/pietroppeter/nimib/blob/dda83a58e1fb2ae53523e29905614f9bfa62a02a/src/nimib.nim#L54

[HugoGranstrom]

With @Clonkk's idea of init procs instead of include it would be possible though: pietroppeter/nimib#45 Not really sure how to workaround it for now 🤔

[zetashift]

I can just call nbInit in types.nim after the imports...not sure if this is best practices though, but hey it compiles

[pietroppeter]

the issue is not nbInit itself but nbPostInit which in the case of nimibook it has an import and import must only be at top level and not inside procs.

I think a workaround for now would be the protect the import os with a custom compile time switch in nbPostInit, push the custom switch in the proc (and pop it at the end of the proc) and add an import os to that file. It is ugly and the nbPostInit mechanism is proving to be really bad so it will be deprecated when we have a workgin alternative.

[HugoGranstrom]

the issue is not nbInit itself but nbPostInit which in the case of nimibook it has an import and import must only be at top level and not inside procs.

Ah that makes sense now, includes just copy-paste 🤦

Ugly solutions are always nice 🤣

[pietroppeter]

I can just call nbInit in types.nim after the imports...not sure if this is best practices though, but hey it compiles

well, my workaround is probably cleaner for the types file but messier to implement. the one you suggest it could mess up other things, but if it works, I am tempted to say, go with this, we need to sort the nbPostInit mess anyway and we will come back and fix this...

[pietroppeter]

mmh, although my guess is that your workaround will create problems the moment you add a second md file (it will put a global nbDoc object in global namespace of types.nim that will be modified by subsequent calls of mdPublish). no it cannot be used, you gotta try the other workaround...

[Clonkk]

The easiest solution would be to generate a Nim file with valid Nimib code from parsing the markdown file.

Then you compile, execute and delete the Nim file.

That way, markdown follow the same process as Nimib file and you don't have to deal with init for now

[pietroppeter]

thinking some more I think we probably could remove the nbPostInit mess already here, without waiting for changes to happen upstream in nimib. I will try to come up with something (or describe better the problem) later this evening.

[zetashift]

After some messing around it seems that my wrong solution also causes a recursive dependency error:

[USING CUSTOM NBPOSTINIT]
..
[USING CUSTOM NBPOSTINIT]
/home/rishi/dev/nim/nimibook/book/index.nim(2, 1) template/generic instantiati
on of `nbInit` from here
/home/rishi/dev/nim/nimibook/src/nimibook/types.nim(5, 1) template/generic ins
tantiation of `nbInit` from here
/home/rishi/dev/nim/nimibook/book/nbPostInit.nim(26, 11) Error: undeclared ide
ntifier: 'load'
This might be caused by a recursive module dependency:
/home/rishi/dev/nim/nimibook/src/nimibook.nim imports /home/rishi/dev/nim/nimi
book/src/nimibook/types.nim
/home/rishi/dev/nim/nimibook/src/nimibook/types.nim imports /home/rishi/dev/ni
m/nimibook/src/nimibook.nim
Error: execution of an external program failed: '/home/rishi/.cache/nim/genboo
k_r/genbook '
stack trace: (most recent call last)
/tmp/nimblecache-3316067695/nimscriptapi_212352811.nim(187, 16)
/home/rishi/dev/nim/nimibook/nimibook.nimble(18, 11) genbookTask
/home/rishi/.choosenim/toolchains/nim-1.4.6/lib/system/nimscript.nim(280, 7) s
elfExec
/home/rishi/.choosenim/toolchains/nim-1.4.6/lib/system/nimscript.nim(280, 7) E
rror: unhandled exception: FAILED: /home/rishi/.choosenim/toolchains/nim-1.4.6
/bin/nim  r -d:release genbook.nim [OSError]
     Error: Exception raised during nimble script execution

[pietroppeter]

ok, nimibPostInit removed with #15
hopefully CI will be green and you can rebase your work on this.

[pietroppeter]

most of the discussion on this happened on nim discord #science channel, to summarize:

• using the bookcfg rebase there were circular dependency issue when trying to use it in mdPublish
• even after solving circular dependency issue, some details about paths made it very tricky, so I went with a PR myself.

sorry, @zetashift to have you work on this and then finally commit myself!