Create a PDF file from Markdown documents (md2pdf) #11257

michelou · 2021-01-30T13:06:32Z

WIP

The pandoc subproject make it possible to create a PDF file (e.g. scala3_reference.pdf) from the Markdown documents of the Scala 3 documentation (internals, reference, usage).

Todo :

fix arguments passed to the Docker container
test (and improve) workflow file pandoc.yaml to run CI job with GitHub Actions.
improve documentation (Docker, GitHub Actions)

Related to :

SethTisue · 2021-01-30T14:37:27Z

I don't think there's anything here I can meaningfully review (I don't know Docker), but I'm delighted to see it happening.

…o scala3-pandoc

michelou · 2021-01-31T09:10:06Z

I don't think there's anything here I can meaningfully review (I don't know Docker), but I'm delighted to see it happening.

That's fair. My primary intent was to let you know things are moving forward. I've removed your name from the reviewers.

liufengyun

Very nice work 👍

liufengyun · 2021-02-02T14:42:12Z

pandoc/README.md

+***
+
+*[mics](https://github.com/michelou/)/January 2021* [**&#9650;**](#top "Back to top")
+<span id="bottom">&nbsp;</span>


The Back to top seems not very useful, maybe remove it.

While your comment is justified for people working with 20+ inch screens, I and others I guess are used to read online docs with a 5.8 inch cell phone, in which case this tiny button is very practical. I let you decide.

Fair enough, it's fine to keep it.

liufengyun · 2021-02-02T14:42:38Z

pandoc/README.md

+<table style="margin-top:-10px; max-width:650px;">
+<tr>
+<td><b>Input directories</b></td>
+<td><code>dotty/docs/docs/&lt;project&gt;/</code><br/><code>dotty/pandoc/data/</code></td>


Suggested change

<td><code>dotty/docs/docs/<project>/</code><br/><code>dotty/pandoc/data/</code></td>

<td><code>docs/docs/<project>/</code><br/><code>pandoc/data/</code></td>

liufengyun · 2021-02-02T14:43:01Z

pandoc/README.md

+</tr>
+<tr>
+<td><b>Output directory</b></td>
+<td><code>dotty/out/pandoc/</code></td>


Suggested change

<td><code>dotty/out/pandoc/</code></td>

<td><code>out/pandoc/</code></td>

liufengyun · 2021-02-02T14:48:09Z

pandoc/data/internals.md

+---
+layout: doc-page
+title: "Scala 3 Internals"
+titlegraphic: ../out/pandoc/images/scala-spiral.png


May it refer to pandoc/data/images/scala-spiral.png directly?

md2pdf starts with a preprocessing phase which transforms :

cross-references found in the Markdown files (e.g. reference) to named anchors

references to image files stored in pandoc/data/images/ (e.g. scala-spiral.png) to relative paths.

In short all references not starting with https:// are transformed before calling the pandoc command line tool. In the output directory out/pandoc/ both directories images/ and src_managed/ are expected to be at the same level :

out/pandoc/images/

out/pandoc/src_managed/

out/pandoc/scala3_reference.pdf

liufengyun · 2021-02-02T14:48:52Z

pandoc/data/internals.md

+layout: doc-page
+title: "Scala 3 Internals"
+titlegraphic: ../out/pandoc/images/scala-spiral.png
+author: Martin Odersky and al.


Suggested change

author: Martin Odersky and al.

author: Martin Odersky et al.

liufengyun · 2021-02-02T15:26:05Z

pandoc/data/reference.md

+layout: doc-page
+title: "Scala 3 Reference"
+titlegraphic: ../out/pandoc/images/scala-spiral.png
+author: Martin Odersky and al.


Suggested change

author: Martin Odersky and al.

author: Martin Odersky et al.

liufengyun · 2021-02-02T15:27:37Z

pandoc/docs/PROJECT.md

+
+This project is located at the root of the [`lampepfl/dotty`][github_dotty] repository, at the same level as the [`Scala3doc`][github_scala3doc] project.
+
+## <span id="structure">Directory structure</span>


It would be nice to merge this with pandoc/README, so that the reader has an overview on first read.

liufengyun · 2021-02-02T15:29:06Z

pandoc/docs/TEXLIVE.md

@@ -0,0 +1,168 @@
+# <span id="top">More about TeX Live</span> <span style="size:25%;"><a href="../README.md" title="Back to README">↩</a></span>
+<!-- created by mics (https://github.com/michelou/) on December 2020 -->


Is this document necessary given that we have the docker image?

liufengyun · 2021-02-02T15:31:22Z

pandoc/extras/build_image.bat

@@ -0,0 +1,218 @@
+@rem ## Created by mics (https://github.com/michelou/) on December 2020.


If image can be built by Linux/MacOS and published, then we don't need the script to build the image on Windows.

My wish here is to offer different ways to developers to generate a PDF file from the Markdown files before they publish their changes (more details per email).

liufengyun · 2021-02-02T15:40:11Z

pandoc/README.md

+
+We provide two commands for MacOS/Ubuntu as well as their equivalents for MS Windows :
+- for *interactive users* :<br/>[`build {<option>|<subcommand>}`](./build) (resp. [`build.bat`](./build.bat) on MS Windows)<br/><span style="font-size:80%;">(option `-help` displays the available options/subcommands).</span>
+- for *automated tasks* (primarily, but not exclusively) :<br/>[`md2pdf [<project>]`](./md2pdf) (resp. [`md2pdf.bat`](./md2pdf.bat) on MS Windows)<br/><span style="font-size:80%;">(default value for `<project>` : `reference`).</span>


I've wondering, instead of having two commands build and md2pdf, can we have the same command for both local machines and cloud CI, assuming that both will use a published docker image?

Your suggestion is right.

The existence of both commands build and md2pdf stems in part from my own background as a long-standing developer. Let me try to advocate the case (for fun) :

as a preamble the existance of both scripts is not directly linked to Docker.

build is a user-friendly wrapper script; it calls md2pdf but it also

provides some options (e.g. -verbose and -debug) and some subcommands (e.g. clean and view on MS Windows); I typically run command build -timer -verbose clean view to generate file scala3_reference.pdf on my Win10 laptop.

provides conditional compilation; the output file (e.g. scala3_reference.pdf) is generated depending on input file timestamps.

md2pdf is the minimal script for running pandoc with the appropriate settings.

In the context of the Dotty project, we can indeed have only one shell script : md2pdf (and may be md2pdf.bat).

NB. I could even omit several ~~type checks~~ environment checks inmd2pdf assuming users can help themselves in case of errors.

- Register class type parameters from outer contexts in typedDefdef. - Remove a condition to support gadt approximation for class type parameters. - Add test cases in tests/pos/class-gadt. - Ignore test cases tests/pos/i4345.scala and i5735.scala (issue scala#11220 and scala#11221)

Co-authored-by: Allan Renucci <allanrenucci@users.noreply.github.com>

Make sure that something is a language import if and only if it looks like one. i.e. is of one of the forms import language.xyz import scala.language.xyz import _root_.scala.language.xyz

Avoid a notation of "owner" when checking for a language feature. Instead, allow qualified names for language features, e.g. `experimental.dependent".

The bit is set if - -Yexplicit-nulls is set - we are not in an `import.unsafeNulls` scope

bishabosha · 2021-07-08T19:13:55Z

Hi @michelou do you have any time to continue working on this?

anatoliykmetyuk · 2021-07-27T14:25:19Z

@michelou what's the status on this one? Are you still working on it? Do you need someone to review it?

michelou · 2021-07-28T08:58:44Z

@michelou what's the status on this one? Are you still working on it? Do you need someone to review it?

Short status is as follows :

I regularly update the locally generated PDF file scala3_reference.pdf (last update: July 23, 2021) on my GitHub repository dotty-examples.
I made no progress with the generation of the PDF file in a remote Docker container because I failed to upload the Docker image (NB. my internet connection is very limited, i.e. download ~10 Mbps / upload ~1 Mbps).

In resume I need a co-worker ready to help me with step 2.

griggt · 2021-07-28T22:33:59Z

@michelou I think I can help with getting the PDF generation working in Docker / CI. I have a branch from quite a while ago where I had this working....once I find it are you ok with me pushing the changes to this PR?

michelou · 2021-12-28T18:19:53Z

@anatoliykmetyuk @liufengyun @griggt Today I updated my locally generated PDF file scala3_reference.pdf .

Next steps :

open one (or two) PR to integrate my changes to the Markdown files.
give another try with the PDF generation built upon a Docker image.

michelou · 2022-04-06T18:53:35Z

Here is an actualized version of the locally generated PDF file scala3_reference.pdf.

@griggt I finally have to bring myself to abandon my attempts with Docker. In short your help is welcome !
As explained in my previous messages all my work is contained in directory DOTTY_PROJECT_ROOT/pandoc/. If you agree we can discuss the next steps offline (aka. via email).

griggt · 2022-04-06T19:00:03Z

@michelou happy to help, but it may be days or weeks before I have a chance to look at this again.

ckipp01 · 2023-05-10T08:24:31Z

Hello! It's been about a year of inactivity here, so I just wanted to check if this should be closed, if someone plans on returning to this, etc?

michelou · 2023-05-14T21:19:35Z

I just updated my GitHub project to (locally) generate the actualized PDF file scala3_reference.pdf.
Otherwise, no progress on the Docker-side.

I plan to look at the recently added pages and submit a few additions/corrections.

add pandoc subproject (md2pdf)

067a4d7

michelou added area:documentation in progress labels Jan 30, 2021

michelou requested review from SethTisue and liufengyun January 30, 2021 13:06

michelou mentioned this pull request Jan 30, 2021

Feature request : add xz-utils package to image pandoc/ubuntu-latex jgm/pandoc#7056

Closed

michelou added 2 commits January 30, 2021 19:13

Merge branch 'scala3-pandoc' of https://github.com/michelou/dotty int…

e7cfae3

…o scala3-pandoc

updated shell script 'build'

3a442e2

michelou removed the request for review from SethTisue January 31, 2021 09:04

liufengyun self-assigned this Feb 2, 2021

liufengyun reviewed Feb 2, 2021

View reviewed changes

michelou and others added 17 commits February 2, 2021 23:10

Merge remote-tracking branch 'upstream/master' into scala3-pandoc

6dfa925

Merge remote-tracking branch 'upstream/master' into scala3-pandoc

8e634ac

addressed subset of review comments

2277b00

Add regression tests for scala#9283 scala#9284 scala#9288 scala#9293

8a0660b

Add regression test for scala#2941

af60155

Update CI status badge URL to reflect workflow name change

1388371

Update compiler/src/dotty/tools/dotc/typer/Typer.scala

7999636

Co-authored-by: Allan Renucci <allanrenucci@users.noreply.github.com>

Fix CI failure in [test_windows_full]

d899873

Disable some scala3doc tests in Windows CI

b55aae5

pages for enums

0b7a4f6

Make sure language imports are detectable syntactically

7620508

Make sure that something is a language import if and only if it looks like one. i.e. is of one of the forms import language.xyz import scala.language.xyz import _root_.scala.language.xyz

Simplifications

63430b9

Avoid a notation of "owner" when checking for a language feature. Instead, allow qualified names for language features, e.g. `experimental.dependent".

Add mode bit to track whether we are in safe nulls mode

2e25572

The bit is set if - -Yexplicit-nulls is set - we are not in an `import.unsafeNulls` scope

Refine initial SafeNulls setting

00e087b

Fix deprecation for java.lang.Deprecated

ce9fe32

Apply requested changes

97b9b8a

Use Mill 0.9.6-16-a5da34

eda52b6

bishabosha unassigned liufengyun Jul 8, 2021

updated external links to docs.scala-lang.org

e8797ff

anatoliykmetyuk assigned michelou Jul 27, 2021

anatoliykmetyuk added stat:wip and removed in progress labels Jul 27, 2021

This was referenced Dec 30, 2021

Online Scala 3 Reference : broken links #14186

Open

more updates to Markdown files #14191

Merged

michelou added 11 commits January 2, 2022 19:01

updated files pandoc.yaml and reference.md

6efbda4

fixed branch

60af122

prints version

b7f64a9

chmod +x

67e85ae

prints version (cnt'd)

0b53581

added printenv job

83e7ad9

updated .md files, updated pandoc.yaml

4cd7159

bash -> sh

3e7db05

updated shell scripts (bash->sh, fonts)

5de9c72

install additional TeX fonts

d4e7651

updated file reference.md

959b112

griggt self-assigned this Apr 6, 2022

ckipp01 marked this pull request as draft February 6, 2023 13:22

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Create a PDF file from Markdown documents (md2pdf) #11257

Create a PDF file from Markdown documents (md2pdf) #11257

michelou commented Jan 30, 2021 •

edited

Loading

SethTisue commented Jan 30, 2021

michelou commented Jan 31, 2021

liufengyun left a comment

liufengyun Feb 2, 2021

michelou Feb 2, 2021

liufengyun Feb 3, 2021

liufengyun Feb 2, 2021

michelou Feb 2, 2021

liufengyun Feb 2, 2021

michelou Feb 2, 2021

liufengyun Feb 2, 2021

michelou Feb 2, 2021 •

edited

Loading

liufengyun Feb 2, 2021

michelou Feb 2, 2021

liufengyun Feb 2, 2021

michelou Feb 2, 2021

liufengyun Feb 2, 2021

michelou Feb 2, 2021

liufengyun Feb 2, 2021

liufengyun Feb 2, 2021

michelou Feb 4, 2021

liufengyun Feb 2, 2021

michelou Feb 3, 2021 •

edited

Loading

bishabosha commented Jul 8, 2021

anatoliykmetyuk commented Jul 27, 2021

michelou commented Jul 28, 2021

griggt commented Jul 28, 2021

michelou commented Dec 28, 2021

michelou commented Apr 6, 2022

griggt commented Apr 6, 2022

ckipp01 commented May 10, 2023

michelou commented May 14, 2023

	<td><code>dotty/docs/docs/<project>/</code><br/><code>dotty/pandoc/data/</code></td>
	<td><code>docs/docs/<project>/</code><br/><code>pandoc/data/</code></td>

	<td><code>dotty/out/pandoc/</code></td>
	<td><code>out/pandoc/</code></td>


		This project is located at the root of the [`lampepfl/dotty`][github_dotty] repository, at the same level as the [`Scala3doc`][github_scala3doc] project.

		## <span id="structure">Directory structure</span>

		@@ -0,0 +1,168 @@
		# <span id="top">More about TeX Live</span> <span style="size:25%;"><a href="../README.md" title="Back to README">↩</a></span>
		<!-- created by mics (https://github.com/michelou/) on December 2020 -->

		@@ -0,0 +1,218 @@
		@rem ## Created by mics (https://github.com/michelou/) on December 2020.

Create a PDF file from Markdown documents (md2pdf) #11257

Are you sure you want to change the base?

Create a PDF file from Markdown documents (md2pdf) #11257

Conversation

michelou commented Jan 30, 2021 • edited Loading

SethTisue commented Jan 30, 2021

michelou commented Jan 31, 2021

liufengyun left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

michelou Feb 2, 2021 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

michelou Feb 3, 2021 • edited Loading

Choose a reason for hiding this comment

bishabosha commented Jul 8, 2021

anatoliykmetyuk commented Jul 27, 2021

michelou commented Jul 28, 2021

griggt commented Jul 28, 2021

michelou commented Dec 28, 2021

michelou commented Apr 6, 2022

griggt commented Apr 6, 2022

ckipp01 commented May 10, 2023

michelou commented May 14, 2023

michelou commented Jan 30, 2021 •

edited

Loading

michelou Feb 2, 2021 •

edited

Loading

michelou Feb 3, 2021 •

edited

Loading