pod2man uses CW font, no longer enabled in groff by default

[puck]

In groff v1.23 which was released this month, the CW font is disabled by default as we discovered in Debian: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1040975 . It appears to be this commit https://git.savannah.gnu.org/cgit/groff.git/commit/?id=15f8188656ef0ebed797eb5981b012b590fc77ad which disabled CW and this commit shows how to re-enable it: https://git.savannah.gnu.org/cgit/groff.git/commit/?id=5278dee79a180a453a73d54ec4cc7e164dafcdd5 .

Running pod2man with groff v1.23 I now get these warnings:

puck@dirk:~$ pod2man /usr/share/perl/5.36.0/version.pod | nroff -man | grep CW          
troff:<standard input>:145: warning: cannot select font 'CW'
troff:<standard input>:182: warning: cannot select font 'CW'
troff:<standard input>:191: warning: cannot select font 'CW'
troff:<standard input>:205: warning: cannot select font 'CW'
troff:<standard input>:207: warning: cannot select font 'CW'
...

While we have resolved this in Debian for now by re-enabling CW, this may hit other operating systems.

It seems that pod2man should use a different font than CW, perhaps R?

[jkeenan]

In groff v1.23 which was released this month, the CW font is disabled by default as we discovered in Debian: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1040975 . It appears to be this commit https://git.savannah.gnu.org/cgit/groff.git/commit/?id=15f8188656ef0ebed797eb5981b012b590fc77ad which disabled CW and this commit shows how to re-enable it: https://git.savannah.gnu.org/cgit/groff.git/commit/?id=5278dee79a180a453a73d54ec4cc7e164dafcdd5 .

Running pod2man with groff v1.23 I now get these warnings:

puck@dirk:~$ pod2man /usr/share/perl/5.36.0/version.pod | nroff -man | grep CW          
troff:<standard input>:145: warning: cannot select font 'CW'
troff:<standard input>:182: warning: cannot select font 'CW'
troff:<standard input>:191: warning: cannot select font 'CW'
troff:<standard input>:205: warning: cannot select font 'CW'
troff:<standard input>:207: warning: cannot select font 'CW'
...

What output put were you getting with this command in earlier versions of groff?

I ask because when I run your command with my existing, installed versions of groff and perl, I get no output. For example:

$ uname -a
Linux bullseye 5.10.0-18-amd64 #1 SMP Debian 5.10.140-1 (2022-09-02) x86_64 GNU/Linux

$ groff --version | head -n 1
GNU groff version 1.22.4

$ perl -V:config_args
config_args='-de -Dprefix=/home/jkeenan/perl5/perlbrew/perls/perl-5.38.0 -Aeval:scriptdir=/home/jkeenan/perl5/perlbrew/perls/perl-5.38.0/bin';

$ pod2man `perldoc -l version` | nroff -man | grep CW 
$ 

[puck]

That's correct, previous versions had the CW font enabled, and would alias it to a font, so no warnings were issued about CW. Although the commentary is that this didn't always work very well.

Version 1.23 of groff changed this behaviour.

[puck]

My sample command was just to highlight the warnings being issued. The man page still renders as plain text if you drop the grep. I haven't tried viewing it as a PDF/DVI/PS/whatever, but I'd imagine that the text that is supposed to be rendered with the CW font will now continue to use the previous font instead.

[Leont]

It seems that pod2man should use a different font than CW, perhaps R?

Absolutely not. Code/verbatim sections must be constant width, making them proportional (R=Roman) is not helpful at all.

If I understand this correctly the GNU people want you to explicitly select a specific font, instead of the generic constant width one. This is inherently unportable and I have no idea why the GNU people want to be this user-hostile. Why break something that has worked literally 50 years? FFS.

[puck]

With my comment about R, I was merely repeating what had been suggested elsewhere.

Searching for "CW" in the groff commits, it appears they've been working on eradicating CW for a number of years. For example: https://lists.gnu.org/archive/html/groff-commit/2020-07/msg00014.html

[Leont]

Searching for "CW" in the groff commits, it appears they've been working on eradicating CW for a number of years. For example: https://lists.gnu.org/archive/html/groff-commit/2020-07/msg00014.html

That sounds like we should probably be using CR instead of CW (the case that uses R first sets the font family to C so then it's equivalent to CR)

[g-branden-robinson]

It seems that pod2man should use a different font than CW, perhaps R?

Absolutely not. Code/verbatim sections must be constant width, making them proportional (R=Roman) is not helpful at all.

Trying to change font families in a terminal (note use of nroff above) is futile. It never works.

If I understand this correctly the GNU people want you to explicitly select a specific font, instead of the generic constant width one. This is inherently unportable and I have no idea why the GNU people want to be this user-hostile. Why break something that has worked literally 50 years? FFS.

Wow, you're getting pretty carried away here. I develop groff and I committed this change, so I'm the "GNU person" you are excoriating here.

And I made the change because "CW" is not a portable font name. You won't find it anywhere in the Version 7 Unix (1979) man pages: that's when and where the man macro package was born. (Man pages date back to Version 1 Unix--or farther than that, even, to the Multics documentation that inspired them--but used a different set of macros that few Unix historians seem to dignify with the term "package".) I invite you to peruse the archives of the Unix Heritage Society (TUHS) and check my claim.

CW as a font name started to come into use in the 1980s. The Unix System III manual documents a cw command as "a preprocessor for troff(1) input files that contain text to be typeset in the constant-width (CW) font" (boldface emphasis added). You can find this (and the related checkcw program) at the TUHS archive as well.

But before one leaps up to proclaim this precedent as binding on everyone forever more (too late), one should also look at what else was going on in the 1980s. Namely, BSD Unix and Adobe PostScript--which supported neither this approach, nor these tools, nor the restriction of a single monospaced typeface. The CW *roff font name was not portable to BSD Unix.

Meanwhile, PostScript popularized (apparently to the point of market dominance in digital fonts for Adobe's partner foundries) the practice of arranging text fonts into families of four styles each: roman (upright), italic (or oblique), bold, and bold-italic (or -oblique). The standard families were of serif, sans serif, and monospaced (serif) forms, better known by the names given to them by Adobe's partner foundries: Times, Helvetica, and Courier, respectively.

AT&T troff, re-written for device-independence by Kernighan circa 1980 (and documented in Bell Labs CSTR #97) was expensive commercial software throughout its developmental life, and BSD Unix sites largely ignored it, often in favor of vtroff, which took the C/A/T-specific output of Version 7 Unix troff ("Ossanna troff") and could be combined with the then well-known Hershey vector fonts to drive Versatec and Benson-Varian plotters.

AT&T's best-known effort to extract licensing revenue from troff was probably the "Documenter's Workbench" (DWB) software suite. Eventually (in the 2000s) this code base was released for public perusal. In it, we can see how AT&T adapted to the wider variety of fonts available in PostScript. Recalling the Cartesian product of family and style established above, we find the following names.

R, I, B, BI (Times)
H, HI, HB, HX (Helvetica)
CW, CI, CB, CX (Courier)

Not perfectly orthogonal. Easy to forget the exceptions to the rough naming rules.

(By the time of DWB 3.3, at least, several other families were available, but I will not discuss them here.)

When James Clark developed groff in the period 1989-1991, he settled on the following alternative naming scheme.

TR, TI, TB, TBI (Times)
HR, HI, HB, HBI (Helvetica)
CR, CI, CB, CBI (Courier)

(AT&T troff did not use identifiers longer than 2 characters in any circumstance. [If there's an exception, I can't think of it right now.])

Further, by adding features to the formatter itself (troff), Clark could manage font families and styles separately. The GNU troff manual documents its approach to font selection.

The reader may be well past the point of asking, "so what?", and up to a point, I can sympathize. This stuff matters because there is a worse problem.

On top of the font names not being portable, there is also no portable way to ask troff whether a font is available. You request a font by name and must then fire blindly.[^1] GNU troff does support such inquiries, and Heirloom Doctools (originally based on the aforementioned DWB 3.3) cloned the feature. But it will not help you on Solaris 10 troff or other descendants of Unix System V troff. (I believe HP-UX still has one.)

It gets worse. troff formatters keep track of the previously selected font, so you can do quick alternations like \fIthis\fP (where we switch to italics for "this" and then back to the "previous" font). But what should happen if the font you switched to can't be selected? Does the identity of the "previous" font change or not? Good luck; Solaris troff handles this differently from everyone else.

Now consider whether our man document can predict what state it will be in after selecting font CW. Or CR. Or anything that isn't R, I, or B. (Strictly, even BI is a gamble.)

The bottom line is that it is not portable to set portions of man pages in a monospaced font.

So, re-evaluating your highly confident assertions:

Absolutely not. Code/verbatim sections must be constant width,

The original implementation of man had no provision for "code/verbatim sections", nor for selection of a constant-width font. See if you can find mention of either in Version 7 Unix's man(7) document.

making them proportional (R=Roman) is not helpful at all.

Nothing is proportional on a terminal emulator. If you read typeset man pages, I applaud you, but the fact remains that there is no means of obtaining monospaced and proportional fonts in the same man(7) document with universal portability to man implementations.

... I have no idea why the GNU people want to be this user-hostile.

I'm so user-hostile I wrote some code for Colin Watson to include in his man-db man pages to help them render on the broad menagerie of systems his project supports.

Why break something that has worked literally 50 years? FFS.

Your frustration would be better directed at yourself for going off without having a command of the facts first.

For what it's worth, I've worked fruitfully with podlator/pod2man maintainer Russ Allbery before. You'll understand if I struggle to hold similar expectations of you.

FFS, as it were.

[^1] You could create a troff register with the nr request, store the value of the current font mounting position in it, attempt a font change, and then compare the new current font mounting position to the stored value. Doing this for every font change would be so horribly tedious that it is no wonder that man(7) authors don't bother. And no one should expect them to.

[g-branden-robinson]

With my comment about R, I was merely repeating what had been suggested elsewhere.

Searching for "CW" in the groff commits, it appears they've been working on eradicating CW for a number of years. For example: https://lists.gnu.org/archive/html/groff-commit/2020-07/msg00014.html

I wouldn't quite say it's a campaign of "eradication", but it is becoming harder to support. The only System V-descended proprietary Unix we (groff developers) were able to directly evaluate for the 1.23.0 release was Solaris. Solaris 11 switched to groff. Solaris 10 was going to be end-of-lifed next year (early 2024) but has received a stay of execution (to 2025). Even with that vote of...confidence... from Oracle, I kind of get the feeling they won't be patching their System V troff for any reason.

The "CW" font name seems to be mostly be cruft in 2023, and there are ways people (or distributors!) can patch their systems to support it in groff if they really need to somehow coexist with a historical troff implementation, or render historical troff documents that require the name and can't be updated.

For instance,

.if t ftr CW CR
.if n ftr CW R

in groff's troffrc file, for general use, or its man.local or mdoc.local files if one wants to restrict this to man pages, should work just fine. The latter (if in nroff mode--that is, formatting for a terminal), is the solution suggested by groff 1.23.0's stock man.local file, and adopted by Debian in its groff 1.23.0-2 package release.

Possibly this is an item I'll need to add to groff's PROBLEMS file, or maybe the underlying issue is that man page authors don't have a good grasp of the font system. man(7) authorship seems to be heavily freighted with cargo-cult practices. I've tried very hard to make the groff_man(7) page a solution to this problem; time will tell if my efforts make a dent.

[puck]

Hi Branden,

First off, thank you for the detailed background on the history of CW, and the reasoning behind the change in groff. I had gathered only fragments of this from my investigation into the disabling by default of CW.

I wouldn't quite say it's a campaign of "eradication", but it is becoming harder to support.

I was not meaning to cause offence to anyone with my use of "eradication", I was merely trying to point out (perhaps poorly) that this has been an on-going work within groff for a number of years. I would have liked to have been able to point to a bug/issue that discussed it but couldn't find one in my, admittedly brief, searching to justify why pod2man should change.

I agree that distributions can re-enable CW, and I'm the one who suggested that in the Debian bug issue for the groff-base package linked to after reading that in the sample config file. ;) I also agree with Colin Watson that Debian should not carry this change long term (nor should other distros), and your background lesson above gives me good reason to more strongly agree with Colin. I see this as a short term fix within Debian while our man pages are updated. Perhaps Debian was the litmus test here since Colin uploaded groff-base 1.23.0-1 within a few weeks of the release of groff 1.23 and I just happened to build a Perl package within hours of groff-base entering Debian unstable.

Possibly this is an item I'll need to add groff's PROBLEMS file [...]

I was a little surprised to find no mention of disabling CW in the release notes, or even really in the ChangeLog. It was only when I grep'ed the installed files that I found information about the change to CW.

However, the packages that use CW should also be changed, hence why I created this bug report.

It seems to me that if the desire is to have pod2man continue to use monospaced fonts when available, then the same logic as included in https://gitlab.com/man-db/man-db/-/commit/bbf7701c4f8269090a12791f3c9bde80d45c8765 is the approach to take within the documents that pod2man generates.

Kind regards, and thank you for continuing to maintain and improve a core tool for so many of us.

PS. I used to have physical manual pages that came with an ICL Unix system, but sadly the manuals were destroyed during a flood causing by a dodgy washing machine.

[g-branden-robinson]

Hi Branden,

First off, thank you for the detailed background on the history of CW, and the reasoning behind the change in groff. I had gathered only fragments of this from my investigation into the disabling by default of CW.

Hi Andrew, and thanks! It is not an easy subject to master. Since I jumped into groff development in 2017 I've found myself having to write more documentation than I ever expected, to gather historical information, correct myths and misconceptions, fix errata (verily, even in storied scripture like CSTR #54), and construct a (one hopes) coherent narrative from which one can learn the software suite, its history, and the reasons behind some of its more bizarre features.

I wouldn't quite say it's a campaign of "eradication", but it is becoming harder to support.

I was not meaning to cause offence to anyone with my use of "eradication", I was merely trying to point out (perhaps poorly) that this has been an on-going work within groff for a number of years.

I took no offense from anything you've written. I was irked by the "Absolutely not/user-hostile/GNU" comment posted by another contributor, a remark that seems to have come directly from a Dunning-Kruger model armchair.

I would have liked to have been able to point to a bug/issue that discussed it but couldn't find one in my, admittedly brief, searching to justify why pod2man should change.

Russ Allbery discussed the issue with the groff community late last year (November, December). Generation of portable man(7) is a challenging problem with many constraints, and Perl has to be portable to as many places as groff would like to be, maybe more.

A point that I don't think is noted above is that this diagnostic message, warning: cannot select font 'CW' really is just a warning. Nothing different is happening when groff renders the document to a terminal, where most people read man pages. (Somewhat to my disappointment, as I've spent much effort improving groff man(7)'s behavior when typesetting.) Further, many people won't see the diagnostic anyway because, IIRC, man-db man(1) throws away the standard error output stream from groff by default.

Where this change, like others in groff 1.23.0, seems to be causing consternation is in automated test harnesses that are extremely, perhaps excessively, sensitive to changes in diagnostic output. When that happens the solution is not to panic but to research the issue, try to understand it, and then decide if it can be ignored. If you design an automated test for extreme sensitivity, expect it to squawk for benign reasons from time to time. A similar issue arose in Debian's continuous integration infrastructure, because a man page had a table entry that was one character cell too wide (overrunning the line length) and I modified the English in the diagnostic message for groff 1.23.0. I reported it, the package maintainer took it even more seriously than I did (which is saying quite a bit, since the issue blocked the progress of Debian's groff 1.23.0 packages into its "testing" suite), and with my explanation and recommended fix was able to swiftly resolve the issue.

That, IMO, is a model for how these sorts of issues should be handled.

I agree that distributions can re-enable CW, and I'm the one who suggested that in the Debian bug issue for the groff-base package linked to after reading that in the sample config file. ;)

Thank you! It renews my hope when people read the bits of assistance I try to prepare for them. :D

I also agree with Colin Watson that Debian should not carry this change long term (nor should other distros),

I'm okay if you adopt more optimism than I do on this front. ;-)

and your background lesson above gives me good reason to more strongly agree with Colin. I see this as a short term fix within Debian while our man pages are updated. Perhaps Debian was the litmus test here since Colin uploaded groff-base 1.23.0-1 within a few weeks of the release of groff 1.23 and I just happened to build a Perl package within hours of groff-base entering Debian unstable.

I was impressed that several distributors got it packaged in the interval between its tagging in Git and our maintainer's release announcement. On the downside, a change many distributors (including Arch and several of its derivatives) made to their man.local and mdoc.local files has caused measurable grief--a change they'd have known to take out if they had read the NEWS file in the source distribution.

Possibly this is an item I'll need to add groff's PROBLEMS file [...]

I was a little surprised to find no mention of disabling CW in the release notes, or even really in the ChangeLog. It was only when I grep'ed the installed files that I found information about the change to CW.

It has been really hard for me to gauge its breadth of use. I don't know of a place that will (1) give me grep-like access to all man pages in all Unix distributions everywhere, or even close to that, and also (2) categorize man pages by the tool that generates them--and many man pages are generated documents, as with pod2man. Readers of this ticket understand that the correct place to change generated man(7) documents in response to the immediate issue is in the tool that generates them. But a lot of users don't understand this, and they can't be blamed. One usually needs experience under one's belt to appreciate that fact.

However, the packages that use CW should also be changed, hence why I created this bug report.

It seems to me that if the desire is to have pod2man continue to use monospaced fonts when available, then the same logic as included in https://gitlab.com/man-db/man-db/-/commit/bbf7701c4f8269090a12791f3c9bde80d45c8765 is the approach to take within the documents that pod2man generates.

I hope Russ will continue to regard me (and the groff mailing list) as a resource for crafting something adequate, if he decides it's worth the trouble.

Kind regards, and thank you for continuing to maintain and improve a core tool for so many of us.

I appreciate your feedback. So far, the world hasn't blown up--people are simply upset by diagnostic messages. Imagine what they would say if they also used MANROFFOPT=-rCHECKSTYLE=4 and man --warnings. ;-)

PS. I used to have physical manual pages that came with an ICL Unix system, but sadly the manuals were destroyed during a flood causing by a dodgy washing machine.

Argh. I recently suffered the loss of a handful of books to water damage myself. It's painful.

The manual was intended to be typeset; some detail is sacrificed on terminals. (man(1), Unix Time-Sharing System Programmer's Manual, Eighth Edition, February 1985)

Regards,
Branden

[g-branden-robinson]

I was a little surprised to find no mention of disabling CW in the release notes, or even really in the ChangeLog. It was only when I grep'ed the installed files that I found information about the change to CW.

Hi Andrew,

I neglected to reply clearly to this point. The reason there was nothing in the release notes (the NEWS file) about this is because there was no change to formatting operations. What happened is that groff 1.23.0 produces diagnostics in more cases when it cannot honor input requests. Here are a couple of relevant commits (1, 2). We're continuing to improve our diagnostic story post-1.23.0 (3).

When I said in my previous comment, "[a] point that I don't think is noted above is that this diagnostic message, warning: cannot select font 'CW' really is just a warning. Nothing different is happening when groff renders the document to a terminal, where most people read man pages", that is what I was getting at.

I hope I've managed to be less opaque.

Regards,
Branden

[g-branden-robinson]

To clarify my comment of 27 July, here is the exact commit responsible for the warning groff now produces that is the topic of this issue.

https://git.savannah.gnu.org/cgit/groff.git/commit/?id=1986da1d4bb11dc0421e004b153729b3d2a2a3ca

[puck]

Hi Branden,

Thank you, I think your explanations are very clear.

I'm not a dev on the perl project, so up to someone else to determine what to do in pod2man.

Another option to resolve this in Debian is, perhaps we raise a bug against lintian since I think you're right that lintian is perhaps a bit twitchy.

Cheers,
Andrew

[g-branden-robinson]

Hi Andrew,

Thank you, I think your explanations are very clear.

I appreciate being told that--it is a major concern of mine, since I consider myself something of a technical writer.

Another option to resolve this in Debian is, perhaps we raise a bug against lintian since I think you're right that lintian is perhaps a bit twitchy.

That's been proposed in a Debian ticket, and is proceeding as far as I know (or at least the intent is formed and no one's been talked out of it).

Regards,
Branden

[puck]

While that Debian ticket was a very interesting (and entertaining) read, it is related to the \- change in behaviour rather than the CW font.

I've checked the bugs for both lintian and groff-base and see no relevant bugs.

[g-branden-robinson]

While that Debian ticket was a very interesting (and entertaining) read, it is related to the \- change in behaviour rather than the CW font.

Whoops. I've been responding to a few different observations about groff 1.23.0 behavior, and managed to get my wires (and URLs) crossed here.

I've checked the bugs for both lintian and groff-base and see no relevant bugs.

I think a lintian ticket was kicked around for a while as an idea but no one actually filed one. So I did, as Debian #1051357.

[jkeenan]

There was considerable informed discussion in this ticket, but AFAICT the only change in code suggested was a change in the default fixed-width font for pod2man from CW to ... something else. In any event, since pod2man comes out of the podlators distribution, which is maintained upstream on CPAN (led by Russ Albery), the place where any change will have to be submitted first is in that distribution's pull request tracker, https://github.com/rra/podlators/pulls.

I will take this ticket for the purpose of closing it within 7 days unless there is serious objection.

[g-branden-robinson]

Hi @jkeenan,

AFAICT the only change in code suggested was a change in the default fixed-width font for pod2man from CW to ... something else.

Yes. I've been piloting an alternative in Thomas Dickey's ncurses package, which, like Perl, needs to be portable to some of the world's crustiest Unix systems.

So far, this definition of a CW string seems to do the trick; I've confirmed that it renders fine with Documenter's Workbench nroff and troff, which I use as a proxy for Unix System V troff since it is closely related to them, and with Heirloom Doctools troff/nroff, mandoc, and of course groff.

In any event, since pod2man comes out of the podlators distribution, which is maintained upstream on CPAN (led by Russ Albery), the place where any change will have to be submitted first is in that distribution's pull request tracker, https://github.com/rra/podlators/pulls.

Sounds reasonable to me.