Answer and enquiry about TMG

[amakukha]

I've just noticed this question in the wiki:

t[1-8].s: McIlroy's(?) TMG?

But didn't find the edit button. So I will just answer here.

Answer:

Yes, this is an early version of TMG for Unix by Douglas McIlroy. He later did a "new TMG" for Unix Version 2 in PDP-11 assembly. This newer implementation was included into Unix until Version 6, but then was abandoned in favor of YACC, a new compiler-compiler developed at Bell Labs. Recently, I ported this "new TMG" from PDP-11 assembly to C (here is my TMG in C repo).

Many routines in this old Unix TMG are named the same as in the later version. Particularly: kput, jget, obuild, ctest, putoct, bundle, table, find, char, string, octal. Some others are recognizable: getname (same as getnam in the later version), parsedo (same as parse), mark (same as smark), roindir (unary *, named .rv on PDP-11), etc. Data structures in t7.s are also recognizable. Though the code is very different from the "new TMG" that I worked with.

Enquiry:

Unfortunately, I didn't notice any TMG programs in this repo. It would be interesting to see, because McIlroy claimed that he wrote TMG for PDP-7 "in its own language" by bootstrapping it up.

I also didn't notice the "driving table" for TMG, which is an assembly output of TMG that would be assembled together with those t[1-8].s files to produce a new compiler. I'd expect that this driving table would be needed to make the TMG code functional.

Didn't find the driving table nor the original TMG program in the PDFs neither.

If anyone is aware of such code, please, let me know.

[sebras]

Didn't find the driving table

Can you explain a bit more about the driving table? Is that available in the later version somewhere?

nor the original TMG program in the PDFs neither.

This list mentions that the t1.s through t7.s are available in 13-120-147.pdf from the original scans. Is that what you are referring to here?

[philbudne]

I've just noticed this question in the wiki:

t[1-8].s: McIlroy's(?) TMG?

I created the wiki page, so likely my typing.
At some point McIlroy confirmed that's what the files were, but the page was never updated.

Enquiry:

Unfortunately, I didn't notice any TMG programs in this repo. It would be interesting to see, because McIlroy claimed that he wrote TMG for PDP-7 "in its own language" by bootstrapping it up.

...

I also didn't notice the "driving table" for TMG, which is an assembly output of TMG that would be assembled together with those t[1-8].s files to produce a new compiler. I'd expect that this driving table would be needed to make the TMG code functional.

Didn't find the driving table nor the original TMG program in the PDFs neither.

If anyone is aware of such code, please, let me know.

I believe everything in the PDFs has been typed in and is in git. We're dealing with a snapshot from an unknown point in time, of unknown completeness.

My recall is:

We don't have any examples of TMG input files.
We don't know if the snapshot predates TMG being re-written in TMG.
We don't know what files we might be missing.

We were exceedingly lucky that the first batch included a complete and functioning system (kernel). I wrote a replacement for the shell, and Warren replaced many utilities. Further luck meant that the second batch (including the t?.s files) included the shell and utility programs). But it seemed our luck ran out when it came to TMG and B.

From the B run-time files in the first batch of PDFs, Robert Swierczek was able to figure out what the threaded code B generated might look like, and bootstrap a B compiler.

My recall is that PDP-11 TMG parsers depend on a library of routines for "actions", but I never dug in and figured out what parts of the PDP-11 TMG implementation we had corresponding PDP-7 bits for.

It's likely that your questions are the answer for the question "What bits are we missing?", but none of us are as familiar with the PDP-11 implementation as you must be.

If we only have "runtime" bits, then perhaps creating a TMG based reconstruction bootstrapped on the PDP-11, or using your C translation could be a path to providing a stand-in for what is missing?

[amakukha]

Can you explain a bit more about the driving table?

Driving table is a generated assembly code which calls all the "actions" (as Phil calls them above). McIlroy calls them builtins.

In other words, driving table is the threaded code: it consists predominantly of call instructions to a range of the available subroutines.

Is that available in the later version somewhere?

Yes, see tmgl.s.
This code is generated by TMG from the file tmgl.t, which is the implementation of TMG in itself.
Both of these files appear to be missing in this repo.

My recall is that PDP-11 TMG parsers depend on a library of routines for "actions"

Correct. Files t1.s through t6.s appear to be implementing those "actions", file t7.s defines data structures and variables, file t8.s defines the macros. But the actual definition of TMG parser appears to be missing.

If we only have "runtime" bits, then perhaps creating a TMG based reconstruction bootstrapped on the PDP-11, or using your C translation could be a path to providing a stand-in for what is missing?

Yeah, it could be done and could be interesting. Though, personally, I would be more interested to see the original implementations of TMG and B in TMG.

[amakukha]

We don't know if the snapshot predates TMG being re-written in TMG.

Also, to be clear, according to McIlroy, he created TMG in TMG from the start. Quote from here:

[McIlroy] … I brought up a compiler writer, a compiler compiler, written in its own language. Bootstrapped it up.
[Mahoney:] Now, is that TMG or did you do that in B?
[McIlroy:] It was TMG. TMG was a begun in itself. There was no B. There was no C. There was an assembler.

And then later parser for B was implemented in TMG.

So, indeed, it's a bit unfortunate that those *.t files are missing…

Thanks for your answers anyway.

[philbudne]

@amakukha I'm wondering if you might be able to advise on any aspect of trying to reconstruct a .t file for the PDP-7 version of TMG?

How similar are the available PDP-7 actions to the ones used in this early (Multics) "tmg.t" file?
https://multicians.org/tmg.html

Have you seen anything in your explorations that might indicate what the PDP-7 version of TMG might have looked like?

One can imagine that it would have been a subset of the Multics version.

Do the available PDP-7 files give you any idea of what the structure of the PDP-7 tmg.s might have been like?

How close (or far) is the above Multics tmg.t file to the language supported by your portable C version of TMG?

[amakukha]

@philbudne , for the "reconstruction" part, I would advise to just backport "new TMG" from PDP-11 to PDP-7 assembly. That way it would be possible to compile the original PDP-11 "TMG in TMG" file. Some of the existing assembly code for PDP-7 TMG could be reused in the process.

I think, it's the most sensible thing to do, because I don't have any information about the changes introduced between PDP-7 TMG and PDP-11 TMG.

Regarding the Multics TMG, I didn't analyse it deeply. The problem with different TMG implementations is that they rely on different routines which are implemented in assembly. For example, I can see from the code that Multics TMG relied on functions INTRN, SOURCE, SKIPCOM, DUMDUM, KILL, etc., the purpose of which is unknown.

That said, all TMG implementations are similar in their basic operation and differ only about as much as different BASIC implementations differ.

[philbudne]

FYI, today got A PDP-7 TMG (if not the original) running using @amakukha TMG in C (https://github.com/amakukha/tmg)

See email at:
https://minnie.tuhs.org/pipermail/pdp7-unix/2021-September/000701.html

Code at:
https://github.com/philbudne/pdp7-unix/tree/tmg