why this hasn't seen activity in like five years

[Ravenslofty]

For ABI reasons, linking to PS2SDK from Rust isn't a feasible approach.

So, five years ago, I hatched a plan: if the data in the official manuals could be encoded in System View Definition then a tool like svd2rust could be used to generate a set of register bindings and remove the misery of manually abstracting over all the things in the PS2. It seemed like a good idea at the time.

Unfortunately, I majorly underestimated the effort it would take to encode data into SVD; the manuals are PDF format, and are too irregular to attempt to parse directly - you would be writing almost as many exceptions as you would be writing parsing rules.

The only way I've found that reliably works is to write the SVD by hand, referencing the manuals. Understandably, pretty much everybody who comes across this project and asks how they can help gets cold feet after I explain this to them.

I don't really know how to resolve this problem, and that's why this project hasn't seen activity in five years.

I don't know if anybody will come across this issue and offer assistance in doing mind-numbing work, but perhaps this can offer an explanation for why things are stuck.

[zachary-cauchi]

Hi @Ravenslofty. I can't say I have much experience doing work of this calibre, but I'd like to at least try and help. Do you have a branch you've been using to work on this or a place to catalogue progress on the file?
Also, do you have a general procedure you've been following that may serve as a useful 'getting started' point to help?

[Ravenslofty]

No, the only branch is master, and the progress is the file itself; it's laid out pretty much in order of the EE User's Manual: if you look for a register name in the SVD and it's not there, then it's probably not implemented.

I'd be happy to give a rough guide to how to write the SVD, but it's quite late in the day, so I'll probably have to do it tomorrow (if I remember).

[zachary-cauchi]

Fair enough, makes sense. If it won't be a problem, that would be great, thanks! I'll read up about the file format in the meantime.

[Ravenslofty]

Let's pick the timers as a starting point.

First we need a pretty basic skeleton:

        <peripheral>
            <name>TIMER</name>
            <version>1</version>
            <description>Programmable Timers. See EE User's Manual, Chapter 4.</description>
            <groupName>TIMER</groupName>
        </peripheral>

Then we read the manual a little:

So, let's sketch these registers:

            <registers>
                <register>
                    <name>T0_MODE</name>
                    <description>Register for setting modes and reading status</description>
                </register>
                <register>
                    <name>T0_COUNT</name>
                    <description>Counter register</description>
                </register>
                <register>
                    <name>T0_COMP</name>
                    <description>Comparison register</description>
                </register>
                <register>
                    <name>T0_HOLD</name>
                    <description>Hold register</description>
                </register>

Let's read the manual a little more:

That's a bunch of fields, let's sketch them too. Most of these can be described with enumeratedValues, such as CLKS:

                        <field>
                            <name>CLKS</name>
                            <description>Clock Selection</description>
                            <bitRange>[1:0]</bitRange>
                            <enumeratedValues>
                                <enumeratedValue>
                                    <name>BUSCLK</name>
                                    <description>BUSCLK (147.456MHz)</description>
                                    <value>0</value>
                                </enumeratedValue>
                                <enumeratedValue>
                                    <name>BUSCLK16</name>
                                    <description>1/16 of the BUSCLK</description>
                                    <value>1</value>
                                </enumeratedValue>
                                <enumeratedValue>
                                    <name>BUSCLK256</name>
                                    <description>1/256 of the BUSCLK</description>
                                    <value>2</value>
                                </enumeratedValue>
                                <enumeratedValue>
                                    <name>HBLNK</name>
                                    <description>External Clock (H-BLNK)</description>
                                    <value>3</value>
                                </enumeratedValue>
                            </enumeratedValues>
                        </field>

but EQUF and OVFF need something else too - <modifiedWriteValues>oneToClear</modifiedWriteValues> tells SVD that clearing this field requires setting that bit to one, not zero.

Let's read the manual a little more:

Now that's just a 16-bit value, so we encode this as a bitRange:

                <register>
                    <name>T0_COUNT</name>
                    <description>Counter register</description>

                    <fields>
                        <field>
                            <name>COUNT</name>
                            <description>Counter Value. The counter value is incremented according to the conditions of the clock and the gate signal specified in the Tn_MODE.</description>
                            <bitRange>[15:0]</bitRange>
                        </field>
                    </fields>
                </register>

Tn_COMP and Tn_HOLD are similar.

Okay, but where are the addresses for these registers? If we go back to Chapter 2, we have the EE memory map, which gives us this:

And so we can specify at the top of TIMER that <baseAddress>0xB0000000</baseAddress>.

"Wait, 0xB0000000?"

MIPS - and the PS2 because of MIPS - has a bunch of overlays of memory. kseg1 is defined to never go through cache, which means we'll always access the device and get an up-to-date value, instead of a stale, cached one.

Then we can add in the various <addressOffset>s for each field relative to the <baseAddress>, and then finally, we can define the other timers by using derived registers with different <addressOffset>s.

[zachary-cauchi]

I see, that clears up a lot of the confusion I had when going through the svd. Thanks for the detailed breakdown! If I may try summarise to make sure I understood:

1. Find a peripheral,component, or similar, that is missing from the svd file.
2. Locate it in the documentation.
3. Create a base peripheral entry for it (or similar if appropriate).
4. Find it's base address:
   a. First by getting it's actual address from the documentation.
   b. Find the kernel-mapped memory segment from which you will be reading/writing to and determine whether it should be cached or not.
   c. Add the memory segment base address to the physical base address.
5. For each component/register/field for it:
   a. Add the appropriate entry/entries inside the block.
   b. Add the physical address offset to the derived base address.

Is that more or less the procedure for a typical peripheral?

Also, with regards to documentation sources, would everything needed be accessible via the PS2-Programming-Docs repo? That's what I've been reading so far (not counting ps2tek).

[Ravenslofty]

I see, that clears up a lot of the confusion I had when going through the svd. Thanks for the detailed breakdown! If I may try summarise to make sure I understood:

1. Find a peripheral,component, or similar, that is missing from the svd file.

2. Locate it in the documentation.

3. Create a base `peripheral` entry for it (or similar if appropriate).

4. Find it's base address:
   a. First by getting it's actual address from the documentation.
   b. Find the kernel-mapped memory segment from which you will be reading/writing to and determine whether it should be cached or not.
   c. Add the memory segment base address to the physical base address.

5. For each component/register/field for it:
   a. Add the appropriate entry/entries inside the block.
   b. Add the physical address offset to the derived base address.

Is that more or less the procedure for a typical peripheral?

Yes, I would say so. There are, however, a notable number of atypical peripherals - the GS is an excellent example. Another thing I would mention is that kseg1 should pretty much always be used; let's not give ourselves problems with caching in the future.

Also, with regards to documentation sources, would everything needed be accessible via the PS2-Programming-Docs repo? That's what I've been reading so far (not counting ps2tek).

Indeed.

[zachary-cauchi]

True, too true. I'm sorry, one more question regarding the timer register: In the ee users manual in chapter 2.2.1, the T0_COUNT register is offset by 0x0000, but in the svd, it's 0x0010. The same pattern occurs for the other timers too.

[Ravenslofty]

You are correct; I must have gotten confused by Chapter 2 being in the order of MODE, COUNT, COMP, HOLD when the hardware order is COUNT, MODE, COMP, HOLD. Good spot.

[zachary-cauchi]

Thanks! I can take care of that if you'd like. Also, I see the FIFO section is pending. How about I begin with that?

[Ravenslofty]

The FIFO is pretty low on the list of things; I think the DMAC GIF is a better starting point.

(the DMAC's already been done, fortunately)

[zachary-cauchi]

Okay, I'll begin with that. Will keep you posted on it.

[zachary-cauchi]

Just added the fields for the GIF_CTRL and GIF_MODE registers, was hoping you could give them a look and give me any feedback before tackling the rest.
Here is my branch compared to master.

[Ravenslofty]

Looks good to me.

[zachary-cauchi]

Just published a PR for the GIF registers. When you can, would you please give it a look? Please do scrutinise it since it's my first time working on SVD files.

[Ravenslofty]

Also, with regards to documentation sources, would everything needed be accessible via the PS2-Programming-Docs repo? That's what I've been reading so far (not counting ps2tek).

Indeed.

I'm going to correct myself, there is a relatively important thing which isn't in there: in the EE User's Manual, there is a reference to some registers which do not exist in the GS User's Manual - IIRC, at least SMODE1.

These were left undocumented because Sony didn't want people fucking with the video config outside of the SetGsCrt syscall. Unfortunately, if we're to avoid depending on the PS2 BIOS, we will need to fuck with the video config.

As far as I know, the only reference for this is, uh, Linux for PlayStation 2 or perhaps the cleaned up driver

[zachary-cauchi]

These were left undocumented because Sony didn't want people fucking with the video config outside of the SetGsCrt syscall. Unfortunately, if we're to avoid depending on the PS2 BIOS, we will need to fuck with the video config.

Ah, interesting. Oh well, no project worth doing is free of complications.

Given it would involve some trickery to properly use as mentioned, intentionally lacks documentation, and the aforementioned syscall is already wrapped and does the work for us, I suppose it's low priority for the time being?

[Ravenslofty]

Thinking about it, I'm not sure I trust the BIOS bindings anymore; #4 migrated from asm! to llvm_asm! and then #8 migrated from llvm_asm! to the new-flavour asm! which has never stabilised for MIPS (and by "MIPS" Rust seems to mean MIPS32r2, not MIPS II/III that the PS2 uses).

I think the only way to make sure it doesn't unusably bitrot would be to write it in Rust proper, which means those registers will need to be encoded in the SVD.

[zachary-cauchi]

Hmm, a fair point. I can try tackle the task of documenting them in the SVD later on today. I can create an issue about the task and keep track of progress there. What do you think?

[Ravenslofty]

Sounds good.

[ayrtonm]

For ABI reasons, linking to PS2SDK from Rust isn't a feasible approach.

@Ravenslofty going back to this for a second. Is the reason basically that ps2sdk relies on an old fork of gcc w/ some patches on top which has ABI-mismatches with MIPS code generated by LLVM or something else? If it's the former then trying to use the gcc rustc backend might be an option. I looked into this two years ago and it also requires a gcc fork to build a patched version of libgccjit.so so that would require some work to combine the two stacks of patches on a version of gcc that works for both the rustc backend and ps2sdk but it would let you avoid the tedious work of writing register bindings. Just a thought though, fwiw for my rust ps1 sdk I went with the tedious register binding approach.

Also how reliable is the code you're generating with rustc rn. Your target JSON uses MIPS II but can that emit any instructions the EE doesn't implement or have othe codegen issues? I skimmed through your clang patch stack and I assumed at least some of those would be required for reliable codegen. I'm asking because there was a similar issue for the ps1 that basically generated incorrect code when returning from a function and iirc the rust psp folks have fixed similar MIPS codegen problems.

[Ravenslofty]

For ABI reasons, linking to PS2SDK from Rust isn't a feasible approach.

@Ravenslofty going back to this for a second. Is the reason basically that ps2sdk relies on an old fork of gcc w/ some patches on top which has ABI-mismatches with MIPS code generated by LLVM or something else? If it's the former then trying to use the gcc rustc backend might be an option.

When I said "old version of GCC", I meant "GCC 3.2.3". It seems like they've managed to port the EE compiler to GCC 10.2, so this might be feasible. However:

I looked into this two years ago and it also requires a gcc fork to build a patched version of libgccjit.so so that would require some work to combine the two stacks of patches on a version of gcc that works for both the rustc backend and ps2sdk but it would let you avoid the tedious work of writing register bindings. Just a thought though, fwiw for my rust ps1 sdk I went with the tedious register binding approach.

I'm really, really not convinced "maintain your own GCC fork, and require users to build their own rustc" is less effort than the "tedious register binding approach".

Also how reliable is the code you're generating with rustc rn. Your target JSON uses MIPS II but can that emit any instructions the EE doesn't implement or have othe codegen issues? I skimmed through your clang patch stack and I assumed at least some of those would be required for reliable codegen. I'm asking because there was a similar issue for the ps1 that basically generated incorrect code when returning from a function and iirc the rust psp folks have fixed similar MIPS codegen problems.

Sure it can; atomic instructions are not implemented in the EE. There's also the "short loop bug", but this appears to have been fixed in all production silicon, so I'm not too worried there. But I don't particularly mind doing emulation of instructions because Prussia will need to install a bunch of exception handlers anyway to fail """gracefully""".

[ayrtonm]

I'm really, really not convinced "maintain your own GCC fork, and require users to build their own rustc" is less effort than the "tedious register binding approach".

Oh yeah I definitely meant developer effort not user effort haha, and even then I'm not saying register binding is a bad approach. It probably lets you create a more ergonomic rust API than wrapping and linking against existing C/asm code anyway. I think the gcc backend might be interesting to consider if for example you could provide a prebuilt libgccjit.so either when running rustc or to a rustc build, but it's probably not too useful to speculate on what that would look like until the gcc backend becomes more integrated rustc and they upstream the rest of their patches to gcc.

Sure it can

Neat! good to hear codegen works