add compilation.yml to enable CI

[bignaux]

A first step for CI. Since iopmod modules are incompatible with latest gcc i use (11.2.0), it allows me to use the old toolchain for this stuff.

[frno7]

This seems interesting. I’m not very familiar with the Git Hub compilation workflow service. Is it something in the Actions menu, having downloadable IRX modules in a tar file? Would you be able to explain, in the commit message, how people can use this and where the downloads will be found?

Note: I’ll be travelling for about a week in the beginning of April, so replies may be a bit slow.

[frno7]

So we’d have a dependency on @TobiX repo, then.

[AKuHAK]

Yes, this is why I didnt made this pr earlier, caus eit is based on 3rd party docker container that can be removed in any moment.

[AKuHAK]

As about describing in Readme where files can be found, @frno7 do you mean just point to the the github official documentation? This is GitHub basic features I dont think that such features should be covered by repo.

[frno7]

@AKuHAK, a couple of sentences about why and where (a URL) one might find the produced IRX files would be good, in the README.md especially. Maybe we ought to have a build pass/fail badge there too? :-)

[frno7]

@AKuHAK, are you proposing frno7/linux#69 as a replacement for this pull request?

[AKuHAK]

@AKuHAK, are you proposing frno7/linux#69 as a replacement for this pull request?

Not a replacement but more an addition. CI (Continuous Integration) is meant to be involved in each repository as a part of repository's internal check and as an easy way for generating precompiled binaries. This PR can be revisited when the necessary preparations will be made (as described in that PR).

[AKuHAK]

@frno7 frno7/linux#69 is only the next step after you create one more repository for holding the main Docker image

[frno7]

@bignaux, additionally, in a separate effort, I think we should attempt to fix whatever problem there is with GCC 11.2.0.

[bignaux]

@frno7 : As i wrote you friday in a mail, i'm not able to boot properly your kernel right now. I figure than address entry point was not the issue, and use wLaunchelf instead of incompatible kernelloader (but i can't pass kernel cmdline). I was disagree with @AKuHAK on the fact "3rd party docker container" would be an issue, the completion of the CI task is optional, and to have reference binaries is very useful for people like me that can't trust his own binaries for now. I use it here https://github.com/bignaux/meta-playstation2/blob/main/recipes-kernel/iopmod/iopmod-bin.bb as a temporary workaround, and we can see later to fix the issue with newest toolchain.

[AKuHAK]

@frno7 If you wish, you can create an empty repository named mipsr5900el-toolchain and I will provide PR for the internal GitHub container registry. So
container: tobix/mipsr5900el-toolchain:latest
can be replaced with
container: ghcr.io/${{ github.repository_owner }}/mipsr5900el-toolchain:main
making it focused in one place and easier for maintaining.

[TobiX]

@AKuHAK Be aware that GitHub Packages (of which the container registry is a part if I understood it correctly) has a limit of 500MB per user (and 1GB transfer) and the Toolchain image is already ~450MB in size

[AKuHAK]

@TobiX thanks for this limitation, Gentoo is really heavy distribution, I personally prefer Alpine for CI

[AKuHAK]

Be aware that GitHub Packages (of which the container registry is a part if I understood it correctly) has a limit of 500MB per user (and 1GB transfer) and the Toolchain image is already ~450MB in size

@TobiX Do you have the script for compiling everything inside your Docker container?

[TobiX]

@TobiX thanks for this limitation, Gentoo is really heavy distribution, I personally prefer Alpine for CI

Me too, but that image is literally just https://github.com/frno7/linux/wiki/Building-an-R5900-cross-compiler-using-Gentoo-Linux in script-form - that's the whole reason it's using Gentoo.

And no, I don't have any script for compiling anything inside that container: I just used it as a dev/experimenting environment.

[frno7]

I’ll be able to revisit this in a couple of weeks, mid-April I’m guessing, once I’m back from travelling.

[AKuHAK]

Cannot compile busybox: @frno7 do you have any thoughts? The log is here.
USE="prefix-guest static" emerge-mipsr5900el-unknown-linux-gnu -v sys-apps/busybox
prefix-guest added to get rid of circular dependency on libcrypt
Error:
/usr/mipsr5900el-unknown-linux-gnu/bin/busybox: cannot execute binary file: Exec format error

[frno7]

@AKuHAK, what does readelf --file-header /usr/mipsr5900el-unknown-linux-gnu/bin/busybox say? Maybe you’re attempting to run an R5900 Busybox on some other processor architecture? You’d need R5900 QEMU in that case, or a real PlayStation 2 Linux. :-)

[AKuHAK]

@frno7 thanks, the wiki sections for that step were a bit confusing.
Successfully build an elf and successfully run it in pcsx2 using @TobiX image:

[bignaux]

I'd rewrite this using gcc-mipsel-linux-gnu ubuntu package instead of container. @AKuHAK seems to work on same stuff so i donno if i don't lose my time right now.

[AKuHAK]

I'd rewrite this using gcc-mipsel-linux-gnu ubuntu package instead of container. @AKuHAK seems to work on same stuff so i donno if i don't lose my time right now.

The action itself will not change much, only the container link will change. Even if we will use the ubuntu package it is still better to grab a Docker container somewhere.
BTW your workflow lacks an artifacts section. Releases will keep only the latest build, sometimes it is useful to check older builds.

[TobiX]

Please be aware that binaries created by a "normal" gcc-mipsel-linux-gnu will probably crash or deadlock on the PS2, caused by hardware errata of that CPU

[frno7]

Please be aware that binaries created by a "normal" gcc-mipsel-linux-gnu will probably crash or deadlock on the PS2, caused by hardware errata of that CPU

Indeed, the R5900 has its short loop erratum, which, by default, isn’t handled by GAS nor GCC for non-mipsr5900el targets. However, the GAS and GCC option -mfix-r5900, available since 2018, will force a fix for the R5900 erratum regardless, with the generated code remaining compatible and safe for all other MIPS variants. Confer GAS commit GAS/MIPS: Add -mfix-r5900 option for the R5900 short loop erratum GCC commit MIPS: Add -mfix-r5900 option for the R5900 short loop erratum for details. Generally speaking it shouldn’t be necessary to have a specific MIPS compiler for the R5900, if appropriate options are given to it, for instance -march=r5900.

That said, this repo is unaffected by the R5900 erratum, since it’s about the input/output processor (IOP) which is a MIPS R3000A, or in later PlayStation 2 models a PowerPC 405GP emulating a MIPS R3000A. Hence -march=r3000 is supplied to GCC here:

iopmod/Makefile

Line 17 in 4f0dc8c

IOP_CFLAGS += -O0 -march=r3000 -EL -msoft-float -fomit-frame-pointer \

There’s issue #1 about the -O0 option, though.

Finally, as previously noted I’m travelling at the moment so replies take a bit more time for me. :-)

[frno7]

@AKuHAK, continuing on frno7/linux#28 (comment), your comment here in #7 (comment) saying

Initrd not found or empty - disabling initrd

seems suggest that the indicated or empty is a likely problem. Are you sure you have a good directory set up for for INITRAMFS?

[AKuHAK]

Yes, I am sure.

[AKuHAK]

initramfs_data.cpio.zip
@frno7 can you test your kernel with this ramdisk ?

[AKuHAK]

seems suggest that the indicated or empty is a likely problem. Are you sure you have a good directory set up for for INITRAMFS?

this error means that the ramdisk isn't properly extracted, not that this ramdisk is improper itself

[frno7]

initramfs_data.cpio.zip

A quick glance, listing its content with xzcat initramfs_data.cpio.xz | cpio -vt | less looks good to me, including file and directory ownerships. Also extraction with xzcat ../initramfs_data.cpio.xz | cpio -vi --no-absolute-filenames is fine. Both /init and /sbin/init scripts are reasonable. Your bin/busybox is present and appears correct, although at 2.6 MB it’s more than 1 MB bigger than mine. Most likely works anyway, but memory pressure increases.

I suppose you have these configured (among probably a few others):

linux % grep XZ .config
CONFIG_HAVE_KERNEL_XZ=y
CONFIG_KERNEL_XZ=y
CONFIG_RD_XZ=y
CONFIG_DECOMPRESS_XZ=y

Indeed, it could be worth checking whether your vmlinuz kernel has its INITRAMFS attached as it’s suppose to. People have been doing similar things, as described in https://lkml.kernel.org/lkml/20090105114730.GD3268@1wt.eu/T/ (although maybe somewhat outdated from 2008, and using gz rather than xz).

@frno7 can you test your kernel with this ramdisk ?

Sure, but not yet as I don’t have my kit set up due to travelling.

[frno7]

@AKuHAK, maybe your total kernel size is too large? There are hard limits on this, not apparently obvious. My vmlinuz is currently about 4.43 MB, and I recall that 6-7 years ago I had to trim it down, by removing unnecessary kernel features and Busybox applications. Once the kernel started properly I didn’t bother trimming it down further. There are most likely a lot of opportunities to reduce its size even further.

Of course, the decompressed kernel must also fit within memory, and so on. We should mention these size limits on the wiki. Incidentally, you may want to check whether wLaunchELF loads the whole ELF properly, if file size is an issue.

Compared with your (2.67 MB) Busbox, I’ve disabled these applications:

ar
ascii
base32
bb
bbconfig
bbsh
bc
crc32
flashcp
flash_eraseall
flash_lock
flash_unlock
ginit
i2ctransfer
lpq
lpr
lzopcat
mim
minips
mkfs.reiser
netcat
nologin
popmaildir
sendmail
svok
tc
telnetd
ts
tune2fs
uncompress
unit
unlzop

My (1.47 MB) Busybox has a few that yours doesn’t have. :-) That’d be these (most of which I think really are unnecessary):

add-shell
beep
bootchartd
crontab
dc
dnsd
fakeidentd
fbsplash
fold
fsck.minix
ftpget
ftpput
hostid
hush
inetd
ipcalc
klogd
logger
logname
mkfs.minix
od
ping6
rdev
readprofile
remove-shell
run-parts
runsv
runsvdir
slattach
smemcap
sulogin
sv
svlogd
syslogd
taskset
tcpsvd
traceroute6
udpsvd
uudecode
uuencode

[AKuHAK]

this is current configuration with default config provided in kernel sources (ps2_defconfig). If I built with this settings i get initrd not found error. Kernel size 4.06Mb

this will change 2 settings in config:

CONFIG_INITRAMFS_COMPRESSION_XZ=y
# CONFIG_INITRAMFS_COMPRESSION_NONE is not set

Kernel size 4.06Mb and I got the same initrd not found error.

But if i select here

# CONFIG_INITRAMFS_COMPRESSION_XZ is not set
CONFIG_INITRAMFS_COMPRESSION_NONE=y
CONFIG_INITRAMFS_COMPRESSION=""

i can see that usr/initramfs_data.cpio is generating and it is almost 7Mb in size. But resulting kernel is 3.99Mb in size
Now it will freeze after Uncompressing linux at load address 80010000

If I disable INITRD completely
# CONFIG_BLK_DEV_INITRD is not set
then kernel will be 1.95Mb and boots just fine.

[AKuHAK]

My (1.47 MB) Busybox has a few that yours doesn’t have.

Do you have a guide on how to change busybox configuration on Gentoo? Currently, I am using the command

USE="prefix-guest static" emerge-mipsr5900el-unknown-linux-gnu -v sys-apps/busybox

I found out that I can use savedconfig, but didn't realize how.

[frno7]

Do you have a guide on how to change busybox configuration on Gentoo?

As I recall I built my R5900 Busybox manually, about 7 years ago, without involving its Gentoo package. My MIPS tools were much more primitive back then. I think your modern approach is much nicer. :-)

Currently, I am using the command

USE="prefix-guest static" emerge-mipsr5900el-unknown-linux-gnu -v sys-apps/busybox

I found out that I can use savedconfig, but didn't realize how.

Marvellous. Busybox has a .config file very similar to the Linux kernel, doesn’t it? So one simply needs to come up with a Busybox .config according to one’s taste, and, as it looks to me, put it in a file /usr/mipsr5900el-unknown-linux-gnu/etc/portage/savedconfig/sys-apps/busybox, to have Gentoo pick it up when given the savedconfig USE flag, no? According to usage it’s possible to have variants with specific package version numbering, etc. You may want to look if you’ve got some variant of this file already. I had one that I don’t recall that I’ve made myself. It’s called /usr/mipsr5900el-unknown-linux-gnu/etc/portage/savedconfig/sys-apps/busybox-1.31.1-r2 and was most likely produced when building the Gentoo Linux system set for the PlayStation 2. Well, since then Busybox has been removed from the Gentoo system set so one might no longer get it by default as I once did. :-)

[TobiX]

Stupid question, slightly off-topic: Maybe create a new GitHub-organization for all Linux-on-PS2 releated repositories and move everything there? Unfortunately, https://github.com/ps2linux is already taken...

[frno7]

@TobiX, when I started out I was hoping to merge as much code as possible with preexisting organisations, such GNU for Binutils and GCC, QEMU, the Linux kernel, and perhaps, eventually, also Gentoo (their experimental download section does have MIPS, as opposed to most other significant Linux distributions, believe it or not), so that everything would be available from the usual channels without additional patches. Maybe @bignaux will make an effort with Yocto as mentioned in frno7/linux#34 (comment).

As mentioned in #7 (comment) the somewhat recent -mfix-r5900 option to GCC makes it possible to compile a generic MIPS II operating system that’s compatible with the R5900, without having any PlayStation 2 specific parts to it. One could make a generic MIPS II Gentoo Linux download out of it, that’s compatible with a range of different MIPS hardware, for instance. Prior to -mfix-r5900, the R5900 and the all other MIPS variants were mutually exclusive.

Of course, this doesn’t preclude setting up a PS2/Linux organisation at Git Hub, but it may explain where at least my focus is. :-)

Issue #1 remains a significant hurdle, for sure. I need to cut down the initial patch submission from 120 patches to less than half, most probably. I’ve also come to the conclusion that it must become an n32 ABI kernel, rather than the current o32 ABI kernel. The reason being the 128-bit multimedia registers of the R5900, which will have grave technical problems with an o32 ABI kernel.

[bignaux]

@TobiX : i plan to create an org when i'll have a working Yocto setup.

About ABI, you can look at https://github.com/bignaux/meta-playstation2/blob/main/conf/machine/tune-r5900.inc is the file to enable new ABI (n32/o32, perhaps 64bits and so) and you just have to config the right DEFAULTTUNE to rebuild with the new ABI setting.

[frno7]

With #9 (comment), I suppose this issue is resolved as well?