guestmount failing on ubuntu

[NathanTP]

Marshal may fail with a somewhat cryptic error on ubuntu systems. This is a known issue and it's being documented here for posterity:

Reproduce:
./marshal build test/command.json

Symptom (e.g.):

subprocess.CalledProcessError: Command 'guestmount -a /scratch/nathanp/firesim/sw/firesim-software/images/br-base.img -m /dev/sda /scratch/nathanp/firesim/sw/firesim-software/disk-mount' returned non-zero exit status 1

In the detailed logs you'll see:

libguestfs: error: /usr/bin/supermin exited with error status 1.

This is an issue with guestmount and ubuntu (marshal uses guestmount to manipulate target filesystems without requiring sudo). The bugs are documented here and here. The solution is to add read privileges to the kernel (e.g. sudo chmod +r /boot/vmlinuz-*).

[NathanTP]

Closed because we can't fix it from mashal, it must be fixed by your sysadmin.

[wallento]

Just to understand as this is a common error: Why would chipyard need access to the host kernel?

[jerryz123]

Chipyard attempts to build firemarshal linux images by default in the chipyard setup script. Firemarshal requires guestmount with sudo to create the images.

You can disable the default firemarshal builds in chipyard's setup script with some flag. I believe it is --skip-marshal.

[wallento]

Yes, but what I mean is from the user experience point of view this error and the explanation is confusing.
Also, what you say actually is the change I was about to propose in a PR, because right now it doesn't sudo:

diff --git a/wlutil/wlutil.py b/wlutil/wlutil.py
index b68631f..ad8a02e 100644
--- a/wlutil/wlutil.py
+++ b/wlutil/wlutil.py
@@ -626,14 +626,14 @@ def mountImg(imgPath, mntPath):
             assert "nfs" not in fstype, f"Guestmount does not support {fstype} filesystems, change mount-dir to a non-NFS filesystem"
 
             pidPath = './guestmount.pid'
-            run(['guestmount', '--pid-file', pidPath, '-o', f'uid={uid}', '-o', f'gid={gid}', '-a', imgPath, '-m', '/dev/sda', mntPath])
+            run(sudoCmd + ['guestmount', '--pid-file', pidPath, '-o', f'uid={uid}', '-o', f'gid={gid}', '-a', imgPath, '-m', '/dev/sda', mntPath])
             try:
                 with open(pidPath, 'r') as pidFile:
                     mntPid = int(pidFile.readline())
                 yield mntPath
             finally:
-                run(['guestunmount', mntPath])
-                os.remove(pidPath)
+                run(sudoCmd + ['guestunmount', mntPath])
+                run(sudoCmd + ['rm', pidPath])
 
             # There is a race-condition in guestmount where a background task keeps
             # modifying the image for a period after unmount. This is the documented

Right now, it does not use sudo and the fix is not obvious. Relying on sudo should probably be documented somewhere.

[wallento]

I think most people come via Chipyard and it should be documented there, that if you want to run it fully with firemarshal you will need sudo, and that you can use -skip-marshal in case your infrastructure does not allow you sudo. (You see I had many interactions with industry folks ;)

Happy to prepare all patches if you agree.

[jerryz123]

That would be a good addition. I'd appreciate a PR with the additional notes if you are able