serial: Add --device virtio-serial,pty support #113

cfergeau · 2024-03-15T10:06:57Z

This new virtio-serial option allocates a pseudo-tty for the VM console. It
can then be accessed using screen for example. This is a bit similar to
the --device virtio-serial,stdio option, except that the console is not
tied to the terminal running vfkit, it's possible to connect/disconnect
from the pseudo-tty from any terminal.

This fixes #48

pkg/vf/virtionet.go uses local adhoc solution to run cleanup tasks when vfkit exits. github.com/onsi/gocleanup achieves the same result in more generic code. Since there are more things I'd like to cleanup at exit time, let's make use of this package. It hasn't been changed in a long time, but its code is a short single file, at worse we can import this file in vfkit repository and make the changes we need. Signed-off-by: Christophe Fergeau <cfergeau@redhat.com>

When using the REST API over a unix socket, this ensures the unix socket is removed from the filesystem when vfkit exits.

This new virtio-serial option allocates a pseudo-tty for the VM console. It can then be accessed using `screen` for example. This is a bit similar to the `--device virtio-serial,stdio` option, except that the console is not tied to the terminal running vfkit, it's possible to connect/disconnect from the pseudo-tty from any terminal. This fixes crc-org#48 Signed-off-by: Christophe Fergeau <cfergeau@redhat.com>

This uses more helper APIs from github.com/pkg/term/termios, and makes the code closer to Apple's recommendations in https://developer.apple.com/documentation/virtualization/running_linux_in_a_virtual_machine?language=objc#:~:text=Configure%20the%20Serial%20Port%20Device%20for%20Standard%20In%20and%20Out This also removes direct use of `syscall` in pkg/vf/virtio.go Signed-off-by: Christophe Fergeau <cfergeau@redhat.com>

openshift-ci · 2024-03-15T10:07:05Z

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please ask for approval from cfergeau. For more information see the Kubernetes Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

anjannath · 2024-03-18T10:49:49Z

tested and working as expected..

% ./out/vfkit \
    --cpus 2 --memory 2048 \
    --bootloader efi,variable-store=efi-variable-store,create \
    --device virtio-blk,path=vfkit-test-image.raw --device virtio-serial,pty
INFO[0000] &{2 2048    {[efi variable-store=efi-variable-store create] true}  [virtio-blk,path=vfkit-test-image.raw virtio-serial,pty] none://  false}
INFO[0000] boot parameters: &{EFIVariableStorePath:efi-variable-store CreateVariableStore:true}
INFO[0000]
INFO[0000] virtual machine parameters:
INFO[0000] 	vCPUs: 2
INFO[0000] 	memory: 2048 MiB
INFO[0000]
INFO[0000] Adding virtio-blk device (imagePath: vfkit-test-image.raw)
INFO[0000] Using PTY (pty path: /dev/ttys002)
INFO[0000] virtual machine is running
INFO[0000] waiting for VM to stop

after this i was able to see the fedora login prompt by running % screen /dev/ttys002

anjannath · 2024-03-18T10:49:56Z

/lgtm

praveenkumar · 2024-03-20T11:45:43Z

I am testing it with crc bundle with the script part of contrib/script with following change but not able to get any login prompt

$ git diff
diff --git a/contrib/scripts/start-crc-bundle.sh b/contrib/scripts/start-crc-bundle.sh
old mode 100644
new mode 100755
index 34a28ee..965a497
--- a/contrib/scripts/start-crc-bundle.sh
+++ b/contrib/scripts/start-crc-bundle.sh
@@ -15,6 +15,6 @@ cp -c ${BUNDLE_PATH}/${DISKIMG} overlay.img
     --initrd "${BUNDLE_PATH}/${INITRD}" \
     --kernel-cmdline "${CMDLINE}" \
     --device virtio-blk,path=overlay.img \
-    --device virtio-serial,logFilePath=start-bundle.log \
+    --device virtio-serial,pty \ 
     --device virtio-net,nat,mac=72:20:43:d4:38:62 \
     --device virtio-rng

$ contrib/scripts/start-crc-bundle.sh /Users/prkumar/.crc/cache/crc_vfkit_4.13.14_arm64
[...]
INFO[0000]                                              
INFO[0000] virtual machine parameters:                  
INFO[0000] 	vCPUs: 2                                    
INFO[0000] 	memory: 2048 MiB                            
INFO[0000]                                              
INFO[0000] Adding virtio-blk device (imagePath: overlay.img) 
INFO[0000] Using PTY (pty path: /dev/ttys000)            <= tried to connect this with screen and nothing
INFO[0000] Adding virtio-net device (nat: true macAddress: [72:20:43:d4:38:62]) 
INFO[0000] Adding virtio-rng device                     
INFO[0000] virtual machine is running                   
INFO[0000] waiting for VM to stop

I will try with fedora also same what @anjannath tested and put another comment.

praveenkumar · 2024-03-20T12:22:18Z

To me even steps mentioned by @anjannath didn't work after screen /dev/ttys000 I just see the blank without any prompt.

cfergeau · 2024-03-20T12:46:06Z

To me even steps mentioned by @anjannath didn't work after screen /dev/ttys000 I just see the blank without any prompt.

You need to make sure you have console=hvc0 on the kernel cmd line if you are booting with external kernel. If I connect late in the boot process, sometimes I cannot get output, not clear why.

anjannath · 2024-03-20T13:16:52Z

I tried again and it is working for me still, maybe you have to attach to the pty quickly otherwise you miss the login prompt and it doesn't re-appear?

pty.mov

but another thing i noticed is that the keyboard only works sometimes, a few times (2 out of 4 runs) during testing keys are not echoed in screen, which i assumed to be the keyboard not working

praveenkumar · 2024-03-20T14:54:39Z

maybe you have to attach to the pty quickly otherwise you miss the login prompt and it doesn't re-appear?

I also tried to attach pty quickly but still nothing, waiting more that 5 mins.

praveenkumar · 2024-03-20T15:08:05Z

but another thing i noticed is that the keyboard only works sometimes, a few times (2 out of 4 runs) during testing keys are not echoed in screen, which i assumed to be the keyboard not working

So after 5-6 retry finally I am able to get the login prompt for fedora and crc bundle but then on crc I am not able to use the keyboard but on fedora able to do.

cfergeau · 2024-03-20T15:13:05Z

but another thing i noticed is that the keyboard only works sometimes, a few times (2 out of 4 runs) during testing keys are not echoed in screen, which i assumed to be the keyboard not working

So after 5-6 retry finally I am able to get the login prompt for fedora and crc bundle but then on crc I am not able to use the keyboard but on fedora able to do.

Maybe virtio: Simplify setRawMode is causing issues. Or maybe it would be better to use https://github.com/pkg/term/blob/1a4a3b719465afccedae7788469c0b54a43417e0/termios/termios.go#L41-L50 , but I could not get this to work correctly.
Really not familiar with pty/terminal input/output :-/

praveenkumar · 2024-03-20T15:36:53Z

but another thing i noticed is that the keyboard only works sometimes, a few times (2 out of 4 runs) during testing keys are not echoed in screen, which i assumed to be the keyboard not working

So after 5-6 retry finally I am able to get the login prompt for fedora and crc bundle but then on crc I am not able to use the keyboard but on fedora able to do.

Maybe virtio: Simplify setRawMode is causing issues. Or maybe it would be better to use https://github.com/pkg/term/blob/1a4a3b719465afccedae7788469c0b54a43417e0/termios/termios.go#L41-L50 , but I could not get this to work correctly. Really not familiar with pty/terminal input/output :-/

I am also not sure what causing the issue but as of now this functionality is quite buggy and we might not able to consume it when required like debugging an broken ssh connection. Do you think we can use tty along with log to a file because in tty I also don't see boot logs ?

openshift-merge-robot · 2024-03-26T08:52:23Z

PR needs rebase.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

cfergeau added 4 commits March 15, 2024 10:45

rest: Use gocleanup

9989850

When using the REST API over a unix socket, this ensures the unix socket is removed from the filesystem when vfkit exits.

openshift-ci bot requested review from anjannath and praveenkumar March 15, 2024 10:07

cfergeau mentioned this pull request Mar 15, 2024

Access guest console when it is running #48

Open

openshift-ci bot assigned anjannath Mar 18, 2024

openshift-ci bot added the lgtm label Mar 18, 2024

openshift-merge-robot added the needs-rebase label Mar 26, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

serial: Add --device virtio-serial,pty support #113

serial: Add --device virtio-serial,pty support #113

cfergeau commented Mar 15, 2024

openshift-ci bot commented Mar 15, 2024

anjannath commented Mar 18, 2024

anjannath commented Mar 18, 2024

praveenkumar commented Mar 20, 2024

praveenkumar commented Mar 20, 2024

cfergeau commented Mar 20, 2024

anjannath commented Mar 20, 2024

praveenkumar commented Mar 20, 2024

praveenkumar commented Mar 20, 2024

cfergeau commented Mar 20, 2024

praveenkumar commented Mar 20, 2024

openshift-merge-robot commented Mar 26, 2024

serial: Add --device virtio-serial,pty support #113

Are you sure you want to change the base?

serial: Add --device virtio-serial,pty support #113

Conversation

cfergeau commented Mar 15, 2024

openshift-ci bot commented Mar 15, 2024

anjannath commented Mar 18, 2024

anjannath commented Mar 18, 2024

praveenkumar commented Mar 20, 2024

praveenkumar commented Mar 20, 2024

cfergeau commented Mar 20, 2024

anjannath commented Mar 20, 2024

praveenkumar commented Mar 20, 2024

praveenkumar commented Mar 20, 2024

cfergeau commented Mar 20, 2024

praveenkumar commented Mar 20, 2024

openshift-merge-robot commented Mar 26, 2024