Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Restore guest agent unix socket functionality #2006

Merged
merged 1 commit into from
Dec 25, 2023

Conversation

afbjorklund
Copy link
Member

@afbjorklund afbjorklund commented Nov 17, 2023

Make both vsock and virtio explicit, instead of hardcoding internal filenames in the agents. Fallback to to the unix.

If neither vsock nor virtio is being used, then the driver silently falls back to forwarding the unix socket over ssh.

This means that new drivers don't have to do anything, unless overriding.

@balajiv113
Copy link
Member

@afbjorklund Thanks for addressing this usecase

How about we expose a util to forward unix sock ?? Drivers lile vbox can use this and follow the same mechanism of dial unix sock

In guest side, we will have last default as unix sock without any additional options.

This way, the guest communication will fully be taken care by drivers only and host will not hardcode any functionality for guest connection

@afbjorklund

This comment was marked as outdated.

@balajiv113
Copy link
Member

Yes i agree on bringing the forwarded sock. But decision should be done from each driver and not on host agent.

The reason is, we are using ForwardUnixGuestAgent just for comparability reasons. Which doesn't give much value for a driver interface and also because of this hostagent has to take care of forwarding unix socket

The advantage of driver deciding is the separation of guest agent connection from host agent and keeping the driver interface clean of other.

I agree some changes needed on vbox for this but it will be very minimal. We could do that just to keep the driver interface clear

@afbjorklund
Copy link
Member Author

afbjorklund commented Nov 18, 2023

I agree, keeping it in the interface was more of a workaround (due to vsock/virtio ports and conn being exposed already)

could just copy the d.VSockPort == 0 && d.VirtioPort == "" to the approximately two places actually needing it

We could add more callbacks (in the driver) to set up and cancel the sockets, and do the implementation in the basedriver.

just didn't like have any exception or boiler plate code in the "empty" driver, it would be nice if it could inherit

@afbjorklund
Copy link
Member Author

afbjorklund commented Nov 18, 2023

Kept the implementation in the hostagent (since it was there before, and had all the functions needed)

We could move it (2-3 functions) to the BaseDriver with some effort, but I am not sure if it is worth spending

localUnix := filepath.Join(a.instDir, filenames.GuestAgentSock)
remoteUnix := "/run/lima-guestagent.sock"
forwardSSH(context.Background(), a.sshConfig, a.sshLocalPort, localUnix, remoteUnix, verbCancel, false)
forwardSSH(ctx, a.sshConfig, a.sshLocalPort, localUnix, remoteUnix, verbForward, false)

d.DialContext(ctx, "unix", filepath.Join(a.instDir, filenames.GuestAgentSock))

There is some similar empty hook in the Register/Unregister callbacks, but that doesn't have shared code

Thanks for keeping the interface clean, it did feel a bit rushed (but interface doesn't expose any variables)

@afbjorklund afbjorklund marked this pull request as draft November 18, 2023 11:43
@afbjorklund

This comment was marked as outdated.

@afbjorklund
Copy link
Member Author

Added some documentation about the previous/fallback functionality, best I could come up with.

@afbjorklund afbjorklund marked this pull request as ready for review November 18, 2023 12:09

var l net.Listener
if _, err := os.Stat(virtioPort); err == nil {
qemuL, err := serialport.Listen(virtioPort)
if virtioPort != "" {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Instead of using virtioPort as a arg, can we simply do os.Stat and fallback to unix sock ?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Having it as an explicit argument was a feature, but it should probably be named serialSock

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The communication medium is internal to us. I don't think there is a need to make it a argument. We can hardcode it like we do for unix sock just to simplify stuff

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If the filename is indeed fixed we can hardcode it, but then it might not even need the constant

@@ -137,6 +142,12 @@ func (d *BaseDriver) ListSnapshots(_ context.Context) (string, error) {
return "", fmt.Errorf("unimplemented")
}

func (d *BaseDriver) ForwardGuestAgent() bool {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My concern is still here.

My point is we don't need this. Instead of treating unix sock as fallback am saying lets treat it as a way drivers can use. But drivers need to decide on the communication medium.

For eg: vz will always work on vsock, virtbox will always work on unix sock (This individual drivers should decide). Hostagent should not have that code

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wanted to keep the hostagent tunneling as a fallback. It was handy when experimenting with external VMs.

For the VBox driver, it can probably use a serial console. Will do a rebase of it someday, it is not important...

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When we say fallback i wanted this to be a universal one not just for un configured driver.

Something like this 4640684. This is just a idea how we can do, based on the error we can fallback to this universal unix sock

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As long as it is done in the hostagent, and not copy/paste (or explicit) in the driver, that would work as well

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure where to cancel the forwarding process, but maybe it would just die when the instance does?

@AkihiroSuda
Copy link
Member

Needs rebase

@afbjorklund afbjorklund marked this pull request as draft November 24, 2023 07:42
@afbjorklund afbjorklund self-assigned this Nov 24, 2023
@afbjorklund
Copy link
Member Author

Needs rewrite, without the changes to the interface (as suggested)

@AkihiroSuda AkihiroSuda removed this from the v0.19.0 milestone Nov 24, 2023
@afbjorklund afbjorklund mentioned this pull request Nov 25, 2023
@afbjorklund
Copy link
Member Author

afbjorklund commented Nov 25, 2023

Rebase done.

  • still passes the virtio socket as an explicit argument (like vsock), instead of fixed path
        daemonCommand.Flags().Int("vsock-port", 0, "use vsock server instead a UNIX socket")
+       daemonCommand.Flags().String("virtio-port", "", "use virtio server instead a UNIX socket")
  • still uses the driver interface to propagate the vsock and virtio fields from the basedriver
func (d *BaseDriver) ForwardGuestAgent() bool {
       // if driver is not providing, use host agent
       return d.VSockPort == 0 && d.VirtioPort == ""
}

So now the old code works again, at the cost of making the interface a bit more muddy...

@afbjorklund afbjorklund marked this pull request as ready for review November 25, 2023 11:14
@AkihiroSuda
Copy link
Member

CI is failing

Make both vsock and virtio explicit, instead of hardcoding
internal filenames in the agents. Fallback to to the unix.

Signed-off-by: Anders F Björklund <anders.f.bjorklund@gmail.com>
Copy link
Member

@AkihiroSuda AkihiroSuda left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks

@AkihiroSuda AkihiroSuda added this to the v0.20.0 milestone Dec 25, 2023
@AkihiroSuda AkihiroSuda merged commit 0e0ec94 into lima-vm:master Dec 25, 2023
24 checks passed
@afbjorklund afbjorklund removed their assignment Apr 9, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants