Tracking issue for MacOS support

[feschber]

Tracking issue for MacOS support

This issue is for tracking MacOS support and the current development status.

TODOs

• Make it compile and get frontend up and running (macos: enable running lan-mouse on macos #42)
• Input Emulation
• Keycode Translation
• Input Capture

[feschber]

#42

[feschber]

Mouse input emulation works, for keyboard emulation the keycodes need to be translated.

[feschber]

#81

[kiwiz]

I took a quick look at Input Capture (I'm primarily interested in the ability to release input). It looks like CGEventTap offers the necessary functionality, but the interface is callback-based. There doesn't seem to be an easy way to marry that with the (polling-based) EventProducer paradigm in the code.

[feschber]

The easy way is to just spawn a thread. And as there doesn't seem to be any non-blocking systemcall for input capturing, this is probably the only way.

What I've been considering, is giving the producer access to the sender_tx channel, which passes the events to the udp_task, directly.
The event-consumer API does not have this problem, since the async consume() function can simply call synchronous code but considering that windows and macos both probably need a different thread for input capture, it might be good to do that at some point.

For now, if you want to work on this, I suggest you just spawn a thread and send the events via a tokio::sync::mpsc::channel to the macos_producer.

[rohitsangwan01]

@feschber really awesome project, just mac is missing, any plan or date to start work on Mac Input capture ?
i am working on a project which might depends on this project, so it would be really great to have Mac full support as well

[feschber]

@feschber really awesome project, just mac is missing, any plan or date to start work on Mac Input capture ?
i am working on a project which might depends on this project, so it would be really great to have Mac full support as well

I can't give any estimate date but encryption and MacOS Input capture are next. Be aware the input capture API is a bit specific to this use case in that it locks the cursor and starts when the screen area is exited (this can not be done manually right now).
I was planning to extract this to a separate crate at some point for use in other projects.

[rohitsangwan01]

@feschber this implementation from Synergy might be helpful or inspiration for Mac support,
Also do you think it would be possible to use lan-mouse as a library for another project ?

[mkdalynn]

Hello,
I'm trying to get this working but am having trouble. I have lan-mouse running on both machines (pop-os 22.04, and mac-os 13). I've added a client to both sides and see it's found the host, but the mouse wont move screens. Input is connected to the pop-os machine.

• tried changing ports to 24800, which previously worked with Barrier
• tried using the local ip of each machine as the hostname

I'm not sure if I have something misconfiguration or if there is something else going on.
Happy to provide any other info.

[feschber]

Hello,
I'm trying to get this working but am having trouble. I have lan-mouse running on both machines (pop-os 22.04, and mac-os 13). I've added a client to both sides and see it's found the host, but the mouse wont move screens. Input is connected to the pop-os machine.

• tried changing ports to 24800, which previously worked with Barrier
• tried using the local ip of each machine as the hostname

I'm not sure if I have something misconfiguration or if there is something else going on.
Happy to provide any other info.

Be aware that Gnome < v45 won't work. I believe pop os does not ship gnome 45 yet so you're out of luck here unfortunately (until the next popos release)

Your log should show an error saying that only the dummy input capture is active.

[mkdalynn]

Be aware that Gnome < v45 won't work. I believe pop os does not ship gnome 45 yet so you're out of luck here unfortunately (until the next popos release)

Your log should show an error saying that only the dummy input capture is active.

Ah, I missed that dependency. Thanks for pointing it out.

[feschber]

Be aware that Gnome < v45 won't work. I believe pop os does not ship gnome 45 yet so you're out of luck here unfortunately (until the next popos release)
Your log should show an error saying that only the dummy input capture is active.

Ah, I missed that dependency. Thanks for pointing it out.

Yeah it might be a good idea to add a hint to the ui as well since this is not entirely obvious.

[feschber]

Also do you think it would be possible to use lan-mouse as a library for another project ?

As I said, I am planning to separate the input emulation and capture into separate crates. There are a few limitations (by design):

• lan-mouse uses scancodes for keyevents. To use them for textinput an additional library is required to translate them to text (and the other way). Probably xkbcommon. But this is out of scope of this project.
• Input Capture does not have a way of being enabled manually. This is partly because of limitations imposed by the input capture portal and partly also because of the way the layer-shell backend is designed (it simply spawns an invisible layershell surface).
• The Event type used internally could use some improvements as it is currently tied pretty closely to the rest of lan-mouse.

So yeah depending on the project it could certainly be used as a library.

[rohitsangwan01]

@feschber ohh that would be great,
Not sure if this is the correct place to describe, Am working on Similar implementation but with Android and IOS and got some success as well, which expands Apple's universal control functionality to Android and IOS as well, here is a demo of older version of this software
but my plan is to build a proper cross platform or atleast support LanMouse as server, if there are any documentation of protocols to implement a third party client app for lanMouse, and if you will split in different crates, then it would be really easy to integrate lanMouse support in Flutter with flutter_rust_bridge

[feschber]

Interesting! I did a prototype android app before (only for Android input capture, not emulation) and it worked quite well.
The network protocol I'm currently using is a custom one and bound to change. Because of that it is also not yet documented.

If you want something to work with, heres a brief overview:

The types are encoded as two bytes - one for the event type (pointer / keyboard, etc.) and one for the event sub-type (e.g. motion / button).
The rest of the data depends on the specific event sub-type.

You can check out https://github.com/feschber/lan-mouse/blob/main/src/event.rs for the current implementation.
It is basically just a type-length-value (TLV) protocol with the length omitted.

Admittedly its a bit ugly at the moment but I plan to rework it anyway so dont spend too much time on it.

In the future, I want to include variable length integer encoding for bandwidth savings and remove some redundant data I was not sure is needed when I started out.
There is also some Event types that are still missing (notably touch input and high precision scroll events but maybe also gamepad events at some point).

[rohitsangwan01]

i see, ok i guess it would be better to invest client integration time after refactoring or atleast initial documentation when protocols will be stable, Thanks for the detailed info

[feschber]

#131

[isleshocky77]

I don't want to make a new issue if this is already known; however, it also may be related to #91 . I'm able to move from the server (Ubuntu 24.04) to the client (mac os); however, I'm unable to double-click the mouse on the client's screen.

Everything else seems to be working so far (minus the ability to come back without using the keyboard shortcuts).

[livexia]

There is a problem when I test with two mac, I found that shift won't work while using another machine's keyboard.

shift + m input just m not M