Add support for meeting apps

[pokey]

The problem

We'd like to have better support for controlling apps such as zoom, teams, google meet, discord, etc.

For example, if I am in a Zoom meeting, then I should be able to say eg "meeting mute" from anywhere in my OS and Talon will know to execute the proper controls to mute the Zoom meeting.

Extra credit

We may want some actions / commands that can be used to sleep talon and unmute meeting at the same time, and vice versa

The solution

To this end, we'd like to add the following:

• A tag called eg meeting that enables various commands for controlling a meeting
• A set of action declarations for muting, unmuting, focusing the active meeting, leaving the meeting, etc
• A set of Python / talon files providing action implementations for the above actions for various combinations of OS and meeting app
• Code that will activate the correct context for the currently active meeting, even if the app is in the background

Notes

We might use these files as a starting point, keeping in mind there are no generic actions defined therein, and it uses some mac-specific / slightly unreliable techniques

Zoom on MacOS

Also note that for Zoom on MacOS specifically, there are 3 obvious ways to control the app:

• Global keyboard shortcuts. Unfortunately the user has to set these up manually
• Pressing the buttons using AX. Unfortunately these disappear unless the user configures them to stick around
• Using AX to press the button at the top
  This is probably the best bet

We should investigate the following:

• Can we inspect / press the menu items without displaying the menu?
• Can we reliably use ax notifications to handle race condition if we do need to display the menu

Don't use ctrl.key_press(app=)

If necessary, we should just focus the app and press the key

[richardmcmillen]

Not sure if this helps but a quick google search found some extensions which set up global shortcuts to mute google meet which is a possible avenue as well? It would be nice to not have to install more extensions though.

https://github.com/mattsimonis/meet-mute/blob/main/ext/manifest.json

[ghouston]

MS Teams recently added a "Temporary Unmute" which is just a push-to-talk keyboard shortcut. https://support.microsoft.com/en-au/office/muting-and-unmuting-your-mic-in-teams-17886394-9a9a-4f04-b4cc-e46589408b28

Unfortunately they picked CTRL+Space (Win) or Option+Space (Mac) which may conflict for some people. I wish they had made this configurable. Its also unfortunate that this shortcut only works when teams is in focus. It really needs to be a global shortcut.

I will experiment and see if I can incorporate this push-to-talk into knausj_talon's apps/teams/teams.talon soon.

I usually spend 4-6 hrs a day in Teams either meeting or pair programing. I've found toggle controls undesirable because they require keeping up with state (mental load) or checking state (manual delays). Instead direct commands easier e.g. raise hand, lower hand vs toggle hand. Until this update, Teams only had a toggle mute. It was a really bad experience. Talon would have to ensure teams had focus then attempt to send the toggle shortcut. Frequently teams got into the wrong state (e.g. having both teams and talon listening) So I stopped using Teams toggle mute/unmute via talon. Instead I just mouse click the mute (vertical mouse isn't a problem for me).

Currently when meeting, I use earbuds so talon cant hear the meeting. In meetings I use the mouse to toggle teams and talon (via talon-hud), Its not ideal. When pair programing, my teammates are used to me speaking talon and cursorless commands unmuted.

[phillco]

One note is that Discord appears to expose some sort of local Unix RPC/websocket API [1], which you can used to read the mute state, so you could build something smarter than just trying to toggle both at the same time (e.g. having Talon always mute if Discord is unmuted, etc). I can definitely attest to the fact that having to manually juggle both creates a lot of mental stress.

[1] https://discord.com/developers/docs/topics/rpc, although I'm a bit confused about the "closed beta" notice because I was able to register an app and get a quick prototype working.