Initial meeting tag support #1172
Conversation
apps/zoom/zoom.py
Outdated
| app.notify("Mute meeting") | ||
|
|
||
| def meeting_unmute(): | ||
| app.notify("Unmute meeting") | ||
|
|
||
| def meeting_exit(): | ||
| app.notify("Exit meeting") |
There was a problem hiding this comment.
@nriley here's where you'd put the implementations of mute / unmute / exit. Need to decide whether we're comfortable using accessibility here. Also prob need to make the tag mac-specific if we use accessibility
apps/zoom/zoom.py
Outdated
|
|
||
|
|
||
| def is_zoom(app): | ||
| return app.bundle == "us.zoom.xos" |
There was a problem hiding this comment.
Todo: make this work on Windows
apps/zoom/zoom.talon
Outdated
There was a problem hiding this comment.
Maybe we should rename this file? Because this file name should really be for when Zoom is focused
|
@nriley feel free to push directly to this branch, or open a PR against it if you can't for whatever reason |
Also adds user.meeting_is_muted. Tested in US English, German and Simplified Chinese localizations.
|
Instead of looking at the menu (with a different title per-language) I can recognize the shape of the "you are muted" icon and determine the mute status this way; it worked across the localizations and meeting window layouts I tried. Still might be some issues (even ignoring the Talon bug) in recognizing the meeting window title. If we just wanted to support toggling mute, we could do it without accessibility. I put in a partial workaround for the bug by focusing another app and back to Zoom. I don't think actually checking the 2nd time should help in most cases but it should help Talon to notice the window for next time. The problem is still though that the tag won't activate, so the voice commands don't do anything (other than "media mute" as @pokey experienced yesterday :). One thing to note while I was testing, which in part inspired my two-microphone solution: it seems that if you mute audio in Zoom, you can't talk to Talon either on the same microphone (works fine with another microphone). Another action that might be helpful would be to allow you to focus the active meeting window since I've certainly had cases where it gets "lost". This wouldn't require accessibility either (at least for Zoom), since my window filtering just uses the title. |
|
Here's a few options of more robust workarounds for the problem of meeting commands not being active since Talon hasn't "seen" the meeting window yet. We could have the meeting commands active if any of the meeting apps are active, using "user.running". Then in the default implementation for "meeting mute", etc., we could "poke" each of the apps (via accessibility to find the window) to see if we're failing to notice an active meeting window. Another option is to poll windows in active meeting apps by accessibility (which I may end up doing in my own repo, so I can accurately switch microphones) and "force" Talon to notice any it hasn't found. I do that in one of my private repos in a specific circumstance, but I'd be a bit worried about doing that by default. What would be great is if the bug could be fixed and I could rip out all these workarounds :-) |
Could not figure out any way to determine mute status that isn't language-specific or talon.experimental.locate, so just notify if we can't figure it out. "Meeting mute" will toggle mute/unmute regardless.
|
Added Teams support — due to its use of Electron this is more complex/asynchronous. Given Teams is pending a rewrite I'm disinclined to spend much more time on this particular implementation, but it at least works for me. Talon seems to be more reliably "see" the meeting window in Teams, so I didn't include the same workaround I did for Zoom. |
This allows you to react to starting/stopping a meeting by overriding these actions, and using actions.next to invoke the default implementation. Also, remove redundant check for a meeting window in Teams and Zoom on Talon launch.
|
Refactored a bit to put the Here's where I override the |
… launch or module reload.
Does this work even when the Zoom meeting controls are set to auto-hide? |
Yes, I think so — the icon I am matching shows up even if the controls are hidden.
If you can find cases where the above mute icon does not appear, then I can implement checking for the mute menu item in addition (like I do on Webex) — unfortunately this will not be robust to non-English UI languages. |
Hmm, that looks like the one that'd show up on your self-view in the gallery view? Wouldn't that:
|
|
Might be. For obvious reasons (only one of me) it's been a bit hard to test. If you have any other thoughts on how to detect when Zoom is muted (particularly in a way that does not require textual matching of localized UI elements) happy to hear them. |
Unfortunately this will break in UI languages other than English.
|
OK, I've moved off my slick-but-broken Zoom mute detection to a method that works in meetings with others joined (yeah, yeah…) and with Self View disabled but is dependent on the UI being in English. More testing appreciated! |
|
Does the Zoom menu bar item need to be enabled for that to work, or does it use the menus for the meeting window? |
|
Yes, it does not use the menu in the top right corner of the screen but instead uses the application menus even if the application is not active. |
… meeting. Also better explain why each identifier is used - the same window is shared with "pre-joining", but if you are switching between the smaller and larger versions of the meeting window, the first identifier exists in both. (Suspect the prior identifier is no longer being used as of Teams 1.6.00.11156.)
|
Since I upgraded either to a recent Talon beta or macOS (or both), this has become incredibly unreliable. (It's blocked on the window activation callback issue in core Talon, but it was at least usable previously.) Will work on fixing it as I have time. |
|
talonvoice/talon#606 is fixed in the latest beta! I tested Zoom and it seems reliable. |
win_openevent doesn't fire until new meeting window is blurred then re-focused talonvoice/talon#606