Timer support?

[thediveo]

I would like to implement (macro) key repeat and long key presses. Any chance for some kind of timer support?

[Gadgetoid]

It's certainly possible.

I'm torn between using

1. a tick function that you can place any time-sensitive code in and use global variables to control the state of, or:

2. supplying the ability - ala Javascript - to call setTimeout and setInterval to register a function to be called.

I'm somewhat in favour of the latter, since it's a little easier to reason about and compartmentalise than relying on one single tick function to do the heavy lifting. That said, a tick function would allow a co-operative multi-tasker to be implemented - more or less- which would be effectively the same as 2. So maybe:

3. do 1. and use it to write the functionality for 2.

[thediveo]

Personally, I would opt for no. 2, with the capability to cancel. That would allow different use cases:

• cyclic timers
• one shot alarms
• re-triggerable alarms, i.e. watch dogs.

My immediate use case would be the second, with cancelling before the alarm goes off ... based on my SHIFT key use case.

Implementation-wise maybe we can use coroutines for running a socket-based sleep using luasocket, and then dispatch from there?

[thediveo]

It would be beneficial if a timer library (module) could be used not only on the Keybow firmware (Pi), but also on a host system, to ease development and testing.

[dglaude]

Related and unrelated, but if I understand the answer to the following, I could have an opinion on timer...

How many instance of Lua are running in KeyBow?
a single one, one per key, one per key stroke, one but multiple Lua function can run concurrently

If I press key 00 and without releasing key 00 and next press key 01, does the treatment of key 01 wait for the end of the treatment of key 00... or does the Lua treatment of key 01 start as soon as pressed, even if key 00 "macro" is still running.

If I press repeatedly on key 00, very fast, faster that the time it take to process key 00 in Lua, does the new call wait and is queued, or does the second press (or release) is send in parallel with previous treatement.

I can imagine things being very messy if the Lua code is more complex than a simple mapping, and start to take time, or use state information for various behaviour. It become a very dangerous place to write code with many things running in parallel.

I think I will write some test script to see in practice how it seems to behave.
And the funny things, is that I could not figure out the answer by reading the C code the last time I did, just because the answer is maybe in Lua, or in between C and Lua.

Thanks.

[thediveo]

You can see how Keybow key handling work from the file keybow.c. Simplified, it sits in a loop, scanning inputs (key input points), scanning for changes, and then calling the Lua key handlers that need to be called one after the other. Plain and simple, no parallelism. Your key here is starting from main() and then find the scanner and dispatcher loop there. It runs as the main thread. The only other thread in keybow.c that I can see is the one that automatically updates the LEDs from a bitmap, if not disabled.

Everything in perfect sequence; that's why my multibow can be kept simple in places: simple key event handling one after the other, even when pressing multiple keys. Pressing and releasing the same key are two unrelated events. If you managed to press and release a key in less than 1ms, the current Keybow software would just miss some key state changes, but nothing will pile up. Sweet and simple, very straightforward.

[Gadgetoid]

I've learned the hard way to be very wary of threads, so Keybow is- indeed- sweet and simple.

Any cooperative multitasking would be managed in the same loop that scans input keys and dispatches events, and any long-running process (whether it's a timer event or a key press/release) will block further input until it completes.

Depending on what overhead it might introduce I think the most flexible approach is probably to have the main() loop call a Lua funtion: tick.

What you put in this function would then handle the dispatching of timers in whichever way you see fit.

[Gadgetoid]

Okay I've added a call to tick in 0cce1a4

If the first attempt to call this function fails (ie it does not exist in your lua file), it'll ignore it for future attempts. This is because I was encountering a segfault with rapid successive checks for an existing function in lua. Since I only need to check it once, though, this seemed like the easiest way to accomplish that without extra API surface.

This function is passed the time in milliseconds since the epoch- could probably adjust this to be script runtime for sanity. In fact- 9657447

[Gadgetoid]

If you want to play with the timer, you can grab a zip of the media-keys branch, or just replace your keybow executable with the one from the sdcard folder in that branch - https://github.com/pimoroni/keybow-firmware/tree/media-keys

[bertlea]

I assume the timer can also allow a delayed action similar to setTimeout in javascript. If so, can someone help to show a sample code that, for example, when the key_00 is pressed (i.e. in the handle_key_00() function, how to use the tick() function to clear the LED lights after 2 seconds (i.e. by trigger a call to keybow.clear_lights() after 2 seconds)?

[thediveo]

Somehow I don't see how to calculate a suitable alarm/trigger time in ms when Lua has no function to get the current time in ms, but only with seconds resolution. So I cannot calculate a sensible now+500ms to repeat a key, but I need a time in ms in order to compare against the current time in ms for every tick(). Did I overlook something? Note that os.clock() * 1000 doesn't work in this case, because it's too coarse.

[thediveo]

Hm, an ugly hack would be to store away the time given to tick() as the current time for calculating relative alarm offsets... 🤔 And from there on, alarm handling would use a priority queue. From what in I've seen, Luaists tend to misuse tables as priority queues... 😳

[thediveo]

🎉 branch feature/time of multibow now has a fully working timer framework on top of tick(t) in place. So we have the elements discussed above in place, that is, 1 (Keybow) and 2 (Multibow). At the moment, there's a new demo "glow" multibow layout with key #0 slowly ramping up and down its brightness under timer control. (Asynchronous key tapping is lacking atm, but now within reach.)

This feature branch of Multibow now supports:

• multiple timers,
• timer after x ms calling user's trigger function,
• timer every x ms calling user's trigger function,
• canceling timers,
• per keymap activation() and deactivation() callbacks in order to set up or tear down timers as keymaps get switched in and out.

When do you plan to release your tick() support to the main branch and an official release?

[thediveo]

Sifting through Keybow's C code, I now notice another peculiar thing: sendHIDReport() uses a usleep(1000) before it returns in order to throttle things in the synchronous case. This will cause tick()s to be missed, so it slightly stutters. Probably not exactly an issue, but has to be taken into account. More so with sending longer text and tapping a key, as this uses multiple usleep(1000)s.

A l_send_text() can delay up to 4ms for each character in the text to type, if there is a need for pressing and releasing SHIFT, and then 2ms for pressing and release the key to be sent itself.

The bottom line here might be that I need to monkey-patch some of Keybow's (intermediate) lua functions related to tapping keys, text, and modifiers in order to make them useful again in combination with timers: that is, these functions need to be monkey-patched in order to add their args to a queue which is then processed piecemeal-wise in the tick() handler, careful to not block (too extensively).

[Gadgetoid]

Sorry for not keeping my eye on this- I’ve been super busy with, admittedly, somehow more exciting stuff!

If you’ve got any suggestions for what I could change in the code to make your life easier, I’m all ears.

Unfortunately the HID report is set up to send on any key change to avoid having the user manually call a “send HID report” function from Lua, but I don’t believe there’s any reason why we couldn’t make this an option for more advanced use-cases so we can avoid some of the delays.

That said- I’m not sure if Shift and a key can be sent as part of the same HID report and not result in some kind of race condition. Worth exploring though!

The other killer is that- while it’s entirely possible -I really want to avoid introducing any threading here. Since any typed sentence is likely to be blocking anyway I don’t think there would be a benefit to threading and queuing HID output- sadly a choppy timer (which could very easily miss scheduled events and be prone to being inacccurate) is probably a better solution than timer events firing mid sentence and causing weird side-effects.

I’m happy to release this ASAP (in practise sometime this coming week) if you’re happy with how it works so far, and we can see how to improve it from there.

I’ve got a sheaf of design documents and specs on my desk for a MIDI controller which will probably need many of the same features- so it’s tike well spent.

Awesome work- by the way!

[thediveo]

While not ready yet, I'm slowly working on asnchronuous and cooperative key and modifier functions in Lua. This way, the existing API including its delays can be kept as is, and the new keybow asnchronuous functions on top would experience jitters, but would be okay with that. I'm trying the old route of cooperative tasking, so no threading. That means that releasing your designated design as is looks great.

If necessary, later improvements can be made when I know better what small set of additional Keybow API Lua-C functions without delay would be benefiting, as I"m now breaking up those things requiring delays to be done in separate ticks. If that works out as expected, I can even monkey-patch the pure Lua key functions to be rerouted to the cooperatively ticking ones...

I'm astonished that 1ms delays are even already enough, but would have expected needing larger delays. Anyway, my idea is to keep the API as is, and then spread longer key operations over multiple ticks in the Multibow layer. This also avoids the critical timing parts you mention. And it doesn't change things for people using and programming Keybow as originally designed.

Multibow is a crazy expedition in running a full blown OS with a scripting layer on top inside a USB keyboard. We're so decadent here, we're just the perfect example to those extremists on YouTube how rotten we are. manic laughter

[Gadgetoid]

Serial IO, keybow_get_millis() and file IO operations in progress: https://github.com/pimoroni/keybow-firmware/tree/file-and-serial-io

Note: The latter requires a new initrd to mount /boot as rw

[Gadgetoid]

I've had success with writing to /boot, which is great. Still pushing on the serial front- I have it working, but the timeout on serial read is large enough that it breaks keyboard input pretty horribly at the moment. I may either have to thread the serial port and queue the input, or reduce the timeout to the smallest possible functional value. I'm trying to avoid threads if I possibly can.

[thediveo]

For multibow, I've now implemented sending USB keystroke piecemeal-wise one after another with each tick(). Under the restrictions outlined above the tick jitter should be within 4 ticks, if I'm not mistaken. Many strokes will run within two ticks instead of one, due to the sleeps discussed above.

Additionally, I've designed a new and hopefully cool keystroke sequence API that is heavily inspired by "chaining" as found in busted's luassert, et cetera. This API schedules the keystrokes to be send in the tick background (no multithreading). So you can now write key sequences as:

mb.keys.tap("keybow").tap(keybow.ENTER)

Need ^ENTER instead?

mb.keys.tap("keybow").ctrl.tap(keybow.ENTER)

Want to repeat yourself?

mb.keys.times(10).tap("keybow is GREAT")

Okay, let's add a final ENTER, but we want to do it in a single chain, ensuring that we repeat only tapping the "keybow" text, but not the ENTER:

mb.keys.times(10).tap("keybow").fin.tap(keybow.ENTER)

Insert delays:

mb.keys.tap("keybow").wait(500).tap(" is").wait(500).tap(" GREAT")

[thediveo]

@Gadgetoid Would it be possible to officially release a new (0.3?) version of the Keybow firmware including the new tick() support? Then I could follow up with an official new release of my Multibow which now does things based on time, such as detecting a long-press SHIFT and sending USB keystrokes via a "background" send queue without blocking everything else. But before an official Keybow firmware release it would not make sense for me to release the new Multibow features.

[thediveo]

@Gadgetoid ping ... Any chance of a release?

[thediveo]

@Gadgetoid I'm slowly loosing any faith in Pimoroni now. I understand that the company has a lot of projects going on simultaneously, and that this doesn't seem to be a high priority project. That's fine. But since I've invested quite some time of mine to actually make good use of the feature I asked for, I'm now really p.o. (is there an official Unicode glyph for this yet?) with no official release from Pimoroni even months after a poc was done and I showed that it is useful, so other buyers of Keybow hardware can use my software in which they expressed interest.

I don't have the resources to keep and maintain a fork of the base firmware, so this looks like I'm now flogging the dead Pimoroni horse here. Next time, I will be more reluctant to recommend any hardware and its software to anyone, as this is a game of hit and miss, but far from at least a 50:50. I consider myself burned enough here and even consider archiving my own project based on a dead feature that Pimoroni doesn't bring onto master releases. I also really consider to close this issue if there's still no reaction from Pimoroni, as they sell the hardware and then quickly loose interest in the software. This is now no different from any cheap Chinese seller throwing their hardware on the market, then walking away. Because it's "open source".

[Gadgetoid]

I'm sorry to let you down. While it's not an excuse, I think it cannot be stressed enough that when you say "they" you're referring to just "me." Pimoroni isn't a huge corporation with dozens of engineers- I'm almost singularly responsible for 117 public repositories and involved in many more projects in the pipeline. Couple this with the fact I've had a new baby since the 15th of March and have also been more than a little distracted by our 32blit project- well it adds up to things like this, while very much still being of interest, taking a not insignificant amount of time to get 'round to.

None of this, however, excuses not getting back to you for three months and for that I'm sorry. Since it's just me I have a practise of leaving PRs (usually finished code) languishing for a couple of weeks so I can look over it again with fresh eyes and catch any silly errors or omissions.

Okay- for not an excuse that sure was a lot of excuses. I'm looking into getting this fix merged and into a release (and not just because of Keybow Mini!) now.

[jezdez]

Hi folks! Reading up about Keybow/Multibow (🤯❗), I'm trying to figure out what is missing for this ticket to be closed.

The tick function got implemented and released as part of 0.0.4 (May 20), the jitter doesn't seem to be a huge deal (right?) since Multibow's timer/tickjob system will just go through the keystrokes synchronously and doesn't use threading. IIUC there is no need for a custom base firmware needed anymore.

@thediveo @Gadgetoid So -- with much respect for your diligence in creating these niche software projects (and yet very exciting for some, like me) -- what's the status of this? Is there anything you need help with?

[Gadgetoid]

@jezdez thank you!

Judging by the message on the multibow GitHub repository it looks like faith was, indeed, lost and it has been abandoned/archived - https://github.com/TheDiveO/multibow

A shame! It's really cool to see third-party software projects based around our projects, and I'm really sad to see it die by the wayside- especially as, for better or worse, I'm largely at fault for dropping the ball here (circumstances notwithstanding).

Maybe @thediveo will return in time? Or another maintainer will pick up this project.