Setup() lifecycle method in Manual Routine method.

[boscs]

Hi bxparks,

To start, thanks for your work, you are amazing. Your lib is a good answer to a lot of problems I was manually solving before.
If I were to design it by hand, I would however add a setup() method that can be overwritten to initialize resources while starting the CoroutineSheduler (with CoroutineSheduler ::setup()).
If I'm not mistaken, this could be done here :

 void setupScheduler() {
     // start of mod
    for (T_COROUTINE** p = T_COROUTINE::getRoot(); (*p) != nullptr;
          p = (*p)->getNext()) {
        (*p)->setupCoroutine();
      }
    // end of mod
      mCurrent = T_COROUTINE::getRoot();
}

This would on the other hand require the addition of the empty lifecycle method setupCoroutine() in the Coroutine class in order for it to be overwritten if needed.

Tho we can emulate the behavior I propose by inserting code before a loop in the coroutine body, I think this would better separate concerns in the code.
This could be useful to open pins or other resources without sacrificing readability.

Let me know what you think, I can submit a pull request if you think that this would be a good thing :)

[bxparks]

I have not personally needed a Coroutine::setupCoroutine() in anything that I have written, but I agree that this could be a useful feature. Just out of curiosity, where and how would you use it?

My initial concern about this feature is that it increases flash memory consumption. The Coroutine::setupCoroutine() method would have to be virtual with an empty default implementation. If an app does not use this feature, adding this virtual method increases flash usage by 4 bytes per Coroutine class, and static memory by 2 bytes per instance (edit: maybe this is 2 extra bytes per coroutine class?) on AVR processors. That is minor so I think we can live with that.

But as soon as an app uses this feature and overrides the Coroutine::setupCoroutine() method, it seems to increase flash memory consumption by 50-60 bytes per coroutine on AVR, and 30-40 bytes per coroutine on 32-bit processors. (This is with the overridden setupCoroutine() doing basically nothing.) The looping code in CoroutineScheduler::setupCoroutines() adds another 20-24 bytes on AVR. A virtual dispatch on AVR seems to cost about 14 bytes per invocation.

So I created an implementation where CoroutineScheduler::setupCoroutines() is a separate method, instead of including it in CoroutineScheduler::setup(). This allows an app to avoid paying for this feature (other than the 4 bytes of flash per coroutine due to the empty virtual method) if it does not need custom setupCoroutine() methods.

You can call these methods directly from the global setup():

MyCoroutine1 coroutine1;
MyCoroutine2 coroutine2;

void setup() {
  coroutine1.setupCoroutine();
  coroutine2.setupCoroutine();
  CoroutineScheduler::setup();
}

Or you can use the CoroutineScheduler::setupCoroutines():

MyCoroutine1 coroutine1;
MyCoroutine2 coroutine2;

void setup() {
 CoroutineScheduler::setupCoroutines(); 
 CoroutineScheduler::setup();
}

Please take a look at the setupCoroutine branch and look for the Coroutine::setupCoroutine() and CoroutineScheduler::setupCoroutines() methods. If this looks good to you, I will merge and push out a new release.

[boscs]

Hi,
thanks for your fast answer!

where and how would you use it?

I'm working with my colleagues on a new kind of hybrid rocket engine :)
we're actually using, among other things, a teensy 4.1 as a control board. However, with separating concern in mind, I build each component in its own file. Those often need to setup pins for basic IO and/or setup protocol for communication (not really another state machine, but something like it) and lastly the calibration logic is also executed once at initialization. In addition, this guaranties that the setup for each component is done after the main init but before the start of the loop.

My initial concern about this feature is that it increases flash memory consumption.

I have to admit that I did not think of that. The teensy is such a comfortable platform (memory wise) that I've been desensitized to "small" memory costs. I should work on that hehe.

Your implementation seems very good to me and its "opt-in memory consumption" is a good thing for low power devices (AVR).
Just one question : Why do you add a level of indirection for it ? CoroutineScheduler::setupCoroutines() only calls CoroutineScheduler::setupCoroutinesInternal(). Is it better for something, or are just planing for the future ?

Thanks again for your lib :)

[bxparks]

That sounds like an interesting project. And the Teensy 4.1 is a beast of a microcontroller. I bought one of those a few months ago, but I haven't had the time to do anything with it. Are you using AceRoutine for anything real-time related? Because this library is cooperative, not preemptive, so if some of your coroutines must respond within a fixed amount of time, it may not be the right library.

With regards to the extra level of indirection in CoroutineScheduler, that's a fine question and I documented the reasons into the source code, which I reproduce below for your entertainment. It's a bit of premature optimization combined with YAGNI, with backward compatibility preventing me from changing it:

/**
 * Class that manages instances of the `Coroutine` class, and executes them
 * in a round-robin fashion. This is expected to be used as a singleton.
 *
 * Design Notes:
 *
 * Originally, this class was intended to be more substantial, for example,
 * I imagined that different scheduling algorithms could be implemented,
 * allowing coroutines to have different priorities. To ensure that there was
 * only a single instance of the `CoroutineScheduler`, a singleton pattern was
 * used. The `getScheduler()` method fixes the problem that C++ does not
 * guarantee the order of static initialization of resources defined in 
 * different files.
 *
 * It seemed cumbersome to require the client code to go through the
 * `getScheduler()` method to access the singleton instance. So each public
 * instance method of the `CoroutineScheduler` is wrapped in a more
 * user-friendly static method to make it easier to use. For example,
 * `CoroutineScheduler::setup()` is a shorthand for
 * `CoroutineScheduler::getScheduler()->setupScheduler()`.
 *
 * As the library matured, I started to put more emphasis on keeping the runtime
 * overhead of the library as small as possible, especially on 8-bit AVR
 * processors, to allow a Coroutine instance to be a viable alternative to
 * writing a normal C/C++ function with complex internal finite state
 * machines. It turned out that the simple round-robin scheduling algorithm
 * currently implemented by `CoroutineScheduler` is good enough, and this class
 * remained quite simple.
 *
 * With its current functionality, the `CoroutineSchedule` does not need to be
 * a singleton because the information that stores the singly-linked list of
 * Coroutines is actually stored in the `Coroutine` class, not the
 * `CoroutineScheduler` class. Because it does not need to be singleton, the
 * `getScheduler()` method is not really required, and we could have just
 * allowed the end-user to explicitly create an instance of `CoroutineScheduler`
 * and use it like a normal object.
 *
 * However, once the API of `CoroutineScheduler` with its static wrapper methods
 * was released to the public, backwards compatibility meant that I could not
 * remove this extra layer of indirection. Fortunately, the none of these
 * methods are virtual, so the extra level of indirection consumes very little
 * overhead, even on 8-bit AVR processors.
 */
template <typename T_COROUTINE>
class CoroutineSchedulerTemplate {

[boscs]

That sounds like an interesting project.

It is indeed very interesting ! We have always a lot of new random issues, so everyday we are learning :)

And the Teensy 4.1 is a beast of a microcontroller.

To me, it is maybe even the last type of controller I buy for general usage. It can freaking do anything ... it's so amazing! Plus lots of goodies are included in such a small form factor. The only drawback to me is energy consumption. But there's no such thing as a free meal ^^'

Because this library is cooperative, not preemptive [...] it may not be the right library.

Yes I get that, but on the other hand, all my "threads" are very short to execute except the one that has real time needs. Even that one is executed in less than a ms which seems small compared to my 10ms per frame budget. My goal is to let my teensy do nothing for 50% of the time, and gain this way a kind of real-time performance.

Thanks for the documentation, it is very interesting !

[bxparks]

Teensy 4.1 is probably powerful enough to run a real RTOS. Did you evaluate any of those by any chance? Don't get me wrong, I'm glad that you find AceRoutine useful. Just curious about pros and cons. I personally don't have any experience with RTOS. (It's on my long list of backlog projects...) All I know is that FreeRTOS seems relatively popular, and I think it's the underlying OS on the ESP32. A quick search gives me something like this: https://forum.pjrc.com/threads/53662-Teensy-4-RTOS

With regards to selection of microcontrollers, I find myself pulled into different tradeoffs, so will often use different microcontrollers. On the one hand, I want to see how small and power efficient I can make things, so I recently started playing around with the ATtiny85. It turns out we can squeeze in a LOT into 8kB of flash and 512kB of ram, if we work around some of the limitations and bloat of the Arduino programming framework.

On the other hand, I have many projects that require internet access, so that means the ESP8266 and ESP32. The Teensy boards don't have built-in WiFi as far as I know. I have some projects that require both WiFi and battery-power, and it turns out some ESP8266 boards have very good sleep modes. My battery-powered temperature monitor lasts 4-5 months on a set of 3xAA NiMH rechargeable batteries. And those ESP8266 boards are cheap as heck. I can get 10 x D1 Mini for the price of one Teensy 4.1. Hard to beat that.

My Teensy 3.2 and 4.1 boards are kinda caught in the middle between those 2 kinds of projects for me, so I personally haven't found much use for them so far.

Anyway, thanks for letting me know about your project. If you happen to discover other interesting uses of the AceRoutine library, I'd love to hear about them. You can use the GitHub Discussions section of this project. As an open source maintainer, it is so rare that I get positive feedback from users of my code, I always appreciate hearing about how my code is used. Good luck on your project!

[boscs]

Hey,
Sorry for the latency of my reply, I got busy and I don't use AceRoutine to schedule my personal time yet ;)

Teensy 4.1 is probably powerful enough to run a real RTOS. Did you evaluate any of those by any chance?

Yes of course, I mean, if the teensy couldn't run an RTOS... what could ?? However, the main goal of this kind of software is to provide consistency in the timing of the execution of your code. Do I really need this ?
I have only one task that has to be executed at prety regular intervals (control loop). But every other task are just some short monitoring/aquisition stuff + one is in charge of comunication. Comunication can be long with the serialization and RPC calls that could take a longer amount of code to execute but apart from that every single task (exept control) are just a few hundred of instructions long. In teensy world, they are at most a few µs long ? (I expect my control loop to execute at 100Hz to 200Hz max ! In addition the algorithm is adaptative to the time elapsed since the last correction so I don't fear any undefined behavior if a frame takes a little too long to compute.
So first I don't think I need a RTOS.
Second, I've found one port of FreeRTOS for Teensy4.0 (I'm on Teensy4.1... so compatibility issues ???) but it seems like a way bigger project than your thing ! what happens if there is nasty bugs there ? It would take days to isolate and reproduce a real-time bug caused by race-conditions in the OS that only show up for me. One big win for your lib is its size. less lines = less pain for me.
Third, I have the luxury to work in a single threaded environment. I love it. I have to admit that I like to isolate my memory in tasks because it lets me reason about it more easily, so threads are fun and all, but communication between them is definitely not. Getting the best of both worlds (lots of small tasks executed sequentially) is nice and, I think, sufficient for me. So I don't think I'll go the interrupt way.
Fourth, I wouldn't have built your lib the same way as you did, and that's what I like the most about it. It is svelte, elegant and built in a very efficient way. Even though I wouldn't have produced myself such lean code, I see myself contributing to it in the future. I understand your code, and I'm mostly able to navigate through it. This is definitely not the case with FreeRTOS. The thing is HUGE and there is lots of way to achieve the same thing. I prefer simplicity even more when it is more efficient.
Lastly, I want to be able to unit-test my tasks without having to simulate untimely interrupts or other realtime stuff.

... selection of microcontrollers, I find myself pulled into different tradeoffs ...

I feel you my friend ^^ Sometimes I don't really understand that such a diversity of board can exist. But then I think about how many we have in our engine alone.
(a colleague of mine is automating the water valves on his in-house plants, and he is using the same type of low power µc than you I think)

On the other hand, I have many projects that require internet access ...

FWIW, the Teensy4.1 comes with an included Ethernet port. it is not supposed to do Power over Ethernet, but you can modify the
thing to allow it I believe.

Anyway, thanks for letting me know about your project. If you happen to discover other interesting uses of the AceRoutine library, I'd love to hear about them. You can use the GitHub Discussions section of this project. As an open source maintainer, it is so rare that I get positive feedback from users of my code, I always appreciate hearing about how my code is used. Good luck on your project!

You're very welcome ! Thank you for offering your work to the community like that :)
Good luck to you to !