strong coupling of a simulator

[piotre13]

Dear all, I'm new to helics and i want to understand how can i perform the following cosimulation:

• simulator A takes as inputs (T_ext, P_actual) and produce as outputs (P_demand, T_in)
• simulator B takes as input (T_ext, P_demand) and produce as outputs (P_actual)
• simulator C sends every time step T_ext to the others

The cosimulation workflows should folow a precise order during a single timestep which is:
C sends T_ext to both A and B, A calculate a P_demand and sends it to B, B uses P_demand calculate P_actual and sends it back to A, A receives P_Actual calculate T_in and publishes it.

my questions are mainly two:

• how can i structure something like this using helics? I imagine there are several possibilities and i would like to know which can be the best one in your opinion;
• can you please tell me if there is a standardazied taxonomy to categorize this kind of situations or simulators with this kind of workflow?

Thanks a lot

[trevorhardy]

Welcome, @piotre13!

HELICS doesn't directly provide the fine-grained timing control to manage the data exchanges your envisioning at a single point in simulated time. There are two work-arounds I can imagine:

1. Do something we call "microstepping" where you approximate one point in simulated time as many points that are very close together (relative to the federates normal timestep size). For example, if your simulation steps at 1 second you can set up your simulators to do time requests on a millisecond scale. When the complex data exchange begins, all the simulators start requesting time one millisecond at a time until the final output has been produced and then move back to requesting time the next integer time in seconds. You'll need to look at your timing carefully to figure out which federates need to keep microstepping and which ones need to move one to the next second based on which data has been exchanged and which ones need to be exchanged.

   There's two flags that will help with this:

   • "only_update_on_change": only considers a change on an input (that will force a time grant) if the input changes. You can set a ("tolerance")[https://docs.helics.org/en/latest/references/configuration_options_reference.html#tolerance-0] value to define what "change" means.
   • "wait_for_current_time_update" - Set this on one federate to for HELICS to grant it time after all other simulators have processed at that time. I'd have to think through your timing more but simulator A may be a candidate for having this set.

2. Use iteration, allowing repeated data exchanges at a single simulated time. Generally we use this to allow simulators to reach a common value for a model parameter they both represent but you can use it in your case, too. We have an example that walks through how to use iteration when making time requests. I suspect you'll have to implement a little logic inside each simulator to determine what it needs to do on each iteration.

Regarding a "standardized taxonomy to categorize this kind of situation", I'm not clear what you're referring to, specifically. Do you mean some kind of notation or language that describes the data exchange graph? I happy to discuss more if you want to clarify.

[phlptp]

In general HELICS is designed to handle a wide assortment of timing and data patterns. Giving the user a lot of flexibility in how they manage the timing. There is no requirement that each federate use the same timing.

So in the case you described above. @trevorhardy highlighted a few of the capabilities.
So lets say you have everything at the same time steps.
Then you could just iterate A and B 3 times and you should end up with the result you desire.

You could set C to run on time steps just before A and B. Then set B to (wait_for_current_time_update). And iterate A twice to calculate each desired parameter. You have a cycle between A and B so you can't use the wait flag on both. This is probably what I would do first since C is a source only. Sounds like A would need to iterate regardless.

There is a no "taxonomy" definition that HELICS could use since the timing is near infinitely flexible and each federate has independent control of its own timing. Ordering taxonomies such as used in FMI or some other standards can be fit into the timing systems in HELICS, but they are subsets of what is possible, so it hasn't been productive to try to put any rigorous definitions to interfederate timing in HELICS. There are times when rigor is useful, others when it isn't.

[piotre13]

Dear @trevorhardy and @phlptp firstly thanks a lot for the prompt response! It is completely clear now what i can do to model this kind of data exchange. In my experience I've encountered a lot of models behaving like the example i've given. That's the reason of my second question on taxonomy, my curiosity was less related to helics itself but more on the theoretic aspects behind simulation and co-simulation. Thus, if you want, i would like to discuss a little bit more about theory. I've seen that simulation models (SU) can be categorized in many different ways based on different viewpoints e.g. the time advancement(1,2), thus i was wandering if you have any suggestion or knowledge on how to classify simulators based on their data exchange workflow. It seems to me that the example i've made advances regularly in time but also need some kind of event (the reception of a sequence of inputs) to perform, so also the time advancement classification of this it is a bit unclear in my mind. If you have any references to suggest for better understanding co-simulation and simulation world I would be grateful.
Second question is what is your personal opinion on models that behave as the example, do you think there is a good practice on model making? are they a typology of models that should be avoided? (eg. the simulator A could be divided into two simulator one for estimating and one for receiveing the actual P_demand, avoiding the iteration and making a unique direct workflow).

Thanks a lot for the patience, but I'm really interested on this platform and the related principles behind co-simulation.

Finally I have a last curiosity that yopu can ignore if it is already too much:
If I'm simulating two SUs that advance with different stepping (eg. A every 60s B every 15 mins) and i want B to receive all the 15 datapoints coming from A or at least an average value of them and not only the last data point how it works on HELICS.
Thanks a lot

[phlptp]

When we were designing the mechanics of HELICS we did dive into some of the theoretical considerations. What we ended up rapidly running into was the practical realities of trying to connect different simulators together really lend itself to being composed in a single theoretic model. Especially when considering the coupling of event driven simulations with regular time stepped simulators such as is seen when coupling a communication simulator like NS-3 with a power flow model. I do however believe that now that we have the nuts and bolts mostly figured out there is room for more rigor and theoretic considerations.

The main way we broke things down in HELICS was two models. The first being a value federate where real values (including units and data types ) are exchanged in a publish and subscribe mechanism. In this model the values have persistence and only a single value is valid at any given time. The second being packet/event driven mimicking packet based communication networks. In this model data being exchanged is packetized and queued, and can thus be delayed, modified, rerouted while in flight, and multiple packets can be sent in a given timestep. These systems can interact in any given cosimulation but it is useful to think of them separately.

When I give tutorials I usually break co-simulations approaches into two different mechanics. A master/client approach or a library approach. Which basically comes down to who has control of primary execution. In the master/client approach you can make certain assumptions about the components/ force them into certain modes of operation, and do a lot of fancy things with ordering, and timing. But it is a lot of work to get certain simulators into that approach, and then you are stuck using that master engine. HELICS chose the library approach which allows a lot more flexibility on the part of federates, but may trade off pure performance in certain situations and more complex internal mechanics.

On you last question, if I wanted to do that I would either capture the values in an endpoint. or data sink, which means they are queued and I have to interpret them manually, or use a microfederate which matches the more frequent time steps and just maintains a running average or just publishes every 15 minutes, then subscribe the slower one to that publication. I have had several situations where a single main federate had a bunch of smaller federates doing measurement computations or other small such things, and with hierarchical communications don't really add much to the overhead.

[piotre13]

Thanks a lot for your precious answers and consideration, I appreciate that. I will take advantage of this discussion thread to ask another question if it is ok.
When dealing with cascades of simulators, like in the picture I'm attaching. So when dealing with more than 2 simulators that have a precise order to be executed and exchange information in series before we can consider the time-step to be concluded, how can I handle it? Is there a proper way in Helics to be sure that each one receives the output of the previous simulator at the same timestep even if they all have the same period?

[trevorhardy]

Yes; @phlptp and I were just discussing this last week. As long as the data exchange dependencies are truly a cascade (B only depends on A, C only depends on B, and D only depends on C), then you can set the wait_for_current_time_update flag on all the federates except A. At a given simulated time t, HELICS will grant A time first and when A makes it's time request, B will then be granted time t and so on. This will happen at every simulated time and will work even if, say, C is asking for a time t+10 while everybody else is asking for time t; in that case simulated time t will be granted to A, then B, then D.

[piotre13]

Thanks I've tried using pub, subs and inputs, and it works but i think, if I'm not missing something, that it does not work with messages (endpoints), can you confirm it?

[phlptp]

basic endpoints are multidirectional so they can send and receive from all other endpoints. So just putting endpoints creates a different situation from the diagram above. So it doesn't work, in the sense that you can't create the above diagram using endpoints with a fixed periodic timestep. If you were to use targetedEndpoints and fix the targets then it should work similarly to the publications/inputs.

One other thing that has been done in the past is if federates do have data cycles but you do in want them to be strictly ordered, but still periodic you use the 'offset' property to add irrelevant delays to each time step. So if the period was 1 second, the first gets offset by 1 nanosecond, the second by 2 nanoseconds and so on. That can work smoothly when a nanosecond is not important and everything else is periodic. Then cycles don't matter.

[trevorhardy]

I haven't tried this specifically in any configuration so let me get that on my to-do list. When you say "with endpoints" are you only using endpoints?

As it happens, @phlptp and I were talking about this as well. If you just use endpoints in a vanilla configuration, since HELICS doesn't know until runtime who is going to send or receive the messages, it considers all federates with an endpoint to be dependent on each other. That is, at time t, HELICS doesn't know if A is going to send a message to be or B send a message to A.

BUT, if the federates have a communication pattern such that you know A always sends messages to B and B always sends messages to C, you can define a target for the endpoint. If you're doing your configuration via JSON, you can see this in our monster example JSON on the "Configuration Options Reference" page in the "endpoint" section. If you're using the API for configuration you use the helicsEndpointAddDestinationTarget() call. If every endpoint has it's targets defined such that you create a cascade of dependencies, you should be able to use the wait_for_current_time_update flag like you could for pubs and subs.

[piotre13]

Dear all, thanks again a lot for your previous responses they have been very illuminating, but still I'm confused and struggling when dealing with some specific cases. I'm attaching two example schemas for the two workflow situations I'm trying to implement. So each of the two workflows has two sims (envelope & hvac) with an iteration loop with a fixed number of data exchanges. then in both cases, envelope and hvac need to publish a final value once they have completed the fixed number of inter-steps they had to do in a single timestep. The only difference between the two is that in the first case they are also both receiving information at each timestep from the source-only weather sims, while in second case the only one receiving it is the envelope.
case1

case2

From this situation, i have still some questions :
1- for the iteration loop I'm still struggling to implement it properly to be sure that first envelope receives T_ext, then performs a calculation and outputs P_req, hvac receives P_req makes calculation and sends back to envelope P_actual. hvac has finished when publishing P_actual and so should also publish Electrical power, while Envelope as soon as it receives P_actual performs another calculation and publishes T_in and it has finished. Do you have any suggestions on how to properly structure the logic inside federates and the proper sets of timings and flags i should use?

2- my main problem is that this data exchange must be ensured and has a fixed number of ordered different steps that must be performed, there is no convergence check and I must be sure that everyone receives and uses the information needed at the exact moment.

3- i will need to replicate this kind of situation so I'm trying to generalize the iteration logic using for example a list of ordered inputs outputs (e.g. first iter P_req, second iter P_actual ...). because this is a really simple case of two models that need to speak like this, but in my mind a very similar logic must be implemented for Controller that need to read the status of a timestep, operate some decision and communicate them in the exact timestep (a controller could also control a model that has already a loop with another e.g. Hvac). So I was wondering is it always a matter of hardcoding logic for each federation configuration or do you think there is an intelligent way of generalizing some basic interactions to reuse them properly?

So sorry for the long question, I'm a PhD student at Politecnico di Torino, and in a few words, I'm trying to build a test-bed framework based on HELICS for testing reinforcement learning strategies on top of Energy simulation scenarios (simply the COSIM federation will be the environment and one or more simulators will be the reinforcement agents.) I know my questions, are mainly out of scope from the primary HELICS purpose but if you think to are interested in something like what I'm trying to do I would be really glad to have the opportunity of a proper conversation with you, to better show my idea, to better understand HELICS timings and data exchange and to ease this thread exchange and maybe outcome something fo the community at the end.

Best regards

[phlptp]

While most likely you could get the setting right to do this with value federates. The situation you describe is much more in the domain of a combination federate. Where Preq and Pactual being sent back and forth is done via endpoints as it really is more of a packet based exchange. The others can be value based. Then hvac and envelope become event driven inside the loop. Then you can ensure precise and ordered sequences based on the communication packets and internal state logic, which could even be variable.

So while it hasn't been used a lot in this way, endpoint based communication does work with iteration just fine, and combining the two types of federates allows patterns of arbitrary or dynamic complexity.

I think we would be happy to set up a chat to discuss this more detail. I am guess our email is available on some of the HELICS docs and we can set something up.