Broker hierarchy on one machine?

[trevorhardy]

As a user guide example I'm trying to get a broker hierarchy example running on a single computer. I recognize that I need to use unique ports for each of the brokers. connecting to federates but I'm less certain I've correctly configured the hierarchy correctly. I've posted what I've got in the HELICS-examples repo. Any suggestions or help on getting this working would be appreciated.

Here are the helics_cli configurations I'm using to launch the co-sim:

Ex43runnerA.json

{
  "name": "main_broker",
  "federates": [    
    {
      "directory": ".",
      "exec": "helics_broker -f 2 --timeout='10s' --brokerport=20200",
      "host": "localhost",
      "name": "brokerA"
    }
  ]
}

Ex43runnerB.json

{
  "name": "Battery",
  "federates": [
    {
      "directory": ".",
      "exec": "helics_broker -f 1 --loglevel=7 --timeout='10s' --brokerport=20300 --broker_address=tcp://127.0.0.1:20200",
      "host": "localhost",
      "name": "brokerB"
    },
    {
      "directory": ".",
      "exec": "python -u Battery.py 1",
      "host": "localhost",
      "name": "Battery",
      "loglevel": 1
    }
  ]
}

Ex43runnerC.json

{
{
  "name": "Controller_and_Charger",
  "federates": [
    {
      "directory": ".",
      "exec": "helics_broker -f 2 --timeout='10s' --brokerport=20400 --broker_address=tcp://127.0.0.1:20200",
      "host": "localhost",
      "name": "brokerC"
    },
    {
      "directory": ".",
      "exec": "python -u Charger.py 1",
      "host": "localhost",
      "name": "Charger",
      "loglevel": 1
    },
    {
      "directory": ".",
      "exec": "python -u Controller.py 1",
      "host": "localhost",
      "name": "Controller",
      "loglevel": 1
    }
  ]
}

BatteryConfig.json:

{
  "name": "Battery",
  "loglevel": 1,
  "coreType": "zmq",
  "coreInit": "--broker_address=tcp://127.0.0.1:20300",
  "period": 60,
  ...
}

ChargerConfig.json

{
  "name": "Charger",
  "loglevel": 1,
  "coreType": "zmq",
  "coreInit": "--broker_address=tcp://127.0.0.1:20400",
  "period": 60,
  ...
}

ControllerConfig.json

{
  "name": "Controller",
  "loglevel": 1,
  "coreType": "zmq",
  "coreInit": "--broker_address=tcp://127.0.0.1:20400",
  "timedelta": 1,
  "uninterruptible": false,
  "terminate_on_error": true,
  "endpoints": [
    {
      "name": "Controller/ep",
      "global": true
    }
  ]
}

Right now, I'm getting errors related to not connecting to the broker from all executables (except BrokerA). Note that everybody is trying to connect to one port higher than what I specified in the configs. Similar error messages to the multi-broker problem I'm having.

Broker B log:

commWarning||25728-VYEin-i70cE-KfGCO-l04Um (0)::zmq broker connection timed out, trying again (2)
commWarning||25728-VYEin-i70cE-KfGCO-l04Um (0)::sending message to tcp://127.0.0.1:20301
commERROR||25728-VYEin-i70cE-KfGCO-l04Um (0)::zmq broker connection timed out after trying 5 times (2)
commERROR||25728-VYEin-i70cE-KfGCO-l04Um (0)::receiver connection failure
Broker is unable to connect

Broker C log:

commWarning||25732-sCgYz-ctSWY-1DWbH-Ip8gy (0)::zmq broker connection timed out, trying again (2)
commWarning||25732-sCgYz-ctSWY-1DWbH-Ip8gy (0)::sending message to tcp://127.0.0.1:20401
commERROR||25732-sCgYz-ctSWY-1DWbH-Ip8gy (0)::zmq broker connection timed out after trying 5 times (2)
commERROR||25732-sCgYz-ctSWY-1DWbH-Ip8gy (0)::receiver connection failure
Broker is unable to connect

Battery log:

[2020-09-29 12:24:37.545] [console] [warning] commWarning||25729-TrUF3-uwHly-YvLGW-t7m7Y (0)::zmq broker connection timed out, trying again (2)
[2020-09-29 12:24:37.545] [console] [warning] commWarning||25729-TrUF3-uwHly-YvLGW-t7m7Y (0)::sending message to tcp://127.0.0.1:20301

Charger log

[2020-09-29 12:24:41.391] [console] [warning] commWarning||25733-3gr2R-YIBiR-O3xQm-g3SOp (0)::zmq broker connection timed out, trying again (2)
[2020-09-29 12:24:41.391] [console] [warning] commWarning||25733-3gr2R-YIBiR-O3xQm-g3SOp (0)::sending message to tcp://127.0.0.1:20401

Controller log

[2020-09-29 12:24:41.705] [console] [warning] commWarning||25734-b3S8X-66bIf-e0yQc-oOfWI (0)::zmq broker connection timed out, trying again (2)
[2020-09-29 12:24:41.705] [console] [warning] commWarning||25734-b3S8X-66bIf-e0yQc-oOfWI (0)::sending message to tcp://127.0.0.1:20401

[trevorhardy]

I'm still not able to get this to work under v3.0.0-alpha.2 and could use a little help when its convenient.

[trevorhardy]

After working with @ebereleim, we've figured out the winning combination of parameters and values valid as of v3.0.0-alpha.2. Theses will be posted in the User Guide Examples but here are the salient points:

Parent (root) broker:
"exec": "helics_broker --loglevel=7 --timeout='10s' --port=25200",

• The port on which the child brokers connect to the root (parent) broker is 25200.

Child brokers:
"exec": "helics_broker -f 2 --loglevel=7 --timeout='10s' --port=25400 --broker_address=tcp://127.0.0.1:25200"

• Address of the parent broker is 127.0.0.1 port 25200
• Port that the federates connect to this broker on is 25400

Federates:
"brokerAddress": "tcp://127.0.0.1:25400",

• Federates connect to their broker at 127.0.0.1 port 25400