Avoiding the use of dynamic map keys in the cluster_formation results of the stable master health indicator

[masseyke]

Part of the stable master history health indicator's results (the cluster_formation section within details) used dynamic keys in a map. This gets rid of that. So now instead of:

"details": {
                "current_master": {
                    "node_id": null,
                    "name": null
                },
                "recent_masters": [
                    {
                        "node_id": "31WBm9iTTRuMyWnBhWNUGA",
                        "name": "master-node-3"
                    }
                ],
                "cluster_formation": {
                    "31WBm9iTTRuMyWnBhWNUGA": "master not discovered or elected yet, an election requires at least 2 nodes with ids from [nADkAeGsT-q12gw89Ga1FA, 31WBm9iTTRuMyWnBhWNUGA, w8v48JvuRsuDCjwBn8KbRw], have only discovered non-quorum [{master-node-3}{31WBm9iTTRuMyWnBhWNUGA}{lJmGYiTPS_W7AJU7csG_gQ}{master-node-3}{127.0.0.1}{127.0.0.1:9301}{dm}]; discovery will continue using [127.0.0.1:9300, 127.0.0.1:9302, 127.0.0.1:9303, 127.0.0.1:9304, 127.0.0.1:9305, [::1]:9300, [::1]:9302, [::1]:9303, [::1]:9304, [::1]:9305] from hosts providers and [{master-node-2}{nADkAeGsT-q12gw89Ga1FA}{logzEHuuTpqwJp-RWssBPw}{master-node-2}{127.0.0.1}{127.0.0.1:9300}{dm}, {master-node-3}{31WBm9iTTRuMyWnBhWNUGA}{lJmGYiTPS_W7AJU7csG_gQ}{master-node-3}{127.0.0.1}{127.0.0.1:9301}{dm}] from last-known cluster state; node term 39, last-accepted version 461 in term 39"
                }
}

We will have:

"details": {
                "current_master": {
                    "node_id": null,
                    "name": null
                },
                "recent_masters": [
                    {
                        "node_id": "31WBm9iTTRuMyWnBhWNUGA",
                        "name": "master-node-3"
                    }
                ],
                "cluster_formation": [
                    {
                        "node_id": "31WBm9iTTRuMyWnBhWNUGA",
                        "cluster_formation_message": "master not discovered or elected yet, an election requires at least 2 nodes with ids from [nADkAeGsT-q12gw89Ga1FA, 31WBm9iTTRuMyWnBhWNUGA, w8v48JvuRsuDCjwBn8KbRw], have only discovered non-quorum [{master-node-3}{31WBm9iTTRuMyWnBhWNUGA}{lJmGYiTPS_W7AJU7csG_gQ}{master-node-3}{127.0.0.1}{127.0.0.1:9301}{dm}]; discovery will continue using [127.0.0.1:9300, 127.0.0.1:9302, 127.0.0.1:9303, 127.0.0.1:9304, 127.0.0.1:9305, [::1]:9300, [::1]:9302, [::1]:9303, [::1]:9304, [::1]:9305] from hosts providers and [{master-node-2}{nADkAeGsT-q12gw89Ga1FA}{logzEHuuTpqwJp-RWssBPw}{master-node-2}{127.0.0.1}{127.0.0.1:9300}{dm}, {master-node-3}{31WBm9iTTRuMyWnBhWNUGA}{lJmGYiTPS_W7AJU7csG_gQ}{master-node-3}{127.0.0.1}{127.0.0.1:9301}{dm}] from last-known cluster state; node term 39, last-accepted version 461 in term 39"
                    }
                ]
}

[elasticsearchmachine]

Pinging @elastic/es-data-management (Team:Data Management)

[andreidan]

In light of #89664 shall we use a different key here instead of node_id ? (to avoid future renamings)
Maybe node ?

Or otherwise, maybe report node_id and node_name moving forward as separate keys? (similar to how we do it in recent_masters ?

What do you think? // cc @dakrone

[masseyke]

Since it's no longer the key, it made sense to me to explicitly put the node_id here. I'm planning on a follow up PR (or PRs) to deal with adding the name, but wanted to keep the scope of this one small.

[andreidan]

Ah ++
Maybe we can tag #89664 in the follow up PR and change the description to reporting both id and name (which I like better than combining the 2)

@dakrone how does that sound to you?

[andreidan]

LGTM, thanks Keith

Please also update the docs for the response with the new structure