Trying to setup talos cluster on HPE server with failed results

[matjazantloga]

Hi,

I've tried, together with teammate Sergey to setup talos cluster in different ways

• via local docker with success
• vmware workstation with fail
• govc on vcenter with fail
• HPE server with fail

This last example , HPE server is maybe simplest and i will just paste my config here and I hope i will get some clue what we are doing wrong

so, i boot my HPE gen9 dl 390 and got Talos up in maintenance mode with this IP 10.99.100.72 (example)

then i run following

talosctl gen config TalosHPE https://10.99.100.190:6443

and got talosconfig controlplane.yaml and worker.yaml

i modified controlplane.yaml with following information

network:
interfaces:
- interface: eth0
dhcp: false
addresses:
- 10.99.100.191/24
routes:
- network: 0.0.0.0/0
gateway: 10.99.100.1

didn't change anything else

then i run

talosctl apply-config --insecure --nodes 10.99.100.72 --file controlplane.yaml

end result in pic, before that talos was restarted and I clearly saw partitions were made etc ...

[matjazantloga]

this post is embarrassment, i pull out some vmware ssd drives but forgot to put into small blank ssd, so it was installed nowhere ;D. Will post later I hope with more greens.

[matjazantloga]

[matjazantloga]

aaa, almost

[smira]

The machine in the screenshot above hasn't been installed, it doesn't have machine config. So the command fails as expected.

[matjazantloga]

this is command i will apply

talosctl apply-config --nodes 10.99.100.130 --endpoints 10.99.100.130 --file controlplane.yaml --talosconfig talosconfig --insecure

---

talosconfig should be ok

most vital parts of controlplane.yaml, maybe i did sth wrong here

[matjazantloga]

and it fails

[matjazantloga]

i still can ping the IP... also no idea why suddenly two GW are shown

[smira]

Probably you need to take a step back and try with a simplified configuration.

I think you configured networking both via machine config and via dashboard. Don't mix everything together, and try to get the machine to boot and get network access to it in maintenance mode, then apply the config.

[matjazantloga]

i don't understand why it gets static IP when i boot into reset maintenance mode, it should get IP from dhcp this was working but after i was doing some configs inside dashboard F3 console it doesn't work anymore, i would like to reset completely ...

also , i've defined /dev/sdc but i see its dealing with /dev/sda which is USB from which i'm loading Talos
i will paste screenshot for this

[matjazantloga]

[matjazantloga]

ok, so i reset network back to dhcp in maintenance mode and will try to apply config to that IP

[matjazantloga]

adding .82 into talosconfig

running with .82 and --insecure

.82 was removed but instead of adding .130, as can be seen in above controlplane.yaml config it adds .83 via dhcp and IP is then not visible and it also mess with sda instead of sdc.... i'm lost :(

[matjazantloga]

100.130 is applied, because this works.
just IP is not shown and for sure it's unhealthy

[matjazantloga]

gw 10.99.100.129 is no where to be found

alltough is configured in controlplane.yaml
prob sth wrong with network part

[matjazantloga]

i just found out that network section of controlplane.yaml is not taken into apply config at all, i just made up some values

and when i do --dry-run no error occurs

[matjazantloga]

not really true, of course network config is taken into apply but what i found strange is this
when i have and for sure should have route config i got this

instead of adding this route its throw out error that route already exists

[matjazantloga]

sth is cooking :)

[matjazantloga]

HPE node added to fresh cluster.
i0m happy :). Can't believe we lost few days bc our internal subnet is also 10.9x.xx.xx

[matjazantloga]

node IP was in conflict with pod and service subnets. we are using same range for our internal IPs. once i changed those pod and service subnets out of our range everything started getting together. [cid:b30338b4-7b2d-492b-8a11-73854138aa9f] [cid:f50c4aab-d3d2-44af-9179-b20557ac41fd] machine section is really simple, no need to complicate here. HPE really has eno1 and /dev/sdb but it figured it out and was happy with above for config start Best regards, s prijaznimi pozdravi Matjaž Antloga, Ortoscale d.o.o.

…

________________________________ From: Miran ***@***.***> Sent: Thursday, August 15, 2024 9:31 PM To: siderolabs/talos ***@***.***> Cc: Matjaž Antloga - Ortoscale ***@***.***>; State change ***@***.***> Subject: Re: [siderolabs/talos] Trying to setup talos cluster on HPE server with failed results (Discussion #9169) Hey, what changed? Can you share the machine section? — Reply to this email directly, view it on GitHub<#9169 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AR5G3FJ2JJQLME3ADM3HNL3ZRT6Z5AVCNFSM6AAAAABMOXIZRSVHI2DSMVQWIX3LMV43URDJONRXK43TNFXW4Q3PNVWWK3TUHMYTAMZVGE4DKMY>. You are receiving this because you modified the open/close state.Message ID: ***@***.***>

[miran248]

Thanks for the reply!
Might have the same issue. That or it's because i tried adding a new machine (with different secrets and config) to the existing omni cluster.
Existing cluster has 5 machines, all with public ipv4 and default config, new machine sits on a private network with a router in front (with public ip) and because of that i had to configure it differently.. Will make a brand new cluster today with all machines on the same setup and hopefully it will work :)
LP

[matjazantloga]

for sure it needs to be secrets from existing config, just edit machine part of controlplane or worker to suit the new machine and appy it