Random errors using OIDC

[MrRulf]

Not bound to any actions I randomly can't use social login on nextcloud, my OIDC provider, Authelia, doesn't show anything helpful (went all the way to trace mode).
I managed to get logs of 3 tries, where the first one worked, while the two after it didn't.
Nextcloud didn't provide any logs on this, at least I wasn't able to find anything.
Configuration should be fine, I followed this guide https://github.com/lldap/lldap/blob/main/example_configs/nextcloud_oidc_authelia.md#nextcloud with the only difference being that I deactivated auto create new users.

Since I wasn't able to find an error in my config or something in the logs I hope maybe somebody here has an idea. Nextcloud is running on TrueNAS Scale as an App from Truecharts Version 27.0.2_22.0.22 (latest), in case that helps.

2023-09-14 11:31:24.519942+00:00time="2023-09-14T13:31:24+02:00" level=debug msg="Authorization Request with id 'af94dfab-c730-4a5e-8934-26188818e49c' on client with id 'nextcloud' is being processed" method=GET path=/api/oidc/authorization remote_ip=89.144.223.76
2023-09-14 11:31:24.521575+00:00time="2023-09-14T13:31:24+02:00" level=debug msg="Authorization Request with id 'af94dfab-c730-4a5e-8934-26188818e49c' on client with id 'nextcloud' was successfully processed, proceeding to build Authorization Response" method=GET path=/api/oidc/authorization remote_ip=89.144.223.76
2023-09-14 11:31:55.875066+00:00time="2023-09-14T13:31:55+02:00" level=debug msg="Authorization Request with id '22a3f7c4-d641-4a46-a2f9-11a020115fa8' on client with id 'nextcloud' is being processed" method=GET path=/api/oidc/authorization remote_ip=89.144.223.76
2023-09-14 11:31:55.877462+00:00time="2023-09-14T13:31:55+02:00" level=debug msg="Authorization Request with id '22a3f7c4-d641-4a46-a2f9-11a020115fa8' on client with id 'nextcloud' using consent mode 'explicit' proceeding to generate a new consent session" method=GET path=/api/oidc/authorization remote_ip=89.144.223.76
2023-09-14 11:31:55.880090+00:00time="2023-09-14T13:31:55+02:00" level=debug msg="Authorization Request with id '22a3f7c4-d641-4a46-a2f9-11a020115fa8' on client with id 'nextcloud' using consent mode 'explicit' authentication level 'two_factor' is sufficient for client level 'two_factor'" method=GET path=/api/oidc/authorization remote_ip=89.144.223.76
2023-09-14 11:31:55.880177+00:00time="2023-09-14T13:31:55+02:00" level=debug msg="Authorization Request with id '22a3f7c4-d641-4a46-a2f9-11a020115fa8' on client with id 'nextcloud' using consent mode 'explicit' is being redirected to 'https://authelia.mydomain.net/consent?id=32c80665-4075-4ddc-bb38-f32f9b6be567'" method=GET path=/api/oidc/authorization remote_ip=89.144.223.76
2023-09-14 11:34:25.908719+00:00time="2023-09-14T13:34:25+02:00" level=debug msg="Authorization Request with id '172d4aeb-6271-4a3d-b5fb-cea3a4cd48a8' on client with id 'nextcloud' is being processed" method=GET path=/api/oidc/authorization remote_ip=89.144.223.76
2023-09-14 11:34:25.911551+00:00time="2023-09-14T13:34:25+02:00" level=debug msg="Authorization Request with id '172d4aeb-6271-4a3d-b5fb-cea3a4cd48a8' on client with id 'nextcloud' using consent mode 'explicit' proceeding to generate a new consent session" method=GET path=/api/oidc/authorization remote_ip=89.144.223.76
2023-09-14 11:34:25.913504+00:00time="2023-09-14T13:34:25+02:00" level=debug msg="Authorization Request with id '172d4aeb-6271-4a3d-b5fb-cea3a4cd48a8' on client with id 'nextcloud' using consent mode 'explicit' authentication level 'two_factor' is sufficient for client level 'two_factor'" method=GET path=/api/oidc/authorization remote_ip=89.144.223.76
2023-09-14 11:34:25.913683+00:00time="2023-09-14T13:34:25+02:00" level=debug msg="Authorization Request with id '172d4aeb-6271-4a3d-b5fb-cea3a4cd48a8' on client with id 'nextcloud' using consent mode 'explicit' is being redirected to 'https://authelia.authelia.net/consent?id=49fa9f86-1d08-4143-8b85-5b0bbc181618'" method=GET path=/api/oidc/authorization remote_ip=89.144.223.76