[http_endpoint] fix incorrect TLSConfig initialization (#32104) #32105

r00tu53r · 2022-06-26T11:48:23Z

What does this PR do?

The PR fixes #32104.

Why is it important?

Without this fix http_endpoint cannot receive events due to TLS handshake errors.

Checklist

My code follows the style guidelines of this project
~~[ ] I have commented my code, particularly in hard-to-understand areas~~
~~[ ] I have made corresponding changes to the documentation~~
~~[ ] I have made corresponding change to the default configuration files~~
~~[ ] I have added tests that prove my fix is effective or that my feature works~~
I have added an entry in CHANGELOG.next.asciidoc or CHANGELOG-developer.next.asciidoc.

Related issues

Closes [http_endpoint] is initialized with incorrect TLSConfig resulting in TLS handshake errors #32104

Logs

Shows logs indicating events being received and then sent to ES

{"log.level":"debug","@timestamp":"2022-06-26T11:26:06.655Z","log.logger":"processors","log.origin":{"file.name":"processing/processors.go","file.line":210},"message":"Publish event: {\n  \"@timestamp\": \"2022-06-26T11:26:06.654Z\",\n  \"@metadata\": {\n    \"beat\": \"filebeat\",\n    \"type\": \"_doc\",\n    \"version\": \"8.4.0\",\n    \"raw_index\": \"logs-auth0.logs-default\"\n  },\n  \"event\": {\n    \"original\": \"{\\\"data\\\":{\\\"client_id\\\":\\\"LAklHBNOozEX61cqlv4MLsqw9gAJXFc2\\\",\\\"client_name\\\":\\\"nonmagic.in\\\",\\\"connection\\\":\\\"google-oauth2\\\",\\\"connection_id\\\":\\\"con_kxg4zO3njp5kwfO0\\\",\\\"date\\\":\\\"2022-06-26T11:25:53.194Z\\\",\\\"details\\\":{\\\"completedAt\\\":1656242753193,\\\"elapsedTime\\\":5227,\\\"initiatedAt\\\":1656242747966,\\\"prompts\\\":[{\\\"completedAt\\\":1656242752839,\\\"connection\\\":\\\"google-oauth2\\\",\\\"connection_id\\\":\\\"con_kxg4zO3njp5kwfO0\\\",\\\"elapsedTime\\\":2400,\\\"identity\\\":\\\"109813294014864971352\\\",\\\"initiatedAt\\\":1656242750439,\\\"name\\\":\\\"federated-authenticate\\\",\\\"stats\\\":{\\\"loginsCount\\\":33},\\\"strategy\\\":\\\"google-oauth2\\\"},{\\\"completedAt\\\":1656242752852,\\\"elapsedTime\\\":4874,\\\"flow\\\":\\\"universal-login\\\",\\\"initiatedAt\\\":1656242747978,\\\"name\\\":\\\"login\\\",\\\"timers\\\":{\\\"rules\\\":4},\\\"user_id\\\":\\\"google-oauth2|109813294014864971352\\\",\\\"user_name\\\":\\\"e2.gummaraj@gmail.com\\\"}],\\\"session_id\\\":\\\"E46yacskL7alITIrMbJzKQ-ASc4fLLY2\\\",\\\"stats\\\":{\\\"loginsCount\\\":33}},\\\"hostname\\\":\\\"dev-jb-t5jv1.us.auth0.com\\\",\\\"ip\\\":\\\"122.179.231.6\\\",\\\"log_id\\\":\\\"90020220626112557618935050173941223507819255590610272258\\\",\\\"strategy\\\":\\\"google-oauth2\\\",\\\"strategy_type\\\":\\\"social\\\",\\\"type\\\":\\\"s\\\",\\\"user_agent\\\":\\\"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/102.0.0.0 Safari/537.36\\\",\\\"user_id\\\":\\\"google-oauth2|109813294014864971352\\\",\\\"user_name\\\":\\\"e2.gummaraj@gmail.com\\\"},\\\"log_id\\\":\\\"90020220626112557618935050173941223507819255590610272258\\\"}\",\n    \"dataset\": \"auth0.logs\"\n  },\n  \"input\": {\n    \"type\": \"http_endpoint\"\n  },\n  \"data_stream\": {\n    \"dataset\": \"auth0.logs\",\n    \"namespace\": \"default\",\n    \"type\": \"logs\"\n  },\n  \"agent\": {\n    \"name\": \"nonmagic.in\",\n    \"type\": \"filebeat\",\n    \"id\": \"ced880e1-9633-4756-a878-970b54978e8f\",\n    \"version\": \"8.4.0\",\n    \"ephemeral_id\": \"f9e77a7d-8887-4da0-87c1-842730c40aee\"\n  },\n  \"ecs\": {\n    \"version\": \"8.0.0\"\n  },\n  \"json\": {\n    \"data\": {\n      \"connection\": \"google-oauth2\",\n      \"client_id\": \"LAklHBNOozEX61cqlv4MLsqw9gAJXFc2\",\n      \"client_name\": \"nonmagic.in\",\n      \"details\": {\n        \"initiatedAt\": 1656242747966,\n        \"completedAt\": 1656242753193,\n        \"elapsedTime\": 5227,\n        \"session_id\": \"E46yacskL7alITIrMbJzKQ-ASc4fLLY2\",\n        \"stats\": {\n          \"loginsCount\": 33\n        },\n        \"prompts\": [\n          {\n            \"connection_id\": \"con_kxg4zO3njp5kwfO0\",\n            \"connection\": \"google-oauth2\",\n            \"elapsedTime\": 2400,\n            \"identity\": \"109813294014864971352\",\n            \"name\": \"federated-authenticate\",\n            \"strategy\": \"google-oauth2\",\n            \"initiatedAt\": 1656242750439,\n            \"stats\": {\n              \"loginsCount\": 33\n            },\n            \"completedAt\": 1656242752839\n          },\n          {\n            \"user_id\": \"google-oauth2|109813294014864971352\",\n            \"user_name\": \"e2.gummaraj@gmail.com\",\n            \"timers\": {\n              \"rules\": 4\n            },\n            \"elapsedTime\": 4874,\n            \"name\": \"login\",\n            \"flow\": \"universal-login\",\n            \"initiatedAt\": 1656242747978,\n            \"completedAt\": 1656242752852\n          }\n        ]\n      },\n      \"connection_id\": \"con_kxg4zO3njp5kwfO0\",\n      \"strategy\": \"google-oauth2\",\n      \"strategy_type\": \"social\",\n      \"user_agent\": \"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/102.0.0.0 Safari/537.36\",\n      \"log_id\": \"90020220626112557618935050173941223507819255590610272258\",\n      \"hostname\": \"dev-jb-t5jv1.us.auth0.com\",\n      \"date\": \"2022-06-26T11:25:53.194Z\",\n      \"ip\": \"122.179.231.6\",\n      \"user_id\": \"google-oauth2|109813294014864971352\",\n      \"user_name\": \"e2.gummaraj@gmail.com\",\n      \"type\": \"s\"\n    },\n    \"log_id\": \"90020220626112557618935050173941223507819255590610272258\"\n  },\n  \"tags\": [\n    \"preserve_original_event\",\n    \"forwarded\",\n    \"auth0-logstream\"\n  ],\n  \"elastic_agent\": {\n    \"version\": \"8.2.2\",\n    \"id\": \"ced880e1-9633-4756-a878-970b54978e8f\",\n    \"snapshot\": false\n  },\n  \"cloud\": {\n    \"region\": \"ap-southeast-2\",\n    \"provider\": \"aws\",\n    \"availability_zone\": \"ap-southeast-2b\",\n    \"service\": {\n      \"name\": \"EC2\"\n    },\n    \"account\": {\n      \"id\": \"144492464627\"\n    },\n    \"image\": {\n      \"id\": \"ami-0b21dcff37a8cd8a4\"\n    },\n    \"instance\": {\n      \"id\": \"i-02a008da037760aec\"\n    },\n    \"machine\": {\n      \"type\": \"t2.micro\"\n    }\n  }\n}","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2022-06-26T11:26:06.655Z","log.logger":"processors","log.origin":{"file.name":"processing/processors.go","file.line":210},"message":"Publish event: {\n  \"@timestamp\": \"2022-06-26T11:26:06.655Z\",\n  \"@metadata\": {\n    \"beat\": \"filebeat\",\n    \"type\": \"_doc\",\n    \"version\": \"8.4.0\",\n    \"raw_index\": \"logs-auth0.logs-default\"\n  },\n  \"input\": {\n    \"type\": \"http_endpoint\"\n  },\n  \"data_stream\": {\n    \"dataset\": \"auth0.logs\",\n    \"namespace\": \"default\",\n    \"type\": \"logs\"\n  },\n  \"elastic_agent\": {\n    \"version\": \"8.2.2\",\n    \"id\": \"ced880e1-9633-4756-a878-970b54978e8f\",\n    \"snapshot\": false\n  },\n  \"agent\": {\n    \"id\": \"ced880e1-9633-4756-a878-970b54978e8f\",\n    \"type\": \"filebeat\",\n    \"version\": \"8.4.0\",\n    \"ephemeral_id\": \"f9e77a7d-8887-4da0-87c1-842730c40aee\",\n    \"name\": \"nonmagic.in\"\n  },\n  \"cloud\": {\n    \"availability_zone\": \"ap-southeast-2b\",\n    \"service\": {\n      \"name\": \"EC2\"\n    },\n    \"account\": {\n      \"id\": \"144492464627\"\n    },\n    \"image\": {\n      \"id\": \"ami-0b21dcff37a8cd8a4\"\n    },\n    \"instance\": {\n      \"id\": \"i-02a008da037760aec\"\n    },\n    \"machine\": {\n      \"type\": \"t2.micro\"\n    },\n    \"region\": \"ap-southeast-2\",\n    \"provider\": \"aws\"\n  },\n  \"tags\": [\n    \"preserve_original_event\",\n    \"forwarded\",\n    \"auth0-logstream\"\n  ],\n  \"json\": {\n    \"log_id\": \"90020220626112559284935050175868251264284974632530870274\",\n    \"data\": {\n      \"ip\": \"3.26.173.2\",\n      \"user_name\": \"e2.gummaraj@gmail.com\",\n      \"log_id\": \"90020220626112559284935050175868251264284974632530870274\",\n      \"connection_id\": \"\",\n      \"hostname\": \"dev-jb-t5jv1.us.auth0.com\",\n      \"user_id\": \"google-oauth2|109813294014864971352\",\n      \"type\": \"seacft\",\n      \"description\": \"\",\n      \"user_agent\": \"Go-http-client/2.0\",\n      \"details\": {\n        \"code\": \"******************************************V6j\"\n      },\n      \"date\": \"2022-06-26T11:25:54.687Z\",\n      \"client_id\": \"LAklHBNOozEX61cqlv4MLsqw9gAJXFc2\",\n      \"client_name\": \"nonmagic.in\"\n    }\n  },\n  \"ecs\": {\n    \"version\": \"8.0.0\"\n  },\n  \"event\": {\n    \"original\": \"{\\\"data\\\":{\\\"client_id\\\":\\\"LAklHBNOozEX61cqlv4MLsqw9gAJXFc2\\\",\\\"client_name\\\":\\\"nonmagic.in\\\",\\\"connection_id\\\":\\\"\\\",\\\"date\\\":\\\"2022-06-26T11:25:54.687Z\\\",\\\"description\\\":\\\"\\\",\\\"details\\\":{\\\"code\\\":\\\"******************************************V6j\\\"},\\\"hostname\\\":\\\"dev-jb-t5jv1.us.auth0.com\\\",\\\"ip\\\":\\\"3.26.173.2\\\",\\\"log_id\\\":\\\"90020220626112559284935050175868251264284974632530870274\\\",\\\"type\\\":\\\"seacft\\\",\\\"user_agent\\\":\\\"Go-http-client/2.0\\\",\\\"user_id\\\":\\\"google-oauth2|109813294014864971352\\\",\\\"user_name\\\":\\\"e2.gummaraj@gmail.com\\\"},\\\"log_id\\\":\\\"90020220626112559284935050175868251264284974632530870274\\\"}\",\n    \"dataset\": \"auth0.logs\"\n  }\n}","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2022-06-26T11:26:06.656Z","log.logger":"processors","log.origin":{"file.name":"processing/processors.go","file.line":210},"message":"Publish event: {\n  \"@timestamp\": \"2022-06-26T11:26:06.656Z\",\n  \"@metadata\": {\n    \"beat\": \"filebeat\",\n    \"type\": \"_doc\",\n    \"version\": \"8.4.0\",\n    \"raw_index\": \"logs-auth0.logs-default\"\n  },\n  \"input\": {\n    \"type\": \"http_endpoint\"\n  },\n  \"elastic_agent\": {\n    \"snapshot\": false,\n    \"version\": \"8.2.2\",\n    \"id\": \"ced880e1-9633-4756-a878-970b54978e8f\"\n  },\n  \"ecs\": {\n    \"version\": \"8.0.0\"\n  },\n  \"event\": {\n    \"original\": \"{\\\"data\\\":{\\\"client_id\\\":\\\"LAklHBNOozEX61cqlv4MLsqw9gAJXFc2\\\",\\\"client_name\\\":\\\"nonmagic.in\\\",\\\"connection\\\":\\\"google-oauth2\\\",\\\"connection_id\\\":\\\"con_kxg4zO3njp5kwfO0\\\",\\\"date\\\":\\\"2022-06-26T11:25:56.728Z\\\",\\\"details\\\":{\\\"allowed_logout_url\\\":[\\\"https://nonmagic.in:8443\\\"],\\\"return_to\\\":\\\"https://nonmagic.in:8443\\\",\\\"session_id\\\":\\\"E46yacskL7alITIrMbJzKQ-ASc4fLLY2\\\"},\\\"hostname\\\":\\\"dev-jb-t5jv1.us.auth0.com\\\",\\\"ip\\\":\\\"122.179.231.6\\\",\\\"log_id\\\":\\\"90020220626112601001935050177966946487135970948540268546\\\",\\\"type\\\":\\\"slo\\\",\\\"user_agent\\\":\\\"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/102.0.0.0 Safari/537.36\\\",\\\"user_id\\\":\\\"google-oauth2|109813294014864971352\\\",\\\"user_name\\\":\\\"e2.gummaraj@gmail.com\\\"},\\\"log_id\\\":\\\"90020220626112601001935050177966946487135970948540268546\\\"}\",\n    \"dataset\": \"auth0.logs\"\n  },\n  \"json\": {\n    \"log_id\": \"90020220626112601001935050177966946487135970948540268546\",\n    \"data\": {\n      \"details\": {\n        \"session_id\": \"E46yacskL7alITIrMbJzKQ-ASc4fLLY2\",\n        \"return_to\": \"https://nonmagic.in:8443\",\n        \"allowed_logout_url\": [\n          \"https://nonmagic.in:8443\"\n        ]\n      },\n      \"log_id\": \"90020220626112601001935050177966946487135970948540268546\",\n      \"user_name\": \"e2.gummaraj@gmail.com\",\n      \"client_name\": \"nonmagic.in\",\n      \"ip\": \"122.179.231.6\",\n      \"connection\": \"google-oauth2\",\n      \"connection_id\": \"con_kxg4zO3njp5kwfO0\",\n      \"client_id\": \"LAklHBNOozEX61cqlv4MLsqw9gAJXFc2\",\n      \"date\": \"2022-06-26T11:25:56.728Z\",\n      \"user_agent\": \"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/102.0.0.0 Safari/537.36\",\n      \"hostname\": \"dev-jb-t5jv1.us.auth0.com\",\n      \"user_id\": \"google-oauth2|109813294014864971352\",\n      \"type\": \"slo\"\n    }\n  },\n  \"tags\": [\n    \"preserve_original_event\",\n    \"forwarded\",\n    \"auth0-logstream\"\n  ],\n  \"data_stream\": {\n    \"namespace\": \"default\",\n    \"type\": \"logs\",\n    \"dataset\": \"auth0.logs\"\n  },\n  \"agent\": {\n    \"id\": \"ced880e1-9633-4756-a878-970b54978e8f\",\n    \"type\": \"filebeat\",\n    \"version\": \"8.4.0\",\n    \"ephemeral_id\": \"f9e77a7d-8887-4da0-87c1-842730c40aee\",\n    \"name\": \"nonmagic.in\"\n  },\n  \"cloud\": {\n    \"provider\": \"aws\",\n    \"availability_zone\": \"ap-southeast-2b\",\n    \"service\": {\n      \"name\": \"EC2\"\n    },\n    \"account\": {\n      \"id\": \"144492464627\"\n    },\n    \"image\": {\n      \"id\": \"ami-0b21dcff37a8cd8a4\"\n    },\n    \"instance\": {\n      \"id\": \"i-02a008da037760aec\"\n    },\n    \"machine\": {\n      \"type\": \"t2.micro\"\n    },\n    \"region\": \"ap-southeast-2\"\n  }\n}","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2022-06-26T11:26:25.550Z","log.logger":"processors","log.origin":{"file.name":"processing/processors.go","file.line":210},"message":"Publish event: {\n  \"@timestamp\": \"2022-06-26T11:26:25.550Z\",\n  \"@metadata\": {\n    \"beat\": \"filebeat\",\n    \"type\": \"_doc\",\n    \"version\": \"8.4.0\",\n    \"raw_index\": \"logs-auth0.logs-default\"\n  },\n  \"cloud\": {\n    \"account\": {\n      \"id\": \"144492464627\"\n    },\n    \"image\": {\n      \"id\": \"ami-0b21dcff37a8cd8a4\"\n    },\n    \"instance\": {\n      \"id\": \"i-02a008da037760aec\"\n    },\n    \"machine\": {\n      \"type\": \"t2.micro\"\n    },\n    \"region\": \"ap-southeast-2\",\n    \"provider\": \"aws\",\n    \"availability_zone\": \"ap-southeast-2b\",\n    \"service\": {\n      \"name\": \"EC2\"\n    }\n  },\n  \"tags\": [\n    \"preserve_original_event\",\n    \"forwarded\",\n    \"auth0-logstream\"\n  ],\n  \"input\": {\n    \"type\": \"http_endpoint\"\n  },\n  \"elastic_agent\": {\n    \"id\": \"ced880e1-9633-4756-a878-970b54978e8f\",\n    \"snapshot\": false,\n    \"version\": \"8.2.2\"\n  },\n  \"ecs\": {\n    \"version\": \"8.0.0\"\n  },\n  \"event\": {\n    \"original\": \"{\\\"data\\\":{\\\"client_id\\\":\\\"LAklHBNOozEX61cqlv4MLsqw9gAJXFc2\\\",\\\"client_name\\\":\\\"nonmagic.in\\\",\\\"connection_id\\\":\\\"\\\",\\\"date\\\":\\\"2022-06-26T11:26:06.814Z\\\",\\\"description\\\":\\\"\\\",\\\"details\\\":{\\\"code\\\":\\\"******************************************6a3\\\"},\\\"hostname\\\":\\\"dev-jb-t5jv1.us.auth0.com\\\",\\\"ip\\\":\\\"3.26.173.2\\\",\\\"log_id\\\":\\\"90020220626112607701267824214133775489830594953818931218\\\",\\\"type\\\":\\\"seacft\\\",\\\"user_agent\\\":\\\"Go-http-client/2.0\\\",\\\"user_id\\\":\\\"google-oauth2|109813294014864971352\\\",\\\"user_name\\\":\\\"e2.gummaraj@gmail.com\\\"},\\\"log_id\\\":\\\"90020220626112607701267824214133775489830594953818931218\\\"}\",\n    \"dataset\": \"auth0.logs\"\n  },\n  \"json\": {\n    \"log_id\": \"90020220626112607701267824214133775489830594953818931218\",\n    \"data\": {\n      \"type\": \"seacft\",\n      \"connection_id\": \"\",\n      \"user_id\": \"google-oauth2|109813294014864971352\",\n      \"details\": {\n        \"code\": \"******************************************6a3\"\n      },\n      \"client_name\": \"nonmagic.in\",\n      \"ip\": \"3.26.173.2\",\n      \"user_agent\": \"Go-http-client/2.0\",\n      \"date\": \"2022-06-26T11:26:06.814Z\",\n      \"user_name\": \"e2.gummaraj@gmail.com\",\n      \"hostname\": \"dev-jb-t5jv1.us.auth0.com\",\n      \"description\": \"\",\n      \"client_id\": \"LAklHBNOozEX61cqlv4MLsqw9gAJXFc2\",\n      \"log_id\": \"90020220626112607701267824214133775489830594953818931218\"\n    }\n  },\n  \"data_stream\": {\n    \"type\": \"logs\",\n    \"dataset\": \"auth0.logs\",\n    \"namespace\": \"default\"\n  },\n  \"agent\": {\n    \"type\": \"filebeat\",\n    \"id\": \"ced880e1-9633-4756-a878-970b54978e8f\",\n    \"version\": \"8.4.0\",\n    \"ephemeral_id\": \"f9e77a7d-8887-4da0-87c1-842730c40aee\",\n    \"name\": \"nonmagic.in\"\n  }\n}","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2022-06-26T11:26:25.552Z","log.logger":"processors","log.origin":{"file.name":"processing/processors.go","file.line":210},"message":"Publish event: {\n  \"@timestamp\": \"2022-06-26T11:26:25.552Z\",\n  \"@metadata\": {\n    \"beat\": \"filebeat\",\n    \"type\": \"_doc\",\n    \"version\": \"8.4.0\",\n    \"raw_index\": \"logs-auth0.logs-default\"\n  },\n  \"tags\": [\n    \"preserve_original_event\",\n    \"forwarded\",\n    \"auth0-logstream\"\n  ],\n  \"input\": {\n    \"type\": \"http_endpoint\"\n  },\n  \"elastic_agent\": {\n    \"id\": \"ced880e1-9633-4756-a878-970b54978e8f\",\n    \"snapshot\": false,\n    \"version\": \"8.2.2\"\n  },\n  \"json\": {\n    \"log_id\": \"90020220626112611444267824219519540016213768202012852242\",\n    \"data\": {\n      \"connection_id\": \"con_kxg4zO3njp5kwfO0\",\n      \"hostname\": \"dev-jb-t5jv1.us.auth0.com\",\n      \"strategy_type\": \"social\",\n      \"type\": \"s\",\n      \"user_name\": \"e2.gummaraj@gmail.com\",\n      \"date\": \"2022-06-26T11:26:06.415Z\",\n      \"connection\": \"google-oauth2\",\n      \"user_id\": \"google-oauth2|109813294014864971352\",\n      \"user_agent\": \"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/102.0.0.0 Safari/537.36\",\n      \"strategy\": \"google-oauth2\",\n      \"log_id\": \"90020220626112611444267824219519540016213768202012852242\",\n      \"client_name\": \"nonmagic.in\",\n      \"ip\": \"122.179.231.6\",\n      \"details\": {\n        \"session_id\": \"7JL9wlgt-bVYT3V5RpJ4NqT3rhkAMPF_\",\n        \"stats\": {\n          \"loginsCount\": 34\n        },\n        \"prompts\": [\n          {\n            \"strategy\": \"google-oauth2\",\n            \"stats\": {\n              \"loginsCount\": 34\n            },\n            \"name\": \"federated-authenticate\",\n            \"connection\": \"google-oauth2\",\n            \"initiatedAt\": 1656242764926,\n            \"elapsedTime\": 1119,\n            \"completedAt\": 1656242766045,\n            \"connection_id\": \"con_kxg4zO3njp5kwfO0\",\n            \"identity\": \"109813294014864971352\"\n          },\n          {\n            \"flow\": \"universal-login\",\n            \"initiatedAt\": 1656242760178,\n            \"completedAt\": 1656242766058,\n            \"user_id\": \"google-oauth2|109813294014864971352\",\n            \"user_name\": \"e2.gummaraj@gmail.com\",\n            \"timers\": {\n              \"rules\": 5\n            },\n            \"elapsedTime\": 5880,\n            \"name\": \"login\"\n          }\n        ],\n        \"initiatedAt\": 1656242760174,\n        \"completedAt\": 1656242766414,\n        \"elapsedTime\": 6240\n      },\n      \"client_id\": \"LAklHBNOozEX61cqlv4MLsqw9gAJXFc2\"\n    }\n  },\n  \"event\": {\n    \"original\": \"{\\\"data\\\":{\\\"client_id\\\":\\\"LAklHBNOozEX61cqlv4MLsqw9gAJXFc2\\\",\\\"client_name\\\":\\\"nonmagic.in\\\",\\\"connection\\\":\\\"google-oauth2\\\",\\\"connection_id\\\":\\\"con_kxg4zO3njp5kwfO0\\\",\\\"date\\\":\\\"2022-06-26T11:26:06.415Z\\\",\\\"details\\\":{\\\"completedAt\\\":1656242766414,\\\"elapsedTime\\\":6240,\\\"initiatedAt\\\":1656242760174,\\\"prompts\\\":[{\\\"completedAt\\\":1656242766045,\\\"connection\\\":\\\"google-oauth2\\\",\\\"connection_id\\\":\\\"con_kxg4zO3njp5kwfO0\\\",\\\"elapsedTime\\\":1119,\\\"identity\\\":\\\"109813294014864971352\\\",\\\"initiatedAt\\\":1656242764926,\\\"name\\\":\\\"federated-authenticate\\\",\\\"stats\\\":{\\\"loginsCount\\\":34},\\\"strategy\\\":\\\"google-oauth2\\\"},{\\\"completedAt\\\":1656242766058,\\\"elapsedTime\\\":5880,\\\"flow\\\":\\\"universal-login\\\",\\\"initiatedAt\\\":1656242760178,\\\"name\\\":\\\"login\\\",\\\"timers\\\":{\\\"rules\\\":5},\\\"user_id\\\":\\\"google-oauth2|109813294014864971352\\\",\\\"user_name\\\":\\\"e2.gummaraj@gmail.com\\\"}],\\\"session_id\\\":\\\"7JL9wlgt-bVYT3V5RpJ4NqT3rhkAMPF_\\\",\\\"stats\\\":{\\\"loginsCount\\\":34}},\\\"hostname\\\":\\\"dev-jb-t5jv1.us.auth0.com\\\",\\\"ip\\\":\\\"122.179.231.6\\\",\\\"log_id\\\":\\\"90020220626112611444267824219519540016213768202012852242\\\",\\\"strategy\\\":\\\"google-oauth2\\\",\\\"strategy_type\\\":\\\"social\\\",\\\"type\\\":\\\"s\\\",\\\"user_agent\\\":\\\"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/102.0.0.0 Safari/537.36\\\",\\\"user_id\\\":\\\"google-oauth2|109813294014864971352\\\",\\\"user_name\\\":\\\"e2.gummaraj@gmail.com\\\"},\\\"log_id\\\":\\\"90020220626112611444267824219519540016213768202012852242\\\"}\",\n    \"dataset\": \"auth0.logs\"\n  },\n  \"ecs\": {\n    \"version\": \"8.0.0\"\n  },\n  \"cloud\": {\n    \"region\": \"ap-southeast-2\",\n    \"provider\": \"aws\",\n    \"availability_zone\": \"ap-southeast-2b\",\n    \"service\": {\n      \"name\": \"EC2\"\n    },\n    \"account\": {\n      \"id\": \"144492464627\"\n    },\n    \"image\": {\n      \"id\": \"ami-0b21dcff37a8cd8a4\"\n    },\n    \"instance\": {\n      \"id\": \"i-02a008da037760aec\"\n    },\n    \"machine\": {\n      \"type\": \"t2.micro\"\n    }\n  },\n  \"data_stream\": {\n    \"dataset\": \"auth0.logs\",\n    \"namespace\": \"default\",\n    \"type\": \"logs\"\n  },\n  \"agent\": {\n    \"id\": \"ced880e1-9633-4756-a878-970b54978e8f\",\n    \"ephemeral_id\": \"f9e77a7d-8887-4da0-87c1-842730c40aee\",\n    \"name\": \"nonmagic.in\",\n    \"type\": \"filebeat\",\n    \"version\": \"8.4.0\"\n  }\n}","service.name":"filebeat","ecs.version":"1.6.0"}

elasticmachine · 2022-06-26T11:48:25Z

Pinging @elastic/security-external-integrations (Team:Security-External Integrations)

mergify · 2022-06-26T11:50:07Z

This pull request is now in conflicts. Could you fix it? 🙏
To fixup this pull request, you can check out it locally. See documentation: https://help.github.com/articles/checking-out-pull-requests-locally/

git fetch upstream
git checkout -b debug/broken-tls-handshake-httpendpoint upstream/debug/broken-tls-handshake-httpendpoint
git merge upstream/main
git push upstream debug/broken-tls-handshake-httpendpoint

elasticmachine · 2022-06-26T12:04:45Z

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS

Expand to view the summary

Build stats

Start Time: 2022-06-26T11:52:19.021+0000
Duration: 78 min 56 sec

Test stats 🧪

Test	Results
Failed	0
Passed	2130
Skipped	166
Total	2296

💚 Flaky test report

Tests succeeded.

🤖 GitHub comments

To re-run your PR in the CI, just comment with:

/test : Re-trigger the build.
/package : Generate the packages and run the E2E tests.
/beats-tester : Run the installation tests with beats-tester.
run elasticsearch-ci/docs : Re-trigger the docs validation. (use unformatted text in the comment!)

andrewkroh

I assume you tested without TLS client authentication. Could you verify client auth (mTLS) is working too.

efd6 · 2022-06-27T00:20:17Z

It looks to me like mTLS should not work with the code that is currently in main; this should fix that.

efd6

LGTM but please wait for @andrewkroh

andrewkroh · 2022-06-27T02:21:53Z

I'm very suspicious of the code in the tlscommon package after taking a quick look with client authentication in mind.

It does not appear to set the ClientCAs with the configured CAs.
I don't see where it sets the ClientAuth based on the configured verification_mode. I assume ClientAuth is required in order for the server to a client certs and advertise the list of CAs that it accepts certs from.
It sets InsecureSkipVerify to true which would skip peer certificate verification even if it were enabled. This would mean that PeerCertificates in the tls.ConnectionState is always empty. https://github.com/elastic/elastic-agent-libs/blob/9e0189f6e2c4ad79365d16f6a50e58dcfd29d740/transport/tlscommon/tls_config.go#L113
In VerifyFull mode it checks the host name in the peer cert against a static value. This makes no sense for client auth. A possible check would be if the cert is valid for the client's IP or possibly reverse DNS host name. https://github.com/elastic/elastic-agent-libs/blob/9e0189f6e2c4ad79365d16f6a50e58dcfd29d740/transport/tlscommon/tls_config.go#L269
In VerifyStrict I don't understand the use case of certificate pinning for client certs. It looks like it would work, but I've never seen this feature in a product.

Hopefully I'm wrong about some of this. I think the tlscommon package needs some unit tests.

r00tu53r · 2022-06-27T03:23:18Z

For Auth0 as expected I don't see events reaching ES with client_authentication: required due to handshake errors. The peer certificates are missing in the connection state. I realised that this could be because the environment is a trial dev environment with out a proper custom domain with a cert that's signed by a CA. These are the working configs for auth0 -

enabled: true
certificate: "/path/to/fullchain.pem"
key: "/path/to/privkey.pem"
verification_mode: full

with client_authentication: none and client_authentication: optional

r00tu53r · 2022-06-27T03:51:46Z

I'm very suspicious of the code in the tlscommon package after taking a quick look with client authentication in mind.

It does not appear to set the ClientCAs with the configured CAs.

I don't see where it sets the ClientAuth based on the configured verification_mode. I assume ClientAuth is required in order for the server to a client certs and advertise the list of CAs that it accepts certs from.

It sets InsecureSkipVerify to true which would skip peer certificate verification even if it were enabled. This would mean that PeerCertificates in the tls.ConnectionState is always empty. https://github.com/elastic/elastic-agent-libs/blob/9e0189f6e2c4ad79365d16f6a50e58dcfd29d740/transport/tlscommon/tls_config.go#L113

In VerifyFull mode it checks the host name in the peer cert against a static value. This makes no sense for client auth. A possible check would be if the cert is valid for the client's IP or possibly reverse DNS host name. https://github.com/elastic/elastic-agent-libs/blob/9e0189f6e2c4ad79365d16f6a50e58dcfd29d740/transport/tlscommon/tls_config.go#L269

In VerifyStrict I don't understand the use case of certificate pinning for client certs. It looks like it would work, but I've never seen this feature in a product.

Hopefully I'm wrong about some of this. I think the tlscommon package needs some unit tests.

I might have misunderstood. But isn't this what you're looking for -

Setting ClientCA here
The connection is verified for each new connection from this callback here
The InsecureSkipVerify is always set true because we are using custom verification method here

I will look a little further into the hostname check and verify strict.

andrewkroh · 2022-06-27T04:52:41Z

The piece I was missing was that there is a LoadTLSServerConfig. I only saw LoadTLSConfig when browsing the source and that is why I thought ClientCAs and ClientAuth was not getting set from the config (LoadTLSClientConfig might have been a better name). Aside from having unusual verify modes for client auth (and probably broken for verify_full) this looks like it should work.

* [http_endpoint] fix incorrect TLSConfig initialization * update Changelog (cherry picked from commit 1a75aea)

* [http_endpoint] fix incorrect TLSConfig initialization * update Changelog (cherry picked from commit 1a75aea) Co-authored-by: Sai Kiran <85323324+r00tu53r@users.noreply.github.com>

* [http_endpoint] fix incorrect TLSConfig initialization * update Changelog

[http_endpoint] fix incorrect TLSConfig initialization (elastic#32104)

ed643d1

r00tu53r added bug Filebeat Filebeat Team:Security-External Integrations labels Jun 26, 2022

r00tu53r requested a review from a team as a code owner June 26, 2022 11:48

r00tu53r self-assigned this Jun 26, 2022

botelastic bot added needs_team Indicates that the issue/PR needs a Team:* label and removed needs_team Indicates that the issue/PR needs a Team:* label labels Jun 26, 2022

update changelog

93cc5e0

r00tu53r added 2 commits June 26, 2022 17:20

Merge branch 'main' into debug/broken-tls-handshake-httpendpoint

294c667

fix conflict

04d1755

andrewkroh reviewed Jun 26, 2022

View reviewed changes

efd6 approved these changes Jun 27, 2022

View reviewed changes

andrewkroh approved these changes Jun 27, 2022

View reviewed changes

r00tu53r added backport-v8.3.0 Automated backport with mergify backport-7.17 Automated backport to the 7.17 branch with mergify 8.4-candidate labels Jun 27, 2022

r00tu53r merged commit 1a75aea into elastic:main Jun 27, 2022

mergify bot pushed a commit that referenced this pull request Jun 27, 2022

[http_endpoint] fix incorrect TLSConfig initialization (#32105)

e5fa6dc

* [http_endpoint] fix incorrect TLSConfig initialization * update Changelog (cherry picked from commit 1a75aea)

mergify bot mentioned this pull request Jun 27, 2022

[7.17](backport #32105) [http_endpoint] fix incorrect TLSConfig initialization (#32104) #32111

Merged

mergify bot pushed a commit that referenced this pull request Jun 27, 2022

[http_endpoint] fix incorrect TLSConfig initialization (#32105)

41a37d8

* [http_endpoint] fix incorrect TLSConfig initialization * update Changelog (cherry picked from commit 1a75aea)

mergify bot mentioned this pull request Jun 27, 2022

[8.3](backport #32105) [http_endpoint] fix incorrect TLSConfig initialization (#32104) #32112

Merged

chrisberkhout pushed a commit that referenced this pull request Jun 1, 2023

[http_endpoint] fix incorrect TLSConfig initialization (#32105)

444d848

* [http_endpoint] fix incorrect TLSConfig initialization * update Changelog

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[http_endpoint] fix incorrect TLSConfig initialization (#32104) #32105

[http_endpoint] fix incorrect TLSConfig initialization (#32104) #32105

r00tu53r commented Jun 26, 2022 •

edited

Loading

elasticmachine commented Jun 26, 2022

mergify bot commented Jun 26, 2022

elasticmachine commented Jun 26, 2022 •

edited by jenkins-beats-ci bot

Loading

Build stats

Test stats 🧪

andrewkroh left a comment

efd6 commented Jun 27, 2022

efd6 left a comment

andrewkroh commented Jun 27, 2022 •

edited

Loading

r00tu53r commented Jun 27, 2022 •

edited

Loading

r00tu53r commented Jun 27, 2022 •

edited

Loading

andrewkroh commented Jun 27, 2022

[http_endpoint] fix incorrect TLSConfig initialization (#32104) #32105

[http_endpoint] fix incorrect TLSConfig initialization (#32104) #32105

Conversation

r00tu53r commented Jun 26, 2022 • edited Loading

What does this PR do?

Why is it important?

Checklist

Related issues

Logs

elasticmachine commented Jun 26, 2022

mergify bot commented Jun 26, 2022

elasticmachine commented Jun 26, 2022 • edited by jenkins-beats-ci bot Loading

💚 Build Succeeded

Build stats

Test stats 🧪

💚 Flaky test report

🤖 GitHub comments

andrewkroh left a comment

Choose a reason for hiding this comment

efd6 commented Jun 27, 2022

efd6 left a comment

Choose a reason for hiding this comment

andrewkroh commented Jun 27, 2022 • edited Loading

r00tu53r commented Jun 27, 2022 • edited Loading

r00tu53r commented Jun 27, 2022 • edited Loading

andrewkroh commented Jun 27, 2022

r00tu53r commented Jun 26, 2022 •

edited

Loading

elasticmachine commented Jun 26, 2022 •

edited by jenkins-beats-ci bot

Loading

andrewkroh commented Jun 27, 2022 •

edited

Loading

r00tu53r commented Jun 27, 2022 •

edited

Loading

r00tu53r commented Jun 27, 2022 •

edited

Loading