Cherry-pick #23858 to 7.x: Panic when using inline SSL certificate or key #23949

ph · 2021-02-09T20:18:54Z

Cherry-pick of PR #23858 to 7.x branch. Original message:

When the key or certificate was smaller than 256bytes the system was
throwing a panic, the problem was generate by a debug message. Instead
of logging part of the keys or certificate in the log we are just
writing "inline".

Fixes: #23820

What does this PR do?

Why is it important?

Checklist

My code follows the style guidelines of this project
I have commented my code, particularly in hard-to-understand areas
I have made corresponding changes to the documentation
I have made corresponding change to the default configuration files
I have added tests that prove my fix is effective or that my feature works
I have added an entry in CHANGELOG.next.asciidoc or CHANGELOG-developer.next.asciidoc.

Author's Checklist

[ ]

How to test this PR locally

Related issues

Use cases

Screenshots

Logs

* Panic when using inline SSL certificate or key When the key or certificate was smaller than 256bytes the system was throwing a panic, the problem was generate by a debug message. Instead of logging part of the keys or certificate in the log we are just writing "inline". Fixes: elastic#23820 * changelog (cherry picked from commit 359cd74)

elasticmachine · 2021-02-09T20:18:57Z

Pinging @elastic/agent (Team:Agent)

elasticmachine · 2021-02-09T20:25:18Z

💔 Tests Failed

the below badges are clickable and redirect to their specific view in the CI or DOCS

Expand to view the summary

Build stats

Build Cause: Pull request #23949 updated
Start Time: 2021-02-09T20:23:24.705+0000
Duration: 47 min 14 sec
Commit: cabd8a4

Test stats 🧪

Test	Results
Failed	1
Passed	45653
Skipped	4799
Total	50453

Trends 🧪

Test errors

Expand to view the tests failures

`Build&Test / filebeat-windows-7-32-windows-7-32-bit / test_syslog_with_udp – filebeat.tests.system.test_syslog.Test`

Expand to view the error details

 AssertionError: assert "'su root' fa... /dev/pts/8 4" == "'su root' fa... /dev/pts/8 0"   Skipping 33 identical leading characters in diff, use -v to show   - dev/pts/8 0   ?           ^   + dev/pts/8 4   ?           ^

Expand to view the stacktrace

 self = <test_syslog.Test testMethod=test_syslog_with_udp>

        def test_syslog_with_udp(self):
            """
            Test syslog input with events from TCP.
            """
            host = "127.0.0.1"
            port = 8080
            input_raw = """
    - type: syslog
      protocol:
        udp:
            host: "{}:{}"
    """
    
            input_raw = input_raw.format(host, port)
            self.render_config_template(
                input_raw=input_raw,
                inputs=False,
            )
    
            filebeat = self.start_beat()
    
            self.wait_until(lambda: self.log_contains("Started listening for UDP connection"))
    
            sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)  # TCP
    
            for n in range(0, 50):
                m = "<13>Oct 11 22:14:15 wopr.mymachine.co postfix/smtpd[2000]:" \
                    " 'su root' failed for lonvick on /dev/pts/8 {}\n"
                m = m.format(n)
                sock.sendto(m.encode("utf-8"), (host, port))
    
            self.wait_until(lambda: self.output_count(lambda x: x >= 1))
            filebeat.check_kill_and_wait()
            sock.close()
    
            output = self.read_output()
>           self.assert_syslog(output[0])

tests\system\test_syslog.py:131: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <test_syslog.Test testMethod=test_syslog_with_udp>
syslog = {'@metadata.beat': 'filebeat', '@metadata.truncated': False, '@metadata.type': '_doc', '@metadata.version': '7.12.0', ...}
has_address = True

    def assert_syslog(self, syslog, has_address=True):
        assert syslog["event.severity"] == 5
        assert syslog["hostname"] == "wopr.mymachine.co"
        assert syslog["input.type"] == "syslog"
>       assert syslog["message"] == "'su root' failed for lonvick on /dev/pts/8 0"
E       AssertionError: assert "'su root' fa... /dev/pts/8 4" == "'su root' fa... /dev/pts/8 0"
E         Skipping 33 identical leading characters in diff, use -v to show
E         - dev/pts/8 0
E         ?           ^
E         + dev/pts/8 4
E         ?           ^

tests\system\test_syslog.py:257: AssertionError

Steps errors

Expand to view the steps failures

`filebeat-windows-7-32-windows-7-32-bit - mage build unitTest`

Took 6 min 38 sec . View more details on here
Description: mage build unitTest

`Error signal`

Took 0 min 0 sec . View more details on here
Description: Error 'hudson.AbortException: script returned exit code 1'

Log output

Expand to view the last 100 lines of log output

[2021-02-09T21:10:14.330Z] 16.78s call     x-pack/filebeat/tests/system/test_xpack_modules.py::XPackTest::test_fileset_file_259_snort
[2021-02-09T21:10:14.330Z] 16.19s call     x-pack/filebeat/tests/system/test_xpack_modules.py::XPackTest::test_fileset_file_065_o365
[2021-02-09T21:10:14.330Z] 10.87s call     x-pack/filebeat/tests/system/test_xpack_modules.py::XPackTest::test_fileset_file_104_panw
[2021-02-09T21:10:14.330Z] 10.84s call     x-pack/filebeat/tests/system/test_xpack_modules.py::XPackTest::test_fileset_file_195_cisco
[2021-02-09T21:10:14.330Z] 10.75s call     x-pack/filebeat/tests/system/test_xpack_modules.py::XPackTest::test_fileset_file_106_panw
[2021-02-09T21:10:14.330Z] 10.74s call     x-pack/filebeat/tests/system/test_xpack_modules.py::XPackTest::test_fileset_file_090_ibmmq
[2021-02-09T21:10:14.330Z] 10.66s call     x-pack/filebeat/tests/system/test_xpack_modules.py::XPackTest::test_fileset_file_088_ibmmq
[2021-02-09T21:10:14.330Z] 10.59s call     x-pack/filebeat/tests/system/test_xpack_modules.py::XPackTest::test_fileset_file_207_cisco
[2021-02-09T21:10:14.330Z] 10.34s call     x-pack/filebeat/tests/system/test_xpack_modules.py::XPackTest::test_fileset_file_107_panw
[2021-02-09T21:10:14.330Z] 10.09s call     x-pack/filebeat/tests/system/test_xpack_modules.py::XPackTest::test_fileset_file_005_tomcat
[2021-02-09T21:10:14.330Z] 10.08s call     x-pack/filebeat/tests/system/test_xpack_modules.py::XPackTest::test_fileset_file_009_gcp
[2021-02-09T21:10:14.330Z] 10.00s call     x-pack/filebeat/tests/system/test_xpack_modules.py::XPackTest::test_fileset_file_030_oracle
[2021-02-09T21:10:14.330Z] 9.96s call     x-pack/filebeat/tests/system/test_xpack_modules.py::XPackTest::test_fileset_file_014_oracle
[2021-02-09T21:10:14.330Z] 9.92s call     x-pack/filebeat/tests/system/test_xpack_modules.py::XPackTest::test_fileset_file_006_zscaler
[2021-02-09T21:10:14.330Z] 9.81s call     x-pack/filebeat/tests/system/test_xpack_modules.py::XPackTest::test_fileset_file_102_f5
[2021-02-09T21:10:14.330Z] 9.79s call     x-pack/filebeat/tests/system/test_xpack_modules.py::XPackTest::test_fileset_file_037_fortinet
[2021-02-09T21:10:14.330Z] 9.72s call     x-pack/filebeat/tests/system/test_xpack_modules.py::XPackTest::test_fileset_file_087_juniper
[2021-02-09T21:10:14.330Z] ================ 305 passed, 292 warnings in 1443.76s (0:24:03) ================
[2021-02-09T21:10:14.330Z] >> python test: Integration Testing Complete
[2021-02-09T21:10:17.960Z] Client: Docker Engine - Community
[2021-02-09T21:10:17.961Z]  Version:           20.10.3
[2021-02-09T21:10:17.961Z]  API version:       1.41
[2021-02-09T21:10:17.961Z]  Go version:        go1.13.15
[2021-02-09T21:10:17.961Z]  Git commit:        48d30b5
[2021-02-09T21:10:17.961Z]  Built:             Fri Jan 29 14:33:13 2021
[2021-02-09T21:10:17.961Z]  OS/Arch:           linux/amd64
[2021-02-09T21:10:17.961Z]  Context:           default
[2021-02-09T21:10:17.961Z]  Experimental:      true
[2021-02-09T21:10:17.961Z] 
[2021-02-09T21:10:17.961Z] Server: Docker Engine - Community
[2021-02-09T21:10:17.961Z]  Engine:
[2021-02-09T21:10:17.961Z]   Version:          20.10.3
[2021-02-09T21:10:17.961Z]   API version:      1.41 (minimum version 1.12)
[2021-02-09T21:10:17.961Z]   Go version:       go1.13.15
[2021-02-09T21:10:17.961Z]   Git commit:       46229ca
[2021-02-09T21:10:17.961Z]   Built:            Fri Jan 29 14:31:25 2021
[2021-02-09T21:10:17.961Z]   OS/Arch:          linux/amd64
[2021-02-09T21:10:17.961Z]   Experimental:     false
[2021-02-09T21:10:17.961Z]  containerd:
[2021-02-09T21:10:17.961Z]   Version:          1.4.3
[2021-02-09T21:10:17.961Z]   GitCommit:        269548fa27e0089a8b8278fc4fc781d7f65a939b
[2021-02-09T21:10:17.961Z]  runc:
[2021-02-09T21:10:17.961Z]   Version:          1.0.0-rc92
[2021-02-09T21:10:17.961Z]   GitCommit:        ff819c7e9184c13b7c2607fe6c30ae19403a7aff
[2021-02-09T21:10:17.961Z]  docker-init:
[2021-02-09T21:10:17.961Z]   Version:          0.19.0
[2021-02-09T21:10:17.961Z]   GitCommit:        de40ad0
[2021-02-09T21:10:17.961Z] Unable to find image 'alpine:3.4' locally
[2021-02-09T21:10:18.528Z] 3.4: Pulling from library/alpine
[2021-02-09T21:10:18.786Z] c1e54eec4b57: Pulling fs layer
[2021-02-09T21:10:19.044Z] c1e54eec4b57: Verifying Checksum
[2021-02-09T21:10:19.044Z] c1e54eec4b57: Download complete
[2021-02-09T21:10:19.303Z] c1e54eec4b57: Pull complete
[2021-02-09T21:10:19.303Z] Digest: sha256:b733d4a32c4da6a00a84df2ca32791bb03df95400243648d8c539e7b4cce329c
[2021-02-09T21:10:19.303Z] Status: Downloaded newer image for alpine:3.4
[2021-02-09T21:10:21.491Z] + python .ci/scripts/pre_archive_test.py
[2021-02-09T21:10:23.393Z] Copy ./x-pack/filebeat/build into build/x-pack/filebeat/build
[2021-02-09T21:10:23.405Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-23949/src/github.com/elastic/beats/build
[2021-02-09T21:10:23.707Z] + rm -rf ve
[2021-02-09T21:10:23.707Z] + find . -type d -name vendor -exec rm -r {} ;
[2021-02-09T21:10:23.719Z] Recording test results
[2021-02-09T21:10:25.745Z] [Checks API] No suitable checks publisher found.
[2021-02-09T21:10:26.112Z] Client: Docker Engine - Community
[2021-02-09T21:10:26.112Z]  Version:           20.10.3
[2021-02-09T21:10:26.112Z]  API version:       1.41
[2021-02-09T21:10:26.112Z]  Go version:        go1.13.15
[2021-02-09T21:10:26.112Z]  Git commit:        48d30b5
[2021-02-09T21:10:26.112Z]  Built:             Fri Jan 29 14:33:13 2021
[2021-02-09T21:10:26.112Z]  OS/Arch:           linux/amd64
[2021-02-09T21:10:26.112Z]  Context:           default
[2021-02-09T21:10:26.112Z]  Experimental:      true
[2021-02-09T21:10:26.112Z] 
[2021-02-09T21:10:26.112Z] Server: Docker Engine - Community
[2021-02-09T21:10:26.112Z]  Engine:
[2021-02-09T21:10:26.112Z]   Version:          20.10.3
[2021-02-09T21:10:26.112Z]   API version:      1.41 (minimum version 1.12)
[2021-02-09T21:10:26.112Z]   Go version:       go1.13.15
[2021-02-09T21:10:26.112Z]   Git commit:       46229ca
[2021-02-09T21:10:26.112Z]   Built:            Fri Jan 29 14:31:25 2021
[2021-02-09T21:10:26.112Z]   OS/Arch:          linux/amd64
[2021-02-09T21:10:26.112Z]   Experimental:     false
[2021-02-09T21:10:26.112Z]  containerd:
[2021-02-09T21:10:26.112Z]   Version:          1.4.3
[2021-02-09T21:10:26.112Z]   GitCommit:        269548fa27e0089a8b8278fc4fc781d7f65a939b
[2021-02-09T21:10:26.112Z]  runc:
[2021-02-09T21:10:26.112Z]   Version:          1.0.0-rc92
[2021-02-09T21:10:26.112Z]   GitCommit:        ff819c7e9184c13b7c2607fe6c30ae19403a7aff
[2021-02-09T21:10:26.112Z]  docker-init:
[2021-02-09T21:10:26.112Z]   Version:          0.19.0
[2021-02-09T21:10:26.112Z]   GitCommit:        de40ad0
[2021-02-09T21:10:36.532Z] Stage "Packaging" skipped due to earlier failure(s)
[2021-02-09T21:10:36.586Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-23949/src/github.com/elastic/beats
[2021-02-09T21:10:37.568Z] Running on worker-1244230 in /var/lib/jenkins/workspace/Beats_beats_PR-23949
[2021-02-09T21:10:37.642Z] [INFO] getVaultSecret: Getting secrets
[2021-02-09T21:10:37.725Z] Masking supported pattern matches of $VAULT_ADDR or $VAULT_ROLE_ID or $VAULT_SECRET_ID
[2021-02-09T21:10:40.029Z] + chmod 755 generate-build-data.sh
[2021-02-09T21:10:40.029Z] + ./generate-build-data.sh https://beats-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/Beats/beats/PR-23949/ https://beats-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/Beats/beats/PR-23949/runs/2 FAILURE 2833501
[2021-02-09T21:10:40.029Z] INFO: curl https://beats-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/Beats/beats/PR-23949/runs/2/steps/?limit=10000 -o steps-info.json
[2021-02-09T21:10:45.194Z] INFO: curl https://beats-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/Beats/beats/PR-23949/runs/2/tests/?status=FAILED -o tests-errors.json
[2021-02-09T21:10:45.949Z] INFO: curl https://beats-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/Beats/beats/PR-23949/runs/2/log/ -o pipeline-log.txt

🐛 Flaky test report

❕ There are test failures but not known flaky tests.

Expand to view the summary

Test stats 🧪

Test	Results
Failed	1
Passed	45653
Skipped	4799
Total	50453

Genuine test errors

💔 There are test failures but not known flaky tests, most likely a genuine test failure.

Name: Build&Test / filebeat-windows-7-32-windows-7-32-bit / test_syslog_with_udp – filebeat.tests.system.test_syslog.Test

blakerouse

Backport looks good.

ph · 2021-02-10T16:03:09Z

Jenkins test this please

ph · 2021-02-11T12:57:45Z

Jenkins test this please

ph · 2021-02-11T19:52:14Z

merging this, talked with @urso he will do a PR to improve the udp test.

ph added [zube]: In Review backport Team:Elastic-Agent Label for the Agent team labels Feb 9, 2021

botelastic bot added needs_team Indicates that the issue/PR needs a Team:* label and removed needs_team Indicates that the issue/PR needs a Team:* label labels Feb 9, 2021

Changelog

cabd8a4

ph requested a review from andrewkroh February 9, 2021 20:22

blakerouse approved these changes Feb 10, 2021

View reviewed changes

ph merged commit 8758310 into elastic:7.x Feb 11, 2021

zube bot added [zube]: Done and removed [zube]: In Review labels Feb 11, 2021

zube bot removed the [zube]: Done label May 13, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Cherry-pick #23858 to 7.x: Panic when using inline SSL certificate or key #23949

Cherry-pick #23858 to 7.x: Panic when using inline SSL certificate or key #23949

ph commented Feb 9, 2021 •

edited by zube bot

Loading

elasticmachine commented Feb 9, 2021

elasticmachine commented Feb 9, 2021 •

edited by jenkins-beats-ci bot

Loading

Build stats

Test stats 🧪

Trends 🧪

`Build&Test / filebeat-windows-7-32-windows-7-32-bit / test_syslog_with_udp – filebeat.tests.system.test_syslog.Test`

`filebeat-windows-7-32-windows-7-32-bit - mage build unitTest`

`Error signal`

Test stats 🧪

Genuine test errors

blakerouse left a comment

ph commented Feb 10, 2021

ph commented Feb 11, 2021

ph commented Feb 11, 2021

Cherry-pick #23858 to 7.x: Panic when using inline SSL certificate or key #23949

Cherry-pick #23858 to 7.x: Panic when using inline SSL certificate or key #23949

Conversation

ph commented Feb 9, 2021 • edited by zube bot Loading

What does this PR do?

Why is it important?

Checklist

Author's Checklist

How to test this PR locally

Related issues

Use cases

Screenshots

Logs

elasticmachine commented Feb 9, 2021

elasticmachine commented Feb 9, 2021 • edited by jenkins-beats-ci bot Loading

💔 Tests Failed

Build stats

Test stats 🧪

Trends 🧪

Test errors

Build&Test / filebeat-windows-7-32-windows-7-32-bit / test_syslog_with_udp – filebeat.tests.system.test_syslog.Test

Steps errors

filebeat-windows-7-32-windows-7-32-bit - mage build unitTest

Error signal

Log output

🐛 Flaky test report

Test stats 🧪

Genuine test errors

blakerouse left a comment

Choose a reason for hiding this comment

ph commented Feb 10, 2021

ph commented Feb 11, 2021

ph commented Feb 11, 2021

ph commented Feb 9, 2021 •

edited by zube bot

Loading

elasticmachine commented Feb 9, 2021 •

edited by jenkins-beats-ci bot

Loading

`Build&Test / filebeat-windows-7-32-windows-7-32-bit / test_syslog_with_udp – filebeat.tests.system.test_syslog.Test`

`filebeat-windows-7-32-windows-7-32-bit - mage build unitTest`

`Error signal`