Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[8.6](backport #33656) [Filebeat] Add parse_aws_vpc_flow_log processor #33705

Merged
merged 1 commit into from
Nov 16, 2022
Merged

[8.6](backport #33656) [Filebeat] Add parse_aws_vpc_flow_log processor #33705

merged 1 commit into from
Nov 16, 2022

Conversation

mergify[bot]
Copy link
Contributor

@mergify mergify bot commented Nov 16, 2022 •
edited by zube bot
Loading

This is an automatic backport of pull request #33656 done by Mergify.

Mergify commands and options

More conditions and actions can be found in the documentation.

You can also trigger Mergify actions by commenting on this pull request:

  • @Mergifyio refresh will re-evaluate the rules
  • @Mergifyio rebase will rebase this PR on its base branch
  • @Mergifyio update will merge the base branch into this PR
  • @Mergifyio backport <destination> will backport this PR on <destination> branch

Additionally, on Mergify dashboard you can:

  • look at your merge queues
  • generate the Mergify configuration with the config editor.

Finally, you can contact us on https://mergify.com

@andrewkroh @mergify
[Filebeat] Add parse_aws_vpc_flow_log processor (#33656)
4a166ce 
This is a processor for parsing AWS VPC flow logs. It requires a user specified log format. It can populate the original flow log fields, ECS fields, or both.

Usage:

```yaml
processors:
  - parse_aws_vpc_flow_log:
      format: version account-id interface-id srcaddr dstaddr srcport dstport protocol packets bytes start end action log-status
  - community_id: ~
```

Benchmark:

```
goos: darwin
goarch: arm64
pkg: github.com/elastic/beats/v7/x-pack/filebeat/processors/aws_vpcflow
BenchmarkProcessorRun/original-mode-v5-message-10                2810948              2138 ns/op            2836 B/op         31 allocs/op
BenchmarkProcessorRun/ecs-mode-v5-message-10                     1914754              3107 ns/op            1908 B/op         41 allocs/op
BenchmarkProcessorRun/ecs_and_original-mode-v5-message-10        1693279              3538 ns/op            3076 B/op         41 allocs/op
```

Co-authored-by: Dan Kortschak <90160302+efd6@users.noreply.github.com>
(cherry picked from commit 1a86e42)
@mergify mergify bot requested review from a team as code owners November 16, 2022 21:08
@mergify mergify bot removed the request for review from a team November 16, 2022 21:08
@mergify mergify bot added the backport label Nov 16, 2022
@mergify mergify bot requested review from rdner and fearful-symmetry November 16, 2022 21:08
@botelastic botelastic bot added the needs_team Indicates that the issue/PR needs a Team:* label label Nov 16, 2022
@botelastic
Copy link

botelastic bot commented Nov 16, 2022

This pull request doesn't have a Team:<team> label.

@mergify mergify bot mentioned this pull request Nov 16, 2022
Merged
8 tasks
@elasticmachine
Copy link
Collaborator

💔 Tests Failed

the below badges are clickable and redirect to their specific view in the CI or DOCS
Pipeline View Test View Changes Artifacts preview preview

Expand to view the summary

Build stats

  • Start Time: 2022-11-16T21:09:14.953+0000

  • Duration: 103 min 55 sec

Test stats 🧪

Test Results
Failed 10
Passed 24034
Skipped 1950
Total 25994

Test errors 10

Expand to view the tests failures

Build&Test / x-pack/metricbeat-pythonIntegTest / test_dashboards – x-pack.metricbeat.tests.system.test_xpack_base.Test
    Expand to view the error details

     failed on setup with "Exception: Timeout while waiting for healthy docker-compose services: elasticsearch,kibana"

    Expand to view the stacktrace

     self = <class 'test_xpack_base.Test'>

    @classmethod
    def setUpClass(self):
        self.beat_name = "metricbeat"
        self.beat_path = os.path.abspath(
            os.path.join(os.path.dirname(__file__), "../../"))
    
        self.template_paths = [
            os.path.abspath(os.path.join(self.beat_path, "../../metricbeat")),
            os.path.abspath(os.path.join(self.beat_path, "../../libbeat")),
        ]
    
>       super(XPackTest, self).setUpClass()

tests/system/xpack_metricbeat.py:19: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
../../metricbeat/tests/system/metricbeat.py:42: in setUpClass
    super().setUpClass()
../../libbeat/tests/system/beat/beat.py:204: in setUpClass
    cls.compose_up_with_retries()
../../libbeat/tests/system/beat/beat.py:222: in compose_up_with_retries
    raise ex
../../libbeat/tests/system/beat/beat.py:218: in compose_up_with_retries
    cls.compose_up()
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

cls = <class 'test_xpack_base.Test'>

    @classmethod
    def compose_up(cls):
        """
        Ensure *only* the services defined under `COMPOSE_SERVICES` are running and healthy
        """
        if not INTEGRATION_TESTS or not cls.COMPOSE_SERVICES:
            return
    
        if os.environ.get('NO_COMPOSE'):
            return
    
        def print_logs(container):
            print("---- " + container.name_without_project)
            print(container.logs())
            print("----")
    
        def is_healthy(container):
            return container.inspect()['State']['Health']['Status'] == 'healthy'
    
        project = cls.compose_project()
    
        with disabled_logger('compose.service'):
            project.pull(
                ignore_pull_failures=True,
                service_names=cls.COMPOSE_SERVICES)
    
        project.up(
            strategy=ConvergenceStrategy.always,
            service_names=cls.COMPOSE_SERVICES,
            timeout=30)
    
        # Wait for them to be healthy
        start = time.time()
        while True:
            containers = project.containers(
                service_names=cls.COMPOSE_SERVICES,
                stopped=True)
    
            healthy = True
            for container in containers:
                if not container.is_running:
                    print_logs(container)
                    raise Exception(
                        "Container %s unexpectedly finished on startup" %
                        container.name_without_project)
                if not is_healthy(container):
                    healthy = False
                    break
    
            if healthy:
                break
    
            if cls.COMPOSE_ADVERTISED_HOST:
                for service in cls.COMPOSE_SERVICES:
                    cls._setup_advertised_host(project, service)
    
            time.sleep(1)
            timeout = time.time() - start > cls.COMPOSE_TIMEOUT
            if timeout:
                for container in containers:
                    if not is_healthy(container):
                        print_logs(container)
>               raise Exception(
                    "Timeout while waiting for healthy "
                    "docker-compose services: %s" %
                    ','.join(cls.COMPOSE_SERVICES))
E               Exception: Timeout while waiting for healthy docker-compose services: elasticsearch,kibana

../../libbeat/tests/system/beat/compose.py:102: Exception

Build&Test / x-pack/metricbeat-pythonIntegTest / test_export_config – x-pack.metricbeat.tests.system.test_xpack_base.Test
    Expand to view the error details

     failed on setup with "Exception: Timeout while waiting for healthy docker-compose services: elasticsearch,kibana"

    Expand to view the stacktrace

     self = <class 'test_xpack_base.Test'>

    @classmethod
    def setUpClass(self):
        self.beat_name = "metricbeat"
        self.beat_path = os.path.abspath(
            os.path.join(os.path.dirname(__file__), "../../"))
    
        self.template_paths = [
            os.path.abspath(os.path.join(self.beat_path, "../../metricbeat")),
            os.path.abspath(os.path.join(self.beat_path, "../../libbeat")),
        ]
    
>       super(XPackTest, self).setUpClass()

tests/system/xpack_metricbeat.py:19: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
../../metricbeat/tests/system/metricbeat.py:42: in setUpClass
    super().setUpClass()
../../libbeat/tests/system/beat/beat.py:204: in setUpClass
    cls.compose_up_with_retries()
../../libbeat/tests/system/beat/beat.py:222: in compose_up_with_retries
    raise ex
../../libbeat/tests/system/beat/beat.py:218: in compose_up_with_retries
    cls.compose_up()
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

cls = <class 'test_xpack_base.Test'>

    @classmethod
    def compose_up(cls):
        """
        Ensure *only* the services defined under `COMPOSE_SERVICES` are running and healthy
        """
        if not INTEGRATION_TESTS or not cls.COMPOSE_SERVICES:
            return
    
        if os.environ.get('NO_COMPOSE'):
            return
    
        def print_logs(container):
            print("---- " + container.name_without_project)
            print(container.logs())
            print("----")
    
        def is_healthy(container):
            return container.inspect()['State']['Health']['Status'] == 'healthy'
    
        project = cls.compose_project()
    
        with disabled_logger('compose.service'):
            project.pull(
                ignore_pull_failures=True,
                service_names=cls.COMPOSE_SERVICES)
    
        project.up(
            strategy=ConvergenceStrategy.always,
            service_names=cls.COMPOSE_SERVICES,
            timeout=30)
    
        # Wait for them to be healthy
        start = time.time()
        while True:
            containers = project.containers(
                service_names=cls.COMPOSE_SERVICES,
                stopped=True)
    
            healthy = True
            for container in containers:
                if not container.is_running:
                    print_logs(container)
                    raise Exception(
                        "Container %s unexpectedly finished on startup" %
                        container.name_without_project)
                if not is_healthy(container):
                    healthy = False
                    break
    
            if healthy:
                break
    
            if cls.COMPOSE_ADVERTISED_HOST:
                for service in cls.COMPOSE_SERVICES:
                    cls._setup_advertised_host(project, service)
    
            time.sleep(1)
            timeout = time.time() - start > cls.COMPOSE_TIMEOUT
            if timeout:
                for container in containers:
                    if not is_healthy(container):
                        print_logs(container)
>               raise Exception(
                    "Timeout while waiting for healthy "
                    "docker-compose services: %s" %
                    ','.join(cls.COMPOSE_SERVICES))
E               Exception: Timeout while waiting for healthy docker-compose services: elasticsearch,kibana

../../libbeat/tests/system/beat/compose.py:102: Exception

Build&Test / x-pack/metricbeat-pythonIntegTest / test_export_ilm_policy – x-pack.metricbeat.tests.system.test_xpack_base.Test
    Expand to view the error details

     failed on setup with "Exception: Timeout while waiting for healthy docker-compose services: elasticsearch,kibana"

    Expand to view the stacktrace

     self = <class 'test_xpack_base.Test'>

    @classmethod
    def setUpClass(self):
        self.beat_name = "metricbeat"
        self.beat_path = os.path.abspath(
            os.path.join(os.path.dirname(__file__), "../../"))
    
        self.template_paths = [
            os.path.abspath(os.path.join(self.beat_path, "../../metricbeat")),
            os.path.abspath(os.path.join(self.beat_path, "../../libbeat")),
        ]
    
>       super(XPackTest, self).setUpClass()

tests/system/xpack_metricbeat.py:19: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
../../metricbeat/tests/system/metricbeat.py:42: in setUpClass
    super().setUpClass()
../../libbeat/tests/system/beat/beat.py:204: in setUpClass
    cls.compose_up_with_retries()
../../libbeat/tests/system/beat/beat.py:222: in compose_up_with_retries
    raise ex
../../libbeat/tests/system/beat/beat.py:218: in compose_up_with_retries
    cls.compose_up()
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

cls = <class 'test_xpack_base.Test'>

    @classmethod
    def compose_up(cls):
        """
        Ensure *only* the services defined under `COMPOSE_SERVICES` are running and healthy
        """
        if not INTEGRATION_TESTS or not cls.COMPOSE_SERVICES:
            return
    
        if os.environ.get('NO_COMPOSE'):
            return
    
        def print_logs(container):
            print("---- " + container.name_without_project)
            print(container.logs())
            print("----")
    
        def is_healthy(container):
            return container.inspect()['State']['Health']['Status'] == 'healthy'
    
        project = cls.compose_project()
    
        with disabled_logger('compose.service'):
            project.pull(
                ignore_pull_failures=True,
                service_names=cls.COMPOSE_SERVICES)
    
        project.up(
            strategy=ConvergenceStrategy.always,
            service_names=cls.COMPOSE_SERVICES,
            timeout=30)
    
        # Wait for them to be healthy
        start = time.time()
        while True:
            containers = project.containers(
                service_names=cls.COMPOSE_SERVICES,
                stopped=True)
    
            healthy = True
            for container in containers:
                if not container.is_running:
                    print_logs(container)
                    raise Exception(
                        "Container %s unexpectedly finished on startup" %
                        container.name_without_project)
                if not is_healthy(container):
                    healthy = False
                    break
    
            if healthy:
                break
    
            if cls.COMPOSE_ADVERTISED_HOST:
                for service in cls.COMPOSE_SERVICES:
                    cls._setup_advertised_host(project, service)
    
            time.sleep(1)
            timeout = time.time() - start > cls.COMPOSE_TIMEOUT
            if timeout:
                for container in containers:
                    if not is_healthy(container):
                        print_logs(container)
>               raise Exception(
                    "Timeout while waiting for healthy "
                    "docker-compose services: %s" %
                    ','.join(cls.COMPOSE_SERVICES))
E               Exception: Timeout while waiting for healthy docker-compose services: elasticsearch,kibana

../../libbeat/tests/system/beat/compose.py:102: Exception

Build&Test / x-pack/metricbeat-pythonIntegTest / test_export_index_pattern – x-pack.metricbeat.tests.system.test_xpack_base.Test
    Expand to view the error details

     failed on setup with "Exception: Timeout while waiting for healthy docker-compose services: elasticsearch,kibana"

    Expand to view the stacktrace

     self = <class 'test_xpack_base.Test'>

    @classmethod
    def setUpClass(self):
        self.beat_name = "metricbeat"
        self.beat_path = os.path.abspath(
            os.path.join(os.path.dirname(__file__), "../../"))
    
        self.template_paths = [
            os.path.abspath(os.path.join(self.beat_path, "../../metricbeat")),
            os.path.abspath(os.path.join(self.beat_path, "../../libbeat")),
        ]
    
>       super(XPackTest, self).setUpClass()

tests/system/xpack_metricbeat.py:19: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
../../metricbeat/tests/system/metricbeat.py:42: in setUpClass
    super().setUpClass()
../../libbeat/tests/system/beat/beat.py:204: in setUpClass
    cls.compose_up_with_retries()
../../libbeat/tests/system/beat/beat.py:222: in compose_up_with_retries
    raise ex
../../libbeat/tests/system/beat/beat.py:218: in compose_up_with_retries
    cls.compose_up()
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

cls = <class 'test_xpack_base.Test'>

    @classmethod
    def compose_up(cls):
        """
        Ensure *only* the services defined under `COMPOSE_SERVICES` are running and healthy
        """
        if not INTEGRATION_TESTS or not cls.COMPOSE_SERVICES:
            return
    
        if os.environ.get('NO_COMPOSE'):
            return
    
        def print_logs(container):
            print("---- " + container.name_without_project)
            print(container.logs())
            print("----")
    
        def is_healthy(container):
            return container.inspect()['State']['Health']['Status'] == 'healthy'
    
        project = cls.compose_project()
    
        with disabled_logger('compose.service'):
            project.pull(
                ignore_pull_failures=True,
                service_names=cls.COMPOSE_SERVICES)
    
        project.up(
            strategy=ConvergenceStrategy.always,
            service_names=cls.COMPOSE_SERVICES,
            timeout=30)
    
        # Wait for them to be healthy
        start = time.time()
        while True:
            containers = project.containers(
                service_names=cls.COMPOSE_SERVICES,
                stopped=True)
    
            healthy = True
            for container in containers:
                if not container.is_running:
                    print_logs(container)
                    raise Exception(
                        "Container %s unexpectedly finished on startup" %
                        container.name_without_project)
                if not is_healthy(container):
                    healthy = False
                    break
    
            if healthy:
                break
    
            if cls.COMPOSE_ADVERTISED_HOST:
                for service in cls.COMPOSE_SERVICES:
                    cls._setup_advertised_host(project, service)
    
            time.sleep(1)
            timeout = time.time() - start > cls.COMPOSE_TIMEOUT
            if timeout:
                for container in containers:
                    if not is_healthy(container):
                        print_logs(container)
>               raise Exception(
                    "Timeout while waiting for healthy "
                    "docker-compose services: %s" %
                    ','.join(cls.COMPOSE_SERVICES))
E               Exception: Timeout while waiting for healthy docker-compose services: elasticsearch,kibana

../../libbeat/tests/system/beat/compose.py:102: Exception

Build&Test / x-pack/metricbeat-pythonIntegTest / test_export_index_pattern_migration – x-pack.metricbeat.tests.system.test_xpack_base.Test
    Expand to view the error details

     failed on setup with "Exception: Timeout while waiting for healthy docker-compose services: elasticsearch,kibana"

    Expand to view the stacktrace

     self = <class 'test_xpack_base.Test'>

    @classmethod
    def setUpClass(self):
        self.beat_name = "metricbeat"
        self.beat_path = os.path.abspath(
            os.path.join(os.path.dirname(__file__), "../../"))
    
        self.template_paths = [
            os.path.abspath(os.path.join(self.beat_path, "../../metricbeat")),
            os.path.abspath(os.path.join(self.beat_path, "../../libbeat")),
        ]
    
>       super(XPackTest, self).setUpClass()

tests/system/xpack_metricbeat.py:19: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
../../metricbeat/tests/system/metricbeat.py:42: in setUpClass
    super().setUpClass()
../../libbeat/tests/system/beat/beat.py:204: in setUpClass
    cls.compose_up_with_retries()
../../libbeat/tests/system/beat/beat.py:222: in compose_up_with_retries
    raise ex
../../libbeat/tests/system/beat/beat.py:218: in compose_up_with_retries
    cls.compose_up()
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

cls = <class 'test_xpack_base.Test'>

    @classmethod
    def compose_up(cls):
        """
        Ensure *only* the services defined under `COMPOSE_SERVICES` are running and healthy
        """
        if not INTEGRATION_TESTS or not cls.COMPOSE_SERVICES:
            return
    
        if os.environ.get('NO_COMPOSE'):
            return
    
        def print_logs(container):
            print("---- " + container.name_without_project)
            print(container.logs())
            print("----")
    
        def is_healthy(container):
            return container.inspect()['State']['Health']['Status'] == 'healthy'
    
        project = cls.compose_project()
    
        with disabled_logger('compose.service'):
            project.pull(
                ignore_pull_failures=True,
                service_names=cls.COMPOSE_SERVICES)
    
        project.up(
            strategy=ConvergenceStrategy.always,
            service_names=cls.COMPOSE_SERVICES,
            timeout=30)
    
        # Wait for them to be healthy
        start = time.time()
        while True:
            containers = project.containers(
                service_names=cls.COMPOSE_SERVICES,
                stopped=True)
    
            healthy = True
            for container in containers:
                if not container.is_running:
                    print_logs(container)
                    raise Exception(
                        "Container %s unexpectedly finished on startup" %
                        container.name_without_project)
                if not is_healthy(container):
                    healthy = False
                    break
    
            if healthy:
                break
    
            if cls.COMPOSE_ADVERTISED_HOST:
                for service in cls.COMPOSE_SERVICES:
                    cls._setup_advertised_host(project, service)
    
            time.sleep(1)
            timeout = time.time() - start > cls.COMPOSE_TIMEOUT
            if timeout:
                for container in containers:
                    if not is_healthy(container):
                        print_logs(container)
>               raise Exception(
                    "Timeout while waiting for healthy "
                    "docker-compose services: %s" %
                    ','.join(cls.COMPOSE_SERVICES))
E               Exception: Timeout while waiting for healthy docker-compose services: elasticsearch,kibana

../../libbeat/tests/system/beat/compose.py:102: Exception

Build&Test / x-pack/metricbeat-pythonIntegTest / test_export_template – x-pack.metricbeat.tests.system.test_xpack_base.Test
    Expand to view the error details

     failed on setup with "Exception: Timeout while waiting for healthy docker-compose services: elasticsearch,kibana"

    Expand to view the stacktrace

     self = <class 'test_xpack_base.Test'>

    @classmethod
    def setUpClass(self):
        self.beat_name = "metricbeat"
        self.beat_path = os.path.abspath(
            os.path.join(os.path.dirname(__file__), "../../"))
    
        self.template_paths = [
            os.path.abspath(os.path.join(self.beat_path, "../../metricbeat")),
            os.path.abspath(os.path.join(self.beat_path, "../../libbeat")),
        ]
    
>       super(XPackTest, self).setUpClass()

tests/system/xpack_metricbeat.py:19: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
../../metricbeat/tests/system/metricbeat.py:42: in setUpClass
    super().setUpClass()
../../libbeat/tests/system/beat/beat.py:204: in setUpClass
    cls.compose_up_with_retries()
../../libbeat/tests/system/beat/beat.py:222: in compose_up_with_retries
    raise ex
../../libbeat/tests/system/beat/beat.py:218: in compose_up_with_retries
    cls.compose_up()
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

cls = <class 'test_xpack_base.Test'>

    @classmethod
    def compose_up(cls):
        """
        Ensure *only* the services defined under `COMPOSE_SERVICES` are running and healthy
        """
        if not INTEGRATION_TESTS or not cls.COMPOSE_SERVICES:
            return
    
        if os.environ.get('NO_COMPOSE'):
            return
    
        def print_logs(container):
            print("---- " + container.name_without_project)
            print(container.logs())
            print("----")
    
        def is_healthy(container):
            return container.inspect()['State']['Health']['Status'] == 'healthy'
    
        project = cls.compose_project()
    
        with disabled_logger('compose.service'):
            project.pull(
                ignore_pull_failures=True,
                service_names=cls.COMPOSE_SERVICES)
    
        project.up(
            strategy=ConvergenceStrategy.always,
            service_names=cls.COMPOSE_SERVICES,
            timeout=30)
    
        # Wait for them to be healthy
        start = time.time()
        while True:
            containers = project.containers(
                service_names=cls.COMPOSE_SERVICES,
                stopped=True)
    
            healthy = True
            for container in containers:
                if not container.is_running:
                    print_logs(container)
                    raise Exception(
                        "Container %s unexpectedly finished on startup" %
                        container.name_without_project)
                if not is_healthy(container):
                    healthy = False
                    break
    
            if healthy:
                break
    
            if cls.COMPOSE_ADVERTISED_HOST:
                for service in cls.COMPOSE_SERVICES:
                    cls._setup_advertised_host(project, service)
    
            time.sleep(1)
            timeout = time.time() - start > cls.COMPOSE_TIMEOUT
            if timeout:
                for container in containers:
                    if not is_healthy(container):
                        print_logs(container)
>               raise Exception(
                    "Timeout while waiting for healthy "
                    "docker-compose services: %s" %
                    ','.join(cls.COMPOSE_SERVICES))
E               Exception: Timeout while waiting for healthy docker-compose services: elasticsearch,kibana

../../libbeat/tests/system/beat/compose.py:102: Exception

Build&Test / x-pack/metricbeat-pythonIntegTest / test_index_management – x-pack.metricbeat.tests.system.test_xpack_base.Test
    Expand to view the error details

     failed on setup with "Exception: Timeout while waiting for healthy docker-compose services: elasticsearch,kibana"

    Expand to view the stacktrace

     self = <class 'test_xpack_base.Test'>

    @classmethod
    def setUpClass(self):
        self.beat_name = "metricbeat"
        self.beat_path = os.path.abspath(
            os.path.join(os.path.dirname(__file__), "../../"))
    
        self.template_paths = [
            os.path.abspath(os.path.join(self.beat_path, "../../metricbeat")),
            os.path.abspath(os.path.join(self.beat_path, "../../libbeat")),
        ]
    
>       super(XPackTest, self).setUpClass()

tests/system/xpack_metricbeat.py:19: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
../../metricbeat/tests/system/metricbeat.py:42: in setUpClass
    super().setUpClass()
../../libbeat/tests/system/beat/beat.py:204: in setUpClass
    cls.compose_up_with_retries()
../../libbeat/tests/system/beat/beat.py:222: in compose_up_with_retries
    raise ex
../../libbeat/tests/system/beat/beat.py:218: in compose_up_with_retries
    cls.compose_up()
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

cls = <class 'test_xpack_base.Test'>

    @classmethod
    def compose_up(cls):
        """
        Ensure *only* the services defined under `COMPOSE_SERVICES` are running and healthy
        """
        if not INTEGRATION_TESTS or not cls.COMPOSE_SERVICES:
            return
    
        if os.environ.get('NO_COMPOSE'):
            return
    
        def print_logs(container):
            print("---- " + container.name_without_project)
            print(container.logs())
            print("----")
    
        def is_healthy(container):
            return container.inspect()['State']['Health']['Status'] == 'healthy'
    
        project = cls.compose_project()
    
        with disabled_logger('compose.service'):
            project.pull(
                ignore_pull_failures=True,
                service_names=cls.COMPOSE_SERVICES)
    
        project.up(
            strategy=ConvergenceStrategy.always,
            service_names=cls.COMPOSE_SERVICES,
            timeout=30)
    
        # Wait for them to be healthy
        start = time.time()
        while True:
            containers = project.containers(
                service_names=cls.COMPOSE_SERVICES,
                stopped=True)
    
            healthy = True
            for container in containers:
                if not container.is_running:
                    print_logs(container)
                    raise Exception(
                        "Container %s unexpectedly finished on startup" %
                        container.name_without_project)
                if not is_healthy(container):
                    healthy = False
                    break
    
            if healthy:
                break
    
            if cls.COMPOSE_ADVERTISED_HOST:
                for service in cls.COMPOSE_SERVICES:
                    cls._setup_advertised_host(project, service)
    
            time.sleep(1)
            timeout = time.time() - start > cls.COMPOSE_TIMEOUT
            if timeout:
                for container in containers:
                    if not is_healthy(container):
                        print_logs(container)
>               raise Exception(
                    "Timeout while waiting for healthy "
                    "docker-compose services: %s" %
                    ','.join(cls.COMPOSE_SERVICES))
E               Exception: Timeout while waiting for healthy docker-compose services: elasticsearch,kibana

../../libbeat/tests/system/beat/compose.py:102: Exception

Build&Test / x-pack/metricbeat-pythonIntegTest / test_start_stop – x-pack.metricbeat.tests.system.test_xpack_base.Test
    Expand to view the error details

     failed on setup with "Exception: Timeout while waiting for healthy docker-compose services: elasticsearch,kibana"

    Expand to view the stacktrace

     self = <class 'test_xpack_base.Test'>

    @classmethod
    def setUpClass(self):
        self.beat_name = "metricbeat"
        self.beat_path = os.path.abspath(
            os.path.join(os.path.dirname(__file__), "../../"))
    
        self.template_paths = [
            os.path.abspath(os.path.join(self.beat_path, "../../metricbeat")),
            os.path.abspath(os.path.join(self.beat_path, "../../libbeat")),
        ]
    
>       super(XPackTest, self).setUpClass()

tests/system/xpack_metricbeat.py:19: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
../../metricbeat/tests/system/metricbeat.py:42: in setUpClass
    super().setUpClass()
../../libbeat/tests/system/beat/beat.py:204: in setUpClass
    cls.compose_up_with_retries()
../../libbeat/tests/system/beat/beat.py:222: in compose_up_with_retries
    raise ex
../../libbeat/tests/system/beat/beat.py:218: in compose_up_with_retries
    cls.compose_up()
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

cls = <class 'test_xpack_base.Test'>

    @classmethod
    def compose_up(cls):
        """
        Ensure *only* the services defined under `COMPOSE_SERVICES` are running and healthy
        """
        if not INTEGRATION_TESTS or not cls.COMPOSE_SERVICES:
            return
    
        if os.environ.get('NO_COMPOSE'):
            return
    
        def print_logs(container):
            print("---- " + container.name_without_project)
            print(container.logs())
            print("----")
    
        def is_healthy(container):
            return container.inspect()['State']['Health']['Status'] == 'healthy'
    
        project = cls.compose_project()
    
        with disabled_logger('compose.service'):
            project.pull(
                ignore_pull_failures=True,
                service_names=cls.COMPOSE_SERVICES)
    
        project.up(
            strategy=ConvergenceStrategy.always,
            service_names=cls.COMPOSE_SERVICES,
            timeout=30)
    
        # Wait for them to be healthy
        start = time.time()
        while True:
            containers = project.containers(
                service_names=cls.COMPOSE_SERVICES,
                stopped=True)
    
            healthy = True
            for container in containers:
                if not container.is_running:
                    print_logs(container)
                    raise Exception(
                        "Container %s unexpectedly finished on startup" %
                        container.name_without_project)
                if not is_healthy(container):
                    healthy = False
                    break
    
            if healthy:
                break
    
            if cls.COMPOSE_ADVERTISED_HOST:
                for service in cls.COMPOSE_SERVICES:
                    cls._setup_advertised_host(project, service)
    
            time.sleep(1)
            timeout = time.time() - start > cls.COMPOSE_TIMEOUT
            if timeout:
                for container in containers:
                    if not is_healthy(container):
                        print_logs(container)
>               raise Exception(
                    "Timeout while waiting for healthy "
                    "docker-compose services: %s" %
                    ','.join(cls.COMPOSE_SERVICES))
E               Exception: Timeout while waiting for healthy docker-compose services: elasticsearch,kibana

../../libbeat/tests/system/beat/compose.py:102: Exception

Build&Test / x-pack/metricbeat-pythonIntegTest / test_health – x-pack.metricbeat.module.enterprisesearch.test_enterprisesearch.Test
    Expand to view the error details

     failed on setup with "DeprecationWarning: The 'warn' method is deprecated, use 'warning' instead"

    Expand to view the stacktrace

     self = <docker.api.client.APIClient object at 0x7f4a8f3c97f0>
response = <Response [500]>

    def _raise_for_status(self, response):
        """Raises stored :class:`APIError`, if one occurred."""
        try:
>           response.raise_for_status()

../../build/ve/docker/lib/python3.9/site-packages/docker/api/client.py:268: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <Response [500]>

    def raise_for_status(self):
        """Raises :class:`HTTPError`, if one occurred."""
    
        http_error_msg = ''
        if isinstance(self.reason, bytes):
            # We attempt to decode utf-8 first because some servers
            # choose to localize their reason strings. If the string
            # isn't utf-8, we fall back to iso-8859-1 for all other
            # encodings. (See PR #3538)
            try:
                reason = self.reason.decode('utf-8')
            except UnicodeDecodeError:
                reason = self.reason.decode('iso-8859-1')
        else:
            reason = self.reason
    
        if 400 <= self.status_code < 500:
            http_error_msg = u'%s Client Error: %s for url: %s' % (self.status_code, reason, self.url)
    
        elif 500 <= self.status_code < 600:
            http_error_msg = u'%s Server Error: %s for url: %s' % (self.status_code, reason, self.url)
    
        if http_error_msg:
>           raise HTTPError(http_error_msg, response=self)
E           requests.exceptions.HTTPError: 500 Server Error: Internal Server Error for url: http+docker://localhost/v1.41/containers/8e812f65f39bbaa7025b9e84d7db712a4bed0a7c326f74b656651924b8589039/start

../../build/ve/docker/lib/python3.9/site-packages/requests/models.py:943: HTTPError

During handling of the above exception, another exception occurred:

self = <Service: elasticsearch>
container = <Container: enterprisesearch_7918a246f6f8_elasticsearch_1 (8e812f)>
use_network_aliases = True

    def start_container(self, container, use_network_aliases=True):
        self.connect_container_to_networks(container, use_network_aliases)
        try:
>           container.start()

../../build/ve/docker/lib/python3.9/site-packages/compose/service.py:643: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <Container: enterprisesearch_7918a246f6f8_elasticsearch_1 (8e812f)>
options = {}

    def start(self, **options):
>       return self.client.start(self.id, **options)

../../build/ve/docker/lib/python3.9/site-packages/compose/container.py:228: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <docker.api.client.APIClient object at 0x7f4a8f3c97f0>
resource_id = '8e812f65f39bbaa7025b9e84d7db712a4bed0a7c326f74b656651924b8589039'
args = (), kwargs = {}

    @functools.wraps(f)
    def wrapped(self, resource_id=None, *args, **kwargs):
        if resource_id is None and kwargs.get(resource_name):
            resource_id = kwargs.pop(resource_name)
        if isinstance(resource_id, dict):
            resource_id = resource_id.get('Id', resource_id.get('ID'))
        if not resource_id:
            raise errors.NullResource(
                'Resource ID was not provided'
            )
>       return f(self, resource_id, *args, **kwargs)

../../build/ve/docker/lib/python3.9/site-packages/docker/utils/decorators.py:19: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <docker.api.client.APIClient object at 0x7f4a8f3c97f0>
container = '8e812f65f39bbaa7025b9e84d7db712a4bed0a7c326f74b656651924b8589039'
args = (), kwargs = {}
url = 'http+docker://localhost/v1.41/containers/8e812f65f39bbaa7025b9e84d7db712a4bed0a7c326f74b656651924b8589039/start'
res = <Response [500]>

    @utils.check_resource('container')
    def start(self, container, *args, **kwargs):
        """
        Start a container. Similar to the ``docker start`` command, but
        doesn't support attach options.
    
        **Deprecation warning:** Passing configuration options in ``start`` is
        no longer supported. Users are expected to provide host config options
        in the ``host_config`` parameter of
        :py:meth:`~ContainerApiMixin.create_container`.
    
    
        Args:
            container (str): The container to start
    
        Raises:
            :py:class:`docker.errors.APIError`
                If the server returns an error.
            :py:class:`docker.errors.DeprecatedMethod`
                If any argument besides ``container`` are provided.
    
        Example:
    
            >>> container = client.api.create_container(
            ...     image='busybox:latest',
            ...     command='/bin/sleep 30')
            >>> client.api.start(container=container.get('Id'))
        """
        if args or kwargs:
            raise errors.DeprecatedMethod(
                'Providing configuration in the start() method is no longer '
                'supported. Use the host_config param in create_container '
                'instead.'
            )
        url = self._url("/containers/{0}/start", container)
        res = self._post(url)
>       self._raise_for_status(res)

../../build/ve/docker/lib/python3.9/site-packages/docker/api/container.py:1109: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <docker.api.client.APIClient object at 0x7f4a8f3c97f0>
response = <Response [500]>

    def _raise_for_status(self, response):
        """Raises stored :class:`APIError`, if one occurred."""
        try:
            response.raise_for_status()
        except requests.exceptions.HTTPError as e:
>           raise create_api_error_from_http_exception(e)

../../build/ve/docker/lib/python3.9/site-packages/docker/api/client.py:270: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

e = HTTPError('500 Server Error: Internal Server Error for url: http+docker://localhost/v1.41/containers/8e812f65f39bbaa7025b9e84d7db712a4bed0a7c326f74b656651924b8589039/start')

    def create_api_error_from_http_exception(e):
        """
        Create a suitable APIError from requests.exceptions.HTTPError.
        """
        response = e.response
        try:
            explanation = response.json()['message']
        except ValueError:
            explanation = (response.content or '').strip()
        cls = APIError
        if response.status_code == 404:
            if explanation and ('No such image' in str(explanation) or
                                'not found: does not exist or no pull access'
                                in str(explanation) or
                                'repository does not exist' in str(explanation)):
                cls = ImageNotFound
            else:
                cls = NotFound
>       raise cls(e, response=response, explanation=explanation)
E       docker.errors.APIError: 500 Server Error for http+docker://localhost/v1.41/containers/8e812f65f39bbaa7025b9e84d7db712a4bed0a7c326f74b656651924b8589039/start: Internal Server Error ("driver failed programming external connectivity on endpoint enterprisesearch_7918a246f6f8_elasticsearch_1 (ddd6508cf16260bdff56cbe6e11418ebf11be16c12839c89fdcf83a3f3b460cf): Bind for 0.0.0.0:9200 failed: port is already allocated")

../../build/ve/docker/lib/python3.9/site-packages/docker/errors.py:31: APIError

During handling of the above exception, another exception occurred:

self = <class 'test_enterprisesearch.Test'>

    @classmethod
    def setUpClass(self):
        self.beat_name = "metricbeat"
        self.beat_path = os.path.abspath(
            os.path.join(os.path.dirname(__file__), "../../"))
    
        self.template_paths = [
            os.path.abspath(os.path.join(self.beat_path, "../../metricbeat")),
            os.path.abspath(os.path.join(self.beat_path, "../../libbeat")),
        ]
    
>       super(XPackTest, self).setUpClass()

tests/system/xpack_metricbeat.py:19: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
../../metricbeat/tests/system/metricbeat.py:42: in setUpClass
    super().setUpClass()
../../libbeat/tests/system/beat/beat.py:204: in setUpClass
    cls.compose_up_with_retries()
../../libbeat/tests/system/beat/beat.py:222: in compose_up_with_retries
    raise ex
../../libbeat/tests/system/beat/beat.py:218: in compose_up_with_retries
    cls.compose_up()
../../libbeat/tests/system/beat/compose.py:66: in compose_up
    project.up(
../../build/ve/docker/lib/python3.9/site-packages/compose/project.py:697: in up
    results, errors = parallel.parallel_execute(
../../build/ve/docker/lib/python3.9/site-packages/compose/parallel.py:108: in parallel_execute
    raise error_to_reraise
../../build/ve/docker/lib/python3.9/site-packages/compose/parallel.py:206: in producer
    result = func(obj)
../../build/ve/docker/lib/python3.9/site-packages/compose/project.py:679: in do
    return service.execute_convergence_plan(
../../build/ve/docker/lib/python3.9/site-packages/compose/service.py:559: in execute_convergence_plan
    return self._execute_convergence_create(
../../build/ve/docker/lib/python3.9/site-packages/compose/service.py:473: in _execute_convergence_create
    containers, errors = parallel_execute(
../../build/ve/docker/lib/python3.9/site-packages/compose/parallel.py:108: in parallel_execute
    raise error_to_reraise
../../build/ve/docker/lib/python3.9/site-packages/compose/parallel.py:206: in producer
    result = func(obj)
../../build/ve/docker/lib/python3.9/site-packages/compose/service.py:478: in <lambda>
    lambda service_name: create_and_start(self, service_name.number),
../../build/ve/docker/lib/python3.9/site-packages/compose/service.py:461: in create_and_start
    self.start_container(container)
../../build/ve/docker/lib/python3.9/site-packages/compose/service.py:647: in start_container
    log.warn("Host is already in use by another container")
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <Logger compose.service (WARNING)>
msg = 'Host is already in use by another container', args = (), kwargs = {}

    def warn(self, msg, *args, **kwargs):
>       warnings.warn("The 'warn' method is deprecated, "
            "use 'warning' instead", DeprecationWarning, 2)
E       DeprecationWarning: The 'warn' method is deprecated, use 'warning' instead

/usr/lib/python3.9/logging/__init__.py:1457: DeprecationWarning

Build&Test / x-pack/metricbeat-pythonIntegTest / test_stats – x-pack.metricbeat.module.enterprisesearch.test_enterprisesearch.Test
    Expand to view the error details

     failed on setup with "DeprecationWarning: The 'warn' method is deprecated, use 'warning' instead"

    Expand to view the stacktrace

     self = <docker.api.client.APIClient object at 0x7f4a8f3c97f0>
response = <Response [500]>

    def _raise_for_status(self, response):
        """Raises stored :class:`APIError`, if one occurred."""
        try:
>           response.raise_for_status()

../../build/ve/docker/lib/python3.9/site-packages/docker/api/client.py:268: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <Response [500]>

    def raise_for_status(self):
        """Raises :class:`HTTPError`, if one occurred."""
    
        http_error_msg = ''
        if isinstance(self.reason, bytes):
            # We attempt to decode utf-8 first because some servers
            # choose to localize their reason strings. If the string
            # isn't utf-8, we fall back to iso-8859-1 for all other
            # encodings. (See PR #3538)
            try:
                reason = self.reason.decode('utf-8')
            except UnicodeDecodeError:
                reason = self.reason.decode('iso-8859-1')
        else:
            reason = self.reason
    
        if 400 <= self.status_code < 500:
            http_error_msg = u'%s Client Error: %s for url: %s' % (self.status_code, reason, self.url)
    
        elif 500 <= self.status_code < 600:
            http_error_msg = u'%s Server Error: %s for url: %s' % (self.status_code, reason, self.url)
    
        if http_error_msg:
>           raise HTTPError(http_error_msg, response=self)
E           requests.exceptions.HTTPError: 500 Server Error: Internal Server Error for url: http+docker://localhost/v1.41/containers/8e812f65f39bbaa7025b9e84d7db712a4bed0a7c326f74b656651924b8589039/start

../../build/ve/docker/lib/python3.9/site-packages/requests/models.py:943: HTTPError

During handling of the above exception, another exception occurred:

self = <Service: elasticsearch>
container = <Container: enterprisesearch_7918a246f6f8_elasticsearch_1 (8e812f)>
use_network_aliases = True

    def start_container(self, container, use_network_aliases=True):
        self.connect_container_to_networks(container, use_network_aliases)
        try:
>           container.start()

../../build/ve/docker/lib/python3.9/site-packages/compose/service.py:643: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <Container: enterprisesearch_7918a246f6f8_elasticsearch_1 (8e812f)>
options = {}

    def start(self, **options):
>       return self.client.start(self.id, **options)

../../build/ve/docker/lib/python3.9/site-packages/compose/container.py:228: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <docker.api.client.APIClient object at 0x7f4a8f3c97f0>
resource_id = '8e812f65f39bbaa7025b9e84d7db712a4bed0a7c326f74b656651924b8589039'
args = (), kwargs = {}

    @functools.wraps(f)
    def wrapped(self, resource_id=None, *args, **kwargs):
        if resource_id is None and kwargs.get(resource_name):
            resource_id = kwargs.pop(resource_name)
        if isinstance(resource_id, dict):
            resource_id = resource_id.get('Id', resource_id.get('ID'))
        if not resource_id:
            raise errors.NullResource(
                'Resource ID was not provided'
            )
>       return f(self, resource_id, *args, **kwargs)

../../build/ve/docker/lib/python3.9/site-packages/docker/utils/decorators.py:19: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <docker.api.client.APIClient object at 0x7f4a8f3c97f0>
container = '8e812f65f39bbaa7025b9e84d7db712a4bed0a7c326f74b656651924b8589039'
args = (), kwargs = {}
url = 'http+docker://localhost/v1.41/containers/8e812f65f39bbaa7025b9e84d7db712a4bed0a7c326f74b656651924b8589039/start'
res = <Response [500]>

    @utils.check_resource('container')
    def start(self, container, *args, **kwargs):
        """
        Start a container. Similar to the ``docker start`` command, but
        doesn't support attach options.
    
        **Deprecation warning:** Passing configuration options in ``start`` is
        no longer supported. Users are expected to provide host config options
        in the ``host_config`` parameter of
        :py:meth:`~ContainerApiMixin.create_container`.
    
    
        Args:
            container (str): The container to start
    
        Raises:
            :py:class:`docker.errors.APIError`
                If the server returns an error.
            :py:class:`docker.errors.DeprecatedMethod`
                If any argument besides ``container`` are provided.
    
        Example:
    
            >>> container = client.api.create_container(
            ...     image='busybox:latest',
            ...     command='/bin/sleep 30')
            >>> client.api.start(container=container.get('Id'))
        """
        if args or kwargs:
            raise errors.DeprecatedMethod(
                'Providing configuration in the start() method is no longer '
                'supported. Use the host_config param in create_container '
                'instead.'
            )
        url = self._url("/containers/{0}/start", container)
        res = self._post(url)
>       self._raise_for_status(res)

../../build/ve/docker/lib/python3.9/site-packages/docker/api/container.py:1109: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <docker.api.client.APIClient object at 0x7f4a8f3c97f0>
response = <Response [500]>

    def _raise_for_status(self, response):
        """Raises stored :class:`APIError`, if one occurred."""
        try:
            response.raise_for_status()
        except requests.exceptions.HTTPError as e:
>           raise create_api_error_from_http_exception(e)

../../build/ve/docker/lib/python3.9/site-packages/docker/api/client.py:270: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

e = HTTPError('500 Server Error: Internal Server Error for url: http+docker://localhost/v1.41/containers/8e812f65f39bbaa7025b9e84d7db712a4bed0a7c326f74b656651924b8589039/start')

    def create_api_error_from_http_exception(e):
        """
        Create a suitable APIError from requests.exceptions.HTTPError.
        """
        response = e.response
        try:
            explanation = response.json()['message']
        except ValueError:
            explanation = (response.content or '').strip()
        cls = APIError
        if response.status_code == 404:
            if explanation and ('No such image' in str(explanation) or
                                'not found: does not exist or no pull access'
                                in str(explanation) or
                                'repository does not exist' in str(explanation)):
                cls = ImageNotFound
            else:
                cls = NotFound
>       raise cls(e, response=response, explanation=explanation)
E       docker.errors.APIError: 500 Server Error for http+docker://localhost/v1.41/containers/8e812f65f39bbaa7025b9e84d7db712a4bed0a7c326f74b656651924b8589039/start: Internal Server Error ("driver failed programming external connectivity on endpoint enterprisesearch_7918a246f6f8_elasticsearch_1 (ddd6508cf16260bdff56cbe6e11418ebf11be16c12839c89fdcf83a3f3b460cf): Bind for 0.0.0.0:9200 failed: port is already allocated")

../../build/ve/docker/lib/python3.9/site-packages/docker/errors.py:31: APIError

During handling of the above exception, another exception occurred:

self = <class 'test_enterprisesearch.Test'>

    @classmethod
    def setUpClass(self):
        self.beat_name = "metricbeat"
        self.beat_path = os.path.abspath(
            os.path.join(os.path.dirname(__file__), "../../"))
    
        self.template_paths = [
            os.path.abspath(os.path.join(self.beat_path, "../../metricbeat")),
            os.path.abspath(os.path.join(self.beat_path, "../../libbeat")),
        ]
    
>       super(XPackTest, self).setUpClass()

tests/system/xpack_metricbeat.py:19: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
../../metricbeat/tests/system/metricbeat.py:42: in setUpClass
    super().setUpClass()
../../libbeat/tests/system/beat/beat.py:204: in setUpClass
    cls.compose_up_with_retries()
../../libbeat/tests/system/beat/beat.py:222: in compose_up_with_retries
    raise ex
../../libbeat/tests/system/beat/beat.py:218: in compose_up_with_retries
    cls.compose_up()
../../libbeat/tests/system/beat/compose.py:66: in compose_up
    project.up(
../../build/ve/docker/lib/python3.9/site-packages/compose/project.py:697: in up
    results, errors = parallel.parallel_execute(
../../build/ve/docker/lib/python3.9/site-packages/compose/parallel.py:108: in parallel_execute
    raise error_to_reraise
../../build/ve/docker/lib/python3.9/site-packages/compose/parallel.py:206: in producer
    result = func(obj)
../../build/ve/docker/lib/python3.9/site-packages/compose/project.py:679: in do
    return service.execute_convergence_plan(
../../build/ve/docker/lib/python3.9/site-packages/compose/service.py:559: in execute_convergence_plan
    return self._execute_convergence_create(
../../build/ve/docker/lib/python3.9/site-packages/compose/service.py:473: in _execute_convergence_create
    containers, errors = parallel_execute(
../../build/ve/docker/lib/python3.9/site-packages/compose/parallel.py:108: in parallel_execute
    raise error_to_reraise
../../build/ve/docker/lib/python3.9/site-packages/compose/parallel.py:206: in producer
    result = func(obj)
../../build/ve/docker/lib/python3.9/site-packages/compose/service.py:478: in <lambda>
    lambda service_name: create_and_start(self, service_name.number),
../../build/ve/docker/lib/python3.9/site-packages/compose/service.py:461: in create_and_start
    self.start_container(container)
../../build/ve/docker/lib/python3.9/site-packages/compose/service.py:647: in start_container
    log.warn("Host is already in use by another container")
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <Logger compose.service (WARNING)>
msg = 'Host is already in use by another container', args = (), kwargs = {}

    def warn(self, msg, *args, **kwargs):
>       warnings.warn("The 'warn' method is deprecated, "
            "use 'warning' instead", DeprecationWarning, 2)
E       DeprecationWarning: The 'warn' method is deprecated, use 'warning' instead

/usr/lib/python3.9/logging/__init__.py:1457: DeprecationWarning

Steps errors 5

Expand to view the steps failures

heartbeat-windows-2022-windows-2022 - mage build unitTest
  • Took 3 min 42 sec . View more details here
  • Description: mage build unitTest
x-pack/metricbeat-pythonIntegTest - mage pythonIntegTest
  • Took 26 min 26 sec . View more details here
  • Description: mage pythonIntegTest
x-pack/metricbeat-pythonIntegTest - mage pythonIntegTest
  • Took 20 min 44 sec . View more details here
  • Description: mage pythonIntegTest
x-pack/metricbeat-pythonIntegTest - mage pythonIntegTest
  • Took 19 min 28 sec . View more details here
  • Description: mage pythonIntegTest
Error signal
  • Took 0 min 0 sec . View more details here
  • Description: Error 'hudson.AbortException: script returned exit code 1'

🐛 Flaky test report

❕ There are test failures but not known flaky tests.

Expand to view the summary

Genuine test errors 10

💔 There are test failures but not known flaky tests, most likely a genuine test failure.

  • Name: Build&Test / x-pack/metricbeat-pythonIntegTest / test_dashboards – x-pack.metricbeat.tests.system.test_xpack_base.Test
  • Name: Build&Test / x-pack/metricbeat-pythonIntegTest / test_export_config – x-pack.metricbeat.tests.system.test_xpack_base.Test
  • Name: Build&Test / x-pack/metricbeat-pythonIntegTest / test_export_ilm_policy – x-pack.metricbeat.tests.system.test_xpack_base.Test
  • Name: Build&Test / x-pack/metricbeat-pythonIntegTest / test_export_index_pattern – x-pack.metricbeat.tests.system.test_xpack_base.Test
  • Name: Build&Test / x-pack/metricbeat-pythonIntegTest / test_export_index_pattern_migration – x-pack.metricbeat.tests.system.test_xpack_base.Test
  • Name: Build&Test / x-pack/metricbeat-pythonIntegTest / test_export_template – x-pack.metricbeat.tests.system.test_xpack_base.Test
  • Name: Build&Test / x-pack/metricbeat-pythonIntegTest / test_index_management – x-pack.metricbeat.tests.system.test_xpack_base.Test
  • Name: Build&Test / x-pack/metricbeat-pythonIntegTest / test_start_stop – x-pack.metricbeat.tests.system.test_xpack_base.Test
  • Name: Build&Test / x-pack/metricbeat-pythonIntegTest / test_health – x-pack.metricbeat.module.enterprisesearch.test_enterprisesearch.Test
  • Name: Build&Test / x-pack/metricbeat-pythonIntegTest / test_stats – x-pack.metricbeat.module.enterprisesearch.test_enterprisesearch.Test

🤖 GitHub comments

Expand to view the GitHub comments

To re-run your PR in the CI, just comment with:

  • /test : Re-trigger the build.

  • /package : Generate the packages and run the E2E tests.

  • /beats-tester : Run the installation tests with beats-tester.

  • run elasticsearch-ci/docs : Re-trigger the docs validation. (use unformatted text in the comment!)

@andrewkroh
Copy link
Member

The failing test in Metricbeat should not be related to this Filebeat change.

@andrewkroh andrewkroh merged commit fc2f198 into 8.6 Nov 16, 2022
@andrewkroh andrewkroh deleted the mergify/bp/8.6/pr-33656 branch November 16, 2022 23:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@rdner rdner Awaiting requested review from rdner rdner is a code owner automatically assigned from elastic/elastic-agent-data-plane

@fearful-symmetry fearful-symmetry Awaiting requested review from fearful-symmetry fearful-symmetry is a code owner automatically assigned from elastic/elastic-agent-data-plane

Assignees

@andrewkroh andrewkroh

Labels
backport needs_team Indicates that the issue/PR needs a Team:* label
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@elasticmachine @andrewkroh