[7.x](backport #27607) feat: add huaweicloud metadata support. #28037

mergify · 2021-09-22T02:59:28Z

This is an automatic backport of pull request #27607 done by Mergify.

Mergify commands and options

More conditions and actions can be found in the documentation.

You can also trigger Mergify actions by commenting on this pull request:

@Mergifyio refresh will re-evaluate the rules
@Mergifyio rebase will rebase this PR on its base branch
@Mergifyio update will merge the base branch into this PR
@Mergifyio backport <destination> will backport this PR on <destination> branch

Additionally, on Mergify dashboard you can:

look at your merge queues
generate the Mergify configuration with the config editor.

Finally, you can contact us on https://mergify.io/

Signed-off-by: colstuwjx <colstuwjx@gmail.com> Co-authored-by: Jaime Soriano Pastor <jaime.soriano@elastic.co> (cherry picked from commit 86004dd)

botelastic · 2021-09-22T02:59:32Z

This pull request doesn't have a Team:<team> label.

elasticmachine · 2021-09-22T03:06:36Z

💔 Tests Failed

the below badges are clickable and redirect to their specific view in the CI or DOCS

Expand to view the summary

Build stats

Start Time: 2021-09-28T15:17:09.926+0000
Duration: 85 min 31 sec
Commit: 1ad10a7

Test stats 🧪

Test	Results
Failed	3
Passed	20613
Skipped	1515
Total	22131

Test errors

Expand to view the tests failures

`Build&Test / x-pack/metricbeat-pythonIntegTest / test_qmgr – x-pack.metricbeat.module.ibmmq.test_ibmmq.Test`

Expand to view the error details

 failed on setup with "compose.service.BuildError: (<Service: ibmmq>, 'manifest for ibmcom/mq:9.1.4.0-r1-amd64 not found: manifest unknown: manifest unknown')"

Expand to view the stacktrace

 self = <Service: ibmmq>, no_cache = False, pull = False, force_rm = False
memory = None, build_args_override = None, gzip = False, rm = True
silent = False, cli = False, progress = None

    def build(self, no_cache=False, pull=False, force_rm=False, memory=None, build_args_override=None,
              gzip=False, rm=True, silent=False, cli=False, progress=None):
        output_stream = open(os.devnull, 'w')
        if not silent:
            output_stream = sys.stdout
            log.info('Building %s' % self.name)
    
        build_opts = self.options.get('build', {})
    
        build_args = build_opts.get('args', {}).copy()
        if build_args_override:
            build_args.update(build_args_override)
    
        for k, v in self._parse_proxy_config().items():
            build_args.setdefault(k, v)
    
        path = rewrite_build_path(build_opts.get('context'))
        if self.platform and version_lt(self.client.api_version, '1.35'):
            raise OperationFailedError(
                'Impossible to perform platform-targeted builds for API version < 1.35'
            )
    
        builder = self.client if not cli else _CLIBuilder(progress)
        build_output = builder.build(
            path=path,
            tag=self.image_name,
            rm=rm,
            forcerm=force_rm,
            pull=pull,
            nocache=no_cache,
            dockerfile=build_opts.get('dockerfile', None),
            cache_from=self.get_cache_from(build_opts),
            labels=build_opts.get('labels', None),
            buildargs=build_args,
            network_mode=build_opts.get('network', None),
            target=build_opts.get('target', None),
            shmsize=parse_bytes(build_opts.get('shm_size')) if build_opts.get('shm_size') else None,
            extra_hosts=build_opts.get('extra_hosts', None),
            container_limits={
                'memory': parse_bytes(memory) if memory else None
            },
            gzip=gzip,
            isolation=build_opts.get('isolation', self.options.get('isolation', None)),
            platform=self.platform,
        )
    
        try:
>           all_events = list(stream_output(build_output, output_stream))

../../build/ve/docker/lib/python3.7/site-packages/compose/service.py:1110: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

output = <generator object APIClient._stream_helper at 0x7fa91d56a7c8>
stream = <_io.TextIOWrapper name="<_io.FileIO name=6 mode='rb+' closefd=True>" mode='r+' encoding='utf-8'>

    def stream_output(output, stream):
        is_terminal = hasattr(stream, 'isatty') and stream.isatty()
        stream = utils.get_output_stream(stream)
        lines = {}
        diff = 0
    
        for event in utils.json_stream(output):
            yield event
            is_progress_event = 'progress' in event or 'progressDetail' in event
    
            if not is_progress_event:
>               print_output_event(event, stream, is_terminal)

../../build/ve/docker/lib/python3.7/site-packages/compose/progress_stream.py:30: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

event = {'error': 'manifest for ibmcom/mq:9.1.4.0-r1-amd64 not found: manifest unknown: manifest unknown', 'errorDetail': {'message': 'manifest for ibmcom/mq:9.1.4.0-r1-amd64 not found: manifest unknown: manifest unknown'}}
stream = <_io.TextIOWrapper name="<_io.FileIO name=6 mode='rb+' closefd=True>" mode='r+' encoding='utf-8'>
is_terminal = False

    def print_output_event(event, stream, is_terminal):
        if 'errorDetail' in event:
>           raise StreamOutputError(event['errorDetail']['message'])
E           compose.progress_stream.StreamOutputError: manifest for ibmcom/mq:9.1.4.0-r1-amd64 not found: manifest unknown: manifest unknown

../../build/ve/docker/lib/python3.7/site-packages/compose/progress_stream.py:62: StreamOutputError

During handling of the above exception, another exception occurred:

self = <class 'test_ibmmq.Test'>

    @classmethod
    def setUpClass(self):
        self.beat_name = "metricbeat"
        self.beat_path = os.path.abspath(
            os.path.join(os.path.dirname(__file__), "../../"))
    
        self.template_paths = [
            os.path.abspath(os.path.join(self.beat_path, "../../metricbeat")),
            os.path.abspath(os.path.join(self.beat_path, "../../libbeat")),
        ]
    
>       super(XPackTest, self).setUpClass()

tests/system/xpack_metricbeat.py:19: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
../../metricbeat/tests/system/metricbeat.py:28: in setUpClass
    super(BaseTest, self).setUpClass()
../../libbeat/tests/system/beat/beat.py:148: in setUpClass
    self.compose_up_with_retries()
../../libbeat/tests/system/beat/beat.py:163: in compose_up_with_retries
    raise e
../../libbeat/tests/system/beat/beat.py:159: in compose_up_with_retries
    self.compose_up()
../../libbeat/tests/system/beat/compose.py:69: in compose_up
    timeout=30)
../../build/ve/docker/lib/python3.7/site-packages/compose/project.py:548: in up
    svc.ensure_image_exists(do_build=do_build, silent=silent, cli=cli)
../../build/ve/docker/lib/python3.7/site-packages/compose/service.py:367: in ensure_image_exists
    self.build(cli=cli)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <Service: ibmmq>, no_cache = False, pull = False, force_rm = False
memory = None, build_args_override = None, gzip = False, rm = True
silent = False, cli = False, progress = None

    def build(self, no_cache=False, pull=False, force_rm=False, memory=None, build_args_override=None,
              gzip=False, rm=True, silent=False, cli=False, progress=None):
        output_stream = open(os.devnull, 'w')
        if not silent:
            output_stream = sys.stdout
            log.info('Building %s' % self.name)
    
        build_opts = self.options.get('build', {})
    
        build_args = build_opts.get('args', {}).copy()
        if build_args_override:
            build_args.update(build_args_override)
    
        for k, v in self._parse_proxy_config().items():
            build_args.setdefault(k, v)
    
        path = rewrite_build_path(build_opts.get('context'))
        if self.platform and version_lt(self.client.api_version, '1.35'):
            raise OperationFailedError(
                'Impossible to perform platform-targeted builds for API version < 1.35'
            )
    
        builder = self.client if not cli else _CLIBuilder(progress)
        build_output = builder.build(
            path=path,
            tag=self.image_name,
            rm=rm,
            forcerm=force_rm,
            pull=pull,
            nocache=no_cache,
            dockerfile=build_opts.get('dockerfile', None),
            cache_from=self.get_cache_from(build_opts),
            labels=build_opts.get('labels', None),
            buildargs=build_args,
            network_mode=build_opts.get('network', None),
            target=build_opts.get('target', None),
            shmsize=parse_bytes(build_opts.get('shm_size')) if build_opts.get('shm_size') else None,
            extra_hosts=build_opts.get('extra_hosts', None),
            container_limits={
                'memory': parse_bytes(memory) if memory else None
            },
            gzip=gzip,
            isolation=build_opts.get('isolation', self.options.get('isolation', None)),
            platform=self.platform,
        )
    
        try:
            all_events = list(stream_output(build_output, output_stream))
        except StreamOutputError as e:
>           raise BuildError(self, six.text_type(e))
E           compose.service.BuildError: (<Service: ibmmq>, 'manifest for ibmcom/mq:9.1.4.0-r1-amd64 not found: manifest unknown: manifest unknown')

../../build/ve/docker/lib/python3.7/site-packages/compose/service.py:1112: BuildError

`Build&Test / x-pack/metricbeat-goIntegTest / TestData – github.com/elastic/beats/v7/x-pack/metricbeat/module/airflow/statsd`

Expand to view the error details

 Failed

Expand to view the stacktrace

 === RUN   TestData
    data_test.go:70: received no events
--- FAIL: TestData (30.01s)

`Build&Test / x-pack/metricbeat-goIntegTest / TestFetch – github.com/elastic/beats/v7/x-pack/metricbeat/module/ibmmq/qmgr`

Expand to view the error details

 Failed

Expand to view the stacktrace

 === RUN   TestFetch
Found orphan containers (metricbeat_7_16_0_1ad10a7e17-snapshot_coredns_1, metricbeat_7_16_0_1ad10a7e17-snapshot_cockroachdb_1) for this project. If you removed or renamed this service in your compose file, you can run this command with the --remove-orphans flag to clean it up.
Building ibmmq
Step 1/7 : ARG IBMMQ_VERSION
Step 2/7 : FROM ibmcom/mq:${IBMMQ_VERSION}
Service 'ibmmq' failed to build: manifest for ibmcom/mq:9.1.4.0-r1-amd64 not found: manifest unknown: manifest unknown
    qmgr_integration_test.go:31: failed to start service 'ibmmq: exit status 1
Found orphan containers (metricbeat_7_16_0_1ad10a7e17-snapshot_coredns_1, metricbeat_7_16_0_1ad10a7e17-snapshot_cockroachdb_1) for this project. If you removed or renamed this service in your compose file, you can run this command with the --remove-orphans flag to clean it up.
Building ibmmq
Step 1/7 : ARG IBMMQ_VERSION
Step 2/7 : FROM ibmcom/mq:${IBMMQ_VERSION}
Service 'ibmmq' failed to build: manifest for ibmcom/mq:9.1.4.0-r1-amd64 not found: manifest unknown: manifest unknown
    qmgr_integration_test.go:31: failed to start service 'ibmmq: exit status 1
Found orphan containers (metricbeat_7_16_0_1ad10a7e17-snapshot_coredns_1, metricbeat_7_16_0_1ad10a7e17-snapshot_cockroachdb_1) for this project. If you removed or renamed this service in your compose file, you can run this command with the --remove-orphans flag to clean it up.
Building ibmmq
Step 1/7 : ARG IBMMQ_VERSION
Step 2/7 : FROM ibmcom/mq:${IBMMQ_VERSION}
Service 'ibmmq' failed to build: manifest for ibmcom/mq:9.1.4.0-r1-amd64 not found: manifest unknown: manifest unknown
    qmgr_integration_test.go:31: failed to start service 'ibmmq: exit status 1
    qmgr_integration_test.go:31: getting host for ibmmq: no container running for service
--- FAIL: TestFetch (7.37s)

Steps errors

Expand to view the steps failures

`x-pack/metricbeat-unitTest - mage build unitTest`

Took 4 min 40 sec . View more details here
Description: mage build unitTest

`x-pack/metricbeat-goIntegTest - mage goIntegTest`

Took 13 min 3 sec . View more details here
Description: mage goIntegTest

`x-pack/metricbeat-goIntegTest - mage goIntegTest`

Took 8 min 23 sec . View more details here
Description: mage goIntegTest

`x-pack/metricbeat-goIntegTest - mage goIntegTest`

Took 10 min 38 sec . View more details here
Description: mage goIntegTest

`List files to upload`

Took 0 min 0 sec . View more details here
Description: ls -l src/github.com/elastic/beats/build/system-tests-*.tar.gz

`x-pack/metricbeat-pythonIntegTest - mage pythonIntegTest`

Took 11 min 21 sec . View more details here
Description: mage pythonIntegTest

`x-pack/metricbeat-pythonIntegTest - mage pythonIntegTest`

Took 6 min 27 sec . View more details here
Description: mage pythonIntegTest

`x-pack/metricbeat-pythonIntegTest - mage pythonIntegTest`

Took 6 min 24 sec . View more details here
Description: mage pythonIntegTest

`List files to upload`

Took 0 min 0 sec . View more details here
Description: ls -l src/github.com/elastic/beats/build/system-tests-*.tar.gz

`Error signal`

Took 0 min 0 sec . View more details here
Description: Error 'hudson.AbortException: script returned exit code 1'

🐛 Flaky test report

❕ There are test failures but not known flaky tests.

Expand to view the summary

Genuine test errors

💔 There are test failures but not known flaky tests, most likely a genuine test failure.

Name: Build&Test / x-pack/metricbeat-pythonIntegTest / test_qmgr – x-pack.metricbeat.module.ibmmq.test_ibmmq.Test
Name: Build&Test / x-pack/metricbeat-goIntegTest / TestData – github.com/elastic/beats/v7/x-pack/metricbeat/module/airflow/statsd
Name: Build&Test / x-pack/metricbeat-goIntegTest / TestFetch – github.com/elastic/beats/v7/x-pack/metricbeat/module/ibmmq/qmgr

🤖 GitHub comments

To re-run your PR in the CI, just comment with:

/test : Re-trigger the build.
/package : Generate the packages and run the E2E tests.
/beats-tester : Run the installation tests with beats-tester.

mergify · 2021-09-28T15:15:17Z

This pull request is now in conflicts. Could you fix it? 🙏
To fixup this pull request, you can check out it locally. See documentation: https://help.github.com/articles/checking-out-pull-requests-locally/

git fetch upstream
git checkout -b mergify/bp/7.x/pr-27607 upstream/mergify/bp/7.x/pr-27607
git merge upstream/7.x
git push upstream mergify/bp/7.x/pr-27607

kaiyan-sheng · 2021-09-28T19:15:43Z

Merging this PR because CI failures are from flaky tests.

feat: add huaweicloud metadata support. (#27607)

06167cb

Signed-off-by: colstuwjx <colstuwjx@gmail.com> Co-authored-by: Jaime Soriano Pastor <jaime.soriano@elastic.co> (cherry picked from commit 86004dd)

mergify bot added the backport label Sep 22, 2021

botelastic bot added the needs_team Indicates that the issue/PR needs a Team:* label label Sep 22, 2021

Update CHANGELOG.next.asciidoc

9d1ff65

kaiyan-sheng self-assigned this Sep 22, 2021

Merge remote-tracking branch 'upstream/7.x' into mergify/bp/7.x/pr-27607

1ad10a7

kaiyan-sheng merged commit c6224d0 into 7.x Sep 28, 2021

kaiyan-sheng deleted the mergify/bp/7.x/pr-27607 branch September 28, 2021 19:15

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[7.x](backport #27607) feat: add huaweicloud metadata support. #28037

[7.x](backport #27607) feat: add huaweicloud metadata support. #28037

mergify bot commented Sep 22, 2021 •

edited by zube bot

Loading

botelastic bot commented Sep 22, 2021

elasticmachine commented Sep 22, 2021 •

edited by jenkins-beats-ci bot

Loading

Build stats

Test stats 🧪

`Build&Test / x-pack/metricbeat-pythonIntegTest / test_qmgr – x-pack.metricbeat.module.ibmmq.test_ibmmq.Test`

`Build&Test / x-pack/metricbeat-goIntegTest / TestData – github.com/elastic/beats/v7/x-pack/metricbeat/module/airflow/statsd`

`Build&Test / x-pack/metricbeat-goIntegTest / TestFetch – github.com/elastic/beats/v7/x-pack/metricbeat/module/ibmmq/qmgr`

`x-pack/metricbeat-unitTest - mage build unitTest`

`x-pack/metricbeat-goIntegTest - mage goIntegTest`

`x-pack/metricbeat-goIntegTest - mage goIntegTest`

`x-pack/metricbeat-goIntegTest - mage goIntegTest`

`List files to upload`

`x-pack/metricbeat-pythonIntegTest - mage pythonIntegTest`

`x-pack/metricbeat-pythonIntegTest - mage pythonIntegTest`

`x-pack/metricbeat-pythonIntegTest - mage pythonIntegTest`

`List files to upload`

`Error signal`

Genuine test errors

mergify bot commented Sep 28, 2021

kaiyan-sheng commented Sep 28, 2021

[7.x](backport #27607) feat: add huaweicloud metadata support. #28037

[7.x](backport #27607) feat: add huaweicloud metadata support. #28037

Conversation

mergify bot commented Sep 22, 2021 • edited by zube bot Loading

botelastic bot commented Sep 22, 2021

elasticmachine commented Sep 22, 2021 • edited by jenkins-beats-ci bot Loading

💔 Tests Failed

Build stats

Test stats 🧪

Test errors

Build&Test / x-pack/metricbeat-pythonIntegTest / test_qmgr – x-pack.metricbeat.module.ibmmq.test_ibmmq.Test

Build&Test / x-pack/metricbeat-goIntegTest / TestData – github.com/elastic/beats/v7/x-pack/metricbeat/module/airflow/statsd

Build&Test / x-pack/metricbeat-goIntegTest / TestFetch – github.com/elastic/beats/v7/x-pack/metricbeat/module/ibmmq/qmgr

Steps errors

x-pack/metricbeat-unitTest - mage build unitTest

x-pack/metricbeat-goIntegTest - mage goIntegTest

x-pack/metricbeat-goIntegTest - mage goIntegTest

x-pack/metricbeat-goIntegTest - mage goIntegTest

List files to upload

x-pack/metricbeat-pythonIntegTest - mage pythonIntegTest

x-pack/metricbeat-pythonIntegTest - mage pythonIntegTest

x-pack/metricbeat-pythonIntegTest - mage pythonIntegTest

List files to upload

Error signal

🐛 Flaky test report

Genuine test errors

🤖 GitHub comments

mergify bot commented Sep 28, 2021

kaiyan-sheng commented Sep 28, 2021

mergify bot commented Sep 22, 2021 •

edited by zube bot

Loading

elasticmachine commented Sep 22, 2021 •

edited by jenkins-beats-ci bot

Loading

`Build&Test / x-pack/metricbeat-pythonIntegTest / test_qmgr – x-pack.metricbeat.module.ibmmq.test_ibmmq.Test`

`Build&Test / x-pack/metricbeat-goIntegTest / TestData – github.com/elastic/beats/v7/x-pack/metricbeat/module/airflow/statsd`

`Build&Test / x-pack/metricbeat-goIntegTest / TestFetch – github.com/elastic/beats/v7/x-pack/metricbeat/module/ibmmq/qmgr`

`x-pack/metricbeat-unitTest - mage build unitTest`

`x-pack/metricbeat-goIntegTest - mage goIntegTest`

`x-pack/metricbeat-goIntegTest - mage goIntegTest`

`x-pack/metricbeat-goIntegTest - mage goIntegTest`

`List files to upload`

`x-pack/metricbeat-pythonIntegTest - mage pythonIntegTest`

`x-pack/metricbeat-pythonIntegTest - mage pythonIntegTest`

`x-pack/metricbeat-pythonIntegTest - mage pythonIntegTest`

`List files to upload`

`Error signal`