Skip to content
This repository has been archived by the owner on May 3, 2024. It is now read-only.

EOS-25693-new: Support Bundle generation is not creating and collecting data for m0reportbug-traces. #1221

Merged
merged 2 commits into from
Nov 16, 2021

Conversation

papan-singh
Copy link
Contributor

@papan-singh papan-singh commented Nov 11, 2021

Problem Statement

  • Problem statement

Design

  • For Bug, Describe the fix here.
  • For Feature, Post the link for design

Coding

Checklist for Author

  • Coding conventions are followed and code is consistent

Testing

Checklist for Author

  • Unit and System Tests are added
  • Test Cases cover Happy Path, Non-Happy Path and Scalability
  • Testing was performed with RPM

Impact Analysis

Checklist for Author/Reviewer/GateKeeper

  • Interface change (if any) are documented
  • Side effects on other features (deployment/upgrade)
  • Dependencies on other component(s)

Review Checklist

Checklist for Author

  • JIRA number/GitHub Issue added to PR
  • PR is self reviewed
  • Jira and state/status is updated and JIRA is updated with PR link
  • Check if the description is clear and explained

Documentation

Checklist for Author

  • Changes done to WIKI / Confluence page / Quick Start Guide

…ata for m0reportbug-traces.

Fix : If /proc/sysrq-trigger is mounted as ro then return.

Signed-off-by: Papan Kumar Singh <papan.kumarsingh@seagate.com>
utils/m0reportbug Outdated Show resolved Hide resolved
@papan-singh
Copy link
Contributor Author

papan-singh commented Nov 12, 2021

Tested in cortx-motr-ios container
[root@cortx-data-headless-svc-ssc-vm-rhev4-1330 ~]# /opt/seagate/cortx/motr/libexec/m0reportbug-bundler -b SB999901_7020210e0f264211b487e46e691cc536_motr -t /var/log/cortx/support_bundle/ -c yaml:///etc/cortx/cluster.conf -s all
probes... OK
misc_files... OK
backtraces_live... OK
coredump_live... WARNING ## live core are not collected on kvm ##
OK
backtraces_cores... OK
binaries... OK
m0traces... OK
addb... OK
addb-files... OK
systemd_journal... No journal files were found.
OK
sysrq_info... WARNING /proc/sysrq-trigger is ro
OK
WARNING /var/crash/: no such directory
WARNING /var/log/crash/: no such directory
WARNING /var/log/seagate/motr/tmpspace/: no such directory
m0traces /opt/seagate/cortx/motr/libexec... OK
m0traces /etc/cortx/log/motr/6afe3f8aaac046e09825de316c91c006/trace... OK (1m54s)
addb-files /etc/cortx/log/motr/6afe3f8aaac046e09825de316c91c006/addb... OK (0m19s)
Please attach these files to the bug report:
m0reportbug-data.tar.xz
m0reportbug-cores.tar.xz
m0reportbug-traces.tar.xz

[root@cortx-data-headless-svc-ssc-vm-rhev4-1330 ~]# cd /var/log/cortx/support_bundle/
[root@cortx-data-headless-svc-ssc-vm-rhev4-1330 support_bundle]# ls -R
.:
motr

./motr:
motr_SB999901_7020210e0f264211b487e46e691cc536_motr.tar
[root@cortx-data-headless-svc-ssc-vm-rhev4-1330 support_bundle]# cd motr/
[root@cortx-data-headless-svc-ssc-vm-rhev4-1330 motr]# tar -xvf motr_SB999901_7020210e0f264211b487e46e691cc536_motr.tar
cortx-data-headless-svc-ssc-vm-rhev4-1330/
cortx-data-headless-svc-ssc-vm-rhev4-1330/_m0reportbug-bundler.stderr
cortx-data-headless-svc-ssc-vm-rhev4-1330/m0reportbug-traces.tar.xz
cortx-data-headless-svc-ssc-vm-rhev4-1330/m0reportbug-data.tar.xz
cortx-data-headless-svc-ssc-vm-rhev4-1330/m0reportbug-cores.tar.xz
[root@cortx-data-headless-svc-ssc-vm-rhev4-1330 motr]# ls -R
.:
cortx-data-headless-svc-ssc-vm-rhev4-1330 motr_SB999901_7020210e0f264211b487e46e691cc536_motr.tar

./cortx-data-headless-svc-ssc-vm-rhev4-1330:
_m0reportbug-bundler.stderr m0reportbug-cores.tar.xz m0reportbug-data.tar.xz m0reportbug-traces.tar.xz
[root@cortx-data-headless-svc-ssc-vm-rhev4-1330 motr]# cd cortx-data-headless-svc-ssc-vm-rhev4-1330/
[root@cortx-data-headless-svc-ssc-vm-rhev4-1330 cortx-data-headless-svc-ssc-vm-rhev4-1330]# ls -R
.:
_m0reportbug-bundler.stderr m0reportbug-cores.tar.xz m0reportbug-data.tar.xz m0reportbug-traces.tar.xz
[root@cortx-data-headless-svc-ssc-vm-rhev4-1330 cortx-data-headless-svc-ssc-vm-rhev4-1330]# tar -xvf m0reportbug-cores.tar.xz
m0reportbug-cores/
m0reportbug-cores/var/
m0reportbug-cores/var/motr/
m0reportbug-cores/binaries/
m0reportbug-cores/binaries/m0client
m0reportbug-cores/binaries/m0d
m0reportbug-cores/binaries/m0ham
m0reportbug-cores/binaries/m0cat
m0reportbug-cores/binaries/m0touch
m0reportbug-cores/binaries/m0cp
m0reportbug-cores/binaries/m0mkfs
m0reportbug-cores/binaries/m0beck
m0reportbug-cores/binaries/libmotr.so.2
m0reportbug-cores/binaries/m0betool
m0reportbug-cores/scripts/
m0reportbug-cores/scripts/motr_data_recovery.sh
[root@cortx-data-headless-svc-ssc-vm-rhev4-1330 cortx-data-headless-svc-ssc-vm-rhev4-1330]# tar -xvf m0reportbug-data.tar.xz
m0reportbug-data/
m0reportbug-data/addb_files/
m0reportbug-data/addb_files/var/
m0reportbug-data/addb_files/var/motr/
m0reportbug-data/addb_files/etc/
m0reportbug-data/addb_files/etc/cortx/
m0reportbug-data/addb_files/etc/cortx/log/
m0reportbug-data/addb_files/etc/cortx/log/motr/
m0reportbug-data/addb_files/etc/cortx/log/motr/6afe3f8aaac046e09825de316c91c006/
m0reportbug-data/addb_files/etc/cortx/log/motr/6afe3f8aaac046e09825de316c91c006/addb/
m0reportbug-data/addb_files/etc/cortx/log/motr/6afe3f8aaac046e09825de316c91c006/addb/m0d_addb__addb.xz
m0reportbug-data/root-misc/
m0reportbug-data/root-misc/etc/
m0reportbug-data/root-misc/etc/motr/
m0reportbug-data/root-misc/etc/motr/confd.xc
m0reportbug-data/root-misc/etc/sysconfig/
m0reportbug-data/root-misc/etc/sysconfig/motr
m0reportbug-data/root-misc/etc/sysconfig/motr-free-space-monitor
m0reportbug-data/systemd-journal.xz
m0reportbug-data/m0reportbug.txt
[root@cortx-data-headless-svc-ssc-vm-rhev4-1330 cortx-data-headless-svc-ssc-vm-rhev4-1330]# tar -xvf m0reportbug-traces.tar.xz
m0reportbug-traces/
m0reportbug-traces/etc/
m0reportbug-traces/etc/cortx/
m0reportbug-traces/etc/cortx/log/
m0reportbug-traces/etc/cortx/log/motr/
m0reportbug-traces/etc/cortx/log/motr/6afe3f8aaac046e09825de316c91c006/
m0reportbug-traces/etc/cortx/log/motr/6afe3f8aaac046e09825de316c91c006/trace/
m0reportbug-traces/etc/cortx/log/motr/6afe3f8aaac046e09825de316c91c006/trace/m0d-0x7200000000000001:0x34/
m0reportbug-traces/etc/cortx/log/motr/6afe3f8aaac046e09825de316c91c006/trace/m0d-0x7200000000000001:0x34/m0trace.93.yaml.xz
m0reportbug-traces/etc/cortx/log/motr/6afe3f8aaac046e09825de316c91c006/trace/m0d-0x7200000000000001:0x34/m0trace.94.yaml.xz
m0reportbug-traces/etc/cortx/log/motr/6afe3f8aaac046e09825de316c91c006/trace/m0d-0x7200000000000001:0x4e/
m0reportbug-traces/etc/cortx/log/motr/6afe3f8aaac046e09825de316c91c006/trace/m0d-0x7200000000000001:0x4e/m0trace.93.yaml.xz
m0reportbug-traces/etc/cortx/log/motr/6afe3f8aaac046e09825de316c91c006/trace/m0d-0x7200000000000001:0x4e/m0trace.599.yaml.xz
m0reportbug-traces/etc/cortx/log/motr/6afe3f8aaac046e09825de316c91c006/trace/m0d-0x7200000000000001:0x41/
m0reportbug-traces/etc/cortx/log/motr/6afe3f8aaac046e09825de316c91c006/trace/m0d-0x7200000000000001:0x41/m0trace.93.yaml.xz
m0reportbug-traces/etc/cortx/log/motr/6afe3f8aaac046e09825de316c91c006/trace/m0d-0x7200000000000001:0x41/m0trace.99.yaml.xz
[root@cortx-data-headless-svc-ssc-vm-rhev4-1330 cortx-data-headless-svc-ssc-vm-rhev4-1330]# ls -R
.:
_m0reportbug-bundler.stderr m0reportbug-cores.tar.xz m0reportbug-data.tar.xz m0reportbug-traces.tar.xz
m0reportbug-cores m0reportbug-data m0reportbug-traces

./m0reportbug-cores:
binaries scripts var

./m0reportbug-cores/binaries:
libmotr.so.2 m0beck m0betool m0cat m0client m0cp m0d m0ham m0mkfs m0touch

./m0reportbug-cores/scripts:
motr_data_recovery.sh

./m0reportbug-cores/var:
motr

./m0reportbug-cores/var/motr:

./m0reportbug-data:
addb_files m0reportbug.txt root-misc systemd-journal.xz

./m0reportbug-data/addb_files:
etc var

./m0reportbug-data/addb_files/etc:
cortx

./m0reportbug-data/addb_files/etc/cortx:
log

./m0reportbug-data/addb_files/etc/cortx/log:
motr

./m0reportbug-data/addb_files/etc/cortx/log/motr:
6afe3f8aaac046e09825de316c91c006

./m0reportbug-data/addb_files/etc/cortx/log/motr/6afe3f8aaac046e09825de316c91c006:
addb

./m0reportbug-data/addb_files/etc/cortx/log/motr/6afe3f8aaac046e09825de316c91c006/addb:
m0d_addb__addb.xz

./m0reportbug-data/addb_files/var:
motr

./m0reportbug-data/addb_files/var/motr:

./m0reportbug-data/root-misc:
etc

./m0reportbug-data/root-misc/etc:
motr sysconfig

./m0reportbug-data/root-misc/etc/motr:
confd.xc

./m0reportbug-data/root-misc/etc/sysconfig:
motr motr-free-space-monitor

./m0reportbug-traces:
etc

./m0reportbug-traces/etc:
cortx

./m0reportbug-traces/etc/cortx:
log

./m0reportbug-traces/etc/cortx/log:
motr

./m0reportbug-traces/etc/cortx/log/motr:
6afe3f8aaac046e09825de316c91c006

./m0reportbug-traces/etc/cortx/log/motr/6afe3f8aaac046e09825de316c91c006:
trace

./m0reportbug-traces/etc/cortx/log/motr/6afe3f8aaac046e09825de316c91c006/trace:
m0d-0x7200000000000001:0x34 m0d-0x7200000000000001:0x41 m0d-0x7200000000000001:0x4e

./m0reportbug-traces/etc/cortx/log/motr/6afe3f8aaac046e09825de316c91c006/trace/m0d-0x7200000000000001:0x34:
m0trace.93.yaml.xz m0trace.94.yaml.xz

./m0reportbug-traces/etc/cortx/log/motr/6afe3f8aaac046e09825de316c91c006/trace/m0d-0x7200000000000001:0x41:
m0trace.93.yaml.xz m0trace.99.yaml.xz

./m0reportbug-traces/etc/cortx/log/motr/6afe3f8aaac046e09825de316c91c006/trace/m0d-0x7200000000000001:0x4e:
m0trace.599.yaml.xz m0trace.93.yaml.xz
[root@cortx-data-headless-svc-ssc-vm-rhev4-1330 cortx-data-headless-svc-ssc-vm-rhev4-1330]#

@papan-singh
Copy link
Contributor Author

tested on 1 node VM cluster
[root@ssc-vm-g3-rhev4-2360 ~]# hctl status
Data pool:
# fid name
0x6f00000000000001:0x34 'storage-set-1__sns'
Profile:
# fid name: pool(s)
0x7000000000000001:0x59 'Profile_the_pool': 'storage-set-1__sns' None None
Services:
srvnode-1.data.private (RC)
[started] hax 0x7200000000000001:0x7 192.168.93.109@tcp:12345:1:1
[started] ioservice 0x7200000000000001:0xa 192.168.93.109@tcp:12345:2:1
[started] ioservice 0x7200000000000001:0x19 192.168.93.109@tcp:12345:2:2
[started] confd 0x7200000000000001:0x28 192.168.93.109@tcp:12345:2:3
[started] s3server 0x7200000000000001:0x2b 192.168.93.109@tcp:12345:3:1
[unknown] m0_client 0x7200000000000001:0x2e 192.168.93.109@tcp:12345:4:1
[unknown] m0_client 0x7200000000000001:0x31 192.168.93.109@tcp:12345:4:2
[root@ssc-vm-g3-rhev4-2360 ~]#

[root@ssc-vm-g3-rhev4-2360 ~]# m0reportbug
probes... OK
misc_files... OK
backtraces_live... OK
coredump_live... WARNING ## live core are not collected on kvm ##
OK
backtraces_cores... OK
binaries... OK
m0traces... OK (0m47s)
addb-files... OK (0m14s)
systemd_journal... OK
sysrq_info... WARNING /proc/sysrq-trigger is ro
OK
backtraces_core /var/crash/... OK
backtraces_core /var/log/crash/... OK
backtraces_core /var/log/seagate/motr/tmpspace/... OK
m0traces /opt/seagate/cortx/motr/libexec... OK
WARNING : no such directory
WARNING : no such directory
'/var/log/motr-rpm-conf-update.log' -> 'm0reportbug-data/motr-rpm-conf-update.log'
Please attach these files to the bug report:
m0reportbug-data.tar.xz
m0reportbug-cores.tar.xz
m0reportbug-traces.tar.xz

@mehjoshi
Copy link

@papan-singh This PR is raised against kubernetes branch. Are you sure that is what we want?

@@ -699,6 +703,10 @@ sysrq_info() {
[t]='show-task-states'
[m]='show-memory-usage'
)
if [[ "ro"==$(cat /proc/mounts | grep /proc/sysrq-trigger | awk '{print substr($4,1,2)}') ]] ; then
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Objective of this command is to identify and skip sysrq/journal log on k8 env.
From the VM output, it is skipping on VM as well. which is not expected I believe.
It could also skip on HW as well.

Could you kindly confirm this.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

tested updated patch on VM and HW

Signed-off-by: Papan Kumar Singh <papan.kumarsingh@seagate.com>
@papan-singh
Copy link
Contributor Author

papan-singh commented Nov 15, 2021

tested updated patch on 1 node VM cluster
[root@ssc-vm-g3-rhev4-2460 ~]# m0reportbug
probes... OK
misc_files... OK
backtraces_live... OK
coredump_live... WARNING ## live core are not collected on kvm ##
OK
backtraces_cores... OK
binaries... OK
m0traces... OK (2m20s)
addb-files... OK (0m29s)
systemd_journal... OK
sysrq_info... OK
backtraces_core /var/crash/... OK
backtraces_core /var/log/crash/... OK
backtraces_core /var/log/seagate/motr/tmpspace/... OK
m0traces /opt/seagate/cortx/motr/libexec... OK
WARNING : no such directory
WARNING : no such directory
'/var/log/motr-rpm-conf-update.log' -> 'm0reportbug-data/motr-rpm-conf-update.log'
Please attach these files to the bug report:
m0reportbug-data.tar.xz
m0reportbug-cores.tar.xz
m0reportbug-traces.tar.xz

@papan-singh
Copy link
Contributor Author

Tested updated patch on HW
[root@sm28-r20 ~]# m0reportbug
probes... OK
misc_files... OK
backtraces_live... OK
coredump_live... OK (0m41s)
backtraces_cores... OK
binaries... OK (0m15s)
m0traces... OK (3m4s)
addb-files... OK
systemd_journal... OK
sysrq_info... OK
backtraces_core /var/crash/... OK
WARNING /var/log/crash/: no such directory
backtraces_core /var/log/seagate/motr... OK (1m6s)
m0traces /opt/seagate/cortx/motr/libexec... OK
WARNING /var/log/seagate/motr/trace: no such directory
WARNING /var/log/seagate/motr/addb: no such directory
'/var/log/motr-rpm-conf-update.log' -> 'm0reportbug-data/motr-rpm-conf-update.log'
Please attach these files to the bug report:
m0reportbug-data.tar.xz
m0reportbug-cores.tar.xz
m0reportbug-traces.tar.xz
You have new mail in /var/spool/mail/root

@yatin-mahajan
Copy link
Contributor

Looks good to me.

Copy link
Contributor

@yatin-mahajan yatin-mahajan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me.

@mehjoshi mehjoshi merged commit ca694ad into kubernetes Nov 16, 2021
madhavemuri pushed a commit that referenced this pull request Nov 29, 2021
…ng data for m0reportbug-traces. (#1221)

Fix : If /proc/sysrq-trigger is mounted as ro then return.

Signed-off-by: Papan Kumar Singh <papan.kumarsingh@seagate.com>
madhavemuri pushed a commit that referenced this pull request Dec 6, 2021
…ng data for m0reportbug-traces. (#1221)

Fix : If /proc/sysrq-trigger is mounted as ro then return.

Signed-off-by: Papan Kumar Singh <papan.kumarsingh@seagate.com>
madhavemuri pushed a commit that referenced this pull request Dec 9, 2021
…ng data for m0reportbug-traces. (#1221)

Fix : If /proc/sysrq-trigger is mounted as ro then return.

Signed-off-by: Papan Kumar Singh <papan.kumarsingh@seagate.com>
madhavemuri pushed a commit that referenced this pull request Dec 10, 2021
…ng data for m0reportbug-traces. (#1221)

Fix : If /proc/sysrq-trigger is mounted as ro then return.

Signed-off-by: Papan Kumar Singh <papan.kumarsingh@seagate.com>
madhavemuri pushed a commit that referenced this pull request Dec 13, 2021
…ng data for m0reportbug-traces. (#1221)

Fix : If /proc/sysrq-trigger is mounted as ro then return.

Signed-off-by: Papan Kumar Singh <papan.kumarsingh@seagate.com>
madhavemuri pushed a commit that referenced this pull request Dec 13, 2021
…ng data for m0reportbug-traces. (#1221)

Fix : If /proc/sysrq-trigger is mounted as ro then return.

Signed-off-by: Papan Kumar Singh <papan.kumarsingh@seagate.com>
madhavemuri pushed a commit that referenced this pull request Dec 13, 2021
…ng data for m0reportbug-traces. (#1221)

Fix : If /proc/sysrq-trigger is mounted as ro then return.

Signed-off-by: Papan Kumar Singh <papan.kumarsingh@seagate.com>
@papan-singh papan-singh deleted the EOS-25693-new branch April 4, 2022 09:34
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants