Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added approaches to move loganalyzer to pytest. #1

Closed
wants to merge 3 commits into from
Closed
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
73 changes: 73 additions & 0 deletions doc/loganalyzer_to_pytest.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,73 @@
#### Requirements
Move existed loganalyzer to be supported by pytest framework.

#### Overview
At this approach all the work will be done on the Ansible host in comparing with current implementation, so script will download extracted syslog file to the Ansible host.
Functionality will give possibility to:
- Add start marker to the DUT syslog. Start Marker will be automatically generated based on the prefix + date. Start marker format will be kept the same as in current implementation
- Configure regular expressions which will be used to found matches in the syslog logs
- Extract DUT syslog based on the markers. Download extracted logs to the Ansible host
- Perform analysis of downloaded syslog and return the result

Current loganalyzer implementation description:
https://github.com/Azure/SONiC/wiki/LogAnalyzer

#### Development
Module "loganalyzer.py" with class "Loganalyzer".

"Loganalyzer" class interface:
- __init__(ansible_host: ansible_host, run_dir="/tmp")
- load_common_config() - Clear previous configured match, expect and ignore. Load regular expressions from common configuration files: match, expect and ignore which are located in some configuration directory or with current module. Save loaded configuration to the self.match_regex, self.expect_regex and self.ignore_regex attributes.
- parse_regexp_file(file_path) - Read and parse regular expressions from specified file. Return list of strings of defined regular expressions.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Probably a @classmethod ?

- run_cmd(callable, *args, **kwargs) - Call function and analyze DUT syslog during function execution. Return the same result as "analyze" function.
- init(marker_prefix: string) - Add start marker to the DUT syslog. Generated marker format: marker_prefix + "%Y-%m-%d-%H:%M:%S"
- analyze() - Extract syslog based on the start marker and copy it to ansible host. Analyze extracted syslog file localy. Return python dictionary object.
Return example:
{"counters": {"match": 1, "expected_match": 0, "expected_missing_match": 0},
"match_files": {"/tmp/syslog": {"match": 0, "expected_match": 32},
Copy link

@wangxin wangxin Jul 9, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Because the log analyzer may be called mutliple times, the logs may be copied to sonic-mgmt container multiple times. I think we need to consider putting the copied files under different temp folders to avoid conflicts.

After analyzing is done, should we cleanup the copied temp log files?

Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"I think we need to consider putting the copied files under different temp folders to avoid conflicts." - Yes it is planned like this.
"After analyzing is done, should we cleanup the copied temp log files?" - Yes we should clean up temp log file.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@yvolynets-mlnx Can the loganalyzer print whole temp file with DEBUG level. I assume we will run with DEBUG on jenkins and this part of log that is analyzed will be visible in output. Useful I think

Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Depends on the DUT health, it colud take many lines. We need more comments about this to decide whether to add it automatically.

"/tmp/syslog1": {"match": 0, "expected_match": 15}
},
"match_messages": {"/tmp/syslog1": ["Message 1", "Message 2", "Message n"],
"/tmp/syslog2": ["Message 1", "Message 2", "Message n"]
}
}
- save_full_log(dest_path) - Download extracted DUT syslog (/tmp/syslog) to the Ansible host folder specified in 'dest_path' input parameter.

Attributes:
- match_regex - list of regular expression strings to match
- expect_regex - list of regular expression strings to expect
- ignore_regex - list of regular expression strings to ignore

Usage example:

from ansible_host import ansible_host
from loganalyzer import Loganalyzer

def test(localhost, ansible_adhoc, testbed):
hostname = testbed['dut']
ans_host = ansible_host(ansible_adhoc, hostname)

loganalyzer = Loganalyzer(ans_host)

# If it is a need to load common search regular expressions. It will load regexp from common files and store values in the match_regex, expect_regex and ignore_regex attributes.
loganalyzer.load_common_config()

# Add start marker to DUT syslog
loganalyzer.init()

# Example: If test need specific search marker it can be added
loganalyzer.match.append("Runtime error: can't parse mac address 'None'")

# Example: Get current match regular expressions
print(loganalyzer.match_regex)

# Example: Remove specific match regular expression
loganalyzer.match_regex.remove("Runtime error: can't parse mac address 'None'")

# Example: read test specific match file and add read strings to the existed match list
loganalyzer.match_regex.extend(loganalyzer.parse_regexp_file("PATH_TO_THE_FILE/FILE.txt"))

# Execute test steps here...

result = loganalyzer.analyze()
assert result["counters"]["match"] == 0, "Failure message\n{}\n{}".format(result["counters"], result["match_messages"])