Fix axe accessibility test violations compare

[mohanqxf2]

I have updated the following to the test_accessibility.py script:

• Changed ‘test_obj.conditional_write’ to ‘test_obj.log_result’
• Updated the logic to fetch only violations[] section from the axe_run results, compare it with snapshot_output.json file violations[] section, and print the results if any violations are found.
• Updated snapshot_util.py with load and save snapshot file.

To run under tests/ dir:
python -m pytest test_accessibility.py

[avinash010]

I see the snapshot test is failing when i ran it today. Also i see "Error processing consolidated log file: [Errno 2] No such file or directory: 'log/consolidated_log.txt'"

- ID: region, Impact: moderate, Description: Ensures all page content is contained by landmarks
2024-10-28 08:32:03.864 | ERROR    | /tests/test_accessibility.py::test_accessibility | FAIL: Accessibility checks for main page failed
New violations found on redirect:
- ID: region, Impact: moderate, Description: Ensures all page content is contained by landmarks
2024-10-28 08:32:06.863 | ERROR    | /tests/test_accessibility.py::test_accessibility | FAIL: Accessibility checks for redirect failed
New violations found on contact page:
- ID: region, Impact: moderate, Description: Ensures all page content is contained by landmarks
2024-10-28 08:32:10.456 | ERROR    | /tests/test_accessibility.py::test_accessibility | FAIL: Accessibility checks for contact page failed
2024-10-28 08:32:10.461 | INFO     | /tests/test_accessibility.py::test_accessibility | 

************************
--------RESULT--------
Total number of checks=3
2024-10-28 08:32:10.466 | INFO     | /tests/test_accessibility.py::test_accessibility | Total number of checks passed=0
----------------------
************************


2024-10-28 08:32:10.473 | INFO     | /tests/test_accessibility.py::test_accessibility | Total number of mini-checks=0
2024-10-28 08:32:10.478 | INFO     | /tests/test_accessibility.py::test_accessibility | Total number of mini-checks passed=0
2024-10-28 08:32:10.483 | ERROR    | /tests/test_accessibility.py::test_accessibility | 
--------FAILURE SUMMARY--------

2024-10-28 08:32:10.487 | ERROR    | /tests/test_accessibility.py::test_accessibility | FAIL: Accessibility checks for main page failed
2024-10-28 08:32:10.490 | ERROR    | /tests/test_accessibility.py::test_accessibility | FAIL: Accessibility checks for redirect failed
2024-10-28 08:32:10.494 | ERROR    | /tests/test_accessibility.py::test_accessibility | FAIL: Accessibility checks for contact page failed
FAILED2024-10-28 08:32:13.507 | INFO     | /conftest.py::test_obj | *************************

Error processing consolidated log file: [Errno 2] No such file or directory: 'log/consolidated_log.txt'


======================================================================================= FAILURES =======================================================================================
______________________________________________________________________________ test_accessibility[chrome] ______________________________________________________________________________
test_accessibility.py:87: in test_accessibility
    assert expected_pass == actual_pass, f"Test failed: {__file__}"
E   AssertionError: Test failed: /home/avinash/qxf2/qxf2_pom/qxf2-page-object-model/tests/test_accessibility.py
E   assert 3 == 0

[avinash010]

Probably we need to discuss this. Do we need to save snapshot if there is no snapshot?

[avinash010]

@mohanqxf2
*I don't see the new snapshots, unless we have new snapshot the tests will continue to fail

• As discussed yesterday we don't need to save snapshot if there are no snapshot existing. You can print a message in load snapshot if file not found create a snapshot by running some command
• As discussed we don't see the reason of failure of the accessibility tests, we need that info
• I see you using print statements, can we write the stuff inside log_result?
• I see an error after test run "Error processing consolidated log file: [Errno 2] No such file or directory: 'log/consolidated_log.txt'"
• Can you figure out cleaner way to implement this using log result

            cleaned_result = json.dumps(violations, ensure_ascii=False, separators=(',', ':'))
            cleaned_snapshot = json.dumps(saved_snapshot, ensure_ascii=False, separators=(',', ':'))

            # Check if there are new violations
            new_violations = snapshot_util.find_new_violations(violations, saved_snapshot)
            if new_violations:
                print(f"New violations found on {page}:")
                for violation in new_violations:
                    print(
                        f"- ID: {violation['id']}, "
                        f"Impact: {violation['impact']}, "
                        f"Description: {violation['description']}"
                    )

            # Snapshot comparison
            snapshot_result = cleaned_result == cleaned_snapshot