Improve test coverage #2351

phoerious · 2018-10-03T18:28:01Z

Description

This PR updates the coverage report generation target, replaces lcov with gcov and adds extensive unit tests to cover the CLI module. It also fixes a bug I discovered during unit test development, where the export command would just dump the raw XML body without decrypting embedded Salsa20-protected values first, making the export basically useless, since passwords are scrambled.

In addition, all uses of qCritical() with untranslatble raw char* sequences were removed in favor of proper locale strings. These are written to STDERR through QTextStreams and support output redirection for testing purposes. With this change, error messages don't depend on the global Qt logging settings and targets anymore and go directly to the terminal or into a file if needed.

Lastly, all CLI commands received a dedicated -h/--help option.

Motivation and context

Generation of unit test coverage reports used to be quite complicated and required a lot of different settings, including a custom CMake build type. This patch updates the coverage CMake module to only require -DWITH_COVERAGE=ON to be set on a normal Debug build in order to create a coverage target.

This patch also moves away from lcov in favor of gcovr, since lcov appears to be broken in GCC 8. However, the routines for generating lcov reports still exist, so provided lcov receives updates and there is sufficient reason to switch back, it is easy to do so.

How has this been tested?

Tests pass and increase coverage by about 5%.

Types of changes

✅ Bug fix (non-breaking change which fixes an issue)
✅ New feature (non-breaking change which adds functionality)

Checklist:

✅ I have read the CONTRIBUTING document. [REQUIRED]
✅ My code follows the code style of this project. [REQUIRED]
✅ All new and existing tests passed. [REQUIRED]
✅ I have compiled and verified my code with -DWITH_ASAN=ON. [REQUIRED]
✅ My change requires a change to the documentation and I have updated it accordingly.
✅ I have added tests to cover my changes.

droidmonkey

The new bootstrap namespace is a great addition!

src/cli/Utils.cpp

droidmonkey · 2018-10-04T00:45:55Z

src/cli/keepassxc-cli.cpp

    if (!Crypto::init()) {
        qFatal("Fatal error while testing the cryptographic functions:\n%s", qPrintable(Crypto::errorString()));
        return EXIT_FAILURE;
    }

    QCoreApplication app(argc, argv);
-    app.setApplicationVersion(KEEPASSX_VERSION);
+    QCoreApplication::setApplicationVersion(KEEPASSX_VERSION);


This might be a good time to change this to KEEPASSXC_VERSION

droidmonkey · 2018-10-04T00:47:00Z

CMakeLists.txt

 include(CLangFormat)

 if(UNIX AND NOT APPLE)
-  find_package(Qt5 COMPONENTS Core Network Concurrent Widgets Test LinguistTools DBus REQUIRED)
+  find_package(Qt5 COMPONENTS Core Network Concurrent Gui Widgets Test LinguistTools DBus REQUIRED)


Can we instead put these components in a list and use that for each find_package?

droidmonkey · 2018-10-04T00:55:50Z

src/core/Tools.cpp

-                return true;
-            }
-        }
+bool hasChild(const QObject* parent, const QObject* child)


Shouldn't we just be using QObject::findChild(name) and passing in child->objectName?

http://doc.qt.io/qt-5/qobject.html#findChild

A quick look at the usages of this function and it seems to be misused. For example, it is used to check if the recycle bin contains an entry. That is inappropriate as you should use the Group function "findEntryByUUID" instead of relying on the pointer value of the entry. See cli/Remove.cpp line 87.

Probably. I just copied it over.

src/core/Tools.cpp

src/format/KdbxReader.cpp

src/format/KeePass2Reader.cpp

droidmonkey · 2018-10-04T01:28:09Z

tests/CMakeLists.txt

@@ -180,7 +183,7 @@ add_unit_test(NAME testrandom SOURCES TestRandom.cpp
 add_unit_test(NAME testentrysearcher SOURCES TestEntrySearcher.cpp
        LIBS ${TEST_LIBRARIES})

-add_unit_test(NAME testcsvexporter SOURCES TestCsvExporter.cpp
+add_unit_test(NAME testcsveporter SOURCES TestCsvExporter.cpp


In pull request keepassxreboot#2351 phoerious updated the coding style for cli components. This commit implements similar changes for the new Create component

Generation of unit test coverage reports used to be quite complicated and required a lot of different settings, including a custom CMake build type. This patch updates the coverage CMake module to only require -DWITH_COVERAGE=ON to be set on a normal Debug build in order to create a coverage target. This patch also moves away from lcov in favor of gcovr, since lcov appears to be broken in GCC 8. However, the routines for generating lcov reports still exist, so provided lcov receives updates and there is sufficient reason to switch back, it is easy to do so.

The CLI module was lacking unit test coverage and showed some severe coding style violations, which this patch addresses. In addition, all uses of qCritical() with untranslatble raw char* sequences were removed in favor of proper locale strings. These are written to STDERR through QTextStreams and support output redirection for testing purposes. With this change, error messages don't depend on the global Qt logging settings and targets anymore and go directly to the terminal or into a file if needed. This patch also fixes a bug discovered during unit test development, where the extract command would just dump the raw XML contents without decrypting embedded Salsa20-protected values first, making the XML export mostly useless, since passwords are scrambled. Lastly, all CLI commands received a dedicated -h/--help option.

In pull request keepassxreboot#2351 phoerious updated the coding style for cli components. This commit implements similar changes for the new Create component

- New Database Wizard [#1952] - Advanced Search [#1797] - Automatic update checker [#2648] - KeeShare database synchronization [#2109, #1992, #2738, #2742, #2746, #2739] - Improve favicon fetching; transition to Duck-Duck-Go [#2795, #2011, #2439] - Remove KeePassHttp support [#1752] - CLI: output info to stderr for easier scripting [#2558] - CLI: Add --quiet option [#2507] - CLI: Add create command [#2540] - CLI: Add recursive listing of entries [#2345] - CLI: Fix stdin/stdout encoding on Windows [#2425] - SSH Agent: Support OpenSSH for Windows [#1994] - macOS: TouchID Quick Unlock [#1851] - macOS: Multiple improvements; include CLI in DMG [#2165, #2331, #2583] - Linux: Prevent Klipper from storing secrets in clipboard [#1969] - Linux: Use polling based file watching for NFS [#2171] - Linux: Enable use of browser plugin in Snap build [#2802] - TOTP QR Code Generator [#1167] - High-DPI Scaling for 4k screens [#2404] - Make keyboard shortcuts more consistent [#2431] - Warn user if deleting referenced entries [#1744] - Allow toolbar to be hidden and repositioned [#1819, #2357] - Increase max allowed database timeout to 12 hours [#2173] - Password generator uses existing password length by default [#2318] - Improve alert message box button labels [#2376] - Show message when a database merge makes no changes [#2551] - Browser Integration Enhancements [#1497, #2253, #1904, #2232, #1850, #2218, #2391, #2396, #2542, #2622, #2637, #2790] - Overall Code Improvements [#2316, #2284, #2351, #2402, #2410, #2419, #2422, #2443, #2491, #2506, #2610, #2667, #2709, #2731]

phoerious added bug new feature tests feature: CLI labels Oct 3, 2018

phoerious added this to the v2.4.0 milestone Oct 3, 2018

phoerious requested a review from droidmonkey October 3, 2018 18:28

phoerious force-pushed the feature/coverage branch 6 times, most recently from dab0949 to 89e5734 Compare October 3, 2018 19:40

phoerious changed the title ~~Feature/coverage~~ Improve test coverage Oct 3, 2018

phoerious force-pushed the feature/coverage branch 7 times, most recently from 64a8c89 to 77d9d2c Compare October 3, 2018 22:12

droidmonkey requested changes Oct 4, 2018

View reviewed changes

phoerious force-pushed the feature/coverage branch 3 times, most recently from 5d0f827 to b82ea87 Compare October 4, 2018 22:01

droidmonkey mentioned this pull request Oct 8, 2018

Add unprotected output switch to keepassxc-cli extract #2374

Closed

phoerious force-pushed the feature/coverage branch 2 times, most recently from b3c90e9 to 90c508d Compare October 19, 2018 18:32

keepassxreboot deleted a comment from phoerious Oct 19, 2018

phoerious added 5 commits October 19, 2018 21:49

Fix tests on Windows

108e4ef

Rename TestClock to MockClock and move it to the mock directory

bea31f9

Implement review feedback

0ca7fd3

Reformat CMakeLists.txt files

77adbef

phoerious force-pushed the feature/coverage branch from 8914d0a to 77adbef Compare October 19, 2018 20:16

droidmonkey approved these changes Oct 19, 2018

View reviewed changes

droidmonkey merged commit c749f70 into develop Oct 19, 2018

droidmonkey deleted the feature/coverage branch October 19, 2018 23:44

SohamG mentioned this pull request May 7, 2019

Enable unprotected output of keepassxc-cli extract #2164

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Improve test coverage #2351

Improve test coverage #2351

phoerious commented Oct 3, 2018

droidmonkey left a comment

droidmonkey Oct 4, 2018

droidmonkey Oct 4, 2018

droidmonkey Oct 4, 2018

phoerious Oct 4, 2018

droidmonkey Oct 4, 2018

Improve test coverage #2351

Improve test coverage #2351

Conversation

phoerious commented Oct 3, 2018

Description

Motivation and context

How has this been tested?

Types of changes

Checklist:

droidmonkey left a comment

Choose a reason for hiding this comment

droidmonkey Oct 4, 2018

Choose a reason for hiding this comment

droidmonkey Oct 4, 2018

Choose a reason for hiding this comment

droidmonkey Oct 4, 2018

Choose a reason for hiding this comment

phoerious Oct 4, 2018

Choose a reason for hiding this comment

droidmonkey Oct 4, 2018

Choose a reason for hiding this comment