Initial Catch2 integration

[dmaslenko]

This is a part of migration on Catch2 test framework.
The initial commit includes:

• New tests2 folder (as temporary name to migrae all tests and then it will be renamed back to the original tests)
• New CMakeLists.txt files for unit and GUI tests.
• Unit and GUI tests for PassphraseGenerator and TOTP classes.

Supports #5473

Type of change

• ✅ Refactor (significant modification to existing code)

[droidmonkey]

Wow that produces a clean test! Do you think this is worth it to keep going?

[dmaslenko]

Yes, definitely we should keep going.

Once I done with both regular and GUI demo tests I will ask you to help with TeamCity to configure the Catch2 installation. Locally I installed it from the git repo as described here but I don't know how to manage the TeamCity.

Once we got working environment and couple demo tests I would love to migrate all the rest tests one by one. Yes, this is a long journey, but I like such kind of job.

For now I have a blocker to run all tests together - new tests does not create a QT application and fails on not initialized app, but standalone run works fine. Probably there is a conflict or the main.cpp is not called, I will debug it...

[phoerious]

Does this work with all tests? Qt has a bunch of routines that test for conditions repeatedly, executing the event loop in between.

[dmaslenko]

Never mind, it works from the command line, something wrong with my CLion.

[dmaslenko]

Keep moving, the main window is already displayed, will implement a simple GUI test to check a generated phase on not open DB.

[droidmonkey]

@phoerious qt creates that event loop using the macros at the top of test files. Catch2 bunches all tests into one or more exe's, you define the main function and at that point is when you would establish the qt event loop.

https://stackoverflow.com/a/56569699

[dmaslenko]

I still have couple ideas to improve tests and add missed test cases for Passphrase Generator but in general it is ready for review.

[droidmonkey]

You are a machine! This is awesome

[dmaslenko]

I am trying to implement new test when an user generated a passphrase and wants to apply it.
For some reason neither Apply nor Close nor x buttons work properly from GUI test. As standalone app it works fine.
See the recorded video from GUI test:

For video I put temporary wait() function to have an ability to click the button manually as well.

Any advise where I need to dig deeply?
The apply method is

void PasswordGeneratorWidget::applyPassword()
{
    saveSettings();
    m_passwordGenerated = true;
    emit appliedPassword(m_ui->editNewPassword->text());
    emit closed();
}

where the emit appliedPassword() completed as expected (I can see the changed password in an entry but emit closed() does not close the current Generator widget.

See my the latest code in this test:

SCENARIO_METHOD(FixtureWithDb, "Passphrase Generator on new Entry", "[gui]")

[droidmonkey]

This could happen if you are blocking the event loop, but that would also cause the whole gui to be unresponsive. This can also happen if the button isn't triggering the function you pasted above. Can you set a breakpoint in that function to see if it gets called?

[dmaslenko]

I made sure the mentioned function is triggered by a breakpoint and additional console logs.
As I mentioned inside the method

void PasswordGeneratorWidget::applyPassword()
{
    saveSettings();
    m_passwordGenerated = true;
    emit appliedPassword(m_ui->editNewPassword->text());
    emit closed();
}

the first emit works as expected: the password was updated in an entry, the second emit does not close the widget.
I even changed the order of emits, like this

    emit closed();
    emit appliedPassword(m_ui->editNewPassword->text());

the same the first emit close does not work, the second emit appliedPassword works fine but should not because the widget is to be closed.

I think something wrong with this mouse click:

            AND_WHEN("User clicks Apply Password button")
            {
                auto* applyPasswordButton = pPwGenWidget->findChild<QPushButton*>("buttonApply");
                QTest::mouseClick(applyPasswordButton, Qt::MouseButton::LeftButton);
                QApplication::processEvents();

because this variant works fine

            AND_WHEN("User clicks Apply Password button")
            {
//                auto* applyPasswordButton = pPwGenWidget->findChild<QPushButton*>("buttonApply");
//                QTest::mouseClick(applyPasswordButton, Qt::MouseButton::LeftButton);
//                QApplication::processEvents();
                QTest::keyClick(pPassPhraseWidget, Qt::Key_Escape);

but this one does not work

            AND_WHEN("User clicks Apply Password button")
            {
                auto* applyPasswordButton = pPwGenWidget->findChild<QPushButton*>("buttonApply");
                QTest::mouseClick(applyPasswordButton, Qt::MouseButton::LeftButton);
                QApplication::processEvents();
                QTest::keyClick(pPassPhraseWidget, Qt::Key_Escape);

even the latest Esc command does not close the widget.

I am continue the investigation...

[droidmonkey]

I think you are getting stuck in the process events call

[dmaslenko]

The password generator has 3 ways to apply a generated password:

• Mouse click on the Apply button
• Ctrl + S
• Ctrl + Enter

I tried all of them and only this shortcut works fine:

QTest::keyClick(pPassPhraseWidget, Qt::Key::Key_S, Qt::ControlModifier);

The mouse click does not work as mentioned above. BTW, I found a similar Stackoverflow question.

The second shortcut does not work as well:

QTest::keyClick(pPassPhraseWidget, Qt::Key::Key_Enter, Qt::ControlModifier);

All of these 3 ways work fine in a standalone app.
It can be a Linux specific issue, I will try to run on MacOS...

[droidmonkey]

I fixed a crash you had in the fixtureWithDb class. I can replicate the issue with the password generator dialog, but it might be because of sequencing in the test case itself. You should add a wait condition to the "isVisible" check.

[dmaslenko]

Thank you, you really helped and unblocked me with the crash.
I recognized this place, originally it was as in TestGUI.cpp: fileDialog()->setNextFileName(m_dbFilePath); but I had some issue and fixed it by deleting of the returned pointer. I don't remember exactly the problem. Anyway, let's keep the fix and I will pay attention if run into an issue again.
However, the issue with clicking on the Apply or Close buttons is still open, a wait(10000) does not help, continue the investigation...