Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Write a POC of ui testing #1600

Closed
wants to merge 11 commits into from
Closed

Conversation

saroad2
Copy link
Member

@saroad2 saroad2 commented Oct 4, 2022

Related to #1436

This PR is a POC for writting UI tests with pyautogui for Toga apps.
Here I tested 3 things:

  1. That when clicking a button the on_press method is called
  2. That when clicking a disabled button on_press method is not called
  3. That one can color a button setting button.style.background_color

This is in no way a production-ready PR, this is just to show how such tests should look.

@freakboy3742
Copy link
Member

Can you elaborate on how to run this test suite? I've tried running it with pytest test_ui, and the test app displays, but then locks up.

We will also need to see this running in the CI environment to demonstrate that it's viable.

@saroad2
Copy link
Member Author

saroad2 commented Oct 5, 2022

I run the tests simply with pytest test_ui, but it seems that the test hang from time (aka, the tests are unstable).
After increasing the sleeping time before and after the tests in the setUp and tearDown method, it passed every time.

(venv) C:\Users\sagi\PycharmProjects\toga>pytest test_ui
========================================================================================================= test session starts =========================================================================================================
platform win32 -- Python 3.10.5, pytest-7.1.3, pluggy-1.0.0
rootdir: C:\Users\sagi\PycharmProjects\toga
collected 3 items                                                                                                                                                                                                                      

test_ui\widgets\test_button.py ...                                                                                                                                                                                               [100%]

========================================================================================================== 3 passed in 2.43s ========================================================================================================== 

@saroad2
Copy link
Member Author

saroad2 commented Oct 5, 2022

It seems like the tests are still hanging in the CI process. I had to stop it manually.
Maybe a larger sleeping time will solve the problem, but I guess there should be a better solution for it.

Copy link
Member

@freakboy3742 freakboy3742 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The fact it's failing in CI is obviously a problem; the other source of concern is the reliance on sleep(). Firstly, the required sleep is going to be highly load and system dependent, which will make the test somewhat unreliable; however, the larger concern is the impact on overall test runtime.

When there's only 3 tests, a 0.5 second setup and teardown doesn't matter too much; but if there's 1000 tests, it means the test suite will take 16 minutes to run. This might be unavoidable, but if we can avoid it, we should.

@codecov
Copy link

codecov bot commented Oct 23, 2022

Codecov Report

Merging #1600 (cf910ab) into main (35033e5) will decrease coverage by 0.05%.
The diff coverage is 60.00%.

Impacted Files Coverage Δ
src/core/src/toga/window.py 97.34% <50.00%> (-0.86%) ⬇️
src/core/src/toga/app.py 81.77% <66.66%> (-0.41%) ⬇️

@freakboy3742
Copy link
Member

Closing this in favour of #1687.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants