Skip to content

Commit

Permalink
Merge pull request #21 from pevogam/pyautogui-controller
Browse files Browse the repository at this point in the history
Add a PyAutoGUI controller backend together with some long awaited
renaming and coverage for all display controllers.
  • Loading branch information
pevogam authored Oct 23, 2020
2 parents dd17e65 + 908ff41 commit 45349b9
Show file tree
Hide file tree
Showing 24 changed files with 1,932 additions and 1,302 deletions.
2 changes: 2 additions & 0 deletions .travis.yml
Original file line number Diff line number Diff line change
Expand Up @@ -51,6 +51,8 @@ addons:
# vncdotool
- xfonts-base
- x11vnc
# pyautogui
- scrot
# install any dependencies and build package
install:
- if [[ $INSTALL_VARIANT == "pip" ]]; then travis_retry pip --default-timeout=60 install -r packaging/pip_requirements.txt; fi
Expand Down
16 changes: 8 additions & 8 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,14 +1,14 @@
# guibot [![Build Status](https://travis-ci.org/intra2net/guibot.svg?branch=master)](https://travis-ci.org/intra2net/guibot) [![Documentation Status](https://readthedocs.org/projects/guibot/badge/?version=latest)](http://guibot.readthedocs.io/en/latest/?badge=latest) [![Language grade: Python](https://img.shields.io/lgtm/grade/python/g/intra2net/guibot.svg?logo=lgtm&logoWidth=18)](https://lgtm.com/projects/g/intra2net/guibot/context:python) [![codecov](https://codecov.io/gh/intra2net/guibot/branch/master/graph/badge.svg)](https://codecov.io/gh/intra2net/guibot)

A tool for GUI automation using a variety of computer vision and desktop control backends.
A tool for GUI automation using a variety of computer vision and display control backends.

## Introduction

In order to do GUI automation you usually need to solve two problems: first, you need to have a way to control and interact with the interface and platform you are automating and second, you need to be able to locate the objects you are interested in on the screen. Guibot helps you do both.

To interact with GUIs, Guibot provides the [`desktopcontrol`](https://github.com/intra2net/guibot/blob/master/guibot/desktopcontrol.py) module which contains a common interface for different desktop control backends, with methods to move the mouse, take screenshots, type characters and so on. The backend to use will depend on how your interface is accessible, but the ones currently supported are qemu, vnctotool, xdotool, and autopy.
To interact with GUIs, Guibot provides the [`controller`](https://github.com/intra2net/guibot/blob/master/guibot/controller.py) module which contains a common interface for different display backends, with methods to move the mouse, take screenshots, type characters and so on. The backend to use will depend on how your interface is accessible, but the ones currently supported are listed below.

To locate an element on the screen, you will need an image representing the screen, a [`target`](https://github.com/intra2net/guibot/blob/master/guibot/target.py) and a [`finder`](https://github.com/intra2net/guibot/blob/master/guibot/finder.py) - the finder looks for the target on the image, and returns the coordinates to the region where that target appears. Finders, like desktop controllers, are wrappers around different backends that Guibot supports (see below), while targets are a representation of the object you are looking for, such as images, text, or patterns.
To locate an element on the screen, you will need an image representing the screen, a [`target`](https://github.com/intra2net/guibot/blob/master/guibot/target.py) and a [`finder`](https://github.com/intra2net/guibot/blob/master/guibot/finder.py) - the finder looks for the target on the image, and returns the coordinates to the region where that target appears. Finders, like display controllers, are wrappers around different backends that Guibot supports (see below), while targets are a representation of the object you are looking for, such as images, text, or patterns.

Finally, to bridge the gap between controlling the GUI and finding elements, the [`region`](https://github.com/intra2net/guibot/blob/master/guibot/region.py) module is provided. It represents a subregion of a screen, and contains methods to locate targets in this region and interact with the graphical interface using the chosen backends.

Expand All @@ -29,12 +29,12 @@ Supported Computer Vision (CV) backends are based on
- [autopy](https://github.com/msanders/autopy)
- AutoPy matching

Supported Desktop Control (DC) backends are based on
Supported Display Controller (DC) backends are based on

- [autopy](https://github.com/msanders/autopy)
- [vncdotool](https://github.com/sibson/vncdotool)
- [qemu](https://github.com/qemu/qemu)
- [xdotool](https://www.semicomplete.com/projects/xdotool)
- [AutoPy](https://github.com/msanders/autopy)
- [PyAutoGUI](https://github.com/asweigart/pyautogui)
- [VNCDoTool](https://github.com/sibson/vncdotool)
- [XDoTool](https://www.semicomplete.com/projects/xdotool)

## Resources

Expand Down
Loading

0 comments on commit 45349b9

Please sign in to comment.