Enable (x,y) coordinates with TagUI steps #365
Labels
Comments
kensoh
added a commit
to tebelorg/TagUI
that referenced
this issue
Mar 27, 2019
Building on aisingapore#361 and Sikuli integration, this is a PR that adds ability for TagUI to interact with UI elements using (x,y) coordinates. Backward compatible with web locators and image locators. For eg, `click (400,200)` will click on the 400th pixel from left of the screen and 200th pixel from top of the screen. Works also for other steps - `rclick`, `dclick`, `hover`, `type`, `select`.
kensoh
added a commit
that referenced
this issue
Mar 27, 2019
Building on #361 and Sikuli integration, this is a PR that adds ability for TagUI to interact with UI elements using (x,y) coordinates. Backward compatible with web locators and image locators. For eg, `click (400,200)` will click on the 400th pixel from left of the screen and 200th pixel from top of the screen. Works also for other steps - `rclick`, `dclick`, `hover`, `type`, `select`.
|
Committed to master. Prior to packaged release, feature available from cutting edge version here - |
This was referenced Mar 28, 2019
kensoh
added a commit
to tebelorg/TagUI
that referenced
this issue
Mar 28, 2019
…p actions Building on the series of enhancements around Sikuli integration aisingapore#361 and aisingapore#365 for greater control of mouse, publishing a PR that adds a new `mouse` step. This step lets user send a low-level mouse down or up event to the user interface. This can be used for complex UI mouse actions. For eg, when dragging and dropping UI elements - ``` // below drags the UI element left by 200 pixels hover some_element.png mouse down target_x = mouse_x() - 200 hover (`target_x`,`mouse_y()`) mouse up ```
kensoh
added a commit
that referenced
this issue
Mar 28, 2019
#368 - New mouse step for low-level mouse down and mouse up Building on the series of enhancements around Sikuli integration #361 and #365 for greater control of mouse, publishing a PR that adds a new `mouse` step. This step lets user send a low-level mouse down or up event to the user interface. This can be used for complex UI mouse actions. For eg, when dragging and dropping UI elements - ``` // below drags the UI element left by 200 pixels hover some_element.png mouse down target_x = mouse_x() - 200 hover (`target_x`,`mouse_y()`) mouse up ```
kensoh
added a commit
to tebelorg/TagUI
that referenced
this issue
Mar 29, 2019
…modifiers Building on the series of enhancements around Sikuli integration aisingapore#368, aisingapore#361 and aisingapore#365 for greater control of mouse, creating and sending a PR that adds a new `keyboard` step. This step lets user send low-level keyboard keystrokes to the operating system user interface, including the special keys and modifier keys. Normal letter characters and numbers can also be entered. Prior to this, users can only use 1. `type page.png as text` and limited to [enter] and [clear], or 2. use `vision` step to send custom commands to perform complex keyboard actions. Below are some examples. **macOS** ``` keyboard [cmd][space] keyboard safari[enter] keyboard [cmd]c keyboard [cmd]v keyboard testing 123 ``` **Windows** ``` keyboard [ctrl][home] keyboard [printscreen] keyboard [ctrl]c keyboard v[ctrl] keyboard testing 456 ``` List of modifier keys - [shift] [ctrl] [alt] [cmd] [win] [meta] List of special keys - [clear] [space] [enter] [backspace] [tab] [esc] [up] [down] [left] [right] [pageup] [pagedown] [delete] [home] [end] [insert] [f1] .. [f15] [printscreen] [scrolllock] [pause] [capslock] [numlock]
kensoh
added a commit
that referenced
this issue
Mar 29, 2019
…371) Building on the series of enhancements around Sikuli integration #368, #361 and #365 for greater control of mouse, creating and sending a PR that adds a new `keyboard` step. This step lets user send low-level keyboard keystrokes to the operating system user interface, including the special keys and modifier keys. Normal letter characters and numbers can also be entered. Prior to this, users can only use 1. `type page.png as text` and limited to [enter] and [clear], or 2. use `vision` step to send custom commands to perform complex keyboard actions. Below are some examples. **macOS** ``` keyboard [cmd][space] keyboard safari[enter] keyboard [cmd]c keyboard [cmd]v keyboard testing 123 ``` **Windows** ``` keyboard [ctrl][home] keyboard [printscreen] keyboard [ctrl]c keyboard v[ctrl] keyboard testing 456 ``` List of modifier keys - [shift] [ctrl] [alt] [cmd] [win] [meta] List of special keys - [clear] [space] [enter] [backspace] [tab] [esc] [up] [down] [left] [right] [pageup] [pagedown] [delete] [home] [end] [insert] [f1] .. [f15] [printscreen] [scrolllock] [pause] [capslock] [numlock]
kensoh
added a commit
to tebelorg/TagUI
that referenced
this issue
Mar 30, 2019
… for new steps Creating and sending a PR for updated TagUI language definitions and translation engine for the new steps (`keyboard`, `mouse`) and helper functions (`mouse_xy()`, `mouse_x()`, `mouse_y()`) created in aisingapore#370, aisingapore#368, aisingapore#361 and aisingapore#365. This lets users use the new steps and helper functions in 21 different languages. More details on the multiple languages feature of TagUI in below link of documentation - https://github.com/kelaberetiv/TagUI#native-languages
kensoh
added a commit
that referenced
this issue
Mar 30, 2019
…eps (#373) Creating and sending a PR for updated TagUI language definitions and translation engine for the new steps (`keyboard`, `mouse`) and helper functions (`mouse_xy()`, `mouse_x()`, `mouse_y()`) created in #370, #368, #361 and #365. This lets users use the new steps and helper functions in 21 different languages. More details on the multiple languages feature of TagUI in below link of documentation - https://github.com/kelaberetiv/TagUI#native-languages
kensoh
added a commit
to tebelorg/TagUI
that referenced
this issue
Mar 30, 2019
…#361 aisingapore#368 # 370 aisingapore#372 Creating and sending a PR to address issues that can arise out of the PRs on aisingapore#365 - enabling (x,y) coordinates locator, aisingapore#361 - helper functions for retrieving x & y coordinates, and aisingapore#372 - update of translation engine to reflect the new steps (aisingapore#368 mouse and aisingapore#370 keyboard) and functions. **change 1** As TagUI is built on CasperJS, it has by default the use of convenience function x ([more details here](http://docs.casperjs.org/en/latest/selectors.html#index-2)), which is used to form XPath selectors to be used within CasperJS. However, with introduction of (x,y) coordinates selectors, it is not hard to see that many users will start using x as a variable and overwriting x variable with the integer value of the x coordinates (be it for calculation of offset or other purpose). When that happens, the default x function would no longer work and that will basically break the automation operation of TagUI. In this PR, the function x is renamed as xps666 instead (X Path Selector 666), a acronym with a random number, so that when user uses variable x, it will not affect TagUI operations. **change 2** In tagui_header.js, the live mode translation engine use of 'mouse_xy()', 'mouse_x()' and 'mouse_y()' strings is broken up into 'mouse_' + 'xy()', 'mouse_' + 'x()' and 'mouse_' + 'y()'. This prevents tagui_parse.php from mistakenly thinking that these functions are used by users and invoking Sikuli visual automation when users do not use these functions in their TagUI scripts.
kensoh
added a commit
that referenced
this issue
Mar 30, 2019
Creating and sending a PR to address issues that can arise out of the PRs on #365 - enabling (x,y) coordinates locator, #361 - helper functions for retrieving x & y coordinates, and #372 - update of translation engine to reflect the new steps (#368 mouse and #370 keyboard) and functions. **change 1** As TagUI is built on CasperJS, it has by default the use of convenience function x ([more details here](http://docs.casperjs.org/en/latest/selectors.html#index-2)), which is used to form XPath selectors to be used within CasperJS. However, with introduction of (x,y) coordinates selectors, it is not hard to see that many users will start using x as a variable and overwriting x variable with the integer value of the x coordinates (be it for calculation of offset or other purpose). When that happens, the default x function would no longer work and that will basically break the automation operation of TagUI. In this PR, the function x is renamed as xps666 instead (X Path Selector 666), a acronym with a random number, so that when user uses variable x, it will not affect TagUI operations. **change 2** In tagui_header.js, the live mode translation engine use of 'mouse_xy()', 'mouse_x()' and 'mouse_y()' strings is broken up into 'mouse_' + 'xy()', 'mouse_' + 'x()' and 'mouse_' + 'y()'. This prevents tagui_parse.php from mistakenly thinking that these functions are used by users and invoking Sikuli visual automation when users do not use these functions in their TagUI scripts.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Building on #361 and Sikuli integration, this is a PR that adds ability for TagUI to interact with UI elements using (x,y) coordinates. Backward compatible with web locators and image locators.
For eg,
click (400,200)will click on the 400th pixel from left of the screen and 200th pixel from top of the screen. Works also for other steps -rclick,dclick,hover,type,select.The text was updated successfully, but these errors were encountered: