Using SikuliX OCR API with Appium
Disclaimer: I have modified OCR.java from https://github.com/Simon-Kaz/AppiumFindByImage to add some more features.
Why shoud you use this?
If your iPhone/Android mobile app does not have an id, name or accessibilityID then chances are that you cannot interact with that element on the screen, especially if Xpath is unreliable or not available. Image recognition with SikuliX api comes to the rescue in this case.
What will you need?
- Appium server
- Appium java client
- SikuliX API Jar v1.1.1 - http://sikulix.com/
How to use this class?
- Take a screenshot of the screen of interest on your iPhone
- Use the Photos app on your Macbook to crop the screenshot and save the image you want to click using Appium (file.jpg). No other screenshot utility worked with SikuliX except Photos
- Note that the coordinates of the matched image need to be divided by 2 to account for the "Retina" display on iPhones since they pack twice the number of pixels.