KeyboardBreaker is a computer vision project made to automatically play the game keyboard breaker.
The project orignally ran by detecting the colour blue, grabbing the area of the letter, converting it to grayscale and then parsing it through pytesseract. However, this was very ineffcient and ended up barely being able to beat the easiest difficulty. Therefore the project pivoted to using a simple neural netowrk. Therefore, tensorflow is required for this project.
There are a few steps to setting this up to work on your machine.
- First install requirements.txt
- Install pytesseract: windows, mac
- You may need to change the MSS version in KeyboardBreaker to your system version.
To play the game simply load up the game and wait till the main menu (after the addictinggames logo). You should now be able to run main.py. This may take a few seconds as it's loading in the tensorflow model. When this is loaded, the screen location will be found and the game will automatically load into a hard mode game and destroy the competition. While a game is won by either side the bot will not attempt to press any more keys.
To train the neural network yourself you'll need to grab images using the datagetter.py module. This currently still runs off of pytesseract so is not as efficient as it could be. Moreover, you also need to manually locate the screen valeus (Sorry!). You will also need to create an output folder in the keyboardbreaker directory. After that, you simply just start a game and let it run. it will automatically play new games and take new screenshots of letters to use to be trained on. Currently pytesseract will predict the letter and place it in the folder. If it is not sure it will put it in unknown. You should manually go into the folders and take out incorrect data if you plan on training this! Currently the best accuracy I could get with all the screen characters on a dataset of 800 training images was 93.5%. See if you can beat me!
Thanks to ClarityCoders for inspiring this project from his fastwords project
Thanks to Praveen for his hsvfinder implementation that I modified to find the correct hsv values for the different shades of blue seen in this project.
Thanks to codebasics for his digit classification video that gave me some good information about the neural network problem space.