A scraper to collect branch information from https://www.srsdistribution.com.
The scraper collects the main branch information, then iterates through each branch URL and collects the individual data. This is then inserted into a google sheet using the gspread module.
- Initialise the Poetry environment.
- In a terminal change directories into the current project.
- run
poetry init
There are a few steps needed to authenticate this application to insert into a Google sheet. The first are of course to have an account and create a sheet, and the rest of the process is explained in detail in the documentation. Note that as this is a bot, the authentication method will be via Service Account. If you are already familiar with the process, then just follow the Quickstart section below!
- Create a project here.
- Enable google drive api
- Enable google sheets aip
- Go to credentials and create a service account.
- Under 'Actions' select 'manage keys' and create a key in JSON. Store the output in a file named 'google_service_account.json' in this repository.
- Copy the service account email.
- Create Google sheet.
- Share the google sheet with the service account email.