bankform.py allows to import bank sector statistics stored as archived DBF files at Bank of Russia website ("public data") to a local MySQL database, aggregate data into reports and save these reports in csv or xlsx format. The script can also import statistics stored locally in text form files ("private data").
Principal steps taken in bankform.py are the following:
- Original archived DBF files are stored at Bank of Russia website
- Download these ZIP/RAR files, unpack to get local DBF files
- Convert DBF files to CSV
- Import CSV files to "raw data" database (large in size, contains all dates and banks)
- Truncate dataset and migrate it to "final use" database (smaller in size, just a few banks)
- Create output reports and write reports to CSV or XLSX files
These steps are shown diagram below:
For more information see:
For full command line interface:
python -m cbr_db.bankform
or
baf.bat
The script below will download data and produce final balance sheet report for Jan 1, 2015 for reporting form 101. The final report files will be located in 'output' folder.
python bankform.py reset database raw
python bankform.py reset database final
python bankform.py download 101 2015-01-01
python bankform.py unpack 101 2015-01-01
python bankform.py make csv 101 2015-01-01
python bankform.py import csv 101 2015-01-01
python bankform.py make dataset 101 2015-01-01
python bankform.py migrate dataset 101
python bankform.py make balance
python bankform.py report balance --xls
Same result shorter:
baf reset database
baf update 101 2015-01
baf make dataset 101 2015-01
baf migrate dataset 101
baf make balance
baf report balance --xlsx
Same result using wrapper script:
python wrapper.py simple 101 2015-01-01
See python wrapper.py --help for additional options.
Check output by dir ..\output (Win) or ls dir ..\output (Linux).
See Testing milestones for more comments about these sample scipts.