Here you will find all the different scripts and tools that we use to generate the data.
As there are several metrics being reported, each with its independent pipeline, the overall data pipeline can seem a bit complex. Therefore, tis file attempts to explain the most relevant processes in use as we believe that transparency is a must and that it can help developers in contributing to the project.
| Folder | Description |
|---|---|
grapher |
Contains output files that power our grapher visualizations |
input |
External files used to compute derived metrics, such as X-per capita, and aggregate groups, such as 'Asia', etc. |
notebook |
Notebooks used for test purposes. |
scripts |
Contains all pipelines. The code is a mixture of R and Python scripts. |
src |
cowidev library. WORK IN PROGRESS |
Note that the folder public/data is not to be modified, as it contains output files generated by this pipeline. Exceptions may include output folder refactor and others.
📁 Find it at
scripts/vaccinations/
It is pobably the most mature and complex process. It includes a python library that provides all the tools to run all country importers.
More info:
📁 Find it at
scripts/testing/
It resembles very much the architecture of the vaccination pipeline, but differs in some key points. The most noticeable difference is that it contains both R and python code. We currently prefer contributions in Python.
More info:
📁 Find it at
cowidev.excess_mortality
Collects excess mortality data and exports it to human-readable format.
More info:
Run it:
python -m cowidev.excess_mortality etl
📁 Find it at
cowidev.yougov
Run it:
python -m cowidev.yougov
📁 Find them at
scripts/
Old README content can be found at README-old.md