Skip to content

Latest commit

 

History

History
170 lines (137 loc) · 9.64 KB

CONTRIBUTE.md

File metadata and controls

170 lines (137 loc) · 9.64 KB
Raw

Contribute - Vaccination data

We welcome contributions to our vaccination dataset! Note that due to the nature of our pipeline, we cannot accept pull requests for manually imported country data. To see which countries are automated and which reaquire manual import, check this file.

Content

About our vaccination data

For details about the development environment, check the details in README.

We are currently collecting vaccination data at country level in the following format:

location date vaccine source_url total_vaccinations people_vaccinated people_fully_vaccinated
Afghanistan 2021-05-30 Oxford/AstraZeneca, Pfizer/BioNTech, Sinopharm/Beijing https://covid19.who.int/ 600152 480226 119926

Where the metrics total_vaccinations, people_vaccinated and people_fully_vaccinated are defined as read here.

Note that for some countries, some metrics can't be reported as these are not be available. This is not ideal but it is OK.

Manufacturer data

Along with the main data, we include vaccine data break by manufacturer for some countries where this data is available.

Each row in the data gives the cumulative number of doses administered for a given date and vaccine manufacturer.

Fields

  • date: Date in format YYYY-MM-DD
  • vaccine: Vaccine manufacturer name. Our convention for vaccine names can be found here. As new vaccines emerge, new conventions will be defined.
  • location: Region name.
  • total_vaccinations: Cumulative number of administered doses up to date for given vaccine.

Example

date vaccine location total_vaccinations
... ... ... ...
2021-06-01 Moderna Lithuania 151261
2021-06-01 Oxford/AstraZeneca Lithuania 333733
2021-06-01 Johnson&Johnson Lithuania 34974
2021-06-01 Pfizer/BioNTech Lithuania 1133371
... ... ... ...

Notes

We only include manufacturer data for countries for which the process can be automated. No manual reports are currently being accepted. This is to ensure scalability of the project.

Age group data

Along with the main data, we include vaccine data break by age groups for some countries where the data is available.

Each row in the data gives the percentage of people within an age group that have received at least one dose. Note that currently there is no standard for which age groups are accepted, as each country may define different ones. As a general rule, we try to have groups in 10 years chunks.

Note that the reported metric is relative, and not absolute.

Fields

  • date: Date in format YYYY-MM-DD
  • age_group_min: Lower bound of the age group.
  • age_group_max: Upper bound of the age group (included).
  • location: Region name.
  • people_vaccinated_per_hundred: Percentage of people within the age group that have received at least one dose.
  • people_fully_vaccinated_per_hundred: Percentage of people within the age group that have been fully vaccinated.

Example

location date age_group_min age_group_max people_vaccinated_per_hundred people_fully_vaccinated_per_hundred
... ... ... ... ... ...
Poland 2021-06-08 18 24 26.77 7.33
Poland 2021-06-08 25 49 36.01 14.2
Poland 2021-06-08 50 59 50.68 30.22
Poland 2021-06-08 60 69 63.05 35.67
Poland 2021-06-08 70 79 77.45 70.7
Poland 2021-06-08 80 59.94 56.55
... ... ... ... ... ...

Notes

We only include age group data for countries for which the process can be automated. No manual reports are currently being accepted. This is to ensure scalability of the project.

Report new data values

To report new values for a country/location, first check if the imports for that country/territory are automated. You can check column automated in this file.

  • If the country imports are automated (TRUE value in file above), note that the new value might be added in next update. Only report new values if the data is missing for more than 48 hours! Report the new data as a pull request.
  • If the country imports are not automated, i.e. data is manually added, (FALSE value in file above) you can report new data in any of the following ways:
    • Open a new issue, reporting the data and the corresponding source.
    • If you plan to contribute regularly to a specific country/location, consider opening a dedicated issue. This way, we can easily back-track the data addded for that country/location.
    • If this seems too complicated, alternatively, you may simply add a comment to thread #230.

Notes

  • We only accept official sources or news correctly citing official sources.
  • We only accept manual reports for country aggregate vaccination data. That is, we currently do not include manufacturer and age vaccination data if no automation is provided.

Add new country automations

To automate the data import for a country, make sure that:

  • The source is reliable.
  • The source provides data in a format that can be easily read:
    • As a file (e.g. csv, json, xls, etc.)
    • As plain text in source HTML, which can be easily scraped.

Steps

Next, follow the steps below:

  1. Decide if the import is batch (i.e. all the timeseries) or incremental (last value). See the scripts in src/vax/batch and src/vax/incremental for more details. Note: Batch is prefered over Incremental.
  2. Create a script and place it based on decision in step 1 either in src/vax/batch or src/vax/incremental. Note that each source is different and there is no single pattern that works for all sources, however you can take some inspiration from the scripts below:
  3. Feel free to add manufacturer/age data if you are automating a batch script and the data is available.
  4. Test that it is working and that it is stable. For this you need to have the library installed. Run
cowid-vax get -c [country-name]
  1. Issue a pull request and wait for a review.

More details: #230, #250

Criteria to accept pull requests

Due to how our pipeline operates at the moment, pull requests are only accepted under certain conditions. These include, but are not limited to, the following:

  • Code improvements / bug fixes. As an example, you can take #465.
  • Updates on the data for countries with automated data imports and incremental processes (this countries are found here). For this case, you can create a PR modifying the corresponding file in output folder. Create the pull request only if the daily update already ran but did not update the corresponding country.

You can of course, and we appreciate it very much, create pull requests for other cases.

Note that files in public folder are not to be manually modified.