JSON to NDJSON Parser

Parse .json to .ndjson

Simple Case

If your fetched file contains a list of json objects, like this:

[
  {
    "name": "VaccinesRUs",
    "website": "https://www.vaccines-r-us.com"
  },
  {
    "name": "Ye Olde Vaccine Shoppe",
    "website": "https://the-vaccine-shoppe.com"
  }
]

You can use a shared parser to convert to ndjson! Simply add a file called parse.yml to your runner directory (runners/<state>/<site>), and set it up using the following template:

• Update state: to be your state's two-letter abbreviation
• Update site: to be the name of your runner directory (e.g., vaccinespotter_org)
• Set parser: to json_list

---
state: al
site: arcgis
parser: json_list

That's it! Test your parser by running:

poetry run event-feed-ingest parse <state>/<site>

Complex Case

If your fetched file contains a list of JSON objects nested in other structure, like this:

[
  {
    "notice": "These locations provide covid vaccines.",
    "locations": [
      {
        "name": "VaccinesRUs",
        "website": "https://www.vaccines-r-us.com"
      },
      {
        "name": "Ye Olde Vaccine Shoppe",
        "website": "https://the-vaccine-shoppe.com"
      }
    ]
  }
]

You need to add a list of attributes to traverse to your config under path:

---
state: al
site: arcgis
parser: json_list
path:
- 0 # The 0th item of the array
- locations # The "location" field for that item

That's it! Test your parser by running:

poetry run event-feed-ingest parse <state>/<site>