The Freedom of Information Act (FOIA) allows constituents to gain access to the data that their government creates. Gaining access to this data comes through a process called a "FOIA request". Making a FOIA request does not guarantee that the request will be fulfilled. There are many reasons that a request may be denied - from the data being unavailable to the data falling under executive privilege.
In Delaware, every government agency is required by law to keep a FOIA log. This log is a historical record of all FOIA requests that have been made to that agency. In an effort to better understand what data was commonly being FOIA requested, Open Data Delaware made a FOIA request to as many agencies as possible for their FOIA logs. Most of these FOIA-log-FOIA-requests were made on February 9, 2019 by David Ginzberg.
As each FOIA-log-FOIA request was fulfilled, it became clear that there were major differences between state agencies in regards to how their FOIA logs are stored. Some agencies sent data as an Excel file, some as Word documents, others as scanned PDFs. Some agencies stored all of their FOIA logs in one master file while others had one file for each year. Each agency tracked different information and called that information something different.
In addition to the many flavors of FOIA log storage we received, most of the FOIA logs were extremely "dirty", meaning that there were small (or major) errors throughout them. As an example, in the FOIA logs for the Office of Management and Budget (which were among the best that we worked with), there is a field dedicated to the Division within the agency that should fulfill the request. There were often multiple spellings of the same Division. For example, one of those Divisions was the Director's Office, which was spelled in 5 different ways:
- Dir Office
- Director office
- Director's office
- Director's Office
- Directors office
In order to perform future analysis on any of the FOIA logs, any of these small errors must be accounted for and corrected.
To see the current status of the different cleaning projects, head over to the projects board. You can see the overall status of the project by heading to the Overall project board. Project boards for each agency or division are linked from there.