Catalyst Cooperative
FERC Form 1 Meta Data #2053
-
|
Dear Catalyst team, Thanks for integrating the Ferc form 1 data. This is amazing! I was able to find all the available Ferc form 1 data from the data dictionaries (https://catalystcoop-pudl.readthedocs.io/en/latest/data_dictionaries/ferc1_db.html). However, I wasn't able to find any further documentations to explain the variables in each of the table. Since the variable names aren't super intuitive, where can I find relevant meta data so that I can have a better understanding on each variable? Thanks you very much. |
Beta Was this translation helpful? Give feedback.
Replies: 1 comment 4 replies
-
|
Sadly, there's precious little documentation of the original FERC Form 1 databases. However, we've archived PDFs of the blank forms going back a number of years, and there's some additional information on the FERC Form 1 data source page. What we usually do is use the page you referenced to figure out what pages of the PDF to look at (which is what utilities are looking at when they fill the form in, presumably) and use that to interpret the table and column names, and what one ought to find in them. We'll start publishing the new XBRL data covering many FERC forms shortly, and it will have much richer metadata! It will only directly apply to the newest data, but we're doing our best to align the old DBF and new XBRL data series, so that we can use the new metadata to annotate the old data as well. |
Beta Was this translation helpful? Give feedback.
-
|
Nice to hear that you are going to publish a new XBRL data. Do you know if that will be published in 2022? Thanks! |
Beta Was this translation helpful? Give feedback.
-
|
Yes, that's definitely the plan! I'm working on release stuff right now. You can follow along on #2077 & #1681 if you like. |
Beta Was this translation helpful? Give feedback.
-
|
Hi @yifeivioletliu! The new XBRL has been published! You can access the raw data on data.catalyst.coop. The raw data isn't well documented for the reasons @zaneselvans stated. However, we have cleaned a couple dozen FERC Form 1 tables from the dbf and xbrl data. You can access the clean tables in the pudl database. The pudl database data dictionary contains table and column level information. |
Beta Was this translation helpful? Give feedback.
-
|
Unfortunately the reshaping of the XBRL data from column to row orientation means that much of the XBRL derived metadata has yet to be associated with the new XBRL data in the PUDL DB, but this is something we're planning on fixing. See #2173 |
Beta Was this translation helpful? Give feedback.
Sadly, there's precious little documentation of the original FERC Form 1 databases. However, we've archived PDFs of the blank forms going back a number of years, and there's some additional information on the FERC Form 1 data source page.
What we usually do is use the page you referenced to figure out what pages of the PDF to look at (which is what utilities are looking at when they fill the form in, presumably) and use that to interpret the table and column names, and what one ought to find in them.
We'll start publishing the new XBRL data covering many FERC forms shortly, and it will have much richer metadata! It will only directly apply to the newest data, but we're doing our best to a…