Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Stop warning about duplicate record IDs & missing columns that don't matter #2054

Merged
merged 7 commits into from
Nov 11, 2022

Conversation

zaneselvans
Copy link
Member

@zaneselvans zaneselvans commented Nov 11, 2022

No description provided.

FERC is not always consistent in its naming of corresponding columns
that appear in both the instant & duration tables. This commit adds
a renaming step to the default FERC1 process_instant_xbrl() and
process_duration_xbrl() methods, that rename them early in the data
transformations. This transform is parameterized using the same
RenameColumns parameter and rename_columns() methods that we're using
elsewhere.
@zaneselvans zaneselvans added ferc1 Anything having to do with FERC Form 1 metadata Anything having to do with the content, formatting, or storage of metadata. Mostly datapackages. xbrl Related to the FERC XBRL transition dbf Data coming from FERC's old Visual FoxPro DBF database file format. labels Nov 11, 2022
@zaneselvans zaneselvans self-assigned this Nov 11, 2022
@codecov
Copy link

codecov bot commented Nov 11, 2022

Codecov Report

Base: 84.2% // Head: 84.2% // Increases project coverage by +0.0% 🎉

Coverage data is based on head (27616b8) compared to base (c6c4866).
Patch coverage: 100.0% of modified lines in pull request are covered.

Additional details and impacted files
@@               Coverage Diff                @@
##           xbrl_integration   #2054   +/-   ##
================================================
  Coverage              84.2%   84.2%           
================================================
  Files                    72      72           
  Lines                  7972    7980    +8     
================================================
+ Hits                   6717    6724    +7     
- Misses                 1255    1256    +1     
Impacted Files Coverage Δ
src/pudl/transform/params/ferc1.py 100.0% <ø> (ø)
src/pudl/transform/classes.py 94.5% <100.0%> (-0.4%) ⬇️
src/pudl/transform/ferc1.py 81.7% <100.0%> (+0.5%) ⬆️

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

☔ View full report at Codecov.
📢 Do you have feedback about the report comment? Let us know in this issue.

@zaneselvans zaneselvans changed the base branch from plant-in-service-xbrl to xbrl_integration November 11, 2022 01:02
@zaneselvans zaneselvans linked an issue Nov 11, 2022 that may be closed by this pull request
14 tasks
@cmgosnell cmgosnell merged commit 33902f8 into xbrl_integration Nov 11, 2022
@zaneselvans zaneselvans changed the title Add aggregation metadata for plant_in_service_ferc1 table Stop warning about duplicate record IDs & missing columns that don't matter Nov 12, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
dbf Data coming from FERC's old Visual FoxPro DBF database file format. ferc1 Anything having to do with FERC Form 1 metadata Anything having to do with the content, formatting, or storage of metadata. Mostly datapackages. xbrl Related to the FERC XBRL transition
Projects
No open projects
Status: 🏗 In progress
Development

Successfully merging this pull request may close these issues.

Transform plant_in_srvce xbrl + dbf
2 participants