Release 2022.11.XX #1681
Merged
Release 2022.11.XX #1681
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Codecov ReportBase: 83.9% // Head: 85.1% // Increases project coverage by
Additional details and impacted files@@ Coverage Diff @@
## main #1681 +/- ##
=======================================
+ Coverage 83.9% 85.1% +1.1%
=======================================
Files 64 72 +8
Lines 6879 8202 +1323
=======================================
+ Hits 5775 6981 +1206
- Misses 1104 1221 +117
Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here. ☔ View full report at Codecov. |
8 tasks
8 tasks
|
Check out this pull request on See visual diffs & provide feedback on Jupyter Notebooks. Powered by ReviewNB |
…the small_plants_2004-2016.xlsx spreadsheet
updates: - [github.com/pre-commit/mirrors-prettier: v3.0.0-alpha.3 → v3.0.0-alpha.4](pre-commit/mirrors-prettier@v3.0.0-alpha.3...v3.0.0-alpha.4) - [github.com/asottile/pyupgrade: v3.1.0 → v3.2.0](asottile/pyupgrade@v3.1.0...v3.2.0)
Updates the requirements on [dask](https://github.com/dask/dask) to permit the latest version. - [Release notes](https://github.com/dask/dask/releases) - [Changelog](https://github.com/dask/dask/blob/main/docs/release-procedure.md) - [Commits](dask/dask@2021.08.0...2022.10.2) --- updated-dependencies: - dependency-name: dask dependency-type: direct:production ... Signed-off-by: dependabot[bot] <support@github.com>
…te-config [pre-commit.ci] pre-commit autoupdate
…/dask-gte-2021.8-and-lt-2022.10.3 Update dask requirement from <2022.10.1,>=2021.8 to >=2021.8,<2022.10.3
Docs housekeeping
Progress on #2012 #2014 * Fixed a bug in how the DBF row numbers that need to be mapped are identified. Now it looks for any time the row_literal associated with a row number has changed from one year to the next, rather than selecting the first instance of each distinct combination of row_literal and row_number. * Also discovered that there's an obscure row_status field that differentiates between annual (A) and quarterly (Q) row literals, and is part of the f1_row_lit_tbl primary key, but it only shows up in association with the f1_schedules_list table. I integrated it but... maybe that table should just be excluded from the row mapping template? * Added some (janky) helper functions to pudl.transform.ferc1 to manage the generation of the row maps. This location is temporary. They should probably become methods of a Ferc1 abstract transformer class for reshaped tables, or maybe end up in a different module. Not sure how they'll end up getting used yet though. * Updated the dbf_to_xbrl.csv file to include all of the possible rows that could need mapping (4270 in total). * Removed the XBRL specific metadata fields from the dbf_to_xbrl.csv file, since they should (hopefully) be available programmatically from the metadata @zschira is extracting from the XBRL taxonomies, and can be joined to this table based on the xbrl_column_stem. * Updated the plant_in_service transform to use the new row map. Need to test on all of the years.
33 tasks
Updates the requirements on [dask](https://github.com/dask/dask) to permit the latest version. - [Release notes](https://github.com/dask/dask/releases) - [Changelog](https://github.com/dask/dask/blob/main/docs/release-procedure.md) - [Commits](dask/dask@2021.08.0...2022.11.1) --- updated-dependencies: - dependency-name: dask dependency-type: direct:production ... Signed-off-by: dependabot[bot] <support@github.com>
…/dask-gte-2021.8-and-lt-2022.11.2 Update dask requirement from <2022.11.1,>=2021.8 to >=2021.8,<2022.11.2
After re-running the full ETL locally, I found that the new row counts matched the ones that were being found by the nightly builds (which were different from what I was finding prior to rebuilding my DB). They were also the row counts that Christina was finding in her DB. So... hopefully they are actually correct! If they once again fail to match up, then maybe we should look at: * Are the contents of the `fuel_ferc1` or `fbp_ferc1` tables being altered somewhere in the course of the validation tests, such that they have a different number of rows when they're checked in isolation vs. when they're checked as part of the overall data validations? * Is the ETL being run with code from the commit that we think it's being run from? Or is it somehow checking out a different version than we expect?
Updates the requirements on [pandas](https://github.com/pandas-dev/pandas) to permit the latest version. - [Release notes](https://github.com/pandas-dev/pandas/releases) - [Changelog](https://github.com/pandas-dev/pandas/blob/main/RELEASE.md) - [Commits](pandas-dev/pandas@v1.4.0...v1.5.2) --- updated-dependencies: - dependency-name: pandas dependency-type: direct:production ... Signed-off-by: dependabot[bot] <support@github.com>
…-records Address missing fuel ferc1 records I'm going to merge this into `dev` without letting the CI finish since I'm going to immediately run the CI on `dev`...
|
@bendnorman @cmgosnell Your two small PRs: #2079 & #2091 seem like they could conceivably be merged in here before the release... do we want to risk that? Or not? |
…/pandas-gte-1.4-and-lt-1.5.3 Update pandas requirement from <1.5.2,>=1.4 to >=1.4,<1.5.3
Add loglevel and logfile to all CLI commands
|
Okay I've merged in @bendnorman's logging standardization, and am holding off on the 860m for the moment. Was thinking I would kick off the nightly build job. |
Add FERC XBRL DBs to datasette deployment.
Updates the requirements on [flake8](https://github.com/pycqa/flake8) to permit the latest version. - [Release notes](https://github.com/pycqa/flake8/releases) - [Commits](PyCQA/flake8@4.0.0...6.0.0) --- updated-dependencies: - dependency-name: flake8 dependency-type: direct:development ... Signed-off-by: dependabot[bot] <support@github.com>
…/flake8-gte-4.0-and-lt-6.1 Update flake8 requirement from <5.1,>=4.0 to >=4.0,<6.1
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.