Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature/handle nans #89

Merged
merged 41 commits into from
Aug 29, 2024
Merged

Feature/handle nans #89

merged 41 commits into from
Aug 29, 2024

Conversation

FelixMau
Copy link
Contributor

Handlling Nans is crucial for data_adapter_oemof to work.

Some data might be missing due to irrelevance this data still has to be filled since oemof.solph multi period cannot take cahnging values including None and fill with defaults. Therefore the Nan handler is necessary.

Continue work from #78

@FelixMau FelixMau marked this pull request as draft June 24, 2024 12:10
@FelixMau
Copy link
Contributor Author

FelixMau commented Jul 5, 2024

Added functionality to find and replace irrelevant nan-data

Searches for where investment is allowed

  • If no Investment is allowed and no capacity is given
    -> Data is replaced by pandas ffil/bfill

If no Investment is allowed and there have been periods with investment before ("decommissioning of investment") this may lead to issues since capacity might still be existing and values will be assigned as they were not intended. If the user wants to use specific data here they should provide it (:

Searches for decomissioned Processes

  • If capacity of a process is 0 other nan data is replaced by ffil/bfill.

Moreover moved replacement of min/max values from #78 to seperate function handle_min_max executed before replacement of find_and_replace_irrelevant_data datapoints.

More documentation has been written in code docstrings.

]
# no investment in decommissioning processes
# no capacity can be set on investment objects
if not any([x in capacity_columns for x in group_df.columns]):
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Kindly asking for opinion on if/else statement since it actually is not needed.

I added it because data where capacity is set and investment is allowed at the same time the nans cannot be handled by the developed logic so far.

@FelixMau FelixMau marked this pull request as ready for review July 5, 2024 10:26
@FelixMau FelixMau requested a review from henhuy July 5, 2024 10:26
Copy link
Contributor

@henhuy henhuy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Did not understand all - left some comments.
Thx for implementing though!

data_adapter_oemof/calculations.py Outdated Show resolved Hide resolved
data_adapter_oemof/calculations.py Outdated Show resolved Hide resolved
data_adapter_oemof/calculations.py Outdated Show resolved Hide resolved
@FelixMau
Copy link
Contributor Author

Adapted to use pd.DataFrame.where in stead to fill in nans.
I hope this approach makes my goal clearer.
Moreover added a test.

@FelixMau FelixMau merged commit 18eefcf into dev Aug 29, 2024
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants