fix clustering bug due to missing countries in n.buses

[p-glaum]

Changes proposed in this Pull Request

filter looking for empty values in n.buses.countries is not working anymore. This leads to errors later on in the cluster_network script. for 37 nodes, the error in the cluster script was that there are more combinations of subnetworks and countries than 37 and for higher cluster numbers this lead to a problem due to a division by 0.

The CI did not catch the mistake in the current master, because we only consider one country for the test CI

Checklist

• I tested my contribution locally and it seems to work fine.
• Code and workflow changes are sufficiently documented.
• Changed dependencies are added to envs/environment.yaml.
• Changes in configuration options are added in all of config.default.yaml.
• Changes in configuration options are also documented in doc/configtables/*.csv.
• A release note doc/release_notes.rst is added.

[martacki]

I just have changed this due to the new pandas version, where .isna() was not working anymore. Which pandas version are you using?

[p-glaum]

I am now also working on the newest pandas version and there isna() is working but it does not filter as I intended it to do. Is the implementation with == "na" working for you?

[p-glaum]

I think to avoid any problems with pandas, we could just do
buses.loc[:, "country"] = ""
and then later filter for
c_nan_b = buses.country == ""

[FabianHofmann]

I think to avoid any problems with pandas, we could just do buses.loc[:, "country"] = "" and then later filter for c_nan_b = buses.country == ""

I agree, but since the country column is of dtype string, we should rather track down where the nans come from and make sure we use "" instead. Then, the filter would be n.buses.country == ""

[p-glaum]

The nan values came from a previous pandas version. In the latest version, those entries are just empty. But we somehow need to filter them and neither isna() nor isnull() are working .

[p-glaum]

So basically with the newest pandas version =="na" is working as Martha did it, so I can close this PR. If pandas should decide to change again how missing values are treated, we could use something as I suggested above.

[fneum]

If it only works for a particular pandas version, this should be reflected in the environment.yaml. Ideally, the minimum version should not be the very latest pandas version. We might need a workaround that works for more versions.

[p-glaum]

The workaround could be the one I suggested. I.e. do buses.loc[:, "country"] = ""
and then later filter for c_nan_b = buses.country == ""

[FabianHofmann]

or just add a .fillna("na") before the comparison