Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remove Tag from Exposures #756

Merged
merged 25 commits into from
Aug 31, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
25 commits
Select commit Hold shift + click to select a range
c0bfb13
exposures.base: remove tag from Exposures
emanuel-schmid Jul 13, 2023
a83c98c
doc: remove tag from Exposures tutorial
emanuel-schmid Jul 13, 2023
e2fa3f4
Entity: no more __str__ overriding
emanuel-schmid Jul 13, 2023
a240c68
remove tag from Exposures
emanuel-schmid Jul 14, 2023
8c578a9
remove dead code
emanuel-schmid Jul 14, 2023
474ef1e
cosmetics
emanuel-schmid Jul 14, 2023
7b0a770
LitPop remove tag, replaced by novel _metadata attribute `description`
emanuel-schmid Jul 14, 2023
5d94a2c
Merge remote-tracking branch 'origin/develop' into feature/remove_tag…
emanuel-schmid Jul 14, 2023
6e2ac62
PEP8
emanuel-schmid Jul 14, 2023
6d4f4e0
reaction to out of the blue failing of readthedocs to build
emanuel-schmid Jul 14, 2023
96537e1
Merge branch 'develop' into feature/remove_tag_exposures
emanuel-schmid Aug 4, 2023
37949bf
adjust readthedocs environment
emanuel-schmid Aug 4, 2023
57463c2
add output secction to prevent readthedocs from buildiing
emanuel-schmid Aug 5, 2023
bded4bb
changelog updated
emanuel-schmid Aug 7, 2023
6add533
chanchangelog updated
emanuel-schmid Aug 7, 2023
a67c2ac
Merge branch 'develop' into feature/remove_tag_exposures
emanuel-schmid Aug 9, 2023
ff75e23
Make description an attribute of Exposures, not just LitPop
emanuel-schmid Aug 11, 2023
1e22bc6
Merge branch 'develop' into feature/remove_tag_exposures
emanuel-schmid Aug 11, 2023
3c6bd74
Merge branch 'develop' into feature/remove_tag_exposures
emanuel-schmid Aug 29, 2023
08a671c
exposures.plots: skip standard title formatting
emanuel-schmid Aug 30, 2023
314948d
forcast: revert dead code removal
emanuel-schmid Aug 30, 2023
6788292
Merge branch 'develop' into feature/remove_tag_exposures
emanuel-schmid Aug 30, 2023
3953502
exposures.base.plots: title must not be None
emanuel-schmid Aug 30, 2023
49de034
remove remaining traces of Exposures.tag
emanuel-schmid Aug 30, 2023
8ce8cba
remove remaining traces of Exposures.tag
emanuel-schmid Aug 30, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,7 @@ Removed:
- Added method `Exposures.centroids_total_value` to replace the functionality of `Exposures.affected_total_value`. This method is temporary and deprecated. [#702](https://github.com/CLIMADA-project/climada_python/pull/702)
- New method `climada.util.api_client.Client.purge_cache`: utility function to remove outdated files from the local file system to free disk space.
([#737](https://github.com/CLIMADA-project/climada_python/pull/737))
- New attribute `climada.entity.exposures.Exposures.description`: used for setting the default title in plots from plotting mathods `plot_hexbin` and `plot_scatter`. [#756](https://github.com/CLIMADA-project/climada_python/pull/756)
- Added advanced examples in unsequa tutorial for coupled input variables and for handling efficiently the loading of multiple large files [#766](https://github.com/CLIMADA-project/climada_python/pull/766)

### Changed
Expand Down Expand Up @@ -58,6 +59,7 @@ Removed:
- `list_dataset_infos` from `climada.util.api_client.Client`: the `properties` argument, a `dict`, can now have `None` as values. Before, only strings and lists of strings were allowed. Setting a particular property to `None` triggers a search for datasets where this property is not assigned. [#752](https://github.com/CLIMADA-project/climada_python/pull/752)
- Reduce memory requirements of `TropCyclone.from_tracks` [#749](https://github.com/CLIMADA-project/climada_python/pull/749)
- Support for different wind speed and pressure units in `TCTracks` when running `TropCyclone.from_tracks` [#749](https://github.com/CLIMADA-project/climada_python/pull/749)
- The title of plots created by the `Exposures` methods `plot_hexbin` and `plot_scatter` can be set as a method argument. [#756](https://github.com/CLIMADA-project/climada_python/pull/756)
- Changed the parallel package from Pathos to Multiproess in the unsequa module [#763](https://github.com/CLIMADA-project/climada_python/pull/763)
- Updated installation instructions to use conda for core and petals [#776](https://github.com/CLIMADA-project/climada_python/pull/776)

Expand All @@ -83,6 +85,7 @@ Removed:
- `Centroids.set_raster_from_pix_bounds` [#721](https://github.com/CLIMADA-project/climada_python/pull/721)
- `requirements/env_developer.yml` environment specs. Use 'extra' requirements when installing the Python package instead [#712](https://github.com/CLIMADA-project/climada_python/pull/712)
- `Impact.tag` attribute. This change is not backwards-compatible with respect to the files written and read by the `Impact` class [#743](https://github.com/CLIMADA-project/climada_python/pull/743)
- `Exposures.tag` attribute. This change is not backwards-compatible with respect to the files written and read by the `Exposures` class [#756](https://github.com/CLIMADA-project/climada_python/pull/756)
- `Hazard.tag` attribute. This change is not backwards-compatible with respect to the files written and read by the `Hazard` class [#767](https://github.com/CLIMADA-project/climada_python/pull/767)
- `impact.tot_value ` attribute removed from unsequa module [#763](https://github.com/CLIMADA-project/climada_python/pull/763)

Expand Down
13 changes: 3 additions & 10 deletions climada/entity/entity_def.py
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,6 @@
from typing import Optional
import pandas as pd

from climada.util.tag import Tag
from climada.entity.impact_funcs.impact_func_set import ImpactFuncSet
from climada.entity.disc_rates.base import DiscRates
from climada.entity.measures.measure_set import MeasureSet
Expand Down Expand Up @@ -106,7 +105,7 @@ def read_mat(self, *args, **kwargs):
self.__dict__ = Entity.from_mat(*args, **kwargs).__dict__

@classmethod
def from_excel(cls, file_name, description=''):
def from_excel(cls, file_name):
"""Read csv or xls or xlsx file following climada's template.

Parameters
Expand All @@ -125,15 +124,14 @@ def from_excel(cls, file_name, description=''):
"""

exp = Exposures(pd.read_excel(file_name))
exp.tag = Tag(file_name=file_name, description=description)

dr = DiscRates.from_excel(file_name)
disc_rates = DiscRates.from_excel(file_name)
impf_set = ImpactFuncSet.from_excel(file_name)
meas_set = MeasureSet.from_excel(file_name)

return cls(
exposures=exp,
disc_rates=dr,
disc_rates=disc_rates,
impact_func_set=impf_set,
measure_set=meas_set,
)
Expand Down Expand Up @@ -178,8 +176,3 @@ def __setattr__(self, name, value):
if not isinstance(value, DiscRates):
raise ValueError("Input value is not (sub)class of DiscRates.")
super().__setattr__(name, value)

def __str__(self):
return 'Exposures: \n' + self.exposures.tag.__str__()

__repr__ = __str__
65 changes: 40 additions & 25 deletions climada/entity/exposures/base.py
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,6 @@
import cartopy.crs as ccrs

from climada.hazard import Hazard
from climada.util.tag import Tag
import climada.util.hdf5_handler as u_hdf5
from climada.util.constants import ONE_LAT_KM, DEF_CRS, CMAP_RASTER
import climada.util.coordinates as u_coord
Expand Down Expand Up @@ -84,8 +83,8 @@ class Exposures():

Attributes
----------
tag : climada.util.tag.Tag
metada - information about the source data
description : str
metadata - description of content and origin of the data
ref_year : int
metada - reference year
value_unit : str
Expand Down Expand Up @@ -121,7 +120,7 @@ class Exposures():
TC. There might be different hazards defined: centr_TC, centr_FL, ...
Computed in method assign_centroids().
"""
_metadata = ['tag', 'ref_year', 'value_unit', 'meta', 'description']
_metadata = ['description', 'ref_year', 'value_unit', 'meta']

vars_oblig = ['value', 'latitude', 'longitude']
"""Name of the variables needed to compute the impact."""
Expand All @@ -142,7 +141,7 @@ def crs(self):
# In case of gdf without geometry, empty or before set_geometry_points was called
return self.meta.get('crs')

def __init__(self, *args, meta=None, tag=None, ref_year=DEF_REF_YEAR,
def __init__(self, *args, meta=None, description=None, ref_year=DEF_REF_YEAR,
value_unit=DEF_VALUE_UNIT, crs=None, **kwargs):
"""Creates an Exposures object from a GeoDataFrame

Expand All @@ -154,8 +153,8 @@ def __init__(self, *args, meta=None, tag=None, ref_year=DEF_REF_YEAR,
Named arguments of the GeoDataFrame constructor, additionally
meta : dict, optional
Metadata dictionary. Default: {} (empty dictionary)
tag : climada.entity.exposures.tag.Tag, optional
Exposures tag. Defaults to the entry of the same name in `meta` or an empty Tag object.
description : str, optional
Default: None
ref_year : int, optional
Reference Year. Defaults to the entry of the same name in `meta` or 2018.
value_unit : str, optional
Expand All @@ -168,11 +167,10 @@ def __init__(self, *args, meta=None, tag=None, ref_year=DEF_REF_YEAR,
self.meta = {} if meta is None else meta
if not isinstance(self.meta, dict):
raise ValueError("meta must be a dictionary")
self.tag = self.meta.get('tag', Tag()) if tag is None else tag
self.description = self.meta.get('description') if description is None else description
self.ref_year = self.meta.get('ref_year', DEF_REF_YEAR) if ref_year is None else ref_year
self.value_unit = (self.meta.get('value_unit', DEF_VALUE_UNIT)
if value_unit is None else value_unit)
self.description = kwargs.pop('description') if 'description' in kwargs else None

# remaining generic attributes from derived classes
for mda in type(self)._metadata:
Expand Down Expand Up @@ -501,7 +499,6 @@ def from_raster(cls, file_name, band=1, src_crs=None, window=None,
Exposures
"""
exp = cls()
exp.tag = Tag(file_name=file_name)
meta, value = u_coord.read_raster(file_name, [band], src_crs, window,
geometry, dst_crs, transform, width,
height, resampling)
Expand All @@ -521,7 +518,7 @@ def from_raster(cls, file_name, band=1, src_crs=None, window=None,

def plot_scatter(self, mask=None, ignore_zero=False, pop_name=True,
buffer=0.0, extend='neither', axis=None, figsize=(9, 13),
adapt_fontsize=True, **kwargs):
adapt_fontsize=True, title=None, **kwargs):
"""Plot exposures geometry's value sum scattered over Earth's map.
The plot will we projected according to the current crs.

Expand All @@ -546,17 +543,19 @@ def plot_scatter(self, mask=None, ignore_zero=False, pop_name=True,
adapt_fontsize : bool, optional
If set to true, the size of the fonts will be adapted to the size of the figure.
Otherwise the default matplotlib font size is used. Default is True.
title : str, optional
a title for the plot. If not set `self.description` is used.
kwargs : optional
arguments for scatter matplotlib function, e.g.
cmap='Greys'. Default: 'Wistia'
cmap='Greys'

Returns
-------
cartopy.mpl.geoaxes.GeoAxesSubplot
"""
crs_epsg, _ = u_plot.get_transformation(self.crs)
title = "\n".join(self.tag.description)
cbar_label = f'Value ({self.value_unit})'
if title is None:
title = self.description or ""
if mask is None:
mask = np.ones((self.gdf.shape[0],), dtype=bool)
if ignore_zero:
Expand All @@ -566,8 +565,13 @@ def plot_scatter(self, mask=None, ignore_zero=False, pop_name=True,
value = self.gdf.value[mask][pos_vals].values
coord = np.stack([self.gdf.latitude[mask][pos_vals].values,
self.gdf.longitude[mask][pos_vals].values], axis=1)
return u_plot.geo_scatter_from_array(value, coord, cbar_label, title,
pop_name, buffer, extend,
return u_plot.geo_scatter_from_array(array_sub=value,
geo_coord=coord,
var_name=f'Value ({self.value_unit})',
title=title,
pop_name=pop_name,
buffer=buffer,
extend=extend,
proj=crs_epsg,
axes=axis,
figsize=figsize,
Expand All @@ -576,7 +580,7 @@ def plot_scatter(self, mask=None, ignore_zero=False, pop_name=True,

def plot_hexbin(self, mask=None, ignore_zero=False, pop_name=True,
buffer=0.0, extend='neither', axis=None, figsize=(9, 13),
adapt_fontsize=True, **kwargs):
adapt_fontsize=True, title=None, **kwargs):
"""Plot exposures geometry's value sum binned over Earth's map.
An other function for the bins can be set through the key reduce_C_function.
The plot will we projected according to the current crs.
Expand Down Expand Up @@ -605,6 +609,8 @@ def plot_hexbin(self, mask=None, ignore_zero=False, pop_name=True,
If set to true, the size of the fonts will be adapted to the size of the figure.
Otherwise the default matplotlib font size is used.
Default is True.
title : str, optional
a title for the plot. If not set `self.description` is used.
kwargs : optional
arguments for hexbin matplotlib function, e.g.
`reduce_C_function=np.average`.
Expand All @@ -615,8 +621,8 @@ def plot_hexbin(self, mask=None, ignore_zero=False, pop_name=True,
cartopy.mpl.geoaxes.GeoAxesSubplot
"""
crs_epsg, _ = u_plot.get_transformation(self.crs)
title = "\n".join(self.tag.description)
cbar_label = f'Value ({self.value_unit})'
if title is None:
title = self.description or ""
if 'reduce_C_function' not in kwargs:
kwargs['reduce_C_function'] = np.sum
if mask is None:
Expand All @@ -628,9 +634,17 @@ def plot_hexbin(self, mask=None, ignore_zero=False, pop_name=True,
value = self.gdf.value[mask][pos_vals].values
coord = np.stack([self.gdf.latitude[mask][pos_vals].values,
self.gdf.longitude[mask][pos_vals].values], axis=1)
return u_plot.geo_bin_from_array(value, coord, cbar_label, title,
pop_name, buffer, extend, proj=crs_epsg,
axes=axis, figsize=figsize, adapt_fontsize=adapt_fontsize,
return u_plot.geo_bin_from_array(array_sub=value,
geo_coord=coord,
var_name=f'Value ({self.value_unit})',
title=title,
pop_name=pop_name,
buffer=buffer,
extend=extend,
proj=crs_epsg,
axes=axis,
figsize=figsize,
adapt_fontsize=adapt_fontsize,
**kwargs)

def plot_raster(self, res=None, raster_res=None, save_tiff=None,
Expand Down Expand Up @@ -842,6 +856,9 @@ def from_hdf5(cls, file_name):
for key, val in metadata.items():
if key in type(exp)._metadata: # pylint: disable=protected-access
setattr(exp, key, val)
if key == 'tag': # for backwards compatitbility with climada <= 3.x
descriptions = [u_hdf5.to_string(x) for x in getattr(val, 'description', [])]
exp.description = "\n".join(descriptions) if descriptions else None
return exp

def read_mat(self, *args, **kwargs):
Expand Down Expand Up @@ -1158,7 +1175,7 @@ def add_sea(exposures, sea_res, scheduler=None):
ref_year=exposures.ref_year,
value_unit=exposures.value_unit,
meta=exposures.meta,
tag=exposures.tag
description=exposures.description,
)


Expand Down Expand Up @@ -1217,5 +1234,3 @@ def _read_mat_metadata(exposures, data, file_name, var_names):
file_name, data[var_names['var_name']['uni']][0][0])
except KeyError:
exposures.value_unit = DEF_VALUE_UNIT

exposures.tag = Tag(file_name)
47 changes: 23 additions & 24 deletions climada/entity/exposures/litpop/litpop.py
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,6 @@
import climada.util.coordinates as u_coord
import climada.util.finance as u_fin

from climada.util.tag import Tag
from climada.entity.exposures.litpop import nightlight as nl_util
from climada.entity.exposures.litpop import gpw_population as pop_util
from climada.entity.exposures.base import Exposures, INDICATOR_IMPF, DEF_REF_YEAR
Expand Down Expand Up @@ -81,7 +80,7 @@ def from_countries(cls, countries, res_arcsec=30, exponents=(1,1),
data_dir=SYSTEM_DIR):
"""Init new LitPop exposure object for a list of countries (admin 0).

Sets attributes `ref_year`, `tag`, `crs`, `value`, `geometry`, `meta`,
Sets attributes `ref_year`, `crs`, `value`, `geometry`, `meta`,
`value_unit`, `exponents`,`fin_mode`, `gpw_version`, and `admin1_calc`.

Parameters
Expand Down Expand Up @@ -183,19 +182,19 @@ def from_countries(cls, countries, res_arcsec=30, exponents=(1,1),
LOGGER.warning('Some countries could not be identified and are ignored: '
'%s. Litpop only initiated for: %s', countries_out, countries_in)

tag = Tag(description=f'LitPop Exposure for {countries_in} at {res_arcsec} as, '
f'year: {reference_year}, financial mode: {fin_mode}, '
f'exp: {exponents}, admin1_calc: {admin1_calc}')
description = (f'LitPop Exposure for {countries_in} at {res_arcsec} as,'
f' year: {reference_year}, financial mode: {fin_mode},'
f' exp: {exponents}, admin1_calc: {admin1_calc}')

exp = cls(
data=Exposures.concat(litpop_list).gdf,
crs=litpop_list[0].crs,
ref_year=reference_year,
tag=tag,
value_unit=get_value_unit(fin_mode),
exponents = exponents,
gpw_version = gpw_version,
fin_mode = fin_mode,
exponents=exponents,
gpw_version=gpw_version,
fin_mode=fin_mode,
description=description
)

try:
Expand Down Expand Up @@ -429,9 +428,9 @@ def from_shape_and_countries(cls, shape, countries, res_arcsec=30, exponents=(1,
else:
raise NotImplementedError('Not implemented for `shape` of type {type(shape)}')

exp.tag.append(Tag(description=f'LitPop Exposure for custom shape in {countries} at '
f'{res_arcsec} as, year: {reference_year}, financial mode: '
f'{fin_mode}, exp: {exponents}, admin1_calc: {admin1_calc}'))
exp.description = (f'LitPop Exposure for custom shape in {countries} at'
f' {res_arcsec} as, year: {reference_year}, financial mode:'
f' {fin_mode}, exp: {exponents}, admin1_calc: {admin1_calc}')
exp.set_gdf(gdf.reset_index())

try:
Expand Down Expand Up @@ -473,7 +472,7 @@ def from_shape(
"""init LitPop exposure object for a custom shape.
Requires user input regarding the total value to be disaggregated.

Sets attributes `ref_year`, `tag`, `crs`, `value`, `geometry`, `meta`,
Sets attributes `ref_year`, `crs`, `value`, `geometry`, `meta`,
`value_unit`, `exponents`,`fin_mode`, `gpw_version`, and `admin1_calc`.

This method can be used to initiated LitPop Exposure for sub-national
Expand Down Expand Up @@ -542,21 +541,21 @@ def from_shape(
elif total_value is not None:
raise TypeError("total_value must be int, float or None.")

tag = Tag(description = f'LitPop Exposure for custom shape at {res_arcsec} as, ' \
f'year: {reference_year}, exp: {exponents}')
description = (f'LitPop Exposure for custom shape at {res_arcsec} as,'
f' year: {reference_year}, exp: {exponents}')

litpop_gdf[INDICATOR_IMPF] = 1

exp = cls(
data=litpop_gdf,
crs=litpop_gdf.crs,
ref_year=reference_year,
tag=tag,
value_unit=value_unit,
exponents = exponents,
gpw_version = gpw_version,
fin_mode = None,
)
data=litpop_gdf,
crs=litpop_gdf.crs,
ref_year=reference_year,
value_unit=value_unit,
exponents=exponents,
gpw_version=gpw_version,
fin_mode=None,
description=description
)

if min(len(exp.gdf.latitude.unique()), len(exp.gdf.longitude.unique())) > 1:
#if exp.gdf.shape[0] > 1 and len(exp.gdf.latitude.unique()) > 1:
Expand Down
Loading