From 2809cbe8e4a8856e272472283970ffba779b7218 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Kai=20M=C3=BChlbauer?= Date: Fri, 24 May 2024 07:56:44 +0200 Subject: [PATCH 1/5] add link to CF conventions on packed data in doc/user-guide/io.rst --- doc/user-guide/io.rst | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/doc/user-guide/io.rst b/doc/user-guide/io.rst index b73d0fdcb51..076a5b58e8c 100644 --- a/doc/user-guide/io.rst +++ b/doc/user-guide/io.rst @@ -416,7 +416,8 @@ read the data. Scaling and type conversions ............................ -These encoding options work on any version of the netCDF file format: +These encoding options (based on `CF Conventions on packed data`_) work on any +version of the netCDF file format: - ``dtype``: Any valid NumPy dtype or string convertible to a dtype, e.g., ``'int16'`` or ``'float32'``. This controls the type of the data written on disk. @@ -435,6 +436,8 @@ converting ``NaN`` to ``-9999``, we would use ``encoding={'foo': {'dtype': 'int16', 'scale_factor': 0.1, '_FillValue': -9999}}``. Compression and decompression with such discretization is extremely fast. +.. _CF Conventions on packed data: https://cfconventions.org/cf-conventions/cf-conventions.html#packed-data + .. _io.string-encoding: String encoding From 9f977591d64742cf314d509391e35a70622b0cbf Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Kai=20M=C3=BChlbauer?= Date: Fri, 24 May 2024 08:02:26 +0200 Subject: [PATCH 2/5] add whats-new.rst entry --- doc/whats-new.rst | 2 ++ 1 file changed, 2 insertions(+) diff --git a/doc/whats-new.rst b/doc/whats-new.rst index 49d39927f2b..b38d07fe49b 100644 --- a/doc/whats-new.rst +++ b/doc/whats-new.rst @@ -38,6 +38,8 @@ Bug fixes Documentation ~~~~~~~~~~~~~ +- Add link to CF Conventions on packed data in doc/user-guide/io.rst (:issue:`9041`, :pull:`9045`). + By `Kai Mühlbauer `_. Internal Changes From fd51b8323324ac42491cef0017c95fb88d0cc767 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Kai=20M=C3=BChlbauer?= Date: Fri, 7 Jun 2024 13:50:31 +0200 Subject: [PATCH 3/5] add more details to type determination --- doc/user-guide/io.rst | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/doc/user-guide/io.rst b/doc/user-guide/io.rst index 076a5b58e8c..3d6bad7a3a9 100644 --- a/doc/user-guide/io.rst +++ b/doc/user-guide/io.rst @@ -428,7 +428,8 @@ version of the netCDF file format: output file, unless explicitly disabled with an encoding ``{'_FillValue': None}``. - ``scale_factor`` and ``add_offset``: Used to convert from encoded data on disk to to the decoded data in memory, according to the formula - ``decoded = scale_factor * encoded + add_offset``. + ``decoded = scale_factor * encoded + add_offset``. Please note, that ``scale_factor`` + and ``add_offset`` must be of same type and determine the type of the decoded data. These parameters can be fruitfully combined to compress discretized data on disk. For example, to save the variable ``foo`` with a precision of 0.1 in 16-bit integers while From f394df08ea4b8244c4671f19df3c4fb8bee096ac Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Kai=20M=C3=BChlbauer?= Date: Fri, 7 Jun 2024 13:59:02 +0200 Subject: [PATCH 4/5] fix whats-new.rst --- doc/whats-new.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/whats-new.rst b/doc/whats-new.rst index 1834d7f535a..6e07caedc5a 100644 --- a/doc/whats-new.rst +++ b/doc/whats-new.rst @@ -44,7 +44,7 @@ Bug fixes Documentation ~~~~~~~~~~~~~ -- Add link to CF Conventions on packed data in doc/user-guide/io.rst (:issue:`9041`, :pull:`9045`). +- Add link to CF Conventions on packed data and sentence on type determination in doc/user-guide/io.rst (:issue:`9041`, :pull:`9045`). By `Kai Mühlbauer `_. From d852c9d797f58ef40eed8af1c45041666d5e96cc Mon Sep 17 00:00:00 2001 From: Justus Magin Date: Tue, 11 Jun 2024 10:53:39 +0200 Subject: [PATCH 5/5] remove a comma --- doc/user-guide/io.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/user-guide/io.rst b/doc/user-guide/io.rst index 3d6bad7a3a9..fd6b7708e48 100644 --- a/doc/user-guide/io.rst +++ b/doc/user-guide/io.rst @@ -428,7 +428,7 @@ version of the netCDF file format: output file, unless explicitly disabled with an encoding ``{'_FillValue': None}``. - ``scale_factor`` and ``add_offset``: Used to convert from encoded data on disk to to the decoded data in memory, according to the formula - ``decoded = scale_factor * encoded + add_offset``. Please note, that ``scale_factor`` + ``decoded = scale_factor * encoded + add_offset``. Please note that ``scale_factor`` and ``add_offset`` must be of same type and determine the type of the decoded data. These parameters can be fruitfully combined to compress discretized data on disk. For