Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update documentation #121

Merged
merged 6 commits into from
Mar 31, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .readthedocs.yml
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
version: 2

python:
version: 3.7
version: "3"
install:
- method: pip
path: .
Expand Down
42 changes: 17 additions & 25 deletions doc/implementation.rst
Original file line number Diff line number Diff line change
Expand Up @@ -53,38 +53,31 @@ For example, every :class:`.Code` is a ``NameableArtefact``; [1]_ this means it
- has a list of :attr:`~.AnnotableArtefact.annotations`

:class:`.IdentifiableArtefact`

- has an :attr:`id <.IdentifiableArtefact.id>`, :attr:`URI <.IdentifiableArtefact.uri>`, and :attr:`URN <.IdentifiableArtefact.urn>`.
- is “annotable”; this means it *also* has the `annotations` attribute of an AnnotableArtefact.

The ``id`` uniquely identifies the object against others of the same type in a SDMX message.
The URI and URN are *globally* unique. See `Wikipedia <https://en.wikipedia.org/wiki/Uniform_Resource_Identifier#URLs_and_URNs>`_ for a discussion of the differences between the two.

:class:`.NameableArtefact`

- has a :attr:`name <.NameableArtefact.name>` and :attr:`description <.NameableArtefact.description>`, and
- is identifiable, therefore *also* annotable.
- has a :attr:`name <.NameableArtefact.name>` and :attr:`description <.NameableArtefact.description>`, and
- is identifiable, therefore *also* annotable.

:class:`.VersionableArtefact`

- has a :attr:`version <.VersionableArtefact.version>` number,
- may be valid between certain times (:attr:`valid_from <.VersionableArtefact.valid_from>`, :attr:`valid_to <.VersionableArtefact.valid_to>`), and
- is nameable, identifiable, *and* annotable.
- has a :attr:`version <.VersionableArtefact.version>` number,
- may be valid between certain times (:attr:`valid_from <.VersionableArtefact.valid_from>`, :attr:`valid_to <.VersionableArtefact.valid_to>`), and
- is nameable, identifiable, *and* annotable.

:class:`.MaintainableArtefact`
- is under the authority of a particular :attr:`maintainer <.MaintainableArtefact.maintainer>`, and
- is versionable, nameable, identifiable, and annotable.

- is under the authority of a particular :attr:`maintainer <.MaintainableArtefact.maintainer>`, and
- is versionable, nameable, identifiable, and annotable.

In an SDMX message, a maintainable object might not be given in full; only as a reference (with :attr:`is_external_reference <.MaintainableArtefact.is_external_reference>` set to :obj:`True`).
If so, it might have a :attr:`structure_url <.MaintainableArtefact.structure_url>`, where the maintainer provides more information about the object.

In an SDMX message, a maintainable object might not be given in full; only as a reference (with :attr:`is_external_reference <.MaintainableArtefact.is_external_reference>` set to :obj:`True`).
If so, it might have a :attr:`structure_url <.MaintainableArtefact.structure_url>`, where the maintainer provides more information about the object.

The API reference for :mod:`sdmx.model` shows the parent classes for each class, to describe whether they are versionable, nameable, identifiable, and/or maintainable.

Because SDMX is used worldwide, an :class:`.InternationalString` type is used in
the IM—for instance, the `name` of a Nameable object is an
``InternationalString``, with zero or more :attr:`localizations <.InternationalString.localizations>` in different locales.
Because SDMX is used worldwide, an :class:`.InternationalString` type is used in the IM—for instance, the `name` of a Nameable object is an InternationalString, with zero or more :attr:`localizations <.InternationalString.localizations>` in different locales.

.. [1] Indirectly, through :class:`Item`.

Expand All @@ -102,15 +95,14 @@ Data
====

:class:`.Observation`
A single data point/datum.

A single data point/datum.
The value is stored as the :attr:`~.Observation.value` attribute.
The value is stored as the :attr:`~.Observation.value` attribute.

:class:`.DataSet`
A collection of Observations, SeriesKeys, and/or GroupKeys.

A collection of Observations, SeriesKeys, and/or GroupKeys.

.. note:: **There are no 'Series' or 'Group' classes** in the IM!
.. note:: **There are no 'Series' or 'Group' classes** in the IM!

Instead, the *idea* of 'data series' within a DataSet is modeled as:

Expand All @@ -137,13 +129,13 @@ Data
:class:`.SeriesKey`
The dimension(s) shared by all Observations in a conceptual series.

:class:`.GroupKey`.
:class:`.GroupKey`
The dimension(s) comprising the group.
These may be a subset of all the dimensions in the DataSet, in which case all matching Observations are considered part of the 'group'—even if they are associated with different SeriesKeys.

GroupKeys are often used to attach AttributeValues; see below.

:class:`AttributeValue`
:class:`.AttributeValue`
Value (:attr:`.AttributeValue.value`) for a DataAttribute (:attr:`.AttributeValue.value_for`).

May be attached to any of: DataSet, SeriesKey, GroupKey, or Observation.
Expand Down Expand Up @@ -243,7 +235,7 @@ SDMX-CSV

Reference: https://github.com/sdmx-twg/sdmx-csv

:mod:`sdmx` **does not** currently support SDMX-CSV.
:mod:`sdmx` **does not** currently support SDMX-CSV; see :issue:`34`, :issue:`36`.

:mod:`sdmx`:

Expand Down
3 changes: 2 additions & 1 deletion doc/whatsnew.rst
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ Next release
Migration notes
---------------

Code such as the following will emit a :class:`DeprecationWarning`:
In order to prepare for future support of SDMX 3.0, code such as the following will emit a :class:`DeprecationWarning`:

.. code-block:: python

Expand Down Expand Up @@ -44,6 +44,7 @@ All changes
- :attr:`.xml.Reader.media_types` and :attr:`.json.Reader.media_types` explicitly indicated supported media types.
- :attr:`.ItemScheme.is_partial` defaults to :data:`None`.
- Add empty/stub :mod:`.format.csv`, :mod:`.reader.csv` (cf. :issue:`34`), and :mod:`.model.v30`.
- Improve readability in :doc:`implementation` (:pull:`121`).

v2.7.1 (2023-03-09)
===================
Expand Down
2 changes: 1 addition & 1 deletion pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -49,7 +49,7 @@ dependencies = [
cache = ["requests-cache"]
docs = [
"IPython",
"sphinx >=4,<6",
"sphinx >=4",
"sphinx-book-theme",
]
tests = [
Expand Down
4 changes: 2 additions & 2 deletions sdmx/model/common.py
Original file line number Diff line number Diff line change
Expand Up @@ -60,7 +60,7 @@ def __init__(self, *, text: _TInternationalStringInit = None, **kwargs):
class AnnotableArtefact(BaseModel):
#: :class:`Annotations <.Annotation>` of the object.
#:
#: :mod:`pandaSDMX` implementation: The IM does not specify the name of this
#: :mod:`.sdmx` implementation detail: The IM does not specify the name of this
#: feature.
annotations: List[Annotation] = []

Expand Down Expand Up @@ -425,7 +425,7 @@ class ItemScheme(MaintainableArtefact, Generic[IT]):
:attr:`items` attribute, in which the keys are the :attr:`~.IdentifiableArtefact.id`
of the Item.

Because this may change in future versions of pandaSDMX, user code should not access
Because this may change in future versions, user code should not access
:attr:`items` directly. Instead, use the :func:`getattr` and indexing features of
ItemScheme, or the public methods, to access and manipulate Items:

Expand Down
4 changes: 2 additions & 2 deletions sdmx/model/v21.py
Original file line number Diff line number Diff line change
Expand Up @@ -563,8 +563,8 @@ class DimensionRelationship(AttributeRelationship):


class GroupRelationship(AttributeRelationship):
# 'Retained for compatibility reasons' in SDMX 2.1; not used by pandaSDMX.
#:
#: “Retained for compatibility reasons in SDMX 2.1 versus 2.0; not used by
#: :mod:`sdmx`.
group_key: Optional["GroupDimensionDescriptor"] = None


Expand Down
22 changes: 19 additions & 3 deletions sdmx/reader/xml.py
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,18 @@
from copy import copy
from itertools import chain, count, product
from sys import maxsize
from typing import Any, Dict, Iterable, Mapping, Optional, Type, Union, cast
from typing import (
Any,
Dict,
Iterable,
Iterator,
Mapping,
Optional,
Tuple,
Type,
Union,
cast,
)

from dateutil.parser import isoparse
from lxml import etree
Expand Down Expand Up @@ -241,8 +252,13 @@ def read_message(

try:
# Use the etree event-driven parser
for event, element in etree.iterparse( # type: ignore [attr-defined]
source, events=("start", "end")
# NB (typing) iterparse() returns tuples. For "start" and "end", the second
# item is etree._Element, but for other events, e.g. "start-ns", it is
# not. types-lxml accurately reflects this. Narrow the type here for the
# following code.
for event, element in cast(
Iterator[Tuple[str, etree._Element]],
etree.iterparse(source, events=("start", "end")),
):
try:
# Retrieve the parsing function for this element & event
Expand Down