Apply new frontend design to Container and deriving classes

docs/source/dev/design.rst

@@ -110,33 +110,23 @@ all meta-data access stores in the ``Attributable`` part of an object and follow
 (future work: use `CRTP <https://en.wikipedia.org/wiki/Curiously_recurring_template_pattern>`_)
 
 openPMD frontend classes are designed as handles that user code may copy an arbitrary amount of times, all copies sharing common resources.
-In an undergoing effort, we are moving to a PIMPL-based pattern to make this design more explicit.
+The internal shared state of such handles is stored as a single ``shared_ptr``.
 This serves to separate data from implementation, demonstrated by the class hierarchy of ``Series``:
 
 * ``SeriesData`` contains all actual data members of ``Series``, representing the resources shared between instances.
   Its copy/move constructors/assignment operators are deleted, thus pinning it at one memory location.
   It has no implementation.
-* ``SeriesInterface`` defines the interface of ``Series``.
-  Its only data member is a pointer to its associated instance of ``SeriesData``.
-  Its purpose is to serve as a parent class for any class that implements the ``Series`` interface.
-  The implementing class should deal with resource management for the instance of ``SeriesData``, ``SeriesInterface`` itself performs no resource management for the data and will assume that the pointer points at a valid instance.
-  (This PIMPL-based design allows us to avoid a similar template-heavy design based on CRTP.)
-* ``SeriesInternal`` is the class created by inheriting directly from ``SeriesData`` and ``SeriesInterface``.
-  It is intended for internal usage, pinned at a memory location just like ``SeriesData``, but carrying an implementation.
-  Its constructor and destructor define the setup and tear-down of shared resources for ``Series``.
-* ``Series`` is a wrapper around a shared pointer to ``SeriesInternal`` and also derives from ``SeriesInterface``.
-  It implements handle semantics, serving as interface to users.
-
-Other classes within our object model of the openPMD hierarchy are planned to follow in this design.
+* ``Series`` defines the interface of the class and is used by client code.
+  Its only data member is a shared pointer to its associated instance of ``SeriesData``.
+  (This pointer-based design allows us to avoid a similar template-heavy design based on CRTP.)
+* A non-owning instance of the ``Series`` class can be constructed on the fly from a ``SeriesData`` object by passing a ``shared_ptr`` with no-op destructor when constructing the ``Series``.
+  Care must be taken to not expose such an object to users as the type system does not distinguish between an owning and a non-owning ``Series`` object.
+* Frontend classes that define the openPMD group hierarchy follow this same design (except for some few that do not define data members).
+  These classes all inherit from ``Attributable``.
+  Their shared state inherits from ``AttributableData``.
+  A downside of this design is that the ``shared_ptr`` pointing to the internal state is redundantly stored for each level in the class hierarchy, ``static_cast``-ed to the corresponding level.
+  All these pointers reference the same internal object.
 
 The class hierarchy of ``Attributable`` follows a similar design, with some differences due to its nature as a mixin class that is not instantiated directly:
 
-* ``AttributableData`` serves the same purpose as ``SeriesData``.
-* ``AttributableInterface`` serves the same purpose as ``SeriesData``.
-* Equivalents to ``SeriesInternal`` and ``Series`` do not exist since a combination of ``AttributableData`` and ``AttributableInterface`` is added as a mixin to other classes.
-  As a common base class exposed to user code, ``AttributableInterface`` is aliased as ``Attributable``.
-* For classes not yet following the PIMPL-based design, ``LegacyAttributable`` wraps around a shared pointer to ``AttributableData`` and derives from ``AttributableInterface``.
-  The ``Attributable`` mixin can be added to those classes by deriving from ``LegacyAttributable``.
-* The ``Attributable`` mixin is added to ``Series`` by deriving ``SeriesData`` from ``AttributableData`` and ``SeriesInterface`` from ``AttributableInterface``.
-
 The ``Series`` class is the entry point to the openPMD-api. An instance of this class always represents an entire series (of iterations), regardless of how this series is modeled in storage or transport (e.g. as multiple files, as a stream, as variables containing multiple snapshots of a dataset or as one file containing all iterations at once).

include/openPMD/IO/IOTask.hpp

@@ -37,11 +37,11 @@
 
 namespace openPMD
 {
-class AttributableInterface;
+class Attributable;
 class Writable;
 
 Writable*
-getWritable(AttributableInterface*);
+getWritable(Attributable*);
 
 /** Type of IO operation between logical and persistent data.
  */
@@ -604,7 +604,7 @@ class OPENPMDAPI_EXPORT IOTask
     { }
 
     template< Operation op >
-    explicit IOTask(AttributableInterface* a,
+    explicit IOTask(Attributable* a,
            Parameter< op > const & p)
             : writable{getWritable(a)},
               operation{op},

include/openPMD/Iteration.hpp

@@ -32,19 +32,86 @@
 
 namespace openPMD
 {
+namespace internal
+{
+    /**
+     * @brief Whether an iteration has been closed yet.
+     *
+     */
+    enum class CloseStatus
+    {
+        ParseAccessDeferred, //!< The reader has not yet parsed this iteration
+        Open,             //!< Iteration has not been closed
+        ClosedInFrontend, /*!< Iteration has been closed, but task has not yet
+                               been propagated to the backend */
+        ClosedInBackend, /*!< Iteration has been closed and task has been
+                              propagated to the backend */
+        ClosedTemporarily /*!< Iteration has been closed internally and may
+                               be reopened later */
+    };
+
+    struct DeferredParseAccess
+    {
+        /**
+         * The group path within /data containing this iteration.
+         * Example: "1" for iteration 1, "" in variable-based iteration
+         * encoding.
+         */
+        std::string path;
+        /**
+         * The iteration index as accessed by the user in series.iterations[i]
+         */
+        uint64_t iteration = 0;
+        /**
+         * If this iteration is part of a Series with file-based layout.
+         * (Group- and variable-based parsing shares the same code logic.)
+         */
+        bool fileBased = false;
+        /**
+         * If fileBased == true, the file name (without file path) of the file
+         * containing this iteration.
+         */
+        std::string filename;
+    };
+
+    class IterationData : public AttributableData
+    {
+    public:
+       /*
+        * An iteration may be logically closed in the frontend,
+        * but not necessarily yet in the backend.
+        * Will be propagated to the backend upon next flush.
+        * Store the current status.
+        * Once an iteration has been closed, no further flushes shall be performed.
+        * If flushing a closed file, the old file may otherwise be overwritten.
+        */
+        CloseStatus m_closed = CloseStatus::Open;
+
+        /**
+         * Whether a step is currently active for this iteration.
+         * Used for file-based iteration layout, see Series.hpp for
+         * group-based layout.
+         * Access via stepStatus() method to automatically select the correct
+         * one among both flags.
+         */
+        StepStatus m_stepStatus = StepStatus::NoStep;
+
+        auxiliary::Option< DeferredParseAccess > m_deferredParseAccess{};
+    };
+}
 /** @brief  Logical compilation of data from one snapshot (e.g. a single simulation cycle).
  *
  * @see https://github.com/openPMD/openPMD-standard/blob/latest/STANDARD.md#required-attributes-for-the-basepath
  */
-class Iteration : public LegacyAttributable
+class Iteration : public Attributable
 {
     template<
             typename T,
             typename T_key,
             typename T_container
     >
     friend class Container;
-    friend class SeriesInterface;
+    friend class Series;
     friend class WriteIterations;
     friend class SeriesIterator;
 
@@ -151,42 +218,32 @@ class Iteration : public LegacyAttributable
     bool
     closedByWriter() const;
 
-    Container< Mesh > meshes;
-    Container< ParticleSpecies > particles; //particleSpecies?
+    Container< Mesh > meshes{};
+    Container< ParticleSpecies > particles{}; //particleSpecies?
 
     virtual ~Iteration() = default;
 private:
     Iteration();
 
-    struct DeferredParseAccess
+    std::shared_ptr< internal::IterationData > m_iterationData{
+        new internal::IterationData };
+
+    inline internal::IterationData const & get() const
     {
-        /**
-         * The group path within /data containing this iteration.
-         * Example: "1" for iteration 1, "" in variable-based iteration
-         * encoding.
-         */
-        std::string path;
-        /**
-         * The iteration index as accessed by the user in series.iterations[i]
-         */
-        uint64_t iteration = 0;
-        /**
-         * If this iteration is part of a Series with file-based layout.
-         * (Group- and variable-based parsing shares the same code logic.)
-         */
-        bool fileBased = false;
-        /**
-         * If fileBased == true, the file name (without file path) of the file
-         * containing this iteration.
-         */
-        std::string filename;
-    };
+        return *m_iterationData;
+    }
+
+    inline internal::IterationData & get()
+    {
+        return const_cast< internal::IterationData & >(
+            static_cast< Iteration const * >( this )->get() );
+    }
 
     void flushFileBased(std::string const&, uint64_t);
     void flushGroupBased(uint64_t);
     void flushVariableBased(uint64_t);
     void flush();
-    void deferParseAccess( DeferredParseAccess );
+    void deferParseAccess( internal::DeferredParseAccess );
     /*
      * Control flow for read(), readFileBased(), readGroupBased() and
      * read_impl():
@@ -199,7 +256,7 @@ class Iteration : public LegacyAttributable
      * allow for those different control flows.
      * Finally, read_impl() is called which contains the common parsing
      * logic for an iteration.
-     * 
+     *
      * reread() reads again an Iteration that has been previously read.
      * Calling it on an Iteration not yet parsed is an error.
      *
@@ -210,47 +267,7 @@ class Iteration : public LegacyAttributable
     void readGorVBased( std::string const & groupPath );
     void read_impl( std::string const & groupPath );
 
-    /**
-     * @brief Whether an iteration has been closed yet.
-     *
-     */
-    enum class CloseStatus
-    {
-        ParseAccessDeferred, //!< The reader has not yet parsed this iteration
-        Open,             //!< Iteration has not been closed
-        ClosedInFrontend, /*!< Iteration has been closed, but task has not yet
-                               been propagated to the backend */
-        ClosedInBackend, /*!< Iteration has been closed and task has been
-                              propagated to the backend */
-        ClosedTemporarily /*!< Iteration has been closed internally and may
-                               be reopened later */
-    };
-
-    /*
-     * An iteration may be logically closed in the frontend,
-     * but not necessarily yet in the backend.
-     * Will be propagated to the backend upon next flush.
-     * Store the current status.
-     * Once an iteration has been closed, no further flushes shall be performed.
-     * If flushing a closed file, the old file may otherwise be overwritten.
-     */
-    std::shared_ptr< CloseStatus > m_closed =
-        std::make_shared< CloseStatus >( CloseStatus::Open );
-
-    /**
-     * Whether a step is currently active for this iteration.
-     * Used for file-based iteration layout, see Series.hpp for
-     * group-based layout.
-     * Access via stepStatus() method to automatically select the correct
-     * one among both flags.
-     */
-    std::shared_ptr< StepStatus > m_stepStatus =
-        std::make_shared< StepStatus >( StepStatus::NoStep );
 
-    std::shared_ptr< auxiliary::Option< DeferredParseAccess > >
-        m_deferredParseAccess =
-            std::make_shared< auxiliary::Option< DeferredParseAccess > >(
-                auxiliary::Option< DeferredParseAccess >() );
 
     /**
      * @brief Begin an IO step on the IO file (or file-like object)
@@ -306,15 +323,15 @@ class Iteration : public LegacyAttributable
 
     /**
      * @brief Link with parent.
-     * 
+     *
      * @param w The Writable representing the parent.
      */
     virtual void linkHierarchy(Writable& w);
 
     /**
      * @brief Access an iteration in read mode that has potentially not been
      *      parsed yet.
-     * 
+     *
      */
     void runDeferredParseAccess();
 };  // Iteration