Implement ros2 agnostic storage API #4

Karsten1987 · 2018-07-12T00:23:44Z

The storage API shall be ROS 2 agnostic and only handles the file access to read and write serialized binary code. It shall be able to describe the binary data with enough meta information to restore a complete message out of it.

Todos:

implement abstract storage API interface: see Design abstract storage interface #8
factory loading the storage plugin to work with: see initial version of plugin based storage api #7
feature a SQLite3 plugin: see Implement SQLite3 storage plugin #9
feature a ROS 1 bag plugin: see Implement ROS 1 bag storage plugin #10

- pass a few strings as const reference - throw error when no topics could be found

* GH-52 Extend db schema to include topic meta data - Two table db layout (messages and topics) - Messages table references topics table but without foreign key for improved write performance - Create_topic must be called for every topic prior to storing a message of this topic. - Sqlite_storage caches all known topics - At least for now the type information is stored as a simple string. * GH-54 Make first rcl subscription prototype work * GH-54 find type name from topic * GH-54 Publish messages from database knowing only topic name and pass topic name by terminal * GH-54 Refactoring of typesupport helpers * GH-54 Use c++ typesupport * GH-54 Use cpp typesupport and rclcpp::Node for publisher * GH-54 Add raw subscription and use in rosbag_record * GH-54 Add Rosbag2Node and Rosbag2Publisher classes and use them in Rosbag2::play * GH-54 Rename Rosbag2Publisher to RawPublisher * GH-54 Minor refactoring of Rosbag2Node * GH-54 Extract and test waiting for topic into its own method * GH-54 Fix read integration tests and linters * GH-55 Refactor Rosbag2Node::create_raw_publisher() * GH-54 Add subscription method to rosbag node * GH-54 Keep subscription alive * GH-54: Extract subscription to correct class * GH-55 Change interface of raw_publisher to match subscriber * GH-54 Add test for rosbag node * GH-54 Unfriend rclcpp class * GH-54 Make test more robust * GH-54 Fix build * GH-54 Minor cleanup and documentation * GH-55 Minor refactoring + TODO comment * GH-54 Change dynamic library folder on Windows * GH-54 Fix build * GH-54 Add shutdown to test * GH-55 Add test helpers methods for usage in multiple tests * GH-55 Add new method to read all topics and types in BaseReadInterface and use it in Rosbag2::play * GH-55 Fix gcc and msvc * GH-54 Rename raw to generic in publisher/subscriber * GH-55 Check that topic and associated type in bag file are well defined before playing back messages * GH-54 Prevent unnecessary error message loading storage * GH-54 Fix memory leak * GH-54 stabilize node test * GH-55 Check if database exists when opening storage with READ_ONLY flag * GH-54 Minor cleanup of subscriber * GH-54 Wait a small amount of time to let node discover other nodes * Add logging to false case * GH-54 Catch exceptions and exit cleanly * Use rmw_serialized_message_t and rcutils_char_array_t consistently * GH-4 Refactoring for correctness - pass a few strings as const reference - throw error when no topics could be found * Improve error messages when loading plugins * alphabetical order * type_id -> type

* final reader class Signed-off-by: Karsten Knese <karsten@openrobotics.org> * adaptations for rosbag2_transport Signed-off-by: Karsten Knese <karsten@openrobotics.org> * address review comments Signed-off-by: Karsten Knese <karsten@openrobotics.org>

* Introduce new SequentialReader interface Signed-off-by: Anas Abou Allaban <allabana@amazon.com> Introduce new SequentialReader interface Signed-off-by: Anas Abou Allaban <allabana@amazon.com> Introduce new SequentialReader interface Signed-off-by: Anas Abou Allaban <allabana@amazon.com> Introduce new SequentialReader interface Signed-off-by: Anas Abou Allaban <allabana@amazon.com> * Address commments: - Rewrite history - Move sequential reader implementation to header/source - Change namespaces - Linting Signed-off-by: Anas Abou Allaban <allabana@amazon.com> * Add visiblity control header Signed-off-by: Anas Abou Allaban <allabana@amazon.com> * Address structural review feedback Signed-off-by: Prajakta Gokhale <prajaktg@amazon.com> * Remove extraneous newline Signed-off-by: Prajakta Gokhale <prajaktg@amazon.com> * Add new BaseReaderInterface * Add new reader interface * Use the interface in sequential reader Signed-off-by: Prajakta Gokhale <prajaktg@amazon.com> * Remove extra newline Signed-off-by: Prajakta Gokhale <prajaktg@amazon.com> * Final reader class implementation (#4) * final reader class Signed-off-by: Karsten Knese <karsten@openrobotics.org> * adaptations for rosbag2_transport Signed-off-by: Karsten Knese <karsten@openrobotics.org> * address review comments Signed-off-by: Karsten Knese <karsten@openrobotics.org> * Make BaseReaderInterface public Signed-off-by: Prajakta Gokhale <prajaktg@amazon.com> * Rebase on writer changes Signed-off-by: Karsten Knese <karsten@openrobotics.org> * Introduce new SequentialReader interface Signed-off-by: Anas Abou Allaban <allabana@amazon.com> Introduce new SequentialReader interface Signed-off-by: Anas Abou Allaban <allabana@amazon.com> Introduce new SequentialReader interface Signed-off-by: Anas Abou Allaban <allabana@amazon.com> Introduce new SequentialReader interface Signed-off-by: Anas Abou Allaban <allabana@amazon.com> * Address commments: - Rewrite history - Move sequential reader implementation to header/source - Change namespaces - Linting Signed-off-by: Anas Abou Allaban <allabana@amazon.com> * Final reader class implementation (#4) * final reader class Signed-off-by: Karsten Knese <karsten@openrobotics.org> * adaptations for rosbag2_transport Signed-off-by: Karsten Knese <karsten@openrobotics.org> * address review comments Signed-off-by: Karsten Knese <karsten@openrobotics.org> * rebase Signed-off-by: Karsten Knese <karsten@openrobotics.org> * structurial changes for rosbag2 Signed-off-by: Karsten Knese <karsten@openrobotics.org> * rosbag2_transport adaptations Signed-off-by: Karsten Knese <karsten@openrobotics.org> * fixes for rebasing Signed-off-by: Karsten Knese <karsten@openrobotics.org> * pragma for windows Signed-off-by: Karsten Knese <karsten@openrobotics.org> * remove unused file Signed-off-by: Karsten Knese <karsten@openrobotics.org>

* Introduce new SequentialReader interface Signed-off-by: Anas Abou Allaban <allabana@amazon.com> Introduce new SequentialReader interface Signed-off-by: Anas Abou Allaban <allabana@amazon.com> Introduce new SequentialReader interface Signed-off-by: Anas Abou Allaban <allabana@amazon.com> Introduce new SequentialReader interface Signed-off-by: Anas Abou Allaban <allabana@amazon.com> * Address commments: - Rewrite history - Move sequential reader implementation to header/source - Change namespaces - Linting Signed-off-by: Anas Abou Allaban <allabana@amazon.com> * Add visiblity control header Signed-off-by: Anas Abou Allaban <allabana@amazon.com> * Address structural review feedback Signed-off-by: Prajakta Gokhale <prajaktg@amazon.com> * Remove extraneous newline Signed-off-by: Prajakta Gokhale <prajaktg@amazon.com> * Add new BaseReaderInterface * Add new reader interface * Use the interface in sequential reader Signed-off-by: Prajakta Gokhale <prajaktg@amazon.com> * Remove extra newline Signed-off-by: Prajakta Gokhale <prajaktg@amazon.com> * Final reader class implementation (#4) * final reader class Signed-off-by: Karsten Knese <karsten@openrobotics.org> * adaptations for rosbag2_transport Signed-off-by: Karsten Knese <karsten@openrobotics.org> * address review comments Signed-off-by: Karsten Knese <karsten@openrobotics.org> * Make BaseReaderInterface public Signed-off-by: Prajakta Gokhale <prajaktg@amazon.com> * Rebase on writer changes Signed-off-by: Karsten Knese <karsten@openrobotics.org> * Introduce new SequentialReader interface Signed-off-by: Anas Abou Allaban <allabana@amazon.com> Introduce new SequentialReader interface Signed-off-by: Anas Abou Allaban <allabana@amazon.com> Introduce new SequentialReader interface Signed-off-by: Anas Abou Allaban <allabana@amazon.com> Introduce new SequentialReader interface Signed-off-by: Anas Abou Allaban <allabana@amazon.com> * Address commments: - Rewrite history - Move sequential reader implementation to header/source - Change namespaces - Linting Signed-off-by: Anas Abou Allaban <allabana@amazon.com> * Final reader class implementation (#4) * final reader class Signed-off-by: Karsten Knese <karsten@openrobotics.org> * adaptations for rosbag2_transport Signed-off-by: Karsten Knese <karsten@openrobotics.org> * address review comments Signed-off-by: Karsten Knese <karsten@openrobotics.org> * rebase Signed-off-by: Karsten Knese <karsten@openrobotics.org> * structurial changes for rosbag2 Signed-off-by: Karsten Knese <karsten@openrobotics.org> * rosbag2_transport adaptations Signed-off-by: Karsten Knese <karsten@openrobotics.org> * fixes for rebasing Signed-off-by: Karsten Knese <karsten@openrobotics.org> * pragma for windows Signed-off-by: Karsten Knese <karsten@openrobotics.org> * remove unused file Signed-off-by: Karsten Knese <karsten@openrobotics.org> * multifile sequential reader Signed-off-by: Karsten Knese <karsten@openrobotics.org>

* ci: update ci to use jammy image Signed-off-by: wep21 <border_goldenmarket@yahoo.co.jp> * ci: add workaround for cppcheck Signed-off-by: wep21 <border_goldenmarket@yahoo.co.jp>

* ci: update ci to use jammy image Signed-off-by: wep21 <border_goldenmarket@yahoo.co.jp> * ci: add workaround for cppcheck Signed-off-by: wep21 <border_goldenmarket@yahoo.co.jp> Signed-off-by: James Smith <james@foxglove.dev>

Signed-off-by: Emerson Knapp <emerson.b.knapp@gmail.com>

dirk-thomas mentioned this issue Jul 12, 2018

rosbag ros2/ros2#533

Closed

4 tasks

Martin-Idel-SI referenced this issue in bosch-io/rosbag2 Sep 4, 2018

GH-4 Refactoring for correctness

e838700

- pass a few strings as const reference - throw error when no topics could be found

Karsten1987 closed this as completed Dec 18, 2018

emersonknapp added a commit that referenced this issue May 14, 2024

Add GITHUB_TOKEN so that CI Action can check out the repo (#4)

93b7494

Signed-off-by: Emerson Knapp <emerson.b.knapp@gmail.com>

userguide-galaxea mentioned this issue Nov 18, 2024

ros2 bag record -a will coredump my program(c++) #1859

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Implement ros2 agnostic storage API #4

Implement ros2 agnostic storage API #4

Karsten1987 commented Jul 12, 2018 •

edited

Loading

Implement ros2 agnostic storage API #4

Implement ros2 agnostic storage API #4

Comments

Karsten1987 commented Jul 12, 2018 • edited Loading

Karsten1987 commented Jul 12, 2018 •

edited

Loading