Skip to content

Commit

Permalink
update readme
Browse files Browse the repository at this point in the history
  • Loading branch information
facontidavide committed Nov 30, 2023
1 parent a88bade commit 634be97
Show file tree
Hide file tree
Showing 2 changed files with 42 additions and 29 deletions.
71 changes: 42 additions & 29 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,43 +4,56 @@
[![codecov](https://codecov.io/gh/facontidavide/data_tamer/graph/badge.svg?token=D0wtsntWds)](https://codecov.io/gh/facontidavide/data_tamer)

When we talk about "logging", most of the time we refer to human-readable
messages (strings) with different severity levels (INFO, ERROR, DEBUG, etc.).
messages (strings) with different severity levels (INFO, ERROR, DEBUG, etc.), being displayed in console or saved in a file.

**DataTamer** solves a different problem: it logs/traces numerical values over time and
periodically takes "snapshots" of these values, to later visualize them as timeseries.
takes periodically "snapshots" of these values, to later visualize them as timeseries.

As such, it is a great complement of [PlotJuggler](https://github.com/facontidavide/PlotJuggler),
the timeseries visualization tool (note, you will need PlotJuggler 3.8 or later).
the timeseries visualization tool (note: you will need PlotJuggler **3.8.2** or later).

**DataTamer** is your "fearless" C++ library to log numerical data:
**DataTamer** is your "fearless" C++ library to log numerical data because:

- Track hundreds or thousands of variables: even 1 million points per second
- You can track hundreds or thousands of variables: even 1 million points per second
should have a negligible CPU overhead.
- Perfect for real-time applications: the code in the "hot" thread has very low latency, no matter how the data is saved.

Kudos to [pal_statistics](https://github.com/pal-robotics/pal_statistics), for inspiring this project.
- It can be used in real-time applications, since the code in the "hot" thread has very low latency, no matter how the data is saved.

Since all the values are aggregated in a single "snapshot", it is particularly
suited to record data in a periodic loop (a very frequent use case in robotics applications).

Kudos to [pal_statistics](https://github.com/pal-robotics/pal_statistics), for inspiring this project.

## How it works

![architecture](concepts.png)

DataTamer can be used to monitor multiple variables in your applications.

**Channels** are used to take a "snapshots of a subset of variables at a given time.
If you want to record at different frequencies, you should use different channels.

DataTamer will forward this data to 1 or multiple **Sinks**;
a sink may save the information in a file (currently, we support [MCAP](https://mcap.dev/))
or publish it using an inter-process communication (for instance, a ROS2 publishers).

You can create your own sinks, if you want.

You can use [PlotJuggler](https://github.com/facontidavide/PlotJuggler) to
visualize your logs or see it in real-time.


## Features

- **Serialization schema is created at run-time**: no need to do any code generation.
- **Serialization schema is created at run-time**: no need to do code generation.
- **Suitable for real-time applications**: very low latency (on the side of the callee).
- **Multi-sink architecture**: recorded data can be forwarded to multiple "backends".
- **Very low serialization overhead**, in the order of 1 bit per traced value.
- The user can enable/disable traced variables at run-time.

Available sinks:

- Direct [MCAP](https://mcap.dev/) recording.
- `DummySink`, mostly useful for debugging and unit testing.
- ROS2 publisher.

## Limitations

- Traced variables can not be added (registered) once the recording starts.
- Variable size vectors are supported, but only for numerical values (not complex types).
- Focused on periodic recording. Not the best option for sporadic, asynchronous events.
- If you use `DataTamer::registerValue` you must be careful about the lifetime of the
object. If you prefer a safer RAII interface, use `DataTamer::createLoggedValue` instead.
Expand All @@ -49,23 +62,19 @@ object. If you prefer a safer RAII interface, use `DataTamer::createLoggedValue`

```cpp
#include "data_tamer/data_tamer.hpp"
#include "data_tamer/sinks/dummy_sink.hpp"
#include "data_tamer/sinks/mcap_sink.hpp"

int main()
{
using namespace DataTamer;

// Start defining one or more Sinks that must be added by default.
// Do this BEFORE creating a channel.
auto dummy_sink = std::make_shared<DummySink>();
ChannelsRegistry::Global().addDefaultSink(dummy_sink);
// Multiple channels can use this sink.
// Data will be saved in mylog.mcap
auto mcap_sink = std::make_shared<MCAP>("mylog.mcap");

// Create a channel (or get an existing one) using the
// global registry (singleton-like interface)
auto channel = ChannelsRegistry::Global().getChannel("my_channel");

// If you don't want to use addDefaultSink, you can attach the sink manually:
// channel->addDataSink(dummy_sink);
// Create a channel
auto channel = LogChannel::create("my_channel");
channel->addDataSink(dummy_sink);

// You can register any arithmetic value. You are responsible for their lifetime
double value_real = 3.14;
Expand Down Expand Up @@ -93,7 +102,13 @@ int main()

# Compilation

## Using Conan
### Compiling with ROS2

Just use colcon :)

## Compiling with Conan (not ROS2 support)

Note that th ROS2 publisher will **NOT** be built when using this method.

Assuming conan 2.x installed. From the source directory.

Expand All @@ -115,6 +130,4 @@ cmake -S . -DCMAKE_BUILD_TYPE=Debug \
cmake --build build/Debug/ --parallel
```

## Using ROS2

TODO
Binary file added concepts.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.

0 comments on commit 634be97

Please sign in to comment.