Semantic Segmentation messages

CMakeLists.txt

@@ -25,8 +25,10 @@ set(msg_files
   msg/Detection2D.msg
   msg/Detection3DArray.msg
   msg/Detection3D.msg
+  msg/LabelInfo.msg
   msg/ObjectHypothesis.msg
   msg/ObjectHypothesisWithPose.msg
+  msg/VisionClass.msg
   msg/Point2D.msg
   msg/Pose2D.msg
   msg/VisionInfo.msg

README.md

@@ -47,6 +47,12 @@ http://wiki.ros.org/message_filters#Policy-Based_Synchronizer_.5BROS_1.1.2B-.5D)
 in your code, as the message's header should match the header of the source
 data.
 
+Semantic segmentation pipelines should use `sensor_msgs/Image` messages for publishing segmentation and confidence masks. This allows systems to use standard ROS tools for image processing, and allows choosing the most compact image encoding appropriate for the task. To transmit the metadata associated with the vision pipeline, you should use the [`/vision_msgs/LabelInfo`](msg/LabelInfo.msg) message. This message works the same as `/sensor_msgs/CameraInfo` or [`/vision_msgs/VisionInfo`](msg/VisionInfo.msg):
+
+1.  Publish `LabelInfo` to a topic. The topic should be at same namespace level as the associated image. That is, if your image is published at `/my_segmentation_node/image`, the `LabelInfo` should be published at `/my_segmentation_node/label_info`. Use a latched publisher for `LabelInfo`, so that new nodes joining the ROS system can get the messages that were published since the beginning. In ROS2, this can be achieved using a `transient local` QoS profile.
+
+2. The subscribing node can get and store one `LabelInfo` message and cancel its subscription after that. This assumes the provider of the message publishes it periodically.
+
 ## Messages
 
   * Classification: pure classification without pose

msg/LabelInfo.msg

@@ -0,0 +1,25 @@
+# Provides meta-information about a visual pipeline.
+#
+# This message serves a similar purpose to sensor_msgs/CameraInfo, but instead
+#   of being tied to hardware, it represents information about a specific
+#   computer vision pipeline. This information stays constant (or relatively
+#   constant) over time, and so it is wasteful to send it with each individual
+#   result. By listening to these messages, subscribers will receive
+#   the context in which published vision messages are to be interpreted.
+# Each vision pipeline should publish its LabelInfo messages to its own topic,
+#   in a manner similar to CameraInfo.
+# This message is meant to allow converting data from vision pipelines that
+#   return id based classifications back to human readable string class names.
+
+# Used for sequencing
+std_msgs/Header header
+
+# An array of uint16 keys and string values containing the association
+#   between class identifiers and their names. According to the amount
+#   of classes and the datatype used to store their ids internally, the
+#   maxiumum class id allowed (65535 for uint16 and 255 for uint8) belongs to
+#   the "UNLABELED" class.
+vision_msgs/VisionClass[] class_map 
+
+# The value between 0-1 used as confidence threshold for the inference.
+float32 threshold

msg/VisionClass.msg

@@ -0,0 +1,12 @@
+# A key value pair that maps an integer class_id to a string class label
+#   in computer vision systems.
+
+# The int value that identifies the class.
+# Elements identified with 65535, the maximum uint16 value are assumed
+#   to belong to the "UNLABELED" class. For vision pipelines using less
+#   than 255 classes the "UNLABELED" is the maximum value in the uint8
+#   range.
+uint16 class_id
+
+# The name of the class represented by the class_id
+string class_name