Extracting in real time metadata

[Ruisute]

Hi, just wondering if this could help me to extract sensor data while streaming ¿It is possible to use the metadata to feed a control system?

Like an FPV drone while in flight (intensive use) or RC car that just need that data for state info and not to have a functional approach (partial use)

[dnewman-gpsw]

We have thought about this, but don't yet have public solution. GPMF timing does extract information from the MP4 wrapper today, whereas livestreaming isn't using MP4. Other then a small amount of precision loss in timing, for where would you hope to extract metadata? What data stream?

[Ruisute]

I would like to extract data from the sensors integrated at the GoPro (integrated IMU: gyroscope, accelerometer...) just to avoid using an external component.

My idea is to record from a GoPro camera and stream video, sensor data to a control hardware (some espressif chip) for my FPV project.

[dnewman-gpsw]

Where would the data be extract from, technically? This is the issue this need to be determined.

[Ruisute]

From the telemetry that came with the MP4 (GoPro video metadata)

Sorry if maybe I didn't understand at all the question.

[dnewman-gpsw]

The problem is when streaming data, there is no MP4, so where to want to extract the telemetry from? There are no standards for this, something new is needed.

[Ruisute]

Do you know if there is another way?

[dnewman-gpsw]

I want to know where a developer using livestreaming would like to see the metadata stored, on a technical level. Otherwise this is too much hand waving.

[Ruisute]

And what about an non technical approach? Just to show a bunch of people how fast the pilot is going at a race, just for streaming purpose

[dnewman-gpsw]

It has to be implemented using technology. That is the problem. We have the data, just need to know which pipe to place it. Remember it will be up to a third party to extract it, so where would a third party expect to extract it? Livestreaming has a sender (GoPro can do) and a receiver (everyone else.) So the issue is on the receivers not having a standard place to store metadata.

[dnewman-gpsw]

In your original question: It is possible to use the metadata to feed a control system?
Sure, what control system, and where is it expecting to read telemetry data?

[realchrisolin]

It's looking like this ticket came from someone without a full grasp of the technical complexities at play here. As someone with a somewhat knowledgeable grasp of the issues at play (not nearly as knowledgeable as you or Daryl), I would posit that if this request is going to be taken seriously, the best solution would be to stream the data over IP/MQTT. The GPMF standard is open source, it's up to developers of relevant software to add real-time streaming support. My understanding is this functionality already exists for STANAG 4609, which is another (standardized) KLV format akin to GPMF.

https://standards.globalspec.com/std/1222789/stanag-4609
https://www.impleotv.com/content/klvstreamer/help/page_klv_in_t_s.html
https://github.com/impleotv/gstreamer-test-klv
https://www.vitec.com/fileadmin/downloads/Collateral/VITEC_MilitarySolutions__Brochure__22-E.pdf