Rework Remote ID settings to new style

[DonLakeFlyer]

This is a major rework of the Remote ID SettingsUI to fit the new UI style. Currently it is available from the expanded page of the Remote ID toolbar indicator. I have left the Remote ID Settings page in App Settings alone for now. It works but is still missing things like correct validation.

The goal was also to try a little bit to make this more usable to someone (like me) who doesn't understand all the gory details of Remote ID. At this point you can consider this still be a major WIP part of my changes.

[DonLakeFlyer]

I have a pile of question on how Remote ID works. I am using a Holybro remote id module connected to Telem2 with PX4 firmware:

• RemoteIDManager:
  • Live Position updates: Is ALLOWED_GPS_DELAY set at 5 seconds some sort of remote id spec requirement? Or is it just an internally picked value for timeouts? Reason I ask is that on Mac/iOs the ability to set the interval for position service updates is not possible. And then default update rate is higher then 5 seconds. Does this mean that these setups will need an external GPS? Or can the delay be bumped up such that Mac/iOs can work from internal GPS?
  • In RemoteIDManager::_sendSystem the altitude for the gcs position is validated as >= 0. Can't altitudes go negative when below sea level?
  • Very confused about how Basic ID works. There are different types like say RFC 4122 which is a UUID format for this. That is a 128 bit value. The mavlink field for this is uint8_t[20]. The setting is a string, and the code does a toLocal8Bit() on the string. I don't see how this will work for all the possible types for this field?
• For the lat/lon/alt fixed values the precision is set to 7 digits of precision in the settings metadata. The default QGC precision for lat/lon 6 digits in all other place and altitude is fewer I think 2. Is this 7 digits of precision a remote id standard or can these be changed to match the way the rest of QGC handles these values?
• Mavlink Protocol
  • Arm Status error messages
    • This seems to provide all sorts of important status. For example I am getting this error: SELF_ID OP_ID LOC OP_LOC.
    • As far as I can tell this means it's missing various messages being sent to it. For example if I turn on Broadcast of Self ID then the SELF_ID error will go away.
    • How is a GCS supposed to know what SELF_ID in the error message means. Does a GCS need to so string compares on this text to provide useful help to the user? If so the Mavlink Open Drone ID spec says nothing about this from a spec standpoint. Those text string would need to be documented somewhere.
    • There is an example of this string compare in the RemoteIDManager already. It does this if (armStatus.error == QString("missing basic_id message")) { to see if the basic id is missing.
    • Having to show the actual string from arm status error to the user to have them figure it out is pretty horrendous usability.
    • Also confused as to why my Remote ID device is saying it wants OP_ID? From what I can see in the code it seems like that should only be required if the region is EU.
• Remote ID Device parameters
  • Since there is no metadata for this all the values are floats which is not correct
  • That would seem to indicate that the only true way to edit these params is with DroneCan editing. Is that correct?
• General Remote ID stuff
  • Basic ID is the ID associated with the vehicle correct, not the operator? Is it not the case that the Basic ID values are parameters on the remote id device itself as well? If so, why is Basic ID needed as a general Remote ID setting? And instead Basic ID should be a one time vehicle setup thing where the Remote ID Device parameter are set correctly?
  • Operator ID is for the person operating the vehicle, correct?

[DonLakeFlyer]

And then once my above questions are answered and I get my head around this better. I want to understand how this all relates to a DIY vehicles versus an off the shelf commercial vehicle. But I'll save that for later.

[DonLakeFlyer]

Reviewers: I'm not looking for a code review at this point. More of a review of my questions as well as a review of the direction the UI is going.

[julianoes]

I think @BluemarkInnovations is probably best to comment on this. I'm not very familiar with it all.

[dronevdl]

For EU remote id there is a checksum. Perhaps QGC can implement checking the checksum.
Note that the -xyz is not to be transmitted, it is only to check that you have a correct operator id. But maybe that mavlink message for drone id includes the -xyz so the remote id module can do the checksum as well.
Also the label on your drone showing your operator id, should not contain the -xyz part.

#!/usr/bin/python3
#
# pip3 install python-stdnum

from stdnum import luhn

oprn = 'FIN87astrdge12k8-xyz'

a = oprn[:3]
b = oprn[3:16]
c = oprn[17:]
print(a,b,c)
print(luhn.calc_check_digit(b[:-1]+c, alphabet='0123456789abcdefghijklmnopqrstuvwxyz'))
test = b[:-1]+c+b[-1:]
print(test)
print(luhn.is_valid(test, alphabet='0123456789abcdefghijklmnopqrstuvwxyz'))

[BluemarkInnovations]

For EU remote id there is a checksum. Perhaps QGC can implement checking the checksum.

See #10970

[BluemarkInnovations]

I have a pile of question on how Remote ID works. I am using a Holybro remote id module connected to Telem2 with PX4 firmware:

• RemoteIDManager:

  • Live Position updates: Is ALLOWED_GPS_DELAY set at 5 seconds some sort of remote id spec requirement? Or is it just an internally picked value for timeouts? Reason I ask is that on Mac/iOs the ability to set the interval for position service updates is not possible. And then default update rate is higher then 5 seconds. Does this mean that these setups will need an external GPS? Or can the delay be bumped up such that Mac/iOs can work from internal GPS?

The standard F3586 MoC allows a delay up to 1 second. https://github.com/opendroneid/opendroneid-core-c?tab=readme-ov-file#comparison

• In RemoteIDManager::_sendSystem the altitude for the gcs position is validated as >= 0. Can't altitudes go negative when below sea level?

I agree, it can go negative.

• Very confused about how Basic ID works. There are different types like say RFC 4122 which is a UUID format for this. That is a 128 bit value. The mavlink field for this is uint8_t[20]. The setting is a string, and the code does a toLocal8Bit() on the string. I don't see how this will work for all the possible types for this field?

Somehow agree. The Serial Number and CAA assigned registration ID are chars. The UTM assigned UUID and specific session ID should be displayed as hex values.

• For the lat/lon/alt fixed values the precision is set to 7 digits of precision in the settings metadata. The default QGC precision for lat/lon 6 digits in all other place and altitude is fewer I think 2. Is this 7 digits of precision a remote id standard or can these be changed to match the way the rest of QGC handles these values?

It has to do with precision. RemoteID standard specifies GNSS receivers with SBAS (WAAS). I guess 6 digits are fine to represent that.

• Mavlink Protocol

  • Arm Status error messages

    • This seems to provide all sorts of important status. For example I am getting this error: SELF_ID OP_ID LOC OP_LOC.
    • As far as I can tell this means it's missing various messages being sent to it. For example if I turn on Broadcast of Self ID then the SELF_ID error will go away.
    • How is a GCS supposed to know what SELF_ID in the error message means. Does a GCS need to so string compares on this text to provide useful help to the user? If so the Mavlink Open Drone ID spec says nothing about this from a spec standpoint. Those text string would need to be documented somewhere.

ArduPilot decided to send all those messages but with empty payload. ArduRemoteID firmware is expecting such behavior. PX4 decided not send empty messages, hence that is why you see a lot of error messages.

* There is an example of this string compare in the RemoteIDManager already. It does this `if (armStatus.error == QString("missing basic_id message")) {` to see if the basic id is missing.
* Having to show the actual string from arm status error to the user to have them figure it out is pretty horrendous usability.

Typically, an end-user only needs to set an operatorID (if it is in the EU), location source for the pilot location. The drone manufacturer has setup all other variables.

* Also confused as to why my Remote ID device is saying it wants OP_ID? From what I can see in the code it seems like that should only be required if the region is EU.

See above. It has to do with the implementation of PX4.

• Remote ID Device parameters

  • Since there is no metadata for this all the values are floats which is not correct
  • That would seem to indicate that the only true way to edit these params is with DroneCan editing. Is that correct?

Not sure what you mean. I think you need to distinguish what an end-user has to configure and what an UAV manufacturer has to do. There is also something like tamper resistant implementation.

• General Remote ID stuff

  • Basic ID is the ID associated with the vehicle correct, not the operator?

Yes associated with the vehicle. The serial number range is registered during the DoC submission at the FAA website.

Is it not the case that the Basic ID values are parameters on the remote id device itself as well?

Yes they are stored also as parameters on the RemoteID device.

If so, why is Basic ID needed as a general Remote ID setting?

In some cases, a second basicID needs to be configured. And in some cases, there is no basic ID configured. If you use a MAVLink interface (and not DroneCAN), how can you edit/enter these values? (You need a user interface.)

And instead Basic ID should be a one time vehicle setup thing where the Remote ID Device parameter are set correctly?

In most cases it it is.

• Operator ID is for the person operating the vehicle, correct?

Yes for the EU only.

[dirksavage88]

@DonLakeFlyer I think there is still a ton of confusion but I can maybe help answer this :

. I want to understand how this all relates to a DIY vehicles versus an off the shelf commercial vehicle. But I'll save that for later.

The DIY vehicles that aren't doing the 'industry stuff' and don't want to fly at night/over people, etc are purely hobby will just need a remote ID module that streams basic stuff and no basic ID, arming status, etc, it's a slap on module that likely has it's own gps and therefore is standalone and has to communication with the autopilot. The community this involves most is hobby FPV pilots and fixed wing modelers.

The DIY academic & industrial side will need something like open drone ID and an integrated module approach (e.g. communicates with the autopilot for emergency status/gps location/arming status) to become compliant if they are using the major open source autopilots. Ardu and PX4. Cognipilot does not yet have aerial vehicle support but I'd hope they also leverage open drone ID instead of reinventing the wheel.

The COTS products like DJI, Autel, Skydio have their own proprietary solutions to become RID compliant, likely just a software update.

The context for RID and the history is an interesting read (if you have the time): https://dirksavage88.medium.com/open-drone-id-is-open-source-softwares-answer-to-unmanned-airspace-regulations-fa4c1412d020

[Davidsastresas]

@DonLakeFlyer, I did the first QGC implementation with a partner of mine a little more than a year ago. Back at the time, we spent some time trying to get familiarized with the requirements to be compliant. But since then we've had no contact with it at all. Back then some of these requirements were not super defined, and some of them were nonsense ( like that 1s GPS delay BluemarkInnovations confirmed, that is why we raised it to 5s because otherwise literally no Android device would consider that good. In fact, we manage a lot of custom builds where we've had to raise that timeout up to 15s, because on some android and windows devices it would really not work otherwise ).

Since then, I know some details changed at the requirements level, like that PR BluemarkInnovations and julianoes worked with recently, but I have been really totally out of anything related to RID since that year and a half, so I think @BluemarkInnovations ( thank you for the great response by the way, I read that article from medium, super interesting! ) is really the best to help you out on your questions, as they are really involved and updated in RID matters.

[BluemarkInnovations]

and some of them were nonsense ( like that 1s GPS delay BluemarkInnovations confirmed, that is why we raised it to 5s because otherwise literally no Android device would consider that good. In fact, we manage a lot of custom builds where we've had to raise that timeout up to 15s, because on some android and windows devices it would really not work otherwise ).

It is good to point out that the RemoteID is not a regular add-on like a cam. F3586 requires a maximum delay of 1 second. Above it, you are not-compliant. So, it means that an UAV manufacturer can submit its DoC at the FAA and get approved. But if at some point in time, the FAA requests for an external review/audit, this point will be non-compliant. In such scenario, the FAA will likely request the manufacturer to improve the UAS on this point to make the product and install base compliant. I think that if non-compliant design-choices have been made, it is good to notify the user/UAV-manufacturer of those issues.

[DonLakeFlyer]

F3586 requires a maximum delay of 1 second.

That would seem to indicate then that for GCS to broadcast its location for use with remote id that an externally connected GPS is required. I'm not sure any internal devices update that fast. Which in turn means that QGC needs to be updated so that it doesn't use internal location services to publish gcs location in OPEN_DRONE_ID_SYSTEM messages. Given how location services works cross-platform, the determination of minimum update interval doesn't seem reliable.

Is LIVE_GNSS location required or can you instead just support TAKEOFF?

[Davidsastresas]

It is good to point out that the RemoteID is not a regular add-on like a cam. F3586 requires a maximum delay of 1 second. Above it, you are not-compliant. So, it means that an UAV manufacturer can submit its DoC at the FAA and get approved. But if at some point in time, the FAA requests for an external review/audit, this point will be non-compliant. In such scenario, the FAA will likely request the manufacturer to improve the UAS on this point to make the product and install base compliant. I think that if non-compliant design-choices have been made, it is good to notify the user/UAV-manufacturer of those issues.

Absolutely! For production we would not recommend otherwise, but sometimes for internal tests it is handy to not rely on extra hardware.

[BluemarkInnovations]

F3586 requires a maximum delay of 1 second.

That would seem to indicate then that for GCS to broadcast its location for use with remote id that an externally connected GPS is required. I'm not sure any internal devices update that fast. Which in turn means that QGC needs to be updated so that it doesn't use internal location services to publish gcs location in OPEN_DRONE_ID_SYSTEM messages. Given how location services works cross-platform, the determination of minimum update interval doesn't seem reliable.

Is LIVE_GNSS location required or can you instead just support TAKEOFF?

Table 3 of the ASTM F3586 MoC states for standard RemoteID UAS: "(4) The unmanned aircraft must broadcast the latitude, longitude,and geometric altitude of the unmanned aircraft and its control station no later than 1.0 seconds from the time of measurement to the time of broadcast."

For newly produced drones you need to implement a standard RemoteID solution. That requires a live operator position. Take-off is only allowed in the EU or for add-ons. Yesterday, I also had a chat with somebody more close to the ASTM standardization. Basically, he said that above requirement does not mean that you need to read out the GNSS receiver at 1 Hz interval. Instead, you could rely on sensor fusion: the combination of GNSS, accelerators, algorithms to provide such resolution. Hence, QGC should not read out directly the GNSS receiver, but instead rely on the Android location APIs available in Google Play services to satisfy this requirement. Would that be a solution?

[DonLakeFlyer]

I will get back to this, but it's going to take a while. All my time is being taken up with prepping for some Africa field testing.

[DonLakeFlyer]

Could I get some help from folks on this thread who are familiar with Remote ID to test my new pull? Try using the new UI only to set up Remote ID.