Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Failsafe RTH disarms the machine in some cases #1351

Closed
digitalentity opened this issue Mar 3, 2017 · 31 comments
Closed

Failsafe RTH disarms the machine in some cases #1351

digitalentity opened this issue Mar 3, 2017 · 31 comments
Labels
Milestone

Comments

@digitalentity
Copy link
Member

digitalentity commented Mar 3, 2017

Per @sppnk

I am having big problems with my failsafe configuration. I am not sure if the openlrsng receiver is missconfigured or it is my Inav configuration.

If I signal FAILSAFE by switch, it works well and do RTH.

But if I turn off the transmitter or go out of range, sometimes the plane disarms in flight and I have no control of it althought I turn on again the transmitter. Is it trying to land? Almost lost a plane yesterday.

I have it configured to RTH. All parameters default. GPS I guess doesnt have any glitches.

When I turned off my transmitter in flight I did that almost when plane was over home position or very close. Several times. I didnt want to test that far away Could be the plane started to land ? BUT I had no control over it (when I switched on of course)

@digitalentity digitalentity added this to the 1.7 milestone Mar 3, 2017
@digitalentity
Copy link
Member Author

digitalentity commented Mar 3, 2017

Sub-bugs:

  • If machine is close to home failsafe RTH won't activate and failsafe will disarm the craft
  • Any exit from RTH_IN_PROGRESS state during failsafe RTH will cause disarm

@digitalentity
Copy link
Member Author

digitalentity commented Mar 3, 2017

Solution:

  1. Smarter interaction between navigation core and failsafe system - don't treat non-active RTH as a failsafe abort. In case of failsafe event if RTH is unable to activate it should fallback to emergency landing.
  2. Emergency landing should be treated as part of Failsare-RTH handling.
  3. Disredard minimum distance to home in case of Failsafe-RTH event.

@raul-ortega
Copy link
Contributor

raul-ortega commented Mar 3, 2017

Is quite hard and sad to see how a friend losses the control of his flying wing caused by a bug that hasn't been tested before the latest version has been released (click the picture to see full video of the crass):

Reptile s800 killed by iNav

Not only failsafe has a bug, also enabling horizon start causing problems. If you want more details of what happens during the flight I will be very pleasant to explain every thing second by second.

We were very excited with the release, but we are seriously thinking of abandoning the idea of continuing being beta tester.

@Redshifft
Copy link

@raul-ortega Your plane was not in Failsafe, it looks more like a momentary power surge that disarmed it.
Look at the picture
power fail copy

@oleost
Copy link
Contributor

oleost commented Mar 3, 2017

@raul-ortega Yes, I believe you suffered a power surge and this caused the flight controller to reboot.

How is the wiring done? Seperate BEC from servos and FC or shared?

Issue is after and reboot in air, the GYRO cannot do its calibration, and all other modes will fail. You need to switch into passthrough.

( I must add that I been an tester since the fixedwing early days of INAV, and have yet to crash and airplane where I can blame it on INAV. I have tested features before the even came into develeopment branch and was far from finished. )

@tonyintn
Copy link

tonyintn commented Mar 3, 2017

I to am curious, according to the video posted it never showed it was in failsafe (RTH) mode, showing disarmed in Horizon mode.

@raul-ortega
Copy link
Contributor

raul-ortega commented Mar 3, 2017

Ok about the shapshot of the video frame, but the crash more or less one minute after that, just when the pilot (my friend) activated failsafe from a switch on the transmiter. This is the explanation of what happened in the previous moments of the crash:

"..I was flying in Acro until minute 4:33, when I activated STAB again, and at minute 5:44 I activated the HOLD, and the plane was responding well and made 2 circles arround the point where it was flying.

In this moment the main problem began. I SWITCHED ON THE FAILSAFE MODE, with the intention of testing the RTH procedure, but instead of returning, It ignored it and followed flying without control, It leaves the return to home and makes a straight flight, before the total loss.

As there were no response from the plane, I changed from HOLD moe to STAB mode at minute 6:45, and I lost control at 6:55."

@raul-ortega
Copy link
Contributor

raul-ortega commented Mar 3, 2017

This is what I think that could be happening after the reset of the board (minute 4:58):

The arming contitions are fullfiled for armed, and continues flying.
Home is established because it has fix and enougth sats.
Failsafe is activated manually, the plane is close to home (distance and altitude), then it disarms the motor thinking that the plane has landed.

If it is as I say, it is really a very dangerous bug.

The resetting of the board is also very strange because that board has been flying before with other aircraft (a multirrotor) with a differnet firmware (cleanflight).

@oleost
Copy link
Contributor

oleost commented Mar 3, 2017

Issues with airplanes is servo that can draw alot of power, especially if they get stuck.

But it would be helpful to share CLI dump and hopefully blackbox log of the event.

@ShikOfTheRa
Copy link
Contributor

I'm not so sure on power reset... see how the GPS data continues to update with no delay. Sure you would see a short blip.

Also I had similar issue with patrike fixedwing a couple of times. I figured out my issue was this...

Flying ok like in vid.
Had a rx glitch which made FC think it was receiving switch input to disarm.
glitch was very short, but because throttle was on like in video, it wouldn't re-arm.
Happened a few times before I figured out. Since then I don't use arm switch.

MAybe that is part of issue?

@raul-ortega
Copy link
Contributor

It is not a power reset, it is caused exactly when the failsafe mode is switched on manually by the pilot, on 6:56 (on time in the osd).

@Redshifft
Copy link

@raul-ortega
Can you list your full hardware specs, Bec, servos OSD etc etc ?

@Redshifft
Copy link

@ShikOfTheRa strange no speed was displayed after the "glitch"
I think it's that dodgy old OSD firmware :D LOL
Another good one I would not discount is people put other flight modes the other side of "dangerous" ones, not thinking they cycle past them when they throw the switch ;)

@ShikOfTheRa
Copy link
Contributor

ShikOfTheRa commented Mar 3, 2017 via email

@raul-ortega
Copy link
Contributor

I have attached the configuration (complete dump):
flip32-crashes-iNav1.6.txt

@raul-ortega
Copy link
Contributor

Please, there is not place for speculations about what causes the disarming, it happens exactly when the pilot activates failsafe mode manually from the transmiter. Everything that appears on the OSD affter that instant is as consecuence of the disarming issue.

@digitalentity
Copy link
Member Author

digitalentity commented Mar 3, 2017

@raul-ortega pity to hear about your friend's crash. I'm not that different - also have lost a plane recently - due to a power failure.

I doubt this is the case of exactly this bug - "disarm-close-to-home bug" was there from day 1 and we never hit it before because it can only manifest very close to home (5m by default). I personally tested failsafe RTH in 1.6-ALPHA's and it was fully functional otherwise I wouldn't have released the new version.

I must admit that not all scenarios were tested - as you can understand it's hard to test momentary GPS loss or power failure. Without a blackbox log it's really impossible to say what was happening under the hood, so we can only guess.

How is your friend's failsafe configured? No pulses? Pre-set value? Other?

@Redshifft
Copy link

@ShikOfTheRa No I was joking, can't see the OSD causing it :) yes jumping across a mode has no doubt caught out a few....
@raul-ortega , That was just a FW dump, do you have a list of the components ? If you have changed your mind and now don't want to discuss the details I think it only fair you delete your post.

@digitalentity
Copy link
Member Author

digitalentity commented Mar 3, 2017

@raul-ortega I know what was the cause of your particular issue - it's totally user error. From your dump I figure that you have

set failsafe_kill_switch = ON

This setting means that failsafe will instantly disarm the machine bypassing everything else you might have condifured - so when you flipped the FAILSAFE switch the plane did exactly what it was configured to do. This is mentioned in the docs https://github.com/iNavFlight/inav/blob/master/docs/Failsafe.md#flight-controller-failsafe-system, however most of us don't read docs

@digitalentity
Copy link
Member Author

I have to remind everyone once more - always test failsafe on the ground first. You'll avoid many hairy moments and save developers some time to figure out something that is not a bug.

@raul-ortega
Copy link
Contributor

@digitalentity, of course that I understand that not all scenarios can be tested, and also that without blackbox is difficult to figure out what caused the motor disarming, but be sure that is not a power reset as somebody guess. I'm providing as much information as we have and you request: the video recorded, the explanation of pilot actions, the CLI, and in this comment also I'm providing information about how the failsafe is configured. And as soon as possible I'll provide the list of components, for sure. My aim is only to try to avoid this disarming issue to happen again. Please, don't feel that this is an attack, rather it is a contribution of people very excited with iNav that get into pain in all senses when something like this happens.

About the receiver, the failsafe is configured with pre-sets (sticks positions and switch for failsafe mode in iNav).

About how the modes are configured on the transmitter:

One 3 positions switch for:

  • ID0: ACRO
  • ID1: HEADHOLD + ALTHOLD
  • ID2: ALTHOLD + POSHOLD

Another 3 positions switch for:

  • ID0: ACRO
  • ID1: HORIZON
  • ID2: FAILSAFE

Combining them we could activate at the same time for example:

ALTHOLD + POSHOLD + HORIZON

or

LTHOLD + POSHOLD + FAILSAFE

@digitalentity
Copy link
Member Author

@raul-ortega no, this is definitely not a power surge - the board did not reboot, GPS coordinates keep updating, AHI is responding correctly - all this won't be the case if the board executed a reboot. Also booting the board takes about 1 second.

I wonder if failsafe kill switch setting should be removed completely. If you can spare a switch for FAILSAFE mode you surely can space a switch for ARMing which will give you the same manual disarm opportunity.

If you want actual signall loss do disarm the machine the best way to do that is to configure failsafe procedure to DROP the machine.

What do you think?

@raul-ortega
Copy link
Contributor

raul-ortega commented Mar 3, 2017

@digitalentity, I didn't read your comment about failsafe_kill_switch, I was writting:

set failsafe_kill_switch = ON

This explains everything, it is a big error of the pilot.

I apologize about all the time you have spent trying to figure out our issue. Please, take into account that we try to do the best from our side every time that a new version is released. We have been trying 1.6 RC1, RC2 and now the final release, and some of our community users has reported issues and also data very useful for the test of this firmware, not only 1.6, also older versions.

@digitalentity
Copy link
Member Author

digitalentity commented Mar 3, 2017

@raul-ortega Apology accepted 😄 I never intended to be mean and I really appreciate that you report bugs and help figuring them out instead of going out and compaling that "INAV crashed my plane, arrrrr!!!".

This is how most opensource projects work - developers make features, try to test them as much as possible, but real-life extensive testing is done by users. That's why we have release-candidate versions - if a serious bug is discovered in an RC - we'll do our best to fix it before final release.

@raul-ortega
Copy link
Contributor

raul-ortega commented Mar 3, 2017

I appreciate too much your effort, and your last comment!!! I'm also a open source develpoer (u360gts/amv-open360tracker). One of this days perhaps I will contribute on iNav.

@digitalentity
Copy link
Member Author

Another 3 positions switch for:

ID0: ACRO
ID1: HORIZON
ID2: FAILSAFE

Having a failsafe activation on the same switch with other flight modes is generally a very bad idea.

Despite the name FAILSAFE is an emergency mode which takes control away from the pilot. Failsafe's intention is to attempt to save the machine if pilot is no longer in control of the craft. This may cause very unwanted behavior (maybe even dangerous if by some chance failsafe is activated on the ground).

@raul-ortega
Copy link
Contributor

I'll translate this message to my community. Thanks again.

@raul-ortega
Copy link
Contributor

Konstantin, I have a last question about my personal set up regarding the failsafe. Is it safe to have it together with automatic arming? The wiki says that If it is configured with pre-sets on the receiver, and the procedure is RHT, I should put stick thortle to 0%. Can I trust on that the engine will not be disarmed? Thanks.

@digitalentity digitalentity changed the title Failsafe RTH disarms the machine if close to home Failsafe RTH disarms the machine in some cases Mar 4, 2017
@digitalentity
Copy link
Member Author

@raul-ortega failsafe will disarm bypassing the RTH if throttle was at zero for 10 seconds (default setting for failsafe_throttle_low_delay). It won't hurt to test on the ground though.

However this bug has two scenarios where failsafe may disarm the machine - one when being almost directly above home; the other is loosing GPS when doing failsafe RTH.

@ukmook
Copy link

ukmook commented Mar 4, 2017

Rather than a kill switch why not have a throttle hold switch which turns the motor or motors off but leaves flight surfaces operational. You could still use stick arming.

@digitalentity
Copy link
Member Author

Ok, possible disarm on failsafe RTH is fixed entirely by #1401

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

7 participants