Crash due to loss of local position

Hello,
We had a customer experience a crash and are not sure what caused it. The drone seemed to lose local position after taking off and having a “Critical navigation failure!” message.
Please find the log at: https://review.px4.io/plot_app?log=a71d1aac-3e80-4923-b14d-9cea302858c3

After this crash the customer was able to hover the drone without issues.

Can someone help me identify why this happened?

Hard to tell exactly what happened from a log.
There are a number of odd things here, but what jumps out to me is that it appears the pilot pull back on the elevator ( throttle ) and pretty much held it there.

Other things. Yours:

Good:


Read this over:

I’d say two things, one look at the checkbits, most are bad, specifically the mag (this fw is also pre emergency yaw recovery estimator IIRC - pretty sure though) and the Mag was calibrated prior to flight (it’s in the logged messages) but as Jim pointed out, the vibration is incredibly high, too high. Likely something was loose. Unlikely a prop on upside since it is a quad and flew for a moment. Also, who knows what the customer did from the time it crashed to the subsequent log. There is absolutely no vibration in that one. Something had to be tightened, checked, etc. I’d check the mag and replace it if any doubt, use the newer firmware which has the emergency yaw recovery https://github.com/PX4/PX4-ECL/pull/754 and most likely, ensure everything is strapped down. A roll like that with that level of vibration is going to just swamp the onboard sensors. You could put a secondary mag on there as well but there isn’t any real interference TBH.

I’ve found that the pilots (or customers) explanation is often unintentionally quite wrong about what actually happened. Our recall especially when things go wrong get distorted.

I also agree with something loose that got tightened could be the explanation.

Thanks for your replies! @jimdgit and @ryanjAA

Before shipping it out to the customer we test flew the same vehicle several times and never had anything like this happen. The reason the pilot put the stick all the way down was in response to the navigation failure warning. He then reported that the drone started to descend very slowly. This makes sense given the landing speed.

It looks to me like the vibrations in the Actuator Controls FFT graph are high given that the drone clipped a tree. It doesn’t appear that the vibration was high over all looking at the raw gyro and accelerometer data. The weirdest part of this was the GPS altitude not following the barometer altitude. Can someone help me understand that?

Ah, ok more to the story then. That makes sense about the vibration and yes, there is not a lot of it on the other parts of the flight. A couple things do come up though on the EKF2. About 6 seconds after takeoff the mag innovations start to increase with the heading innovation getting very large (~150 degrees) about 5-8 seconds before the reset. That is the same time the mag test ratio skyrockets (which makes sense) and the GPS accuracy drops (velocity test ratio increases). I’m of the mind that it’s possible GPS accuracy decreased due to the close proximity of the trees and low altitude and the MAG readings were not great (although it was calibrated prior). I don’t see a mag flip though. It seems like the time the tree was hit was probably right when the modes were changed (z axis accel doubles right there). EKF velocity fails though and so does the mag so while it could be hardware (unlikely), looks like due to where it was flown and GPS accuracy decreased, together leading to the navigational error.

a71d1aac-3e80-4923-b14d-9cea302858c3.ulg.pdf (130.0 KB)

That’s an interesting program you’re using to get the plots, can I ask what it is?

It’s a little scary that the drone can pass all checks, arm, take off, then fail due to the location that it is flown. I guess at least it’s something we can recommend to the customer on how to operate the vehicle. Thanks!