X8 Crash Transitioning MANUAL to HOLD

Large X8, about 14kg, running 1.6.0dev EKF2 on Pixhawk 2.1, crashed when going from MANUAL to HOLD / AUTO_LOITER. We are having trouble determining the cause. May be related to the estimator or multicopter position control. The first transition, no crash, the vehicle yaws, not following the setpoint. The second transition, the FC commands a large pitch that I cannot explain either. Thanks for looking @LorenzMeier

Crash Log:
http://logs.px4.io/plot_app?log=607261cf-dd41-49cc-86a8-74cddc18996a

Manual Flight Log for Performance Reference:
http://logs.px4.io/plot_app?log=8419de38-75db-47ef-9bfa-77ab8e487c24

Notes:

  • 53:47 – First GPS Hold engaged
    – Pitch, roll, yaw angles instantly snap to 0 deg
    – 53:49.25 – Yaw estimate rises to 55 deg
    – Nothing strange in pitch/roll
    – Raw magnetic field strength dips on Y, Z axes

  • 54:16 – RP (Remote Pilot) regain manual control

  • 54:40.750 – Second GPS Hold engaged
    – Roll angle setpoint snaps to 0 deg
    – Pitch angle setpoint nearly to 15 deg!
    – Pitch Rate diverges first, then Roll Rate
    – Yaw angle not an issue??
    – Raw Raw magnetic field strength oscillatory divergence?!

  • 53:43.400 – RP attempts to regain control
    – By now, Roll and Pitch have diverged, Yaw starting to diverge
    – Actuator PWM saturation – commanding min and max

  • 54:46.500 – First impact
    – Indicated by harsh Z acceleration
    – Divergence β€œstops” upon β€œlanding” on the ground

  • 54:46.800 – RC Lost Link indicated

  • 54:47.180 – RC Lost Link detected

  • 54:47.200 – RTL engaged, throttle increase to takeoff
    – Large yaw error – Setpoint: 0 deg, Estimated: 57 deg

  • 54:47.625 – Throttle saturated at 94%
    – Only Output 6, others are well below PWM 1900

  • 54:53 – Throttle reduced to 75%

  • 55:01.740 – Second impact
    – Small oscillations leading up
    – Impact implied by log cutting out

Pilot Remarks:

  1. RP manual takeoff
  2. First GPS Hold engaged
    β€’ Motors spin down slightly, small altitude drop
    β€’ Yaw correction?
    β€’ Stabilizes, holds position successfully
  3. RP regains manual control
  4. Second GPS Hold engaged
    (a) Same drop behavior, except divergent mode excited
    (b) Large oscillations in Pitch and Roll
    Β© RP attempts to regain manual control unsuccessfully
    (d) Vehicle descends oscillating from about 20m
    (e) Vehicle impacts tall grass, somehow upright on landing gear
    (f) Throttle appears to be at zero, motors sound off
  5. Unintentional Takeoff due to loss of link safety param
    (a) Unknown to pilot at time – receiver had been ripped off due to tall
    grass
    (b) Motors spin up, vehicle takes off out of tall grass
    Β© RP has zero control input
    (d) Vehicle appears to ascend to around 10m, levels altitude
    (e) Vehicle stability starts to diverge again
    (f) Impact in short grass/mud, around 20m right

After digging through the data a little more, it seems that the first transition may have yawed due to the EKF from what I’ve read on other topics. At the second transition, the FC commands pitch and the vehicle seems to follow the command which ultimately causes it to become unstable. Could this be position control tuning issues? Is there a guide for tuning the relevant parameters? Am I on the right track as to what occurred?