Aggressive response in position mode

We were out setting up large (~1.5m wheelbase, 17.5kg) hexcopter 3 days ago. It flew okay (but not great) in attitude and position mode. We even completed a simple mission. Log here:

https://review.px4.io/plot_app?log=6d501b35-ea18-4b58-a8d6-ecff80718f10

We continued setup/tuning today, and made some improvements in attitude mode. Some settings that may be particularly relevant:

PWM_MIN from 1150 to 1250
PWM_MAX from 1875 to 1950
THR_MDL_FAC from 0 to 0.5

these were determined by analyzing thrust stand data and selecting value that gave the most linear response for this propulsion combo

MC_AIRMODE from 0 to 1
MPC_THR_HOVER from 0.44 to 0.25 - this is a better representation of reality
IMU_GYRO_CUTOFF from 40 to 24
IMU_ACCEL_CUTOFF from 20 to 10
MC_DTERM_CUTOFF from 20 to 10

Flight log form today is here:

https://review.px4.io/plot_app?log=76a37954-2ddb-4a25-9502-abee4ca20d5c

Despite the apparent improvements in attitude mode, position hold was NOT usable. Immediately upon entering position mode aggressive pitch/roll behavior would start. The heading was good, and given the other responses I am confident this is NOT toilet bowl.

Is is possible that the thrust linearization in paricular could cause this sort of thing? Do we just need to tune down the position controller to compensate?