I think I mainly changed the the order of the motors, because it’s easier for me to handle and analyse (at least it seems easier to me). But I think there shouldn’t really be an issue with using the original order and then just assign them to their corresponding PWM outputs.
I did a few new tests, one of them using the original motor order, as you suggested while keeping the rest of the setup (orientation and calibration) the same. The image below gives an overview of the setup. It didn’t really change much, meaning physically the same motors spin up when tilting the drone. This can also be observed in the flight log: https://review.px4.io/plot_app?log=cb6d8b28-5c1c-4a91-b3a6-bc40f38367c9
You can see that motors 4 and 6 (rear motors) spin faster on a negative pitch (nose down), which is the same as before. So the actuator configuration in itself seems to be consistent.
Next test I did was with a completely new setup, where the drone is on it’s back, resulting in a no-angle orientation of the pixhawk. Then I reassigned the actuators and changed the motor axis from downwards to upwards. See the image below for an overview of the setup.
This is the log of a preflight test using configuration 2: https://review.px4.io/plot_app?log=550755bd-5b5e-4315-8cfd-bc4a55b69332
The first thing I noticed during testing already, and which got approved by the log, is that the motors spin much slower in general and seem to be clamped. The clamping seems logical to me, because I don’t command any thrust, just arm in stabilized, lift it in the air and tilt. Besides that, it looks like the correct motors spin up. Like on the first positive pitch (nose up) motors 4 and 6 start to spin up. Also the rest of the motor outputs seem to be consistent with the movement of the drone.
The combination of both tests led me to the conclusion, that the setup of the geometry was wrong. I always assumed that the small diagram in the Actuators tab in QGC corresponds to a top view of the whole drone (with the coordinate system showing the NED body coordinate system). But it seems like I completely misunderstood this and it rather shows all the actuators with respect to the pixhawk itself, so it’s a top view on the pixhawk. In my configuration with the pixhawk being attached upside-down to the bottom of the frame, I actually have to turn my drone upside down and assign the actuators from a bottom view. So I did the exact thing I just described, basically using the actuator configuration from configuration 2 from above (also with upwards motor axis, as seen from the pixhawk). The whole sensor configuration is done in the drones normal orientation, so with roll=180° and the drone being level when standing on its feet. The log file shows similar results to configuration 2, also with correct response from all the motors.
Log File: https://review.px4.io/plot_app?log=55914a11-e867-456e-ab64-44e57c76a5e4
I will need to do a real flight test to verify all this, but at the moment it seems promising to me.