New mixer and PID gain tuning is required
check: Creating new mixer model for Octo-H airframe
I recommend first reading the guides on PX4 mixer:
https://docs.px4.io/master/en/concept/mixing.html
https://docs.px4.io/master/en/dev_airframes/adding_a_new_frame.html