Unable to control all PWM channels with custom mixer setup

I am currently trying to set up a new mixer for an underwater ROV using px4 and a pixhawk. But I do only get PWM output on four channels when using this mixer:

M: 3
O: 10000 10000 0 -10000 10000
S: 0 0 -6000 -6000 0 -10000 10000
S: 0 1 6000 6000 0 -10000 10000
S: 0 4 4000 4000 0 -10000 10000

M: 3
O: 10000 10000 0 -10000 10000
S: 0 0 -6000 -6000 0 -10000 10000
S: 0 1 -6000 -6000 0 -10000 10000
S: 0 4 -4000 -4000 0 -10000 10000

M: 3
O: 10000 10000 0 -10000 10000
S: 0 0 6000 6000 0 -10000 10000
S: 0 1 6000 6000 0 -10000 10000
S: 0 4 -4000 -4000 0 -10000 10000

M: 3
O: 10000 10000 0 -10000 10000
S: 0 0 6000 6000 0 -10000 10000
S: 0 1 -6000 -6000 0 -10000 10000
S: 0 4 4000 4000 0 -10000 10000

M: 2
O: 10000 10000 0 -10000 10000
S: 0 2 9000 9000 0 -10000 10000
S: 0 3 5000 5000 0 -10000 10000

M: 2
O: 10000 10000 0 -10000 10000
S: 0 2 9000 9000 0 -10000 10000
S: 0 3 -5000 -5000 0 -10000 10000

Loading this mixer on /dev/pwm_output0 only makes motor 1-4 controllable, and the pwm info command gives the following output:

device: /dev/pwm_output0
channel 1: 1500 us (alternative rate: 400 Hz failsafe: 1500, disarmed: 1500 us, min: 1100 us, max: 1900 us, trim: 0.00)
channel 2: 1500 us (alternative rate: 400 Hz failsafe: 1500, disarmed: 1500 us, min: 1100 us, max: 1900 us, trim: 0.00)
channel 3: 1500 us (alternative rate: 400 Hz failsafe: 1500, disarmed: 1500 us, min: 1100 us, max: 1900 us, trim: 0.00)
channel 4: 1500 us (alternative rate: 400 Hz failsafe: 1500, disarmed: 1500 us, min: 1100 us, max: 1900 us, trim: 0.00)
channel 5: 0 us (alternative rate: 400 Hz failsafe: 0, disarmed: 1500 us, min: 1100 us, max: 1900 us, trim: 0.00)
channel 6: 0 us (alternative rate: 400 Hz failsafe: 0, disarmed: 1500 us, min: 1100 us, max: 1900 us, trim: 0.00)
channel 7: 0 us (alternative rate: 400 Hz failsafe: 0, disarmed: 1500 us, min: 1100 us, max: 1900 us, trim: 0.00)
channel 8: 0 us (alternative rate: 400 Hz failsafe: 0, disarmed: 1500 us, min: 1100 us, max: 1900 us, trim: 0.00)
channel group 0: channels 1 2
channel group 1: channels 5 6 7 8
channel group 2: channels 3 4

The four first channels is configured exactly as expected, but I can not get it to configure all channels. and it is only the first four channels that are controllable by sending messages to to control_group_0.

I have also tried the IO_pass mixer. Using this I can control 6 motors (and not 8 as I would expect).

Is there something in the mixer configuration that I am missing? Or is this a bug?

Can you try this with PX4 master? There are some recent mixer changes there that might impact this.

I have now tried the same with a clean and new version of PX4 master. Got the exact same behavior. Both with my custom mixer, and with the stock IO_pass.main.mix.

The strange thing is that if I reduce the number of simple mixers for each motor I manage to get my mixer to control 6 pwm outputs. (However not the way I want as i am missing mixers)

I have now tried the fork with new mixer configuration posibilities: https://github.com/PX4/Firmware/pull/6357
I experience the same problem there, it do however seem like all mixers are set up. The command mixer config /dev/pwm_output0 gives:

M: 3
O: 10000 10000 0 -10000 10000
S: 0 0 -5999 -5999 0 -10000 10000
S: 0 1 5999 5999 0 -10000 10000
S: 0 5 3999 3999 0 -10000 10000
M: 3
O: 10000 10000 0 -10000 10000
S: 0 0 -5999 -5999 0 -10000 10000
S: 0 1 -5999 -5999 0 -10000 10000
S: 0 5 -3999 -3999 0 -10000 10000
M: 3
O: 10000 10000 0 -10000 10000
S: 0 0 5999 5999 0 -10000 10000
S: 0 1 5999 5999 0 -10000 10000
S: 0 5 -3999 -3999 0 -10000 10000
M: 3
O: 10000 10000 0 -10000 10000
S: 0 0 5999 5999 0 -10000 10000
S: 0 1 -5999 -5999 0 -10000 10000
S: 0 5 3999 3999 0 -10000 10000
M: 2
O: 10000 10000 0 -10000 10000
S: 0 2 9000 9000 0 -10000 10000
S: 0 3 5000 5000 0 -10000 10000
M: 2
O: 10000 10000 0 -10000 10000
S: 0 2 9000 9000 0 -10000 10000
S: 0 3 -5000 -5000 0 -10000 10000

the four working mixers have their scalers changed by 1, relative to the input mixers…

@LorenzMeier @bkueng is there a mixer limit @eirikhex is hitting?

What if you tried it via the FMU (AUX) instead of IO (MAIN)?

@eirikhex @dagar I’m pretty sure this mixer is too big for IO (it can’t fit in memory). @eirikhex if you can post the output of px4io status after the mixer has been loaded then we know.

Tried with the FMU now, and the mixer works there.
It seems like a good explanation that it is to big to fit in the memory, as all other large mixers I have tried show the same behavior (IO_pass, AAVVTWFF etc. ). The compact multi-copter mixers (e.g octo_+ ) makes me able to get a pwm signal to all motors.

If that is the case, am I correct to assume that my options is to use the aux pwm outputs or make a custom mixer similar to the specialized Multicopter mixers?

The px4io status outputs:

WARN [px4io] loaded
protocol 4 hardware 2 bootloader 3 buffer 64B crc 0xdc7956eb
8 controls 8 actuators 18 R/C inputs 2 analog inputs 0 relays
40 bytes free
status 0x1f01 OUTPUTS_ARMED SAFETY_OFF RC_FAIL FMU_FAIL MIXER_OK ARM_SYNC INIT_OK FAILSAFE
alarms 0x0030 FMU_LOST RC_LOST
vservo 16 mV vservo scale 10000
vrssi 1438
actuators 0 0 0 0 -25000 -25000 -25000 -25000
servos 1500 1500 1500 1500 0 0 0 0
reversed outputs: [________] trims: r: 0.0000 p: 0.0000 y: 0.0000
0 raw R/C inputs
R/C flags: 0x0000
mapped R/C inputs 0x0000
ADC inputs 7 1673
features 0x0008 RSSI_ADC
arming 0x0435 FMU_DISARMED IO_ARM_OK MANUAL_OVERRIDE_OK INAIR_RESTART_OK ALWAYS_PWM_ENABLE OVERRIDE_IMMEDIATE
rates 0x00ff default 50 alt 400 sbus 72
debuglevel 0
controls 0: 0 0 0 0 0 0 0 0
controls 1: 0 0 0 0 0 0 0 0
controls 2: 0 0 0 0 0 0 0 0
controls 3: 0 0 0 0 0 0 0 0
input 0 min 1000 center 1500 max 2000 deadzone 10 assigned 255 options 0x0000
input 1 min 1000 center 1500 max 2000 deadzone 10 assigned 255 options 0x0000
input 2 min 1000 center 1500 max 2000 deadzone 10 assigned 255 options 0x0000
input 3 min 1000 center 1500 max 2000 deadzone 10 assigned 255 options 0x0000
input 4 min 1000 center 1500 max 2000 deadzone 10 assigned 255 options 0x0000
input 5 min 1000 center 1500 max 2000 deadzone 10 assigned 255 options 0x0000
input 6 min 1000 center 1500 max 2000 deadzone 10 assigned 255 options 0x0000
input 7 min 1000 center 1500 max 2000 deadzone 10 assigned 255 options 0x0000
input 8 min 1000 center 1500 max 2000 deadzone 0 assigned 255 options 0x0000
input 9 min 1000 center 1500 max 2000 deadzone 0 assigned 255 options 0x0000
input 10 min 1000 center 1500 max 2000 deadzone 0 assigned 255 options 0x0000
input 11 min 1000 center 1500 max 2000 deadzone 0 assigned 255 options 0x0000
input 12 min 1000 center 1500 max 2000 deadzone 0 assigned 255 options 0x0000
input 13 min 1000 center 1500 max 2000 deadzone 0 assigned 255 options 0x0000
input 14 min 1000 center 1500 max 2000 deadzone 0 assigned 255 options 0x0000
input 15 min 1000 center 1500 max 2000 deadzone 0 assigned 255 options 0x0000
input 16 min 1000 center 1500 max 2000 deadzone 0 assigned 255 options 0x0000
input 17 min 1000 center 1500 max 2000 deadzone 0 assigned 255 options 0x0000
failsafe 1500 1500 1500 1500 0 0 0 0
disarmed values 1500 1500 1500 1500 1500 1500 1500 1500

You have 40 bytes free (see the output). However, its not hard to get quite a bit space back. I just need to get around to it. In the meanwhile I’ve filed an issue for it: https://github.com/PX4/Firmware/issues/6428

Did some additional checking on the memory. When I take away the mixers for the motors not working I still have 40 bytes of free memory. When i take away the mixers from the motors working the free memory is increasing. My three scaler mixers seems to use 144 bytes each, and the two scaler mixers 112 bytes each. So I need 184 bytes of additional memory to be able to load all my mixers…

Is there any low-hanging fruit I could implement that is straightforward but just needs some attention? (or failing that is it easy to remove ex, support for one of the rc input types until a more permanent solution is ready) I have a relatively complicated (7ch, 2-3 mixers on each) vehicle that this is a real bottleneck on.

You could try finishing this branch:

You can also try these instructions: https://dev.px4.io/advanced-gdb-debugging.html#identifying-large-memory-consumers

This is what I would attempt, work out which ones are excessive or redundant and deal with them. A likely low-hanging fruit for IO is described in this issue: