SETUP:
Pixhawk
Raspberry Pi 4
Opti-track motion capture system
Handheld Transmitter
F350 Quadcopter Frame
Flight Mode 1: Manual
In this flight mode, I’m physically using the handheld transmitter. In this mode I’m having no issues, I’m able to operate the aircraft with no issues.
Flight Mode 2: Position
In this mode I’m able to throttle up the aircraft and get it to hover for maybe a few seconds. At this point what occurs is that the aircraft will throttle up and increase in altitude, pause, and then throttle up again, and then pause/hover, then continue to climb. I have the altitude fail safe set at 2 meters, but it doesn’t appear to kick in. So I have to toggle the kill switch to kill the flight before it hits the ceiling. (very high ceilings).
Flight Mode 3: Software Control
In this mode, I’m obviously using software to control the Pixhawk flight operations. collecting positioning data from the MAVROS topics, I’m also setting target coordinates via MAVROS setpoint_raw local publisher. The same thing in Position mode is occurring here as well. If I give a basic translation command of half a meter on the X plane, it will move to that coordinate and then start throttling up, uncontrollably. Again I have to throw the kill switch. (side note, I’ve gotten really good at throwing the kill switch, then enabling the switch so the aircraft doesn’t crash too hard. Lots of timing, but many broken propellers.)
This does not occur 100% of the time. For instance, today, I came in and enabled the motion capture system, and started my pre-flight tests. Check manual flight (Passed), Check Position flight (Failed), check simple software test (Passed), check advanced application (Passed). To be fair the simple software test is roughly the same as the advanced application in regards to the flight characteristics, just a lot more going involved. For roughly 2/3 hours the only failure was the first Position test. I only moved passed it, because I needed to get working on the software and by luck the issue didn’t repeat itself.
as I said for roughly 2/3 hours I had no issues. I was operating the aircraft with no issues, it was all oriented around simple half meter and one meter translations on the X and Y access. Maybe one meter on the Z. Most of the time I was making changes in the software so there wasn’t a lot of flights in the time frame. So this was all on a single battery.
well at this point the battery was almost empty, so I pulled the battery and replaced it with a fresh battery. Went right back to software testing, and bam, the throttling issue started again. The only difference was a battery swap. I obviously needed to relaunch all the applicable ROS nodes, but it was all setback up the way it was before. Now not only does Position mode not work, but both the simple and advanced software control is failing.
I have to say this just feels like there is an issue somewhere related to the aircrafts vertical positioning. And I just can’t understand where. i’ve checked and double checked the Opti-track positioning system, No issues. If I move the aircraft manually, the system follows it will not issues. When I’m flying it, it also appears to track it with no issues. But when I enable Position or use the software were I’m reliant on the MoCap (motion capture), it can have issues with the vertical positioning. I haven’t seen an issue on the MoCap computer, it all looks fine.
Does anyone have any insight that could be helpful?
I have the ulg log file, What information will most helpful? I’ve posted below some of the output that the ulg logged. The data is for roughly 30 seconds, as I wanted to capture what was happening immediately before, during, and after the crash. So I go everything running, on the SBC, and then reset the pixhawk, so that the log would be specifically for the operation. Once the pixhawk was back up, verified with QGroundControl, I executated the flight operations and then the crash.
Console Output
sercon: Registering CDC/ACM serial driver
sercon: Successfully registered the CDC/ACM serial driver
HW arch: PX4_FMU_V5
HW type: V500
HW version: 0x00000000
HW revision: 0x00000000
FW git-hash: e0f016c2b3db160284bbaaadaf70bef84592d81e
FW version: Release 1.10.1 (17433087)
OS: NuttX
OS version: Release 7.29.0 (119341311)
OS git-hash: 427238133be2b0ecd068a11e886ee8fdbc31f6dc
Build datetime: Jan 21 2020 19:03:45
Build uri: localhost
Toolchain: GNU GCC, 7.2.1 20170904 (release) [ARM/embedded-7-branch revision 255204]
PX4GUID: 000200000000383937383138510600280043
MCU: STM32F76xxx, rev. Z
INFO [param] selected parameter default file /fs/mtd_params
INFO [tune_control] Publishing standard tune 1
Board defaults: /etc/init.d/rc.board_defaults
ERROR [safety_button] not starting (use px4io for safety button)
ERROR [safety_button] Task start failed (-1)
WARN [rgbled] no RGB led on bus #1
WARN [rgbled_ncp5623c] no RGB led on bus #1
Board sensors: /etc/init.d/rc.board_sensors
WARN [mpu6000] no device on bus #3 (SPI1)
MPU6000 on SPI bus 1 at 0 (1000 KHz)
BMI055_ACCEL on SPI bus 1 at 3 (10000 KHz)
BMI055_GYRO on SPI bus 1 at 2 (10000 KHz)
INFO [ist8310] no device on bus 1
INFO [ist8310] no device on bus 2
WARN [hmc5883] no device on bus 1 (type: 2)
WARN [hmc5883] no device on bus 2 (type: 2)
WARN [hmc5883] no device on bus 4 (type: 2)
WARN [qmc5883] no device on bus 1 (type: 2)
WARN [qmc5883] no device on bus 2 (type: 2)
WARN [qmc5883] no device on bus 4 (type: 2)
WARN [lis3mdl] no device on bus 2
WARN [lis3mdl] no device on bus 2
WARN [lis3mdl] no device on bus 2
WARN [ak09916] AK09916 mag init failed
ERROR [ak09916] driver start failed
IST8310 on I2C bus 3 at 0x0e (bus: 100 KHz, max: 400 KHz)
MS5611_SPI on SPI bus 4 at 0 (20000 KHz)
ERROR [battery_status] no ADC found: /dev/adc0 (2)
Board extras: /etc/init.d/rc.board_mavlink
INFO [mavlink] mode: Config, data rate: 800000 B/s on /dev/ttyACM0 @ 57600B
Starting MAVLink on /dev/ttyS1
INFO [mavlink] mode: Normal, data rate: 1200 B/s on /dev/ttyS1 @ 57600B
Starting MAVLink on /dev/ttyS2
INFO [mavlink] mode: Onboard, data rate: 46080 B/s on /dev/ttyS2 @ 921600B
WARN [px4io] unknown argument:
INFO [px4io] default PWM output device
ERROR [commander] Manual kill-switch engaged
INFO [init] Mixer: /etc/mixers/quad_x.main.mix on /dev/pwm_output0
INFO [init] Mixer: /etc/mixers/pass.aux.mix on /dev/pwm_output1
INFO [logger] logger started (mode=all)
NuttShell (NSH)
nsh> e[KINFO [ecl/EKF] EKF aligned, (EV height, IMU buf: 14, OBS buf: 9)
INFO [ecl/EKF] EKF commencing external vision position fusion
INFO [ecl/EKF] EKF commencing external vision yaw fusion
INFO [logger] Start file log (type: full)
INFO [logger] Opened full log file: /fs/microsd/log/2021-04-16/20_51_24.ulg
INFO [logger] closed logfile, bytes written: 331907
ERROR [commander] Manual kill-switch disengaged
INFO [logger] Start file log (type: full)
INFO [logger] Opened full log file: /fs/microsd/log/2021-04-16/20_51_33.ulg
INFO [logger] closed logfile, bytes written: 288082
INFO [logger] Start file log (type: full)
INFO [logger] Opened full log file: /fs/microsd/log/2021-04-16/20_55_49.ulg
Pre Flight:
PID COMMAND CPU(ms) CPU(%) USED/STACK PRIO(BASE) STATE FD
0 Idle Task 129204 32.900 216/ 512 0 ( 0) READY 3
1 hpwork 0 0.000 344/ 1260 249 (249) w:sig 3
2 lpwork 271 0.000 1032/ 1516 50 ( 50) w:sig 8
3 init 924 0.000 2032/ 2604 100 (100) w:sem 3
4 wq:manager 0 0.000 408/ 1252 243 (243) w:sem 4
184 wq:att_pos_ctrl 22458 6.400 4768/ 6596 244 (244) w:sem 4
17 dataman 57 0.000 752/ 1180 90 ( 90) w:sem 4
20 wq:lp_default 240 0.000 812/ 1700 205 (205) READY 4
28 wq:hp_default 3836 1.200 1184/ 1596 243 (243) w:sem 4
139 wq:SPI1 73657 23.300 1056/ 1396 254 (254) w:sem 4
154 wq:I2C3 1174 0.400 820/ 1244 246 (246) w:sem 4
157 wq:SPI4 1131 0.400 600/ 1396 251 (251) w:sem 4
183 sensors 8437 2.400 1328/ 1964 237 (237) w:sem 9
690 log_writer_file 449 6.700 824/ 1164 60 ( 60) READY 4
187 commander 2742 0.700 2208/ 3212 140 (140) READY 6
190 wq:rate_ctrl 22036 6.400 1296/ 1596 255 (255) w:sem 4
191 commander_low_prio 5 0.000 576/ 2996 50 ( 50) w:sem 6
310 mavlink_if1 3987 1.100 1648/ 2484 100 (100) w:sig 4
313 mavlink_rcv_if1 1864 0.300 2800/ 3916 175 (175) w:sem 4
333 mavlink_if2 19151 6.000 1832/ 2492 100 (100) READY 4
334 mavlink_rcv_if2 10088 3.600 2784/ 3916 175 (175) w:sem 4
360 px4io 10623 3.400 1016/ 1484 240 (240) w:sem 4
645 navigator 278 0.100 896/ 1764 105 (105) w:sem 4
684 logger 1444 2.400 3080/ 3644 233 (233) RUN 4
Processes: 24 total, 6 running, 18 sleeping, max FDs: 20
CPU usage: 64.80% tasks, 2.30% sched, 32.90% idle
DMA Memory: 5120 total, 1536 used 1536 peak
Uptime: 320.949s total, 129.205s idle
Post Flight:
PID COMMAND CPU(ms) CPU(%) USED/STACK PRIO(BASE) STATE FD
0 Idle Task 134949 30.569 216/ 512 0 ( 0) READY 3
1 hpwork 0 0.000 344/ 1260 249 (249) w:sig 3
2 lpwork 311 2.897 1032/ 1516 50 ( 50) w:sig 8
3 init 924 0.000 2032/ 2604 100 (100) w:sem 3
4 wq:manager 0 0.000 408/ 1252 243 (243) w:sem 4
184 wq:att_pos_ctrl 23778 7.392 4768/ 6596 244 (244) w:sem 4
17 dataman 57 0.000 752/ 1180 90 ( 90) w:sem 4
20 wq:lp_default 244 0.000 812/ 1700 205 (205) w:sem 4
28 wq:hp_default 4056 1.198 1184/ 1596 243 (243) w:sem 4
139 wq:SPI1 77880 23.276 1056/ 1396 254 (254) w:sem 4
154 wq:I2C3 1242 0.399 820/ 1244 246 (246) w:sem 4
157 wq:SPI4 1196 0.399 600/ 1396 251 (251) w:sem 4
183 sensors 8889 2.697 1328/ 1964 237 (237) w:sem 9
690 log_writer_file 1730 5.494 824/ 1164 60 ( 60) w:sem 4
187 commander 2872 0.599 2208/ 3212 140 (140) w:sig 6
190 wq:rate_ctrl 23202 6.393 1296/ 1596 255 (255) w:sem 4
191 commander_low_prio 6 0.000 576/ 2996 50 ( 50) w:sem 6
310 mavlink_if1 4202 1.098 1672/ 2484 100 (100) w:sig 4
313 mavlink_rcv_if1 1923 0.299 2800/ 3916 175 (175) w:sem 4
333 mavlink_if2 20243 5.994 1832/ 2492 100 (100) READY 4
334 mavlink_rcv_if2 10704 3.396 2784/ 3916 175 (175) READY 4
360 px4io 11233 3.396 1016/ 1484 240 (240) w:sem 4
645 navigator 290 0.099 896/ 1764 105 (105) w:sem 4
684 logger 1877 2.397 3080/ 3644 233 (233) RUN 4
Processes: 24 total, 4 running, 20 sleeping, max FDs: 20
CPU usage: 67.43% tasks, 2.00% sched, 30.57% idle
DMA Memory: 5120 total, 1536 used 1536 peak
Uptime: 339.107s total, 134.949s idle