PX4 Erase CAL_ACC, CAL_GYRO params

#1

Hello!

Have a great problem with CUAV Pixhack V3. When get cold such a strange bug appeared:

I make all the calibration process in warm conditions. Via QGC I see params for 2 accels and 2 gyros. All of them are enabled and ACC0 and GYRO0 defined as prime type. When I take copter to cold conditions 5-10 degrees, it shows in QGC in startup bad accelerometer and gyro health. For a long time I tried to calibrate them again, but each time the scenario was the same. Only than I found out, that all params in Sensor Calibration tab became empty. It seems, that on startup when sensors are cold and very inconsistent - something happens and erasing all calibration params.

I solved this problem by using only 1 accel and 1 gyro. Acc0 and Gyro0 are disabled and Acc1, Gyro1 ID selected as prime. But it’s strange solution. Can it be solved in firmware anyhow?

Thank you.
Sorry if my english is not very good.

#2

That’s strange. What does that look like? Do you have a screenshot?

That’s odd. Did you check the calibration parameters in the advanced param tab and check the values? And make sure to refresh them after a calibration to see if they have been calibrated and therefore changed?

#3

Yes, It’s very strange.

You can see screenshots. At first it was calibrated. Params are written in it’s places.
After that I put it in cold for a few minutes and than start agai. You can see all params are zero. And thats all.

I have 5 bords. And all of them work the same.

#4

Odd. @david_s5 do you know more about how the params are stored on CUAV Pixhack V3?

#5

I found out one interesting moment now!

Calibration parameters are not erased! Only CAL_ACC_ID is erased. If I put back CAL_ACC0_ID and CAL_ACC1_ID and restart - it shows back all the calibration!

I dont understand, how it works.

#6

@Takeshi Can you capture and post the boot output from the console for cold and warm?

#7

@JulianOes I am wondering if the board ID is not being detecting unit correctly at boot. V2,V3 detection was and is a hack. We added Pixhack V3 on top of it. There is a dependency on sensors power sequencing that got removed at one point because of the boards that bootlooped. This all my need to be revisited.

1 Like
#8

Please, explain me how to log boot from console… or what you mean. I can do it from QGC?

#9

@Takeshi see https://dev.px4.io/en/debug/system_console.html

#10

WARM START:

FMUv2 ver 0xA : Rev 0 V2M
[boot] Fault Log info File No 4 Length 3177 flags:0x01 state:1
[boot] Fault Log is Armed
sercon: Registering CDC/ACM serial driver
sercon: Successfully registered the CDC/ACM serial driver
HW arch: PX4FMU_V2
HW type: V2M
HW version: 0x0009000A
HW revision: 0x00000000
FW git-hash: 82aa24adfca29321cfd1209e287eab6c2b16780e
FW version: Release 1.8.1 (17302015)
OS: NuttX
OS version: Release 7.22.0 (118882559)
OS git-hash: 63775322bf25adb406594f8e610122fe0cef2f7a
Build datetime: Oct 20 2018 00:19:12
Build uri: BUILD_URI
Toolchain: GNU GCC, 7.2.1 20170904 (release) [ARM/embedded-7-branch revision 255204]
MFGUID: 3834373233355106002d0031
MCU: STM32F42x, rev. 3
UID: 2D0031:33355106:38343732
[hardfault_log] Fault Log is Armed
INFO [tune_control] Publishing standard tune 1
INFO [param] selected parameter default file /fs/mtd_params
rgbled on I2C bus 2 at 0x55 (bus: 100 KHz, max: 100 KHz)
nsh: rgbled_pwm: command not found
MS5611_SPI on SPI bus 4 at 3 (20000 KHz)
WARN [ms5611] no device on bus 4
MS5611_SPI on SPI bus 1 at 3 (20000 KHz)
WARN [bst] no devices found
WARN [hmc5883] no device on bus 1 (type: 2)
WARN [lis3mdl] no device on bus 2
WARN [hmc5883] no device on bus 2 (type: 1)
WARN [mpu6000] no device on bus #3 (SPI1)
MPU6000 on SPI bus 4 at 4 (1000 KHz)
INFO [mpu6000] accel cutoff set to 30.00 Hz
INFO [mpu6000] gyro cutoff set to 30.00 Hz
ERROR [l3gd20] driver start failed
WARN [lsm303d] SPI init failed
ERROR [lsm303d] driver start failed
MPU6000 on SPI bus 1 at 4 (1000 KHz)
INFO [mpu6000] accel cutoff set to 30.00 Hz
INFO [mpu6000] gyro cutoff set to 30.00 Hz
HMC5883_SPI on SPI bus 1 at 5 (11000 KHz)
WARN [hmc5883] no device on bus 1 (type: 3)
INFO [load_mon] stack check enabled
INFO [px4io] default PWM output device
INFO [mavlink] mode: Normal, data rate: 1200 B/s on /dev/ttyS1 @ 57600B
INFO [mavlink] mode: OSD, data rate: 1000 B/s on /dev/ttyS2 @ 57600B
px4flow [223:100]
WARN [px4flow] scanning I2C buses for device…
INFO [mavlink] mode: Config, data rate: 800000 B/s on /dev/ttyACM0 @ 57600B
INFO [init] Mixer: /etc/mixers/hexa_x.main.mix on /dev/pwm_output0
INFO [init] Mixer: /etc/mixers/mount.aux.mix on /dev/pwm_output1
INFO [logger] logger started (mode=all)

COLD START:

FMUv2 ver 0xA : Rev 0 V2M
[boot] Fault Log info File No 4 Length 3177 flags:0x01 state:1
[boot] Fault Log is Armed
sercon: Registering CDC/ACM serial driver
sercon: Successfully registered the CDC/ACM serial driver
HW arch: PX4FMU_V2
HW type: V2M
HW version: 0x0009000A
HW revision: 0x00000000
FW git-hash: 82aa24adfca29321cfd1209e287eab6c2b16780e
FW version: Release 1.8.1 (17302015)
OS: NuttX
OS version: Release 7.22.0 (118882559)
OS git-hash: 63775322bf25adb406594f8e610122fe0cef2f7a
Build datetime: Oct 20 2018 00:19:12
Build uri: BUILD_URI
Toolchain: GNU GCC, 7.2.1 20170904 (release) [ARM/embedded-7-branch revision 255 204]
MFGUID: 3834373233355106002d0031
MCU: STM32F42x, rev. 3
UID: 2D0031:33355106:38343732
[hardfault_log] Fault Log is Armed
INFO [tune_control] Publishing standard tune 1
INFO [param] selected parameter default file /fs/mtd_params
rgbled on I2C bus 2 at 0x55 (bus: 100 KHz, max: 100 KHz)
nsh: rgbled_pwm: command not found
MS5611_SPI on SPI bus 4 at 3 (20000 KHz)
WARN [ms5611] no device on bus 4
MS5611_SPI on SPI bus 1 at 3 (20000 KHz)
WARN [bst] no devices found
WARN [hmc5883] no device on bus 1 (type: 2)
WARN [lis3mdl] no device on bus 2
WARN [hmc5883] no device on bus 2 (type: 1)
WARN [mpu6000] no device on bus #3 (SPI1)
MPU6000 on SPI bus 4 at 4 (1000 KHz)
INFO [mpu6000] accel cutoff set to 30.00 Hz
INFO [mpu6000] gyro cutoff set to 30.00 Hz
ERROR [l3gd20] driver start failed
WARN [lsm303d] SPI init failed
ERROR [lsm303d] driver start failed
MPU6000 on SPI bus 1 at 4 (1000 KHz)
INFO [mpu6000] accel cutoff set to 30.00 Hz
INFO [mpu6000] gyro cutoff set to 30.00 Hz
HMC5883_SPI on SPI bus 1 at 5 (11000 KHz)
WARN [hmc5883] no device on bus 1 (type: 3)
INFO [load_mon] stack check enabled
INFO [px4io] default PWM output device
INFO [mavlink] mode: Normal, data rate: 1200 B/s on /dev/ttyS1 @ 57600B
INFO [mavlink] mode: OSD, data rate: 1000 B/s on /dev/ttyS2 @ 57600B
px4flow [223:100]
WARN [px4flow] scanning I2C buses for device…
INFO [mavlink] mode: Config, data rate: 800000 B/s on /dev/ttyACM0 @ 57600B
INFO [init] Mixer: /etc/mixers/hexa_x.main.mix on /dev/pwm_output0
INFO [init] Mixer: /etc/mixers/mount.aux.mix on /dev/pwm_output1
INFO [logger] logger started (mode=all)

NuttShell (NSH)
nsh> WARN [ekf2] accel id changed, resetting IMU bias
WARN [ekf2] gyro id changed, resetting IMU bias
ERROR [sensors] Accel #0 fail: TIMEOUT!
ERROR [sensors] Sensor Accel #0 failed. Reconfiguring sensor priorities.
WARN [sensors] Remaining sensors after failover event 0: Accel #0 priority: 1
WARN [sensors] Remaining sensors after failover event 0: Accel #1 priority: 255
ERROR [sensors] Gyro #0 fail: TIMEOUT!
ERROR [sensors] Sensor Gyro #0 failed. Reconfiguring sensor priorities.
WARN [sensors] Remaining sensors after failover event 0: Gyro #0 priority: 1
WARN [sensors] Remaining sensors after failover event 0: Gyro #1 priority: 255
WARN [commander_tests] PREFLIGHT FAIL: NO MAG SENSOR #0

1 Like
#11

@Takeshi Thank you!

Well it may be the board ID, but it is not changing from COLD to WARM. Nothig listed in the boot is.

However,

The code you are running is tag: v1.8.1 Date: Wed Jul 25 13:18:10 2018 -0400

in August this PR https://github.com/PX4/Firmware/pull/10231 was added that addressed the board ID issue.

Would you please test current master and see if the problem persists. If it does, please post the boot output from the console here as well.

1 Like
#13

Great! It works good! It seems, the problem is solved! Thank you for your help.

I will put boot log here for exapmle. May be it will be usefull.

WARM START:

FMUv2 ver 0xA : Rev 0 V2M
[boot] Fault Log info File No 4 Length 3177 flags:0x01 state:1
[boot] Fault Log is Armed
sercon: Registering CDC/ACM serial driver
sercon: Successfully registered the CDC/ACM serial driver
HW arch: PX4FMU_V2
HW type: V2M
HW version: 0x0009000A
HW revision: 0x00000000
FW git-hash: 82aa24adfca29321cfd1209e287eab6c2b16780e
FW version: Release 1.8.1 (17302015)
OS: NuttX
OS version: Release 7.22.0 (118882559)
OS git-hash: 63775322bf25adb406594f8e610122fe0cef2f7a
Build datetime: Oct 20 2018 00:19:12
Build uri: BUILD_URI
Toolchain: GNU GCC, 7.2.1 20170904 (release) [ARM/embedded-7-branch revision 255204]
MFGUID: 3834373233355106002d0031
MCU: STM32F42x, rev. 3
UID: 2D0031:33355106:38343732
[hardfault_log] Fault Log is Armed
INFO [tune_control] Publishing standard tune 1
INFO [param] selected parameter default file /fs/mtd_params
rgbled on I2C bus 2 at 0x55 (bus: 100 KHz, max: 100 KHz)
nsh: rgbled_pwm: command not found
MS5611_SPI on SPI bus 4 at 3 (20000 KHz)
WARN [ms5611] no device on bus 4
MS5611_SPI on SPI bus 1 at 3 (20000 KHz)
WARN [bst] no devices found
WARN [hmc5883] no device on bus 1 (type: 2)
WARN [lis3mdl] no device on bus 2
WARN [hmc5883] no device on bus 2 (type: 1)
WARN [mpu6000] no device on bus #3 (SPI1)
MPU6000 on SPI bus 4 at 4 (1000 KHz)
INFO [mpu6000] accel cutoff set to 30.00 Hz
INFO [mpu6000] gyro cutoff set to 30.00 Hz
ERROR [l3gd20] driver start failed
WARN [lsm303d] SPI init failed
ERROR [lsm303d] driver start failed
MPU6000 on SPI bus 1 at 4 (1000 KHz)
INFO [mpu6000] accel cutoff set to 30.00 Hz
INFO [mpu6000] gyro cutoff set to 30.00 Hz
HMC5883_SPI on SPI bus 1 at 5 (11000 KHz)
WARN [hmc5883] no device on bus 1 (type: 3)
INFO [load_mon] stack check enabled
INFO [px4io] default PWM output device
INFO [mavlink] mode: Normal, data rate: 1200 B/s on /dev/ttyS1 @ 57600B
INFO [mavlink] mode: OSD, data rate: 1000 B/s on /dev/ttyS2 @ 57600B
px4flow [223:100]
WARN [px4flow] scanning I2C buses for device…
INFO [mavlink] mode: Config, data rate: 800000 B/s on /dev/ttyACM0 @ 57600B
INFO [init] Mixer: /etc/mixers/hexa_x.main.mix on /dev/pwm_output0
INFO [init] Mixer: /etc/mixers/mount.aux.mix on /dev/pwm_output1
INFO [logger] logger started (mode=all)

NuttShell (NSH)
nsh>

COLD START:

FMUv2 ver 0xA : Rev 0 V2M
[boot] Fault Log info File No 4 Length 3177 flags:0x01 state:1
[boot] Fault Log is Armed
sercon: Registering CDC/ACM serial driver
sercon: Successfully registered the CDC/ACM serial driver
HW arch: PX4FMU_V2
HW type: V2M
HW version: 0x0009000A
HW revision: 0x00000000
FW git-hash: 82aa24adfca29321cfd1209e287eab6c2b16780e
FW version: Release 1.8.1 (17302015)
OS: NuttX
OS version: Release 7.22.0 (118882559)
OS git-hash: 63775322bf25adb406594f8e610122fe0cef2f7a
Build datetime: Oct 20 2018 00:19:12
Build uri: BUILD_URI
Toolchain: GNU GCC, 7.2.1 20170904 (release) [ARM/embedded-7-branch revision 255 204]
MFGUID: 3834373233355106002d0031
MCU: STM32F42x, rev. 3
UID: 2D0031:33355106:38343732
[hardfault_log] Fault Log is Armed
INFO [tune_control] Publishing standard tune 1
INFO [param] selected parameter default file /fs/mtd_params
rgbled on I2C bus 2 at 0x55 (bus: 100 KHz, max: 100 KHz)
nsh: rgbled_pwm: command not found
MS5611_SPI on SPI bus 4 at 3 (20000 KHz)
WARN [ms5611] no device on bus 4
MS5611_SPI on SPI bus 1 at 3 (20000 KHz)
WARN [bst] no devices found
WARN [hmc5883] no device on bus 1 (type: 2)
WARN [lis3mdl] no device on bus 2
WARN [hmc5883] no device on bus 2 (type: 1)
WARN [mpu6000] no device on bus #3 (SPI1)
MPU6000 on SPI bus 4 at 4 (1000 KHz)
INFO [mpu6000] accel cutoff set to 30.00 Hz
INFO [mpu6000] gyro cutoff set to 30.00 Hz
ERROR [l3gd20] driver start failed
WARN [lsm303d] SPI init failed
ERROR [lsm303d] driver start failed
MPU6000 on SPI bus 1 at 4 (1000 KHz)
INFO [mpu6000] accel cutoff set to 30.00 Hz
INFO [mpu6000] gyro cutoff set to 30.00 Hz
HMC5883_SPI on SPI bus 1 at 5 (11000 KHz)
WARN [hmc5883] no device on bus 1 (type: 3)
adc sample timeout
INFO [load_mon] stack check enabled
INFO [px4io] default PWM output device
INFO [mavlink] mode: Normal, data rate: 1200 B/s on /dev/ttyS1 @ 57600B
INFO [mavlink] mode: OSD, data rate: 1000 B/s on /dev/ttyS2 @ 57600B
px4flow [223:100]
WARN [px4flow] scanning I2C buses for device…
INFO [mavlink] mode: Config, data rate: 800000 B/s on /dev/ttyACM0 @ 57600B
INFO [init] Mixer: /etc/mixers/hexa_x.main.mix on /dev/pwm_output0
INFO [init] Mixer: /etc/mixers/mount.aux.mix on /dev/pwm_output1
INFO [logger] logger started (mode=all)

NuttShell (NSH)
nsh> WARN [commander_tests] PREFLIGHT FAIL: NO MAG SENSOR #0

1 Like
#14

Not great (((

Put in cold for a long time ( - 7 C aprox) - the same situation on start. CAL_ACC0_ID = 0, CAL_ACC1_ID = 0. The same for Gyro.

Nothing has changed finally.

#15

Interesting thing… If I start COLD with serial wire connected ang logging in console - it starts well. Nothing erases from params. After I switch off serial wire and start it only with usb cable - bad things happen.

I also noticed, that when touching usb connector a little - Warning appears ( Accel 0 Time Out, Gyro 0 Time Out)… than disappear in some time.

#16

@Takeshi is it running master? I aks because the git has on both posts is the same 82aa24adfca29321cfd1209e287eab6c2b16780e and the board is still reporting as a mini “FMUv2 ver 0xA : Rev 0 V2M”

The issue you are having may not be related, but we need to establish a good baseline first

#17

Fw can be fetched with QGC (I think it may be labled 1.8xxx dev) or from
http://ci.px4.io:8080/blue/organizations/jenkins/PX4_misc%2FFirmware-compile/detail/master/453/artifacts/

#18

I updated QGC to latest version and fetch Fw from it. It is labled as 1.8.2 Stable there. I can upload custom file from your link. But I don’t know what exactly to use.

I thought the problem was solved because of start up with serial cable connected. It’s no problem with it. Don’t know why.

#19

You have to tick Advanced, then selected “Developer Build”

#20

Yes, this time works better! It seems, that the board is recognized correctly and all sensors are good!

Thank you very much! You saved all my project!

For information:

WARM START:

Pixhack V3 detected, forcing to fmu-v3
FMUv2 ver 0xE : Rev 0 V30
[boot] Fault Log info File No 4 Length 3177 flags:0x01 state:1
[boot] Fault Log is Armed
sercon: Registering CDC/ACM serial driver
sercon: Successfully registered the CDC/ACM serial driver
HW arch: PX4_FMU_V3
HW type: V30
HW version: 0x0009000E
HW revision: 0x00000000
FW git-hash: fce35ba9d27c33cc65e1215682785debbd99d361
FW version: 1.9.0 40 (17367104)
OS: NuttX
OS version: Release 7.22.0 (118882559)
OS git-hash: d07a1d459ee9b5b56dbf06a160e8faa4c49fcd10
Build datetime: Dec 19 2018 17:29:16
Build uri: localhost
Toolchain: GNU GCC, 7.2.1 20170904 (release) [ARM/embedded-7-branch revision 255 204]
PX4GUID: 0001000000003131363933375106004e0039
MCU: STM32F42x, rev. 3
[hardfault_log] Fault Log is Armed
INFO [tune_control] Publishing standard tune 1
INFO [dataman] Unknown restart, data manager file ‘/fs/microsd/dataman’ size is 350216 bytes
INFO [load_mon] stack check enabled
INFO [param] selected parameter default file /fs/mtd_params
rgbled on I2C bus 2 at 0x55 (bus: 100 KHz, max: 100 KHz)
WARN [rgbled_ncp5623c] no RGB led on bus #2
nsh: rgbled_pwm: command not found
Board init: /etc/init.d/rc.board
WARN [hmc5883] no device on bus 1 (type: 2)
WARN [lis3mdl] no device on bus 2
INFO [ist8310] no device on bus 1
INFO [ist8310] no device on bus 5
WARN [hmc5883] no device on bus 2 (type: 1)
WARN [mpu6000] no device on bus #3 (SPI1)
MPU6000 on SPI bus 4 at 4 (1000 KHz)
L3GD20 on SPI bus 4 at 1 (11000 KHz)
LSM303D on SPI bus 4 at 2 (11000 KHz)
MPU6000 on SPI bus 1 at 4 (1000 KHz)
HMC5883_SPI on SPI bus 1 at 5 (11000 KHz)
WARN [hmc5883] no device on bus 1 (type: 3)
px4flow [40:100]
INFO [px4flow] scanning I2C buses for device…
MS5611_SPI on SPI bus 4 at 3 (20000 KHz)
MS5611_SPI on SPI bus 1 at 3 (20000 KHz)
WARN [bst] no devices found
INFO [mpu9250] Bus probed: 2
WARN [mpu9250] no device on bus 2
INFO [mavlink] mode: Config, data rate: 800000 B/s on /dev/ttyACM0 @ 57600B
Starting Main GPS on /dev/ttyS3
Starting MAVLink on /dev/ttyS1
INFO [mavlink] mode: Normal, data rate: 1200 B/s on /dev/ttyS1 @ 57600B
INFO [px4io] default PWM output device
INFO [init] Mixer: /etc/mixers/hexa_x.main.mix on /dev/pwm_output0
INFO [init] Mixer: /etc/mixers/mount.aux.mix on /dev/pwm_output1
INFO [logger] logger started (mode=all)

NuttShell (NSH)
nsh> WARN [commander] Preflight Fail: Compass #0 uncalibrated
INFO [ecl/EKF] EKF aligned, (pressure height, IMU buf: 22, OBS buf: 14)

COLD START:

Pixhack V3 detected, forcing to fmu-v3
FMUv2 ver 0xE : Rev 0 V30
[boot] Fault Log info File No 4 Length 3177 flags:0x01 state:1
[boot] Fault Log is Armed
sercon: Registering CDC/ACM serial driver
sercon: Successfully registered the CDC/ACM serial driver
HW arch: PX4_FMU_V3
HW type: V30
HW version: 0x0009000E
HW revision: 0x00000000
FW git-hash: fce35ba9d27c33cc65e1215682785debbd99d361
FW version: 1.9.0 40 (17367104)
OS: NuttX
OS version: Release 7.22.0 (118882559)
OS git-hash: d07a1d459ee9b5b56dbf06a160e8faa4c49fcd10
Build datetime: Dec 19 2018 17:29:16
Build uri: localhost
Toolchain: GNU GCC, 7.2.1 20170904 (release) [ARM/embedded-7-branch revision 255 204]
PX4GUID: 0001000000003131363933375106004e0039
MCU: STM32F42x, rev. 3
[hardfault_log] Fault Log is Armed
INFO [tune_control] Publishing standard tune 1
INFO [dataman] Unknown restart, data manager file ‘/fs/microsd/dataman’ size is 350216 bytes
INFO [load_mon] stack check enabled
INFO [param] selected parameter default file /fs/mtd_params
rgbled on I2C bus 2 at 0x55 (bus: 100 KHz, max: 100 KHz)
WARN [rgbled_ncp5623c] no RGB led on bus #2
nsh: rgbled_pwm: command not found
Board init: /etc/init.d/rc.board
WARN [hmc5883] no device on bus 1 (type: 2)
WARN [lis3mdl] no device on bus 2
INFO [ist8310] no device on bus 1
INFO [ist8310] no device on bus 5
WARN [hmc5883] no device on bus 2 (type: 1)
WARN [mpu6000] no device on bus #3 (SPI1)
MPU6000 on SPI bus 4 at 4 (1000 KHz)
L3GD20 on SPI bus 4 at 1 (11000 KHz)
LSM303D on SPI bus 4 at 2 (11000 KHz)
MPU6000 on SPI bus 1 at 4 (1000 KHz)
HMC5883_SPI on SPI bus 1 at 5 (11000 KHz)
WARN [hmc5883] no device on bus 1 (type: 3)
px4flow [40:100]
INFO [px4flow] scanning I2C buses for device…
MS5611_SPI on SPI bus 4 at 3 (20000 KHz)
MS5611_SPI on SPI bus 1 at 3 (20000 KHz)
WARN [bst] no devices found
INFO [mpu9250] Bus probed: 2
WARN [mpu9250] no device on bus 2
INFO [mavlink] mode: Config, data rate: 800000 B/s on /dev/ttyACM0 @ 57600B
Starting Main GPS on /dev/ttyS3
Starting MAVLink on /dev/ttyS1
INFO [mavlink] mode: Normal, data rate: 1200 B/s on /dev/ttyS1 @ 57600B
INFO [px4io] default PWM output device
INFO [init] Mixer: /etc/mixers/hexa_x.main.mix on /dev/pwm_output0
INFO [init] Mixer: /etc/mixers/mount.aux.mix on /dev/pwm_output1
INFO [logger] logger started (mode=all)

NuttShell (NSH)
nsh> INFO [ecl/EKF] EKF aligned, (pressure height, IMU buf: 22, OBS buf: 14)
WARN [commander] Preflight Fail: Compass #0 uncalibrated

#21

@Takeshi - Great to hear!

Now that we have a proper HW detection, we have a good baseline to test with.

Given what you noted previously on the the effects of temperature, it would be good to do some more temperature based testing to rule out any HW issues.