Pixhawk 4 temperature compensation failure

I’ve performed the off-board temperature compensation on a Pixhawk 2 and a Pixracer without any problems. However, now I’m trying to run an off-board temperature compensation on a Holybro Pixhawk 4. I’ve collected the log, and it appears to be a valid log… but when I pass it to process_sensor_caldata.py, I get an error. I’ve tried several different logs, and they all seem to report the same error.

found gyro 0 data
found gyro 1 data
found accel 0 data
found accel 1 data
found baro 0 data
/home/user/.local/lib/python2.7/site-packages/numpy/lib/polynomial.py:627: RuntimeWarning: invalid value encountered in true_divide
lhs /= scale
Traceback (most recent call last):
File “process_sensor_caldata.py”, line 242, in
coef_gyro_1_x = np.polyfit(temp_rel,sensor_gyro_1[‘x’],3)
File “/home/user/.local/lib/python2.7/site-packages/numpy/lib/polynomial.py”, line 628, in polyfit
c, resids, rank, s = lstsq(lhs, rhs, rcond)
File “/home/user/.local/lib/python2.7/site-packages/numpy/linalg/linalg.py”, line 2236, in lstsq
x, resids, rank, s = gufunc(a, b, rcond, signature=signature, extobj=extobj)
ValueError: On entry to DLASCL parameter number 4 had an illegal value

Anyone else seeing this problem?

Incidentally, I noticed that when I get this error, a partial PDF file is getting generated but it’s invalid and won’t open.

Any chance you can link to the log that doesn’t work and even better also to one that works? That would help a lot to debug the issue.

Sure, here are the log links:

Pixracer:
https://logs.px4.io/plot_app?log=0ff02a24-b765-410b-870c-ffc294d9ebfb

Pixhawk 4:
https://logs.px4.io/plot_app?log=19623805-517a-43ae-b5e6-c304f93415dd

A little background, both of these logs were captured from the stock flight controllers, with only the modifications specified in the temp compensation docs here:
https://docs.px4.io/en/advanced_config/sensor_thermal_calibration.html
Both flight controllers were placed in an environmental chamber at the same time, and the temperature was run from -40C to 60C during the log capture.

The PX4 flight review works for both logs, but the Pixhawk 4 log gets an error when I pass it to the “process_sensor_caldata.py” python script.

Thanks for your help.

Thanks for the quick response, I’ll try to find out what happens :+1:

MaEtUgR- I found that if I modify the python script to ignore Gyro #1, then it completes without any errors, and I’m able to view the PDF results for the remaining sensors. It seems that there’s some bad data in the Gyro #1 series that is not handled very well by the python script.

Good finding! Sorry I didn’t get to it earlier. So then there might be a problem with gyro logging because you weren’t able to use it now. Do you notice any apparent problem with the gyro data series? And there needs to be exception handling in the python script to catch that kind of problem and inform the user.

Gyro 0 looks fine on the data plot, and gyro 1 is the one I had to ignore in the python code. Accel 0 looks fine, but accel 1 has only a few fragments of data at a very small temperature range, most of the data is missing. Finally, baro 0 data plot works fine.

Flight review here. Keep in mind that the Pixhawk 4 was sitting stationary in a temperature chamber during the entire test as the temperature was ramped from -40 C to +60 C.
https://review.px4.io/plot_app?log=d851b073-02a9-4908-ad19-c458d99e3e21

Here’s the PDF result of my temp compensation test.

Apparently the Pixhawk 4 is generating bad data in the log for Gyro 1 and Accel 1. When viewing the flight review log, there appears to be some clipping on the “Roll Angular Rate”, this might be related to the bad data. As near as I can tell, Gyro/Accel 0 is the ICM20689, and Gyro/Accel 1 is the BMI055. If this is true, then the bad data appears to be coming from the BMI055 driver, perhaps it’s just a logging bug?

In summary- The python script doesn’t have an exception handler for bad data, but this is a secondary issue. The bigger problem is that the Pixhawk 4 generates bad data in the log for Accel/Gyro 1

Can you plot the temperatures of the different sensors with Flightplot? https://docs.px4.io/en/log/flight_log_analysis.html#flightplot-desktop

Thanks for the suggestion Lorenz, I tried Flightplot with the Pixhawk 4 log, and everything seems to be working normally. I didn’t see any problems with the data. I wonder why the python script for temp compensation chokes on it then.

@MaEtUgR @LorenzMeier I have made some progress on this issue. I used the pyulog library to read the log files of both the pixracer and the pixhawk 4, and convert them to CSV. Then I loaded them into a spreadsheet, and found out that the temperature values associated with the IMU on the Pixracer look fine, but the temperature values on the Pixhawk 4 are bad for Accel/Gyro #1 (I believe it’s the BMI055). For Gyro #1, the temperature is always 0, and for Accel #1, the temperature fluctuates between 23.00 and 23.99 for the entire test. I probably could have discovered this with Flightplot, but at the time I didn’t know what I was looking for.

This explains why the temperature compensation script chokes on this data. I believe this points to a bug in the BMI055 driver where it is not reading or logging temperature correctly.

1 Like

@MaEtUgR @LorenzMeier
I’ve created an issue for this:

I may have a fix for the driver code shortly.

Nice finding, thanks for the report. So do I summarize that correctly?

  • The external temperature calibration on the pixhawk 4 doesn’t work
  • The python script crashes because the gyroscope(/accelerometer) #1 data sequence is erroneous
  • The erroneous sequence is the temperature reported by the BMI055

Hence the python script should output a useful error message such that if it ever happens again the user knows what failed immediately and the reported temperature of the BMI055 needs to be corrected (tracked in https://github.com/PX4/Firmware/issues/11854).

@MaEtUgR Your summary is correct. My PR with the BMI055 driver fix was accepted, and it will be included in the upcoming 1.9.0 release.

Perfect, thanks a lot for your contribution! :thumbsup: And sorry I couldn’t help you immediately.
Link to PR: https://github.com/PX4/Firmware/pull/11867