Strange magnetometer surges. MAG #0 fail: STALE!

Hello all,

there are strange magnetometer surges (1-3 seconds long) happen occasionally:

Log file link: click log_mag_jumps.ulg — Yandex.Disk

Firmware version: PX4 v1.10. Hardware: Pixracer; I use one external magnetometer (HERE2) and turn off the internal one.
These jumps are followed by the message
[sensors] MAG #0 fail: STALE!. During the “jumps”, mag indeed outputs a fixed value by all 3 axis:

I know that I haven’t provided much data nor steps to reproduce, and also this “bug” is very subtle. I am creating this post in hope that maybe someone has encountered the same issue. Any advice/questions are welcome.

Thanks for sharing. This does not look good. How did you configure the Here2? Is it used over I2C or UAVCAN?

Here2 connected over I2C. UAVCAN was disabled.

I have seen spikes with Here 2 over I2C but never timeouts like this. Have you checked that the I2C lines are not too long and not close to other wiring? Does it happen when you move the cables in a way?

In general they recommend to use the Here 2 over UAVCAN but I think this issue should tracked down nevertheless.

Some more data points that would be interesting:

  • Does it happen on other hardware, e.g. not with a Pixracer but something else?
  • Does anything else in the log file have these pauses?
  • Does it happen with other Here 2 hardware, or do you only have one?
  • Does it still happen with a fast (more expensive) SD card? (There could be some other effects be happening.)

Could you upload that log to please, so I can have a look? Thx.

Log file:

This will certainly require more time to investigate and answer all of your questions. I will keep this updated.

Thx. Alternatively, you could try 1.11 beta and see if it is fixed. There have been quite some changes to these drivers, so maybe you’ll be lucky.

I continue to see the these glitches. Some of them are П-shape like on the plots above, followed by MAG #0 fail: STALE! message, and some look like pikes:

Comparing the x-axis data to the corresponding INTERNAL mag:

@CarlOlsson can you please specify the model of magnetometer you use (is it here2?), do you connect it over I2C or UAVCAN and what is the voltage on your board?

This was HMC5883 over i2c

I have the same behavior.
My configuration is Pixracer R15, PX4-1.8.2, LIS3MDL internal magnetometer is used for priority and no external magnetometer is used.

While flying: Strange magnetometer surges happen! and errrormessage “MAG #0 fail: STALE!”
Then switch to another mag:
2:14:834 WARNING [sensors] Remaining sensors after failover event 0: Mag #0 priority: 1
2:14:834 WARNING [sensors] Remaining sensors after failover event 0: Mag #1 priority: 50

Did anyone handle this, If there are some bugs in the firmware driver please let me know.

PX4 v1.8 is very old. You should update to v1.13. Likely this problem has been resolved in the meantime.