[Release] AI Flight Integrity Observer: Measuring PX4 Offboard degradation under controlled AI inference lag

PX4 Autopilot
1

Hi PX4 community,

I’d like to share AI Flight Integrity Observer (AFIO), an open-source ROS 2 / PX4 runtime assurance observer for Offboard autonomy.

AFIO is designed to make one failure surface visible:

When a companion computer is running VIO, visual tracking, neural planning, or heavy perception workloads, is PX4 still receiving fresh, temporally consistent, physically realizable Offboard intent?

It does not modify PX4, replace failsafe logic, publish recovery commands, or intercept control. It is observe-only.

Why this exists

A. Degraded communication / RF / telemetry link volatility

In degraded communication environments, the Offboard stream may not simply disappear. It may become bursty, stale, or highly jittered while still producing syntactically valid messages.

A flight controller may still receive /fmu/in/trajectory_setpoint, but the control intent may no longer be temporally healthy.

AFIO exposes this through standard ROS diagnostics:

setpointAgeMs
setpointJitterMs
staleStreams
causalAlignment
dominantCause

These signals can be consumed by external autonomy managers, dashboards, or fail-safe state machines.

B. Edge-AI / vision stack overload

VIO, object detection, tracking, neural planning, mapping, and semantic navigation can create bursty companion-compute load.

When inference latency spikes, the Offboard setpoint publisher can become delayed or irregular even while PX4 itself remains healthy.

AFIO detects this as:

SETPOINT_JITTER
SETPOINT_STALE
STALE_STREAM
POSITION_RESPONSE_MISMATCH
COMMAND_RESPONSE_MISMATCH

How AFIO works

AFIO sits as a passive observer on the ROS 2 / PX4 boundary.

It subscribes to:

/fmu/in/trajectory_setpoint
/fmu/in/offboard_control_mode
/fmu/out/vehicle_odometry

It publishes:

/diagnostics
  ai_flight_integrity/flight_execution_integrity

CSV labels
  setpointAgeMs
  setpointJitterMs
  flightResidual
  totalResidual
  dominantCause
  diagnosticLevelName

The core is a NARH-inspired boundary residual observer. NARH here means Non-Associative Residual Hypothesis. AFIO does not claim to inspect PX4’s internal solver order. Instead, it applies the residual-auditing idea at the Offboard boundary: intent stream, feedback stream, stream freshness, and Offboard semantic mode are evaluated together.

Controlled AI-lag injection test

Real UAV hardware stacks vary widely: ELRS / 4G / 5G / WiFi links, Jetson Nano / NX / Orin, RK3588, NPU boards, DDS settings, PX4 versions, and different Offboard planners.

A single real-drone benchmark is useful, but not broadly generalizable by itself. So AFIO includes a decoupled ai_latency_injector_node for controlled-variable testing.

The observer does not read the injector state. It has no access to ai_lag_ms. It only sees the PX4-facing boundary topics.

In a Gazebo / PX4 SITL circular tracking mission:

Injected AI lag Outcome Dominant cause setpointJitterMs p95 flightResidual p95
0 ms 100% OK 100% NONE 50.9 ms 0.214
30 ms 100% OK 100% NONE 57.2 ms 0.237
80 ms 100% OK 100% NONE 64.4 ms 0.264
150 ms 100% WARN 96.9% SETPOINT_JITTER 119.0 ms 0.483
300 ms WARN + ERROR SETPOINT_JITTER / STALE_STREAM 277.2 ms 1.107

Interpretation:

0–80 ms:
  The system remains aligned. AFIO stays quiet.

150 ms:
  The Offboard boundary enters a measurable degraded state.
  AFIO emits SETPOINT_JITTER before a hard failure.

300 ms:
  The Offboard stream crosses a freshness boundary.
  AFIO reports STALE_STREAM / RESYNCING through standard ROS diagnostics.

What I’d like feedback on

For PX4 / ROS users running VIO, visual tracking, neural planners, or heavy perception on companion computers:

  1. How do you currently detect temporary DDS jitter, companion-compute overload, or Jetson/RK/NPU thermal throttling during Offboard autonomy?
  2. Would a standard /diagnostics + CSV label format be useful for regression testing and post-flight review?

Links:

Open Source Robotics Foundation: https://discourse.openrobotics.org/t/release-ai-flight-integrity-observer-measuring-px4-offboard-degradation-under-controlled-ai-inference-lag/55530?u=zc_liu

GitHub: GitHub - ZC502/ai_flight_integrity_observer: A zero-intrusive ROS 2 / PX4 runtime assurance observer for Offboard autonomy under degraded AI compute and communication conditions. · GitHub

  • License: Apache-2.0

  • Compatibility: ROS 2 Humble / Jazzy + PX4 v1.14 / v1.15

2

Thanks for sharing this, and for open sourcing it. A few things in here are genuinely well done. The Best Effort / Volatile QoS note is the kind of thing only someone who has actually been bitten by the PX4 bridge writes, and the px4_msgs branch and dds_topics.yaml caveats, plus the PX4 monotonic vs companion clock timesync warning, are all correct and useful. The core idea, a passive observer that watches setpoint freshness, jitter, and tracking residual at the offboard boundary, is a real and useful piece of runtime assurance.

I want to offer one honest piece of feedback in good faith. I went looking for the AI part and could not find it. There is no model, no weights, and no training step. The detection is a weighted sum of timing and tracking residuals compared against fixed thresholds, which is classic rule based fault detection and perfectly fine on its own. The loadProfile / cpuLoadPercent / aiInferenceLatencyMs fields are, by your own README, manually supplied parameters, and the latency injector is a synthetic delay rather than inference. So calling it as AI based actually undersells how solid the watchdog is, while inviting scrutiny that distracts from the good engineering underneath.

If it was mine I would rename it something like Offboard Boundary Integrity Observer and let the engineering speak for itself. It is good enough that it does not need the framing.

Thanks again for putting it out there. We appreciate it.

3

Thanks Farhang — this is very fair feedback, and I appreciate the careful read.​

You are right: the observer itself is not an AI model, The core is a deterministic runtime assurance observer: setpoint freshness, jitter, stream staleness, Offboard mode semantics, and tracking residuals are combined into a residual-based diagnostic layer.​
The “AI” in the original naming was meant to refer to the upstream autonomy workload being audited — VIO, visual tracking, neural planning, perception inference, companion-compute load, etc. — not to the mechanism of the observer itself. But I agree that this is ambiguous and can distract from the actual engineering.​ I’ll update the README to make this explicit .​

The latency injector is also intended as a controlled-variable test harness, not an inference benchmark. The goal was to isolate the PX4 Offboard boundary response to timing degradation before substituting real VIO / YOLO / tracking workloads as upstream delay sources.​

Thanks again — this is exactly the kind of feedback I was hoping to get from the PX4 community.

4

AFIO sounds quite intriguing. for real-world applications, different lag conditions can significantly influence performance. it reminds me of now commercial air pilot training adjusts to different flying conditions. i recently experienced their training and it’s top-notch for understanding and adapting to various scenarios in aviation. maybe aligning some AFIO tests with such adaptive training ideas could be beneficial