Auto-pilot navigation for drones has evolved beyond merely tracing predefined GPS waypoints. In this era of anticipated UAV deployment in uncontrolled environments, the UAV has to comprehend the dynamic environment, detect objects, maintain a consistent position relative to that environment, and take adequate compensatory measures when the environment deviates from the planned path. In my opinion, this is an important application where AI can add value. However, it raises a big engineering question: What operations would be controlled by AI and which operations by the deterministic flight-control logic?
From Predefined Routes to Adaptive Navigation
Classical missions: When the mission environment and the position knowledge are consistent. The drone will move along a trajectory pre-defined by waypoints, which will be interpreted by a flight controller (that will manage stabilization, navigation, and mission planning/exec). In reality, the mission environments are seldom perfectly consistent.
For instance, during the mission, the vehicle will need to be capable of making decisions when it encounters a newly discovered obstacle, loses positioning accuracy, loses visibility of features, perceives motion around it, and encounters external weather parameters that may impact its pre-defined flight plan.
Artificial Intelligence potentially helps build in an adaptiveness component on top of those classical missions. Instead of building the world based on static hypotheses then processing it to execute a plan, UAV perception can continuously assess sensor measurements and provide information to higher-level intelligence in charge of deciding if a given action (moving along the planned path) is still appropriate.
Sensor Fusion Is Critical
Reliable autonomy starts with perception.
One sensor is not enough to give sufficient information in all operating conditions. For instance, the technical information provided by Drone mentions that there is a combination of various technologies like LiDAR, radar, cameras, GPS, and inertia sensors instead of relying on one source of information.
Different sensors solve different parts of the problem.
LiDAR can provide detailed spatial information and support 3D mapping.
Radar can contribute distance and motion information and may remain useful in conditions such as rain, fog, dust or low visibility.
Cameras provide rich visual information that computer-vision systems can use for object recognition and classification.
IMUs provide information about acceleration and rotational movement, while GNSS can provide global positioning where reliable reception is available.
The interesting part is not simply adding more sensors. It is determining how their measurements should be combined. Sensor fusion can allow information from one sensor to validate or complement another. ZenaDrone’s discussion of LiDAR-radar integration makes this point particularly well: combining different sensing modalities can provide more complete environmental awareness and reduce dependence on the limitations of any individual sensor.
AI-Based Obstacle Detection
To give you a practical example of perception and flight control intersecting, imagine a drone trying to avoid an obstacle. Spotting the obstacle is the easy part. The drone has to first figure out where it is relative to it. Data from its sensors need to be analyzed to generate an action that might be to, for example:
- reduce speed,
- hover,
- stop,
- change altitude,
- calculate another route,
- or abort the current mission.
Obstacle avoidance material on the drone talks about a same chain: sensing, SLAM, an avoidance algorithm, and so forth up through sensor-fusion, a flight-controller, and real-time path-planning - but AI is where things start to get fascinating: at the perception stage, computer vision models don’t just need to know something is out there - they may need the ability to know what it is, reporting useful contextual details into higher navigation level software. “Knowing something’s out there” - that’s one thing. “That thing’s a wall, a tree, another vehicle, a person, or a power cable…
Or, hey - was that another flying object?”
Navigation Without Reliable GPS
Another interesting feature is the ability to operate autonomously in GPS-denied or degraded environments. An obvious example are indoor facilities, but similar challenges may appear in the proximity of large structures or other areas where the satellite positioning technology is ineffective.
One potential solution in the domain of autonomous flight is Simultaneous Localization and Mapping or, in short, SLAM. In contrast to merely using GPS coordinates as reference points, such technology allows an aircraft to create or update a map of the environment and determine its location within it.
The technology is also interesting from another perspective, as it implies the use of computer vision algorithms.
A drone could potentially use a combination of perception modules to understand the environment while SLAM or another technique handles the navigation data. It would allow the aircraft to update its course in real-time based on the latest information. However, the use of AI should not be limited to the possibilities offered by visual data. A more plausible application would be to use the AI-assisted environmental understanding in combination with traditional estimators and flight control algorithms to yield the best possible result.
Real-Time Path Planning
Once an obstacle or environmental change has been identified, the UAV still needs to decide what to do next.
Real-time path planning can evaluate available routes and select an alternative based on factors such as obstacle location, mission objectives and available flight resources. The drone’s technical discussion describes navigation systems dynamically updating routes as new obstacles are detected rather than relying exclusively on the original path.
This suggests a useful separation of responsibilities:
Sensors → Perception → AI/Computer Vision → Path Planning → Flight Controller → Aircraft
Rather than allowing an AI model to directly control motors, AI could operate higher in the autonomy stack. The flight controller would still be responsible for stable and predictable aircraft control, while AI provides additional information or higher-level navigation decisions. For me, this separation is particularly important when thinking about safety.
Decision-Making Requires More Than AI
One of the risks of discussing “AI-powered drones” is assuming that every autonomous decision should be delegated to machine learning.
I don’t think that is necessarily the right direction.
Machine-learning models can be extremely useful for perception, classification, pattern recognition, and processing large amounts of sensor information. ZenaDrone’s AI material similarly discusses computer vision, sensor fusion and real-time processing as mechanisms for improving situational awareness and responsiveness.
But flight systems also need predictable fallback behavior.
What happens if:
- the AI model has low confidence?
- LiDAR and camera observations disagree?
- visibility suddenly deteriorates?
- processing latency increases?
- a sensor becomes unavailable?
- the environment contains an object the model has never encountered?
In those situations, the safest decision might not be the most “intelligent” one.
Hovering, slowing down, returning to a known position, switching navigation modes, or handing control back to an operator may sometimes be better than allowing the AI system to continue making increasingly uncertain decisions.
Where Should AI Sit in the Autonomy Stack?
This is the part that I am most interested in discussing with people working on autonomous UAV systems. As perception models improve, technically, it becomes possible to grant the AI more control over navigation. But it doesn’t necessarily mean that it is the best architecture.
A layered approach seems more practical:
Flight controller: stabilization and safety-critical vehicle control.
Navigation/estimation: position, velocity, orientation, and trajectory information.
Perception: cameras, LiDAR, radar, and other sensor processing.
AI layer: object recognition, environmental interpretation, and potentially higher-level decisions.
Mission layer: objectives, route changes and operator interaction.
This would allow AI to make the aircraft better informed without making the entire flight stack dependent on probabilistic AI output.
Questions for the PX4 Community
I’m interested in how others are approaching this separation in real systems.
Where do you think AI should sit within an autonomous PX4-based navigation stack?