I’m currently working on a project involving computer vision on drones. It involves installing a camera and other relevant sensors on a platform to monitor and detect wildfires. This is still more of a proof of concept and not necessarily to be deployed on the field. Still, I had some doubts about what frame to choose. I’m currently looking at some of the Holybro development kits (PX4 Development Kit - X500 v2 – Holybro Store). I also wanted to ask if it’s worth getting the PX4 development kit or just the frame itself. If anybody has any feedback or recommendations regarding the best frame for this project or anything else please let me know. Basically:
Working on a project involving computer vision, considering expanding into autonomous pre-programmed flight paths in the future
Need recommendations on what frame/kit to choose
Want to know it it’s worthwhile getting the PX4 dev kit or just the frame itself
How to implement computer vision (any camera accessories or gimbals available in the market?)
Well, it all depends on what exactly should be achieved.
Simply flying a pre-defined flight-path and and looking for fires is one thing, …and reasonably easy done.
But detecting fires, sending an alarm message and adjusting flight path accordingly is a different matter. For that you might end up using a companion computer, perhaps with AI, possibly thermal imaging camera and a custom made script to talk to the flight controller.
Or using ROS and integrating certain function that way.
then it becomes possibly an issue in regards to weight.
Also, what area should be covered ? = Flight time and payload considerations.
There are many variables and unless you start narrowing it down it is difficult to provide a suitable answer.
I was thinking about using a raspberry pi or another companion computer to run a computer vision program to detect smoke from fires through a camera. I was also thinking about using a CO2 sensor as an additional layer of confirmation for fire detection. Regarding area to be covered, this is more of a test bench. After I can prove the concept works on a quadcopter drone I was planning to move on to a fixed-wing UAV. Does that narrow it down a bit?
I can save you some time and hassle:
a few years ago I’ve done some object detection using a Raspberri-Pi 3B+.
Due to limited CPU power it manages to process less than 2 picture frames per second. This is not enough to get reliable, quality results whilst flying. - Might be alright for a stationary device.
As part of my research back then, I’ve found this handy little comparison:
(Thanks to the autor, as is not mine).
Thanks a lot, I will look also into more powerful platforms like an NVIDIA Jetson board. I will probably proceed with the X500 frame since it has the necessary payload and flight-time for me right now. Regarding a camera to use for the computer vision aspect, do you have any thoughts on that? My plan was to utilize a camera gimbal that could freely rotate, but I’m not familiar with any commercial products.
Re. Computer: I’ve ended up with a mini PC. As I could use the more powerful CPU and also install a suitable version of Linux. There are many projects out there based on Linux and less issue with proprietary limitations.
Re. Camera: Good question. - That’s where initial research pays off.
A Raspberry-Pi camera has the benefit of supplying raw image data. This allows to process specific elements you are interested in. I.e. looking more on the IR spectrum in order to detect potential heat / fire sources.(There are also a number of projects out there relating to this).
However, this brings us back to limited processing power and another issue is the Raspberry-Pi camera are known to cause significant signal interference in regards to flight controller. - ( It’s mostly the ribbon cable connecting the camera causing this).
There are combination optical and thermal imaging cameras out there, but very expensive.
The issue with any standard camera is, the signal is usully a full stream of video. But when trying to look for something specific, like IR elements, it is now difficult to break that video stream down or filter specific elements out. = Lots of processing power and software resources.
Thanks for your insight. I’ve been doing some research on the cameras and it seems that most gimbal cameras are connected directly to the flight controller. Is there any way to bypass this and instead connect the camera to the companion computer? I want to use the companion computer to control the camera gimbal, but I’m not sure if that’s possible.
Update: I found this thread by Jaeyoung Lim (On realtime control of gimbal from companion computer - #8 by ernsten) that uses MAVROS to control a camera gimbal through what I assume to be a companion computer (didn’t fully comprehend). I’ll try to dig deeper into this to see if it’ll be possible using this. Credit to him.
Although I found a way to control the gimbal through MAVLINK, I still don’t know how to get the video feed from the camera to the companion computer in order to run object recognition software. Does anyone know how to do this?
