Pixhawk+PX4: Cannot get Optical Flow to work!

Hi - this is a plea for help hereā€¦ I have scoured the internet over and over looking for answers, but to no avail. I have used this site to get started: http://dev.px4.io/flow_lidar_setup.html

Setup: Pixhawk with 3DR Optical Flow Camera connected over I2C

I have GPS Off, LPE set. I see Optical Flow data coming into QGC using the analyzers. I see this when the drone motors are disarmed, and moving it around manually over the ground. From what I understand, the LPE module will take data from multiple sensor sources and fuse them together. GPS is OFF, it can take off, and I can put it into Position Control mode (no rejection).

However, the drone will drift away, making no attempt at all to maintain position. Camera is focused as per several guides. One thing I need to try is using a telemetry system to see Flow data in QGC as it is drifting. I assume it should be ok since it works when disarmed, and I am able to select Pos Ctrl without GPS.

Soā€¦ I feel like there is some other parameter I am not setting, somewhere. Since I have never seen this work, I am unclear how ā€˜hardā€™ this should be. I followed the web instructions, but no luck!

I have asked on DIYDrones but no help there.

Hopefully someone can provide some experience! Please! This is making me nuts.

What distance sensor are you using? The sonar of the PX4FLOW or a separate range sensor?

I would definitely recommend that you look at the data in flight with a telemetry module.
You should look at the quality measurement in the flow message as well as the distance from your range sensor in DISTANCE_SENSOR.current_distance. Further, look at the local position estimates and the computed flow.

LPE also sends log messages to QGC whenever flow is initialized or times out due to bad measurements. This should also give you an idea if itā€™s currently running or not.

Thanks Nicolas for the reply! Much appreciated. Bunch of simple questions belowā€¦

Range sensor is the SF11-C, which is reported to be supported in the latest PX4 builds.
https://www.parallax.com/product/28054

So not using the Sonar, although I donā€™t know how to ā€˜disableā€™ it. Does the SF11 automatically override it when I set SENS_EN_SF0X Set to 1?

Does Optical Flow REQUIRE a range sensor to work? Just curiousā€¦ it was the reason I bought the SF11 in the first place, but it didnā€™t help. I do see Range sensor data, and it changes when I wave my hand in front of it.

How do I know if LPE is actually USING the flow data? Are there any other parameters I need to set that are NOT mentioned in
http://dev.px4.io/flow_lidar_setup.html ?

I am determined to get this to work indoors over a patterned carpet!

Derek

So not using the Sonar, although I donā€™t know how to ā€˜disableā€™ it. Does the SF11 automatically override it when I set SENS_EN_SF0X Set to 1?

SF11 should take precedence over the sonar on the PX4FLOW module. You can easily test this by looking at the distance measurements in QGC and covering up one sensor but no the other.

Does Optical Flow REQUIRE a range sensor to work?

Yes it does. Without an estimate of how far the ground is, there is no way of scaling the translational speed measured by the flow module (closer to the ground, the same translational speed produces higher flow)

How do I know if LPE is actually USING the flow data?

In QGC, you will see something along the lines of [lpe] flow init when the flow quality and distance measurements are sufficient for flow fusion. You should also see similar messages for the distance sensor: [lpe] lidar init.
When the flow quality or height above ground is insufficient for too long of a time, you will see [lpe] flow timeout. Flow will then not be used to estimate local position until you see [lpe] flow init again. You can look at this fusion logic here: PX4-Autopilot/src/modules/local_position_estimator/sensors/flow.cpp at main Ā· PX4/PX4-Autopilot Ā· GitHub

Excellent info! Glad I bought that SF11 then on a hunch.

We shall look at the code, thanks for pointing me in the right area.

In QGC, I am a little new to logging. The messages you refer to are in MAVLINK messages? I would like to know what the recommended Widget is to see theseā€¦ Sorry for newbie question. Then I will RESTEST!

Derek

The mavlink log messages are displayed in QGC under ā€œvehicle messagesā€, which looks like a megaphone (or red exclamation point if there are critical messages) as shown here: https://donlakeflyer.gitbooks.io/qgroundcontrol-user-guide/content/QuickStart/QuickStart.html (section ā€œconnecting the droneā€)

The flow init/timeout messages are critical messages, which means they will be shown in a yellow pop up at the top of the QGC screen. If you have older critical messages, you might not see the newest ones. Click on the ā€œvehicle messagesā€ symbol to see all messages.

Ok Nicolas - I have some work to do then. I will report back with the flow stats when in flight, using telemetry and QGC. Appreciate your help so far!

Derek

Ok - so it works! I think the patterned carpet I tried today made the quality go up. Not sure. But with telemetry I can see the success.

One issue I do have though is that the Lidar will come and go, even when stationary, and I see occasional messages:

lidar fault, beta 157.08
lidar fault, beta 107.06

What does that mean? It seems like this causes flow to stop sometimes.

Derek

I see it in source but not sure what it signifies, or if it impacts indoor positioning directly.

https://pixhawk.ethz.ch/docs/lidar_8cpp_source.html

// fault detection
98 float beta = (r.transpose() * (S_I * r))(0, 0);
99
100 if (beta > BETA_TABLE[n_y_lidar]) {
101 if (_lidarFault < FAULT_MINOR) {
102 _lidarFault = FAULT_MINOR;
103 mavlink_and_console_log_info(&mavlink_log_pub, ā€œ[lpe] lidar fault, beta %5.2fā€, double(beta));
104 }
105
106 // abort correction
107 return;
108

That means that the measurement produced by the lidar is considered to be an outlier and is not fused into the estimator. You can observe the lidar measurements in the QGC Analyze widget, look for DISTANCE_SENSOR. If you are getting a lot of bad measurements from the lidar, your height above ground estimate will be bad which will in turn affect your flow performance. However, as long as youā€™re not seeing flow timeout, it is still being used for position estimation.

I will have to correlate when I see these errors to what is actually physically happening at that moment.

Thanks Nicolas!

Soā€¦ problems are back.

I see LIDAR working with correct distance.
I see Flow Quality of 240+

Pixhawk LED remains blue and I get the dreaded ā€˜Reject Position Controlā€™. I wish there was some indication WHY it was rejected. I donā€™t see any indication the FLOW is starting, but I donā€™t know whyā€¦

Sorry to be having so much trouble. I didnā€™t think it would be so difficult.

EDIT: After I wrote this, I plugged in the battery and now FLOW is working! I didnā€™t do anything? Not sure why there is such inconsistency.

USB doesnā€™t provide enough power for all electronics. It is only designed to power Pixhawk, not a lot of peripherals. In hindsight we should not have powered the peripherals from USB in the first place to make this explicit.

However, when I proposed that change there were a lot of furious developers / users who wanted to power on the bench via USB only. The good news is that USB 3.0 offers more current which is again enough, so maybe thatā€™s the long-term fix.

I donā€™t remember if I had the battery unplugged when i was experiencing those issues. But I definitely try to plug in the battery first, then USB. I have not had many issues like that again, so maybe I did not have the battery inā€¦

However, I do have another question. I have it set to LPE, Optical Flow and Lidar enabled. But this time I also enabled GPS and went outside. The two times I flew the drone, it was working seemingly fine, but then the drone would start to travel away, completely ignoring my RC controller. Both times it crashed and I was frantically trying to stop it. I still had the drone in Position Control mode, and in hindsight I should have switched to Manual to possibly regain control. So it is quite frightening to fly as I am afraid it will either hurt someone or fly away! The second time this happened, I was flying it at 5 feet and following right behind it. It was moving smoothly then took off into some bushes thankfully, but it was completely ignoring my transmitter.

So, the question is: If I want to fly outdoors with GPS, is there a recommended setting? Should I have switched from LPE to INAV? I feel like GPS, Lidar, Optical Flow maybe caused some issue and the drone was trying to compensate and fly away on its own. I remember when I was using INAV and GPS alone (no Lidar yet), I never had this issue.

Derek

Hi, Derek

Iā€™ve been attempting to get some PX4 drones to function in position hold using a PX4FLOW as well but without a LIDAR. Iā€™m facing a similar drift issue which I believe is a result of bad local position data (which may in turn be caused by bad distance sensor measurements).

Did you ever find a solution to your position hold problems?

Hi - yes, it turns out the LIDAR is REQUIRED. It does work now and was able to hold position indefinitely. I used it outside as well over my brick patio (for a pattern), and GPS off with no antenna. Worked great. I guess the issue tends to be over grassy areas, as it all looks the same. So it would not be usableā€¦ But indoors over a patterned carpet was good.

Glad to hear itā€™s working well for you. Iā€™ll have to give LIDAR a shot when I manage to acquire a module (considering a Lidar-Lite v3 from Sparkfun currently). Thanks for the update!