This weekend my team, partners, and I completed approximately 1,700 flights on v1.13, primarily, on the CubePilot CubeOrange platform. Our codebase is based on the following commit from March 4th: 601c588294
During these 1,700 flights, we had 3 quads fail. After further inspection of the logs below, it looks like all of the quads failed in exactly the same way:
Nominal flight, nothing was visually wrong with the quads. Then,
Started yawing hard (all in the same direction); logs show full thrust
Maintained a relatively stable attitude with constant yaw; rapid fall straight down
Consistent with single motor cutout
We haven’t ruled out mechanical failures, but it is curious to us that they all failed in a near identical fashion. Can anyone take a look at these and see if there is something glaring in the logs that our team might have missed?
We are worried that there may be an issue after the actuator output stage that could have cause these three quad to fail in a very similar fashion.
Did you find any persistent hardware problem after “hard landing”, like did one motor not work? Or did everything work as normal again once power cycled?
To answer your question, once power cycled, all motors spun up - but with a caveat:
We did observe that 1 of the 4 motors on the Houston drone had some damage but was still able to spin and provide load. What is unclear is whether or not this motor was damaged before or after impact. My intuition tells me that the motor damage was due to impact, because this drone had flown successfully for a year without any significant issue, and I don’t see wobbling or any strange behavior in the logs that would suggest that the motor was about to fail.
Here’s what we found with the Houston drone so far:
One of the four motors was damaged, but could still spin and provide a load.
When we took the motor apart, it looks like a single winding from one of the poles was trapped between the motor and stator. The winding wasn’t catching anything on the rotor allowing it to still spin freely.
it is unclear whether or not this happened on impact with the ground
ESC testing on this unit show that the ESCs can handle 100% power bursts and 75% power for a sustained period of time.
This suggested that there is nothing wrong with the ESC’s power delivery and that all ESCs were healthy at the time of the fall
There was no prop damage
All wiring harnesses were plugged in and are not damaged
The battery voltage at the time of the fall was stable and the health of the packs are monitored every charge cycle. Also, the battery power bus is fed directly to the ESCs and the ESCs haven’t shown any sign that an individual ESC was struggling with power delivery when we put the drone under load on the bench.
Your PIDs are totally off, I guess no one looked at your logs?
Can you post a log from a successful flight?
Got to PID Analysis and look then click the link on that page for Documentation and examples.
But much better that the crashed ones.
Yaw is not even logging, not sure why that would be. Slow/Bad SD card???
You need to focus on tuning. There is now an auto tune feature you might try,
Check this out:
In my experience just because a motor can spin does mean it’s good. When it doubt swap it out.
If you don’t have a motor test stand you need one. Tyto Robotics (Formally RCbenchmark) makes a relatively inexpensive one that will completely parametrize your motors. I would test every motor when new and then after a crash re-test them.
Motor output 4 through 7 are not controlling or connected to anything. I was curious about this too, because we are configured as a Quad X frame. I took a look back at some logs from our previous firmware, base on v1.11.2, and we’ve always been outputting 8 channels even though only output 0 through 3 are being used.
Good point on the vibration as well. With this airframe, I normally see higher vibration than I’d like, but some units are worse than others. If you look at the spectral density, most of the vibration is coming from higher frequencies, above 80 Hz, which should be directly from the motors/props. I’m not sure if you have insight on this but it has been an open question has been, are we using the vibration isolated IMU or non-vibration isolated IMU for our primary sensors? And does that matter in a multi-EKF configuration? (these are CubeOranges)
I took a look at the actuator output graphs, as we discussed in the dev call. Here’s my take on which motor failed (based on Actuator Outputs (EXTRA) graph clamping):
Houston: Output 2 (Motor 3)
Aus #1: Output 2 (Motor 3)
Aus #2: Output 3 (Motor 4)
If true, then in all three cases, the one of the clockwise motors failed causing a counter-clockwise yaw (as supported in the logs). Furthermore, the damaged motor from the Houston drone was motor #3. This evidence further supports mechanical failure rather than PWM output failure from the main output channels.
We will still be doing burn-in testing of some new drones to try and replicate.
It would great if someone can double check my analysis.
Did you solve the problem?
In my lab we had a similar problem: one of the motors overheated during the flight, asking for more power and causing the motor on the opposite side to compensate (like in your logs if motor 3 and motor 2 were on the opposite sides). Then we changed the broken motor but we didn’t achieve a good flight. Afterwards we discovered that the assembly of the new motor caused a rotation of the arm, thus we had always a non-vertical thrust from one motor which produced a yaw variation that the remaining motors wanted to compensate.
