Unstable flight with coaxial octocopter


I’m designing a coxial octocopter which needs to be used for a research project. To try and get understand coaxial octocopter better I made a simulation in Gazebo using one. I managed to get it to fly, but it flies very unstable. I noticed in the flight log that the roll and pitch angle estimate is different from the ground truth. Not sure what can cause this, so any ideas is appreciated. I have also linked the flight log.

Furthermore, I also discovered that if I lower the weight of the simulated drone a bit it will start flying even more unstable and crash. The drone will also takeoff even just using the arm command. Again any ideas for the reason are appreciated.

Here is another update about the drone becoming very unstable if the weight of the drone is changed from 4 kg to 3.5 kg. It starts oscillating in several different areas. Not sure of the course of this.

An identical flight with all the same parameters except the weight changed from 3.5kg to 4kg is shown here. https://review.px4.io/plot_app?log=17995461-71ba-42f2-865b-90f565a7a4c7
It produces a very different flight and is able to safely hover.