Customizing the pixhawk for other aircraft types

Hi there,
I have been working on a new UAV structure, referred to as swash mass UAV. It consists of a double-blade coaxial shaft rotor and four swash masses that allow changing the orientation and maneuvering the UAV. As you see, it has a different structure from the default structure of pixhawk ( Plus Multi-rotor structure).
Here is my question: I need to build a flight control system for this structure and pixhawk does not support this kind of structure. My mission is to fly the multi-rotor drone by using pixhawk and I want to know how can I customize the pixhawk for such a different structure. Should I use MATLAB or other tools? I am new in this environment and I am open to hearing any kind of suggestion!
Thank you folk!
Umut D√ľndar