PX4 Custom MAVLINK Help

I’m trying to make PX4 firmware and add a very simple MAVLINK message. To put it kindly, the tutorial to make expand on the PX4 Firmware is absolutely useless, but I’ll go on about that later. Just read the whole post because there are a lot “kinda works” and “if I do X then Y works but Z breaks” type problems.

I can build on Windows and Linux just fine, add a uORB message just fine, but cannot for the life of me get MAVLINK messages to work.

Here’s my uORB message (mav_test.msg):

uint64 timestamp
uint8 mav_data

It’s included, it’s building fine.

Here’s my custom_messages.xml:

<?xml version="1.0"?>
<mavlink>
  <include>ardupilotmega.xml</include>
  <dialect>2</dialect>
  <messages>
    <message id="9101" name="MAV_TEST">
      <description>Testing MAVLINK</description>
	  <field type="uint64_t" name="timestamp">Timestamp</field>
      <field type="uint8_t" name="mav_data">MAV Data</field>
    </message>
  </messages>
</mavlink>

So far, this is building fine with a few exceptions I’ll explain later.

Here are my includes in mavlink_messages.cpp:

#include <uORB/topics/mav_test.h>
#include <v2.0/custom_messages/mavlink_msg_mav_test.h>

Here’s my function in mavlink_messages.cpp:

class MavlinkStreamMavTest : public MavlinkStream
{
public:
        const char *get_name() const override
        {
                return MavlinkStreamMavTest ::get_name_static();
        }
        static constexpr const char *get_name_static()
        {
                return "MAV_TEST";
        }
        static constexpr uint16_t get_id_static()
        {
                return MAVLINK_MSG_ID_MAV_TEST;
        }
        uint16_t get_id() override
        {
                return get_id_static();
        }
        static MavlinkStream *new_instance(Mavlink *mavlink)
        {
                return new MavlinkStreamMavTest(mavlink);
        }
        unsigned get_size() override
        {
                return MAVLINK_MSG_ID_MAV_TEST_LEN + MAVLINK_NUM_NON_PAYLOAD_BYTES;
        }
private:
        /* do not allow top copying this class */
        MavlinkStreamMavTest(MavlinkStreamMavTest &) = delete;
        MavlinkStreamMavTest &operator = (const MavlinkStreamMavTest &) = delete;
protected:
        explicit MavlinkStreamMavTest(Mavlink *mavlink) : MavlinkStream(mavlink)
        {}
        bool send(const hrt_abstime t) override
        {
                mavlink_mav_test_t msg{};
                msg.mav_data = uint8_t(1234);
                mavlink_msg_mav_test_send_struct(_mavlink->get_channel(), &msg);
                return true;
        }
};

I included the lines to add the stream just after and to enable the stream in …/init.d-posix/rcS, and they have never given me any issue. Those are just fine and not a part of the problem.

Now, if I build the program is it stands with an include in the custom_messages.xml and copy over the files, I get tons of errors on build. I’m going to include an image below. It’s a lot.


Now, if I remove the include in the XML, generate files, then move them over, I can build the firmware and it will run and send the message just fine. But the problem is QGroundControl refuses the message and drops it. I brought this up with DonLakeFlyer who mentioned it may be an issue with CRC. Lo and behold, there’s no CRC data for the custom message being included in the program. That’s in v2.0/custom_messages/custom_messages.h. That doesn’t get included because if you include v2.0/custom_messages/mavlink.h, nothing ever gets included because MAVLINK_H is already defined and everything in the file is skipped. So I have to include the message file directly to get things like the message ID, which obviously leaves out important data like CRC data. It’s a circular situation where you can’t include all the data.

There might be a fix for this as is, but it would include manual splicing of files, which I have not been successful with and is not something that a user should have to do anyway.

There is so much not working here it’s ridiculous. There is zero help in the tutorial or online I can find after over 100 hours of working this. The fact that following the steps of the tutorial causes even more issues than fucking around and trying random other things is a problem. This is all I have to go off of and no amount of tweaking, file splicing, comparing to other questions online, etc. gets me any closer.

Hi, I had the same problem. The right way is modify mavlink messages in your fork of mavlink library. This library is built automatically and deployed to c_library_v2. Also make your own fork. And set up deployment there. This will ensure, that all files will be created rightly.

Then you can change submodule URL of PX4Firmware/mavlink/include/mavlink/v2.0/ pointing to your fork. And then try it again.

And a note, you must set publishing of your message also in this file with maximal rate limit.

What does using a fork do versus just a cloned version of MAVLINK? I’ve tried modifying MAVLINK separately and with the downloaded submodule with the same results. Ultimately a fork is absolutely not an option for a few reasons, so it’s obviously important I do it without a public fork.

It is a better approach for that. It is not the main problem.

QGC refusess your messages… Are you sure, that QGC know your message definition? (You built QGC yourself?)

Before using QGC, try to receive these messages using pymavlink. It is more obvious there. But you must be also sure, that your message is known by pymavlink.

Then my next question is, do I need to build QGC with the custom firmware to just see the message? I’ve seen you don’t need a custom version just to see the message come in, which does seem odd.

Sorry I’m not really sure about that. I infer from how mavlink protocol works (and looks). QGC must know your message definition to decode. Because mavlink message (transmitted data) does not include message structures.

I couple more questions:

How do I properly add my custom firmware to QGC? I’ve added the custom_messages directory to the CMakeLists.txt in the root directory and think I found a place to add custom_messages.h, but I’m not sure it’s right.

Did you just straight up follow the tutorial listed above? Because at this point it leaves out so much info that I’m just beyond confused. They explain nothing, assume so much, and just… it’s ridiculous. Nothing here makes sense at all.

I can’t answer for the first question. I don’t know yet. Hopefully tomorrow :slight_smile: or till the end of the weekend… Yesterday I learned how to compile QGC.

I trayed to follow listed tutorials. But it did not fully work. In the beginning, I was a little bit confused too. I plan to propose some modifications to these tutorials.

I highly recommend that you first try to receive the message using pymavlink. For me it is more transparent how it works :slight_smile: Do you have some experiences with pymavlink?

The reason your xml has crc issues is because of a known bug (https://github.com/ArduPilot/pymavlink/pull/339 is a fix) - basically if you have more than one level of include, things don’t currently work properly. You’re triggering this because you include ardupilotmega which itself has includes.

The root cause of this problem is as auturgy indicates above ^^^. The docs don’t mention this because they assume you’re creating a dialect off common.xml, and also I’d hoped this bugfix would be merged by now.

MAVLink creates a CRC at generation time for every message based on the field types, names and order (some docs on this here). This means that for a new message you will need to have the same library on both ends of the communication.

The latest version of the tutorial may have changed a little (can’t remember) see https://dev.px4.io/master/en/middleware/mavlink.html

You’re right that it isn’t great. We’re a community, would love a PR to improve it hint hint.

2 Likes

I have a ton of notes and private documentation I’ve written up for QGC and PX4. This stuff is my job, part of which is document my custom work and filling the gaps of the public documentation. Much of this I’m sure I’ll add in a pull request when I feel like I have a solid grasp and I’m not spitting out incorrect information.

Thanks for the help!

The way around it at the moment, if you need the ardupilot messages, would be to use Ardupilotmega as the top level xml, with your dialect added to it as an include (multiple includes isn’t the problem, nested includes is), and rebuild from that.
Not ideal, but will get you moving.

So instead of having custom.xml include ardupilotmega.xml, have ardupilotmega.xml include custom.xml?

Yes. That way you get ardupilotmega, common, and your custom.