Data Comets: A new flight log analysis tool

Data Comets: Open Source PX4 ULog Analysis Tool

Help us improve Data Comets with this survey!

I am very excited to share with everyone here a visualization tool I have been working on for PX4 ULogs.
First things first here is a teaser of the tool in action!

You can try the tool for yourself with the hosted version: http://www.datacomets.com/
And you can read more about the tool and get a local version (for best performance and larger logs) on the Github Page: https://github.com/dsaffo/DataComets

The main idea behind Data Comets is that flight data is recored in space as well as time. So why don’t we show where data happened instead of just when. Data Comets allows you to encode flight data onto the flight path itself as a color map. Encoding jamming or noise could show you areas with bad gps signal, or encoding velocity will show you how the UAV speeds up or slows down to get to all the setpoints.

Data Comets also lets you filter the flight data shown by time using the timeline. This lets you focus on certain times of the flight, but this also lets you animate the flight through interaction.

The tool is also interactive and allows for hover for details on demand, linking charts, and pinning data to compare.

I would like this discussion page to be a place to hear your feedback as well as answer any questions about the tool you may have.

Data Comets was developed as a part of a information visualization design study for UAV flight log analysis. We are currently in the process of writing a journal paper for the study and at this stage we are hoping to evaluate Data Comets and the overall design study.

I am very excited to release this to the community and would like to grow and improve this tool with everyone.

Hope you enjoy and look forward to getting your input!
David

Also want to give a special thank you to Daniel Agar, Anthony Daniel, and Jun Kim for all their help with this study and project.

2 Likes

This is VERY cool!

FYI, added small section using your readme text to PX4 Flight Analysis docs here: http://docs.px4.io/master/en/log/flight_log_analysis.html#data-comets

Feel free to extend if you think more needs to be said at this level (I assume you don’t want a full page on this in the user guide because then you would have to maintain it in multiple places).

Nice work!

The first two features that came to my mind that I would like to have:

  • Being able to change the size of the map and graphs. Depending on what I want to have a closer look at I would like to have the graphs take up most of my screen and not the map. For other aspects it’s cool to have a big map, so being able to change the window sizes would be really helpful

  • Being able to mix and match different graphs on the same plot, similar to how the Flightplot tool is handling it.

Keep up the good work! :+1:

Thank you I hope you enjoy using the tool!

I do plan on implementing a pop out window for the charts that will show a chart focused view. I have heard this from a few people now and I do think it is a good idea.

The seconds one is a little bit tricky. I played around with the idea of being able to drag charts onto on another to plot them simultaneously, however it is really difficult to plot data points with multiple scales together without potentially creating misleading or hard to read visualizations.

That is one of the reasons for the pinning functionality so you can pin two charts to appear inline with each other and you can compare them that way. What I am hearing is this is a little un-natural to people which is why some of the overview charts are plotted together when they share the same scale.

It is an interesting problem and can definitely be a cool project for the future so I will keep it in mind.
Thank you again!

Thank you for doing that! Everything you included looks good to me, but I will make a pull request if I think of anything else.

1 Like

have windows laptop and android tablet. would like to run in the field on local without requiring internet link.
.
I don’t know about python… is this my solution?

Yes the local version is your solution for running it on a windows machine, there is currently no native android solution if you do not have an internet connection.
To be more comprehensive than the github README
You will need to install python3
and git

then using your terminal in a directory you want the folder
git clone https://github.com/dsaffo/DataComets.git
cd DataComets
pip install -r requirements.txt
python DataCometsLocal.py

It should launch a browser for you automatically and you can then drag or select the .ulg files.

Everything should work offline except for the map tiles so you will just see the flight path on a grey background. I will work to have a way to save offline maps for future versions as it seems like something people will find useful.

let me know if that works for you or not!