Hello,
I’m trying to flight using local_planner on Intel Aero Drone.
But it seems like the drone is not recognizing the obstacles.
I have to say that simulation is working and everything is fine, but
when I’m on real flight the drone is flying , but not trying to avoid obstacles.
I’m also tried to configure ROS_MASTER_URI & ROS_IP and running rviz on my laptop,
as explained in previous post, but its not works for me.
Best Regards
Evgeny
To connect RVIZ on you laptop with your drone, you need to do the following:
In my opinion it does not make any sense to go out flying if you cannot connect to your drone. You need to perform benchtests first, to verify that your system is setup correctly
@baumanta , Thanks for fast response.
I’m connected to the network through the router and I can reach my drone using SSH.
For now I’m slightly modified the code and I’m toggling the goal position every 20 sec.
This is the only way to check if the drone actually “see” the obstacle…
Can you please explain how to perform benchtests ?
Best Regards
Evgeny
To perform bench tests I would suggest you perform all the steps I suggested and connect RViz. In Rviz you can see the pointcloud of the realsense. You can point the drone towards objects and see whether they are detected and if they are at the right orientation in space (to see whether you set the static transform camera->fcu correctly).
I don’t know what you mean with “I changed the code” and why you would want to do that. But I would suggest you first get the current master running before experimenting as it is verified that this code works.
Unfortunately I cant open Rviz with this configuration:
I’m also tried to set ROS_HOSTNAME on laptop and on the drone.
In this case Rviz is opened, but I can see only empty grid.
I’m connected to the same local network using simple home router.
I can ping each other by IP and also by host name. And I also tried to
connect to another router with the same result.
Best Regards
Evgeny
In the case where you set the hostname, if rviz opened, then there must be a connection to the rosmaster running on the drone. I suggest you repeat all the steps of setting ROS_MASTER_URI, ROS_IP and ROS_HOSTNAME and instead of trying to open rviz, type rostopic list on you laptop. Can you see a list of topics? e.g. /local_pointcloud ?
I configured ROS_MASTER_URI, ROS_IP and ROS_HOSTNAME
But I can’t see the list of topics when I try rostopic list command on my laptop.
On the drone I can see all topics when local_planner is running.
Is there local_planner_aero.launch file should be changed when I’m running local_planner ?
Best Regards
Evgeny
If you can see all the topics on you drone, then the node is running (if you type rostopic hz /local_pointcloud is should give you a value between 10-20Hz). The issue must lie in the environment variables such that you laptop cannot connect to the rosmaster on the drone
It working now. It was Rviz configuration issue.
I just set the Fixed Frame field to “/local_origin” and now I can see
the point cloud and objects. Everything looks fine. Is there any additional steps
that should be taken before flight ? What you suggest to check in order to figure out
why the drone is not trying to avoid obstacles ?
Best Regards
Evgeny
That sounds very good! So now you know that generally your system is working. If you have a router I would suggest you take that out to the field and try the same there and look if you realsense still sees objects outside. One thing that often goes wrong is that the auto-exposure is not on. then you will get completely overexposed images and not be able to see anything. Once you ruled that out, I would suggest you write a short logging script to record a rosbag something like:
OUTPUT_FILE_NAME="/data/rosbags/log_OA_$(date +%Y_%m_%d–%H_%M_%S)"
rosbag record __name:=logger -O $OUTPUT_FILE_NAME /adapted_waypoint /bounding_box /camera_front/color/camera_info /camera_front/color/image_raw /camera_front/realsense2_camera_manager/parameter_descriptions /camera_front/realsense2_camera_manager/parameter_updates /complete_tree /current_setpoint /goal_position /histogram_image /local_planner_node/parameter_descriptions /local_planner_node/parameter_updates /local_pointcloud /mavros/companion_process/status /mavros/local_position/pose /mavros/local_position/velocity /mavros/obstacle/send /mavros/setpoint_position/local /mavros/setpoint_velocity/cmd_vel /mavros/setpoint_velocity/cmd_vel_unstamped /mavros/state /mavros/trajectory/desired /mavros/trajectory/generated /mavros/trajectory/path /original_waypoint /path_actual /path_waypoint /reprojected_points /rosout /rosout_agg /smoothed_waypoint /take_off_pose /tf /tf_static /tree_path --split --size 4000
of course you nee to adapt the camera name in the camera topics to how the topic are called in your case. Then you go out and fly and record the rosbag during your mission. If somehow it still does not work you just get back to me with the rosbag 
Yesterday I was outside and recorded bag file.
Not pointed drone to objects, just hovering… It looks like outside
it’s not see objects. There is no data in local_pointcloud.
Another strange thing that when I’m flying in position control with
Collision Prevention enabled MPC_COL_PREV_D = 3
I get Collision warnings from PX4, but the drone is not stops behind the obstacle.
That sounds like your drone sees nothing, or most likely a little bit of noise (therefore the collision warning statements). Di you check whether the auto exposure on the realsense camera was really enabled? You can enable it over rqt reconfigure (in laptop terminal set ROS_IP and ROS_MASTER_URI, then type rosrun rqt_reconfigure rqt_reconfigure) or over ssh ( shh into drone and type rosrun dynamic_reconfigure dynparam set /camera/realsense2_camera_manager rs435_depth_enable_auto_exposure 1
Does that help?
I can try it only tomorrow.
Will inform you later about this test.
But I have R200 camera, rs435_depth_enable_auto_exposure looks like D435 camera parameter.
Best Regards
Evgeny
Sure, that was an example command. You will have to change the command to the namespace of your camera (which is not necessarily /camera) and to the exact parameter name but I assume it will have a similar name for R200
Ok, i tested it today. But have some new problems.
First - Resetting auto exposure parameter doesn’t helps,
but switching preset to OFF (see picture) does helps.
Is there some option to calibrate R200 ? How point cloud
of calibrated camera should look ? Because if I do reset to auto exposure
at home I get very dense point cloud, but if its good or not - I don’t know.
Second- drone flies aside when I pointing in Rviz left or right.
I think it should head to flight direction - as in simulation.
Basically, your pointcloud is good if it shows correctly the objects in front of your drone and otherwise contains as little noise as possible. What I see in the RVIZ image you attached does not look very good to me, but of course I do not know what it is supposed to show as I don’t know whether it was pointing towards an object or not.
Maybe as an input, here is a picture from a pointcloud from one of our systems. It has a bigger FOV as 3 cameras are attached (Realsense D435) and the colors are postprocessed and show distance.
On the picture I sent It’s not pointed to any object.
I will try to attach some pictures for comparing later. But It seems like
it’s strongly depends on ambient light and maybe on other factors…
Need to find the way to calibrate/tune it automatically, otherwise It’s not
sure any time you flight is there avoidance working or not.
P. S. Do you know why the drone flies aside and no heading flight direction
when I’m pointing it left or right through Rviz ?
Best regards
Evgeny
Yes, you would need to contact Intel for that, I cannot support you on Realsense issues.
For the second question, I’m not sure what you mean. You would have to provide a rosbag with all relevant topics if you want me to look at it. But actually I would solve the realsense issue first. It’s not really worth debugging planner behavior if you feed it with garbage data
If I running local_planner and control It through Rviz with 2D Nav Goal ,
I want to send it left or right its flying aside instead change yaw and flight to
the pointed direction. If I point it to go back or straight its flying ok because it already
in right orientation. In this situation I cant point the drone to the obstacle.
This behavior is differ from simulation when the drone always heading to the flight direction.
Evgeny
After playing with camera parameters I get some sufficient results. Only enabling r200_lr_auto_exposure_enabled doesn’t helps and it need some additional tuning according amount of ambient light. I captured cloud at evening and attaching it here:
On the flight it is some strange behaviour that also present in simulation.
The drone , when approaching the obstacle, is flying around same position and not going to the goal position. Here the bag file: https://drive.google.com/open?id=1VaQbNUDmHMLKjHao3AFN0nO0dJeizsOa
Another issue ,that I mentioned before, is when on real flight the vehicle is not
changing it’s orientation when goal position changed and just flying aside. Maybe it’s some PX4 Parameters need to be changed… Here the bag file:
https://drive.google.com/open?id=1gj8d2TnCoU420iSI6RGpiCC2GvscyRtR
In simulation , when goal is changed, the drone is turning to the flight direction and then flying.
For any case I recorded it also: https://drive.google.com/open?id=1RzKUB_DIKwIdOXaREQ6mMwMbdoNfQ7PZ
Best Regards
Evgeny
