Hi,
I’m setting setup camera triggering with a point grey flea3 camera and have been looking at the docs here (which are great btw). The camera is set to auto exposure and depending on the light sometimes the shutter speed goes much longer than the trigger cycle time (e.g. I started out simply triggering at 30Hz or 33.3 ms cycle time and sometimes the shutter speed would be 60-70 ms). The result is that there were missed triggers. I haven’t dug in super deep but assume the autopilot still sent out the CAMERA_TRIGGER msg even though the camera didn’t acknowledge that trigger and am trying to figure out if that will cause things to get out of sync.
I haven’t got it to work yet, but I know the flea3 supports sending out a pulse when it actually begins to take the image. Is it possible to configure the GPIO of the autopilot to look for that pulse and upon detecting it then send out the CAMERA_TRIGGER message? I assume that functionality doesn’t currently exist but wonder if it’s possible without too much work (which I would be happy to help out with). Or is there a better way to make sure things robustly stay in sync?
Also, I’m a little curious-- on the docs page I reference above the comment
To be clear, the idea here isn’t to take an IMU measurement exactly at the same time as we take a picture but rather to correctly time stamp our images so as to provide accurate data to our VIO algorithm
Why is that case? Why is it preferable to just correctly timestamp the images instead of taking a snapshot of the IMU right as you take the picture? A naive guess would be that it has something to do with taking the picture is not an instantaneous event but is an event with a non-zero duration and doing it this way allows the user to decide what to do.
The camera feedback signal can trigger the capture module to accurately timestamp the images. This functionality is under testing and will be introduced in this PR. Feel free to help us testing it and give feedbacks.
About VIO, the EKF takes high rate IMU measurements and low rate images. It doesn’t need to have an IMU measurement when the image is taken, but it needs to know when the image is taken and when the IMU data is sampled and can deal with the delay between them (IMU is used for preduction and image for update step).
That is fantasic. One minor snag that may make it hard for me to test though is that I’m using a PixRacer–if I’m reading the comments right it only works with FMU pin 5 (not Aux pin 5). Is that right? If I can use Aux pin 5 I should be able to test no problem.
That is great–I’ve been using the original pixhawk and a pixhawk 2.1 along with ArduCopter. I’ve wanted to switch to the px4 flight stack (which I think is much better) for a while and am finally able to along with switching to a pixracer. How do I go about pulling those changes in? Here’s what I’ve done so far:
Installed the toolchain and all the prereqs
Forked the PX4/Firmware repo to my own github account and cloned my fork locally.
Added the PX4/Firmware as an upstream remote on my local copy
added drivers/camera_capture to Firmware/cmake/configs/nuttx_px4fmu-v4_default.cmake
built firmware via make px4fmu-v4_default
I wasn’t able to use make px4fmu-v4_default upload to upload the firmware as the pixracer just sat waiting to be reset (couldn’t figure out how to do that) so I copied airframes.xml, parameters.xml and px4fmu-v4_default.px4 to my windows partition where I have QGroundControl installed (dev environment is an Ubuntu Virtualbox VM) and flashed with QGroundControl.
After rebooting it after the flash, the parameters only appear in QGC if you search for CAM_CAP. They don’t show up under the Camera Control Group.
As sort of a first order test I opened QGC mavlink inspector and verified initially no CAMERA_TRIGGER messages are being sent and then shorted the signal pins on outputs 5 and 6 together (i.e. feed the camera trigger directly into the capture input) and expected to see the autopilot send the message out at 30 Hz. However I observed that it was not.
Configuration was
TRIG_PINS=6
TRIG_MODE=time based, always on
TRIG_INTERVAL=33.3ms
TRIG_ACT_TIME = 1ms
CAM_CAP_DELAY=0
CAM_CAP_EDGE=0
CAM_CAP_FBACK=1
CAM_CAP_MODE=0.
I hooked an oscilloscope up to output 6 and it appears that when the capture feedback is enabled, the trigger signal on pin 6 stops being sent out. The only way to get it back is setting CAM_CAP_FBACK=0 and rebooting the autopilot.
@jasonbeach did you solve this issue? The reason for this behavior is that capture channel 5 is actually pin 6 on the fmu outputs. So basically your trigger was deactivated on pin 6 to allow the capture driver to use it?
Out during testing I discovered that with TRIG_PINS=6 the vehicle wouldn’t even arm and made lots of angry beeps when powering on. Changing it back to 56 allows it to work. So the plan at least while using the pixracer is to just let the camera (a point grey flea3) run in free running mode which will allow us to roughly control the shutter speed via software and use the strobe from the camera to trigger the autopilot. I should then be able to use the feedback mechanism, but haven’t had a chance to test it.
