Logging Software Done! (Almost)

Ok, so it has been a while since my last post. I've been working on getting some robust logging software working on the logging PC. The software is designed to capture from multiple +Point Grey Research cameras at 1280x960 15fps, a +PrimeSense P1080 depth camera at 15fps, a Phidget IMU and a GPS receiver.

Spot the devices...

The outline of the program is a multithreaded logger that uses 4 CPU threads (done with OpenMP)
Thread 0 - Point Grey image acquisition and writing data to disk
Thread 1 - Primesense P1080 Depth and Colour image acquisition
Thread 2 - Read new Phidget data
Thread 3 - Read new GPS data

The most important part of the data is the multi-viewpoint camera data from the Point Grey Cameras as synchronisation between these images is key to accurate reliable 3D reconstruction. As soon as all cv::Mat objects from each camera have returned, Thread 0 writes the image data from each camera as well as writing the image data from the primesense camera and logging the phidget and GPS info to CSV files.

A neat feature of the logger is the dynamic delay function that aims to keep a maximum frame rate.
A timer is started at the beginning of image acquisition and stopped at the end of all the file writing. If this processing time is less than the desired frame rate of 66.67ms (15fps) then a dynamic delay is calculated simply as SleepTime = MAX(DesiredDelay-ProcessingTime,2); cv::waitKey(SleepTime); This dynamic delay means the program doesn't query the camera faster than it's configured for and gives us a nice steady frame rate. The MAX(n,2) protects against -'ve' time values and means it always pauses for at least 2ms to allow for video buffer to draw to frames to the screen. Obviously if the ProcessingTime is >= DesiredDelay then it is not possible to obtain the selected frame rate and the program just runs as fast as it can.

There is still a problem that is unresolved. The USB GPS receiver appears to flood the USB bus. Without the GPS connected, two USB Point Grey FireFly cameras happily work at 1280x960 15fps. When the GPS is added to the system it causes errors on the bus causing frames to be dropped. Now the really odd thing is these errors only appear to happen when the cameras auto adjust exposure. Fixing the shutter and gain values programmatically solves the issue. However this isn't a viable solution as the cameras are going to be mounted on a vehicle that will be driven through scenes that have wildly varying lighting conditions.

This image shows the output of three different cameras with identical fixed shutter and gain values. The top left is the high spec Flea2 and the top right/lower left are the lower cost FireFly cameras. It's interesting to note that the Flea2 has wider spectral sensitivity as it can see the structured light pattern from the PrimeSense camera.

Sort of good...

 In this one we can see a boat load of errors coming from cameras 1-2, the FireFly cameras.

...not so good...

 Here the camera shutter and gain values have been fixed independently from one another. While this looks good, it is not a solution due to the varying lighting conditions it will operate it.

...sort of good.

It is almost complete. Just waiting to get a RS-232 GPS device to remove the USB bus from the equation and hopefully that'll just work...