Computer vision

I am interested in employing computer vision directly embedded on a drone, mostly using OpenCV (but not limited to it). That means I need a powerful enough board for the vision computations, which may nowadays be satisfied with an embedded GPGPU approach. I looked around and found nVidia Jetson TK1 (300 GFLOPS with CUDA) and Odroid-XU3 (100 GFLOPS with OpenCL) to be probably the best boards for my needs. According to my findings, both boards should be able to do both mono and stereo vision tasks in real time. The Jetson has a nice wiki with some performance info.

My question is - what are the options for making a computer vision work on a drone in real time? The vision algorithms should be able to connect back to the autopilot and control the both video and CV camera or fly path altogether for object tracking or collision avoidance.

I can think of 2 ways of making CV work in real time:

  1. Use an existing autopilot solution (standalone APM/Pixhawk board, Navio with RPi etc.) and interface (via GPIO?) from the CV board back to the autopilot board (to be able to change flight path etc.). This requires 2 boards, which means the drone is heavier and draws more power. It would also require developing the SW interface between both boards.

  2. Run Navio directly on the CV board. A single board would be needed. This may be better for weight and power. It would probably include porting APM to the CV board and connecting the HW together - the Jetson has some GPIO which may probably be used (?), but I am not sure about the Odroid-XU3.

I generally consider 2 cases for the computer vision tasks: mono camera and stereo camera. I am particularly interested in the stereo vision, but making mono work is a goal too. The stereo vision will probably further complicate the setup, because a good camera synchronization is needed. I found a relatively cheap global shutter USB camera with an external trigger (can do the synchro) - it is the e-con Systems See3CAM_10CUG. This camera was reported to be working on the Jetson TK1 (link here) and Odroid-XU (not XU3, link here). However, it requires a custom kernel in both cases which may complicate everything even further. Anyways, there may be other cameras (probably of scientific grade, more expensive - Point Grey etc.) that work flawlessly with these boards, I have just not investigated them into great depth.

Could you please comment on the options (2 boards vs. 1 board, mono vs. stereo) above? Which board setup apart from cameras would you recommend? Could you help with making Navio + APM work on those CV boards? Are there any other options that may include some other hardware I did not list above? I see you work on the Odroid-C1 port so maybe you could get some support from Hardkernel for the XU3… However, the nVidia Jetson has much better performance so I actually would like to use it over Odroid.

I noticed Parrot did some 3D reconstruction from stereo demo at CES 2015 (here and here) with Jetson most likely as an extra board. Also 3D Robotics parthered with Intel and seems to be experimenting with object tracking on Edison.

Thanks for your insights in advance!

@kozuch even though extremely powerful unfortunately both XU3 and TK1 lack peripherals such as I2C,SPI, UART to make them usable for APM. What is great about using it with Navio+ is that you can simply connect boards over Ethernet. Porting APM to a new board is also a challenging task, Odroid’s to our sadness have poorly supported peripherals drivers.

For the most challenging tasks I would probably go for TK1 and Navio+ Rpi2 connected over Ethernet.

Board choice would really depend on how much processing power you would need for your project.

@ivereninov

Thanks for a reply. Looking at Jetson TK1 specs I see a note “UART and i2c available through an expansion port” (GPIO meant probably). More info is on the Jetson wiki here: UART and i2c. I found some info about SPI too. The resources are quite scattered all over the wiki so there might be actually even more info available I think.

For the possible Jetson TK1 APM port I think there may be some assistance from nVidia possible because I think they really are trying to market their mobile GPUs for GPGPU so a working autopilot would definitely raise their marketing potential for computer vision applications… Also, the Jetson board is probably completely different grade than Odroid (regarding the issues you mention) since nVidia is huge company an may have much bigger budgets than Hardkernel may even think of…

Regarding the Ethernet - could you please elaborate here a bit how that is supposed to work? If I understand it correctly APM should be able to accept commands over Ethernet? I think a rather low latency would be handy for the CV board to be able to navigate the autopilot - can Ethernet do that?

@kozuch,
This is Anitha, part of Camera team at e-con Systems.

The kernel changes that we have mentioned in our articles are to be done to enable the USB 3.0 mode in Jetson TK1 kit, otherwise the USB host in the jetson Kit will work in USB 2.0 mode.

These changes has to be done if you have to enable the USB 3.0 operation mode, It is not something which has to be done specifically only with our cameras

Also you can check our recently launched e-CAM130_CUTK1 - 13MP MIPI Camera for Jetson TK1 kit or See3CAM_CU130 - 13MP USB 3.0 Camera

1 Like