Use output data from object recognition script/thread as input data to custom Arducopter flight mode? (Yaw/roll controller based on line found in image/position underneath cable)

Navio / Edge Flight stack
1

So I’m using the current Navio2 image and newest Arducopter quad firmware.

I am doing a project where I have a Navio2 with an upwards facing PiCamera and I want to make the drone position itself underneath a hanging cable and oriented parallel to the line.

I am running an OpenCV script where I can find the angle and distance from left camera frame edge to the line found in the image (HoughLines) so I have the input data I need for a yaw and roll controller.

I was wondering if I could somehow get this angle and distance data straight from my OpenCV script (somehow store it locally in memory or something?) and read it into a custom flight mode in Arducopter, where I could code a controller that would take this data as input and then yaw and roll accordingly. I know that I could write something in Python and send yaw and roll commands through dronekit but I was thinking if it would be possible to go around the MAVLink communication and treat the OpenCV output data as just another “sensor” input that the Arducopter code can read and react to. I would think that this would be better for a custom attitude controller since there is not a delay in sending the data through a mavlink message.

All help is appreciated!

2

Well, of course that’s possible, but when you already have the angles it is closer to movement control than sensor data. Using the DroneKit seems like the right way for the task, I suggest to try it first, there shouldn’t be a noticeable delay.

3

What you are wanting is possible, but there is no simple plug and play answer. You will need to create the sensor(open cv python program to process pixels information, then output useable data), then use drone kit to send those command to the aircraft. There is no way to go around mavlink, However, it is less painful with dronekit. I am doing similar things with optical flow. No custom flight mode is needed, Guided mode is already used for “off board” control.