Emlid

Community Forum

Project HawkEye - Cloud Control Station


(Johan) #1

Hello Community!

I have been lurking around this forum for a while and I think its time for me to share what Im up to.

I am currently building a Cloud Control Station for my quadcopter which the goal is to be able to manage one or many drones constantly connected to the Internet and ready for take off with the possibility to stream video using gstreamer connected to a video server for multiple viewers.

The actual CCS would be very simple in its design with just a few actions such as

  • MODE (set modes)
  • ARM
  • TAKEOFF
  • RTL
  • LAND
  • GOTO (GUIDED)
  • SPEED
  • ALTITUDE

First version of the PoC that is shown on the video is a web server based on Python using DroneKit and the version right now Im working on now has been re-written to use pure Node.js and Mavlink messages to communicate with the ArduCopter on TCP port 5760.

The hardware Im using right now has not been optimized for flight time but for functionality.
My Quad

  • S500 FR4 Glass Fiber Quad Copter 480mm
  • Raspberry Pi 3
  • Navio2
  • 4G/Wifi router
  • Logitech C900 HD

In terms of connectivity, I will be using SSH tunnels between the Drones and CCS.

Prototype in action, first test video.


(Mikhail) #2

Hey Johan! That’s pretty cool. How is the experience with WebRTC?


(Johan) #3

Thank you Mikhail!
Its working pretty well. I did for a longer time do Peer-to-Peer WebRTC sessions before I switched to a video server due to the fact that you can have one stream source that multiple users can view and is a lot more stable than a P2P session.

For a longer period Firefox was the goto browser for a smooth video, but since Chrome did some changes in the video decoding a few month ago its working as well in Chrome.

So the flow is as following
C900–> RPi --> GStreamer --> (RTP) Janus Gateway – WebRTC --> Browser ( Chrome )


(Mikhail) #4

I’m interested what are the latency values for both variants (p2p and server)?


(Johan) #5

I dont have any specific numbers, it all comes down to network connection and encoding and decoding.
You can try for yourself if you want, the PoC P2P version at least, change channel to a unique number :slight_smile:

Video sender
https://www.sounden.se/video/drone?channel=1

Video receiver
https://www.sounden.se/video/operator?channel=1

Right now Im using Pusher as a signaling service, would work with Socket.IO or similar too.


(Marcello) #6

Are you using DroneKit to feed info such as the speed, altitude, etc? Because you mention both that you are using DroneKit and SSH? Are you using sending the telemetry data using DroneKit over SSH?


(Johan) #7

Hi Marcello,
In the PoC I made with the video attached to the initial post is using DroneKit and also running the webserver to connect to it directly on the Raspberry Pi and connected from the computer via WiFi, not an optimal solution but I just wanted to put together some existing components.

In the version Im buildning now I will be tunneling Mavlink messages from the APM (ArduCopter - Ardupilot) via SSH between the user and UAV. All those Mavlink messages will contain all information that is usually displayed in a GSC. The Goal is to make it as simple as possible to remote control UAV´s, so all data that the mavlink provides will probably not be available at a glance.

Here are my thoughts of how to put it together :wink:

Some Ideas for the UI Login

Some Ideas for the UI UAV Management

//Johan


(Marcello) #8

Wow love that your keeping it real and staying active. I also really love the super slick interface, I’m surprised by how many web dev have clunky portals. I agree the SSH is a much more clever way than dronekit. However, dronekit may still allow some additional features, maybe could be incorporated as a “module.” Im fairly new to web development and Im still getting the drift of swift, so pardon me for possibly basic questions.

I was curious as to what you were using socket.io for, is this to send the mavlink data in real time between the client and web server? Also Im very curious what your using Ruby for it says webapps what do you have in mind for this?

I would be willing to help in anyway possible, as I said unfortunately I’m relatively new to web dev beyond bootstrap and css which I’ve already dived in to my liking (I’m sure I could get better at handling CSS). However, if you need any beta testers or anything I would be happy to! :slight_smile:

As always keep up the great work :thumbsup:


(Johan) #9

Thank you for the kind words @Deathstroke5467!

The DroneKit-Python or mavproxy is not an option for me right now since Im not using Python at all in this setup. Yes, the Socket.IO is the communication layer between the drone and client, basically an interpreter from web to mavlink.

The cloud solution will consist of three parts.

  1. Mainly Bash shell-scripts that is used to register the UAV at the HawkEye service and keep the tunnel online. Should also control start-stop GStreamer with different video resolutions ( this I have written a small C binary that responds to messages to a specific TCP port )

  2. Rails Web app that provides configuration settings such all available tunnel servers, dynamically assign unique ports to the tunnels. This will also be where the user interacts with the HawkEye services, user admin/mission controls etc.

  3. Node.JS - The node will be a middleware gluing the previous two functions. They will scalable and work as tunnel servers and also be responsible for the real-time communication between the user and the UAV.

Yes, I would love testers who are ready to crash a drone or two when I have gotten a bit further :wink:

I made some really good progress tonight so now Im able to register new UAV´s in the from the command line on the RPi´s and establish tunnels that is accessable externally for testing using other Ground Control software.

Ill keep you posted!


(Fredrik Falkman) #10

Way to go Johan! i Will kopp following your progress with great interest.
Adding a link to my Cloud Control Station project call for anyone to see how a solution like this could be used for good such as sea rescue: http://www.surtsey.org/projects/cloud-control-station/


(Johan) #11

Hellu!
You might think I have been lazy this weekend, no not really!

Great progress on the platform. This weekend I have been focusing on putting all pieces together to establish required connectivity in terms of setting up SSH tunnels to the node servers and authenticate the raspberry pi automagically.

The connectivity is basically done, it will set up two tunnels for each UAV.
One for MAVlink connectivity and another one for a daemon I made called Dcontrol.

The platform will dynamically assign a tunnel server and two unique ports when starting the UAV client on the RPi.

The purpose of the Dcontrol is to execute commands locally on the UAV, such as start video, stop video, reboot. This commands will be initiated by the user in the webUI and transmitted to the RPi through the dcontrol ssh tunnel.

Right now Im testing out different resolutions on the video stream currently 360p, 720p and 1080p.
The latency of the 360p is below 1 second which is good, but it gets higher with the resolutions.

The video is just my desktop being recorded with Quicktime when I did the first flight this weekend (2017-04-08)

ps. dont mind the UI this is just a functional test with some buttons :wink:


(Johan) #12

Hello again!

After some silence since due to the hard on this last weeks.
Here are some updates from the project.

Currently implementing Front-End/UI in React on the Ruby on Rails backend.
Next steps is to actually connect it to the communication server who receives the MAVlink communication between user and UAV.

The process of getting your UAV added to the platform is very easy.

  • Login to your RPi (ssh)
  • Clone a public git repo
  • Run installer script
  • Add you AUTH_TOKEN in the config file (which you get from the platform when signing up)
  • Run the start script

By doing this you will create 3 SSH tunnels

  • One tunnel for SSH access
  • One tunnel for MAVlink tcp traffic
  • One tunnel for D Command (a custom binary daemon) which you can send commands such as start/stop video etc.

For the MAVlink tunnel you can also connect with other GCS if you like as long as you have internet access.

Hope you like it.
To be continued.


(Johan) #13

Hello, some update from the project.

Working with the UI/UX, its really hard to nail since its so much information and controls that are needed.
Here are the latest update of the design, hope you like it.

My plan is to have a closed beta in end of May, let me know if you are interested to try it out.
Requirements for testing the platform is basically a RPi with internet connection (Wifi,4G) and compatible APM that runs ArduPilot.

Best Regards,
Johan


(Richard Balauro) #14

Great effort Johan! I am a new product developer, Would love to test it for competition prototype phase. Looks promising


(Johan) #15

Thank you Richard,
I will let you know when I have something that others can test :slight_smile:

//Johan


(Николай Л) #16

Hi Johan,

How is your project? ) Is it possible to test it?


(Marcello) #17

@11113

I had worked with Johan on the shell scripts, you can find the bitbucket here. Though it hasn’t been updated in about 6 months now. I also checked the website darone.io and it looks like the backend is no longer there as I cannot log in. Only the landing page seems to be working and 2 drones are registered, so it seems it is either dead in the water or has been put on the back burner for now.

I remember Johan had been working on this for some sort of search and rescue organization so perhaps either it was not received as well as expected or was put on hold as he got busy with other things. I’ve tried contacting him once I can try again as I was interested myself. I’ll tag him her just in case he still checks @johan.

I am planning on making my own implementation of this project but it is low on my current list of projects and I have many other web dev things to work on before I get there.


(Николай Л) #18

Marcello, thank you for info!

Anyway, I’m making my own solution to communicate with Ardupilot from web-server.
Now it’s possible to send MAVlink to server, get data from messages and translate it to web-client. Also it’s possible to transmit video. MAVlink and video use socket.io to communicate with server and web-client.

If anybody needs this, here is the repos:

One for board and one for server.


(Marcello) #19

@11113 Awesome, thats actually how I was going to build mine, by forwarding mavlink to server. Darone actually used SSH access, which both has it’s advantages and disadvantages. I thought it to be too complicated. I may make my own implementation down the road but I will definitely follow your project!

Maybe I can help you out with the project if you need anything? :slight_smile:
I’ve been working a lot on React a lot so we might be able to throw together a nice frontend.