Quadplane ghost spinning of rotors

antonvh · September 19, 2017, 6:32pm

Image and ArduPilot version.
Emlid image, Arduplane 3.8.2 and 3.8.0b5, binaries compiled by emlid.
Detailed problem description. How does observed behavior differ from the expected one?
Quad motors spin up randomly and unexpectedly when arduplane is disabled. Everything seems fine but every 10-20s they shortly spin up and stop again.
Your step by step actions.
I calibrated the ESCs. Verified that by connecting them directly to the Taranis X4R-SB without navio. No ghost spinning up here. Solid performance of the ESCs.

The 4 quad ESCs are 40Amp optos with SimonK. They are fed 5V through the navio servo rail by a fifth ESC for the back propeller.

I have Q_ENABLE = 1 and Q_ESC_CALIBRATE = 1. The calibration procedure works fine. The ESCs respond to throttle correctly when Q_ESC_CALIBRATE = 1 and the plane is ARMED.

The weirdness starts when the plan is disarmed. Random spins and inconsitent throttle. It is driving me completely crazy. I replaced all quad ESCs with fresh ones, just to be sure, but that had no effect. I replaced the navio with a fresh one from a sealed box. No effect.

When looking at MAVLink data Servo_raw on channels 5,6,7,8 show a neat 982 pwm.

I also tried turning on PWM_TYPE = 2: ONESHOT125. Again mavlink shows neat 125 output when disarmed, but the motors twitch. And again in Q_ESC_CAL = 1 and Q_STABILIZE mode the ESCs respond fine to throttle output. No Twitching.

It scares me, because if the motors would have props it could potentially cost fingers. I’m totally mystified. No idea where to look next.

Picture of the setup and connection scheme.
It’s simple and standard. 4 opto ESCs connected to servo out 5,6,7&8 on the navio. 1 UBEC ESC on 3 to power the rail. X4R-SB rx on PPM/SB in. There is a radio modem on the USB of the RPi. But I unplugged it to be sure and the problem stayed. And there is of course a power module and a GPS antenna.
If using any additional hardware please provide exact make and model.

schuermannsebastian · September 20, 2017, 4:22am

Hi,

with arducopter you have to put servos on output 9-14, because activating servo functions on 5-8 causes the motors on 1-4 to twitch. This sounds like that problem. Try to connect all the ESCs on output 9-13.

antonvh · October 25, 2017, 5:44pm

I’m on the verge of throwing the Navio2 out of the window and getting a pixhawk. The twitching just won’t go away.

Here are the steps I took:

New Navio2, out of the box
Downloaded latest Emlid stretch image
apt-get update and apt-get upgrade
Removed all connections to the navio2 except power module, telemetry on USB, rx on cppm in, digital coreless servo on SERVO5, ESC with 5v 2A BEC on SERVO3.
Configured ArduPlane 3.8 in emlidtool
Set Q_ENABLE to 1

And voila: the servo on channel 5 starts to twitch.

Then I tried (in vain)

removing the telemetry
moving to channels 10-14 with function 33-36 (quad motor outputs)
moving the ESC bec to channel 10
rebooting
removing the rx
adding a glitchbuster with a 4700microfarat condensator over +5 and GND

This makes the glitch go away:

Running arducopter on the Navio2
Going into ESC calibration mode by rebooting after full throttle

Now the coreless digital servo’s move 1:1 with the throttle stick, no glitches. So the 3.8 quadplane code compiles badly on Navio2. No Idea how to fix that… :(((

NikitaTomilov · October 26, 2017, 11:01am

Hello!

Can you provide us with the full information of your setup?

Arduplane config you are using now (including plane type, which motors/servos are connected to which pins and so on)
RCIO module’s output from dmesg while the glitch occures

antonvh · October 26, 2017, 11:40am

Sorry. I was quite frustrated yesterday evening, because it had taken me so much time and checking to figure this out.

Here is the test setup. Will post the rest later.

antonvh · October 26, 2017, 12:09pm

I did never run

emlidtool rcio update

I will try that too. Never knew it could be done.

Do you want the output from this?
dmesg | grep -i rcio

NikitaTomilov · October 26, 2017, 12:24pm

Yes, do this and see if it helps. It should bring new PWM frequency configs to RCIO MCU.

Yes, it should show PWM channel configuration results.

antonvh · October 26, 2017, 3:41pm

[    3.533299] rcio_core: loading out-of-tree module taints kernel.
[    3.553620] rcio spi1.0: rcio_pwm: Advanced frequency configuration is not supported on this firmware
[    3.569885] rcio spi1.0: rcio_status: Firmware CRC: 0x793d141e
[    3.570567] rcio spi1.0: rcio_status: Could not read board type
[    3.576255] usbcore: registered new interface driver brcmfmac
[    3.780790] brcmfmac: Firmware version = wl0: Aug  7 2017 00:46:29 version 7.45.41.46 (r666254 CY) FWID 01-f8a78378
[    4.048727] random: crng init done
[    4.290567] IPv6: ADDRCONF(NETDEV_UP): intwifi0: link is not ready
[    4.290578] brcmfmac: power management disabled
[    4.741297] Adding 102396k swap on /var/swap.  Priority:-1 extents:1 across:102396k SSFS
[    4.879427] smsc95xx 1-1.1:1.0 enxb827eb78aedd: hardware isn't capable of remote wakeup
[    4.879739] IPv6: ADDRCONF(NETDEV_UP): enxb827eb78aedd: link is not ready
[    5.312791] IPv6: ADDRCONF(NETDEV_CHANGE): intwifi0: link becomes ready
[    8.396920] Please note that only frequency changes on pins 0, 4, 8 and 12 count. This error could occur if you put a servo and a motor on the same pwm group. For additional information please refer to the documentation.
[    8.396939] Please note that only frequency changes on pins 0, 4, 8 and 12 count. This error could occur if you put a servo and a motor on the same pwm group. For additional information please refer to the documentation.
[    8.396955] Please note that only frequency changes on pins 0, 4, 8 and 12 count. This error could occur if you put a servo and a motor on the same pwm group. For additional information please refer to the documentation.
[    8.397011] Please note that only frequency changes on pins 0, 4, 8 and 12 count. This error could occur if you put a servo and a motor on the same pwm group. For additional information please refer to the documentation.
[   74.874345] Please note that only frequency changes on pins 0, 4, 8 and 12 count. This error could occur if you put a servo and a motor on the same pwm group. For additional information please refer to the documentation.
[   74.874562] Please note that only frequency changes on pins 0, 4, 8 and 12 count. This error could occur if you put a servo and a motor on the same pwm group. For additional information please refer to the documentation.
[   74.874771] Please note that only frequency changes on pins 0, 4, 8 and 12 count. This error could occur if you put a servo and a motor on the same pwm group. For additional information please refer to the documentation.
[   77.698467] Please note that only frequency changes on pins 0, 4, 8 and 12 count. This error could occur if you put a servo and a motor on the same pwm group. For additional information please refer to the documentation.
[   77.698486] Please note that only frequency changes on pins 0, 4, 8 and 12 count. This error could occur if you put a servo and a motor on the same pwm group. For additional information please refer to the documentation.
[   77.698523] Please note that only frequency changes on pins 0, 4, 8 and 12 count. This error could occur if you put a servo and a motor on the same pwm group. For additional information please refer to the documentation.
[   77.698544] Please note that only frequency changes on pins 0, 4, 8 and 12 count. This error could occur if you put a servo and a motor on the same pwm group. For additional information please refer to the documentation.
[  299.911249] Please note that only frequency changes on pins 0, 4, 8 and 12 count. This error could occur if you put a servo and a motor on the same pwm group. For additional information please refer to the documentation.
[  299.911264] Please note that only frequency changes on pins 0, 4, 8 and 12 count. This error could occur if you put a servo and a motor on the same pwm group. For additional information please refer to the documentation.
[  299.914593] Please note that only frequency changes on pins 0, 4, 8 and 12 count. This error could occur if you put a servo and a motor on the same pwm group. For additional information please refer to the documentation.
[  299.914612] Please note that only frequency changes on pins 0, 4, 8 and 12 count. This error could occur if you put a servo and a motor on the same pwm group. For additional information please refer to the documentation.
[  299.917934] Please note that only frequency changes on pins 0, 4, 8 and 12 count. This error could occur if you put a servo and a motor on the same pwm group. For additional information please refer to the documentation.

antonvh · October 26, 2017, 3:41pm

…and when I try to update:

pi@navio:~ $ sudo emlidtool rcio check
current: 0x793d141e
local: 0xcaec2284

You need to update. Please run:
emlidtool rcio update
pi@navio:~ $ sudo emlidtool rcio update
updating firmware using /lib/firmware/rcio.fw
Traceback (most recent call last):                          | 0.0% Complete
  File "/usr/local/bin/emlidtool", line 11, in <module>
    sys.exit(main())
  File "/usr/local/lib/python3.5/dist-packages/emlid/emlidtool.py", line 243, in main
    emlidtool.parse_args()
  File "/usr/local/lib/python3.5/dist-packages/emlid/emlidtool.py", line 170, in parse_args
    getattr(self, args.command)()
  File "/usr/local/lib/python3.5/dist-packages/emlid/emlidtool.py", line 219, in rcio
    self.func(rcio, args)
  File "/usr/local/lib/python3.5/dist-packages/emlid/emlidtool.py", line 224, in func
    args.func(args)
  File "/usr/local/lib/python3.5/dist-packages/emlid/util/util.py", line 25, in wrapped
    return fn(*args, **kwargs)
  File "/usr/local/lib/python3.5/dist-packages/emlid/emlidtool.py", line 88, in update_rcio
    updater.update(quiet_update=False)
  File "/usr/local/lib/python3.5/dist-packages/emlid/util/util.py", line 25, in wrapped
    return fn(*args, **kwargs)
  File "/usr/local/lib/python3.5/dist-packages/emlid/rcio/updater.py", line 29, in update
    self.connect()
  File "/usr/local/lib/python3.5/dist-packages/emlid/rcio/updater.py", line 49, in connect
    print_progress_bar(10, 100, prefix='Getting connection')
  File "/usr/local/lib/python3.5/dist-packages/emlid/util/util.py", line 67, in print_progress_bar
    print('\r%s |%s| %s%% Complete' % (prefix, bar, percent), end='\r')
UnicodeEncodeError: 'ascii' codec can't encode characters in position 21-25: ordinal not in range(128)

NikitaTomilov · October 26, 2017, 4:00pm

This line means that you should update rcio firmware to get rid of some software limitations that may cause troubles in your particular configuration.

You should be fine after doing

sudo pip3 install -U emlidtool

as it is described here: Raspbian Stretch Release for Navio with ArduCopter-3.5 and ROS Kinetic - #10

antonvh · October 30, 2017, 7:31am

That does not work:

pi@navio:~ $ sudo pip3 install -U emlidtool
Exception:
Traceback (most recent call last):
  File "/usr/share/python-wheels/urllib3-1.19.1-py2.py3-none-any.whl/urllib3/connectionpool.py", line 350, in _make_request
    self._validate_conn(conn)
  File "/usr/share/python-wheels/urllib3-1.19.1-py2.py3-none-any.whl/urllib3/connectionpool.py", line 837, in _validate_conn
    conn.connect()
  File "/usr/share/python-wheels/urllib3-1.19.1-py2.py3-none-any.whl/urllib3/connection.py", line 323, in connect
    ssl_context=context)
  File "/usr/share/python-wheels/urllib3-1.19.1-py2.py3-none-any.whl/urllib3/util/ssl_.py", line 324, in ssl_wrap_socket
    return context.wrap_socket(sock, server_hostname=server_hostname)
  File "/usr/lib/python3.5/ssl.py", line 385, in wrap_socket
    _context=self)
  File "/usr/lib/python3.5/ssl.py", line 760, in __init__
    self.do_handshake()
  File "/usr/lib/python3.5/ssl.py", line 996, in do_handshake
    self._sslobj.do_handshake()
  File "/usr/lib/python3.5/ssl.py", line 641, in do_handshake
    self._sslobj.do_handshake()
socket.timeout: _ssl.c:704: The handshake operation timed out

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/share/python-wheels/urllib3-1.19.1-py2.py3-none-any.whl/urllib3/connectionpool.py", line 594, in urlopen
    chunked=chunked)
  File "/usr/share/python-wheels/urllib3-1.19.1-py2.py3-none-any.whl/urllib3/connectionpool.py", line 353, in _make_request
    self._raise_timeout(err=e, url=url, timeout_value=conn.timeout)
  File "/usr/share/python-wheels/urllib3-1.19.1-py2.py3-none-any.whl/urllib3/connectionpool.py", line 313, in _raise_timeout
    raise ReadTimeoutError(self, url, "Read timed out. (read timeout=%s)" % timeout_value)
requests.packages.urllib3.exceptions.ReadTimeoutError: HTTPSConnectionPool(host='www.piwheels.hostedpi.com', port=443): Read timed out. (read timeout=15)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/pip/basecommand.py", line 215, in main
    status = self.run(options, args)
  File "/usr/lib/python3/dist-packages/pip/commands/install.py", line 353, in run
    wb.build(autobuilding=True)
  File "/usr/lib/python3/dist-packages/pip/wheel.py", line 749, in build
    self.requirement_set.prepare_files(self.finder)
  File "/usr/lib/python3/dist-packages/pip/req/req_set.py", line 380, in prepare_files
    ignore_dependencies=self.ignore_dependencies))
  File "/usr/lib/python3/dist-packages/pip/req/req_set.py", line 487, in _prepare_file
    req_to_install, finder)
  File "/usr/lib/python3/dist-packages/pip/req/req_set.py", line 428, in _check_skip_installed
    req_to_install, upgrade_allowed)
  File "/usr/lib/python3/dist-packages/pip/index.py", line 465, in find_requirement
    all_candidates = self.find_all_candidates(req.name)
  File "/usr/lib/python3/dist-packages/pip/index.py", line 423, in find_all_candidates
    for page in self._get_pages(url_locations, project_name):
  File "/usr/lib/python3/dist-packages/pip/index.py", line 568, in _get_pages
    page = self._get_page(location)
  File "/usr/lib/python3/dist-packages/pip/index.py", line 683, in _get_page
    return HTMLPage.get_page(link, session=self.session)
  File "/usr/lib/python3/dist-packages/pip/index.py", line 792, in get_page
    "Cache-Control": "max-age=600",
  File "/usr/share/python-wheels/requests-2.12.4-py2.py3-none-any.whl/requests/sessions.py", line 501, in get
    return self.request('GET', url, **kwargs)
  File "/usr/lib/python3/dist-packages/pip/download.py", line 386, in request
    return super(PipSession, self).request(method, url, *args, **kwargs)
  File "/usr/share/python-wheels/requests-2.12.4-py2.py3-none-any.whl/requests/sessions.py", line 488, in request
    resp = self.send(prep, **send_kwargs)
  File "/usr/share/python-wheels/requests-2.12.4-py2.py3-none-any.whl/requests/sessions.py", line 609, in send
    r = adapter.send(request, **kwargs)
  File "/usr/share/python-wheels/CacheControl-0.11.7-py2.py3-none-any.whl/cachecontrol/adapter.py", line 47, in send
    resp = super(CacheControlAdapter, self).send(request, **kw)
  File "/usr/share/python-wheels/requests-2.12.4-py2.py3-none-any.whl/requests/adapters.py", line 423, in send
    timeout=timeout
  File "/usr/share/python-wheels/urllib3-1.19.1-py2.py3-none-any.whl/urllib3/connectionpool.py", line 643, in urlopen
    _stacktrace=sys.exc_info()[2])
  File "/usr/share/python-wheels/urllib3-1.19.1-py2.py3-none-any.whl/urllib3/util/retry.py", line 315, in increment
    total -= 1
TypeError: unsupported operand type(s) for -=: 'Retry' and 'int'

this host is down: www.piwheels.hostedpi.com
(Why are you using it anyway instead of a package manager from github?)

404 here: https://github.com/emlid/emlidtool

NikitaTomilov · October 30, 2017, 11:27am

Can’t reproduce it, please try updating again. The mentioned host is up and running.

If it still doesn’t work I will post instruction on how to upgrade the firmware manually.

antonvh · November 9, 2017, 12:47pm

Yes! It works now. Finally clean flight with the Quad motors!

Thanks for the help.