RT kernel: Unusual dwc_otg high 8k/s interrupt rate using quite some CPU?

Navio / Edge Flight stack
1

I am just trying a fresh emlid-raspberrypi2-raspbian-rt-20150401.img.xz kernel image on a Raspberry Pi 2.

I notice, that in top of an idling machine, the top three processes are interrupt handlers, using quite a chunk of load:

   66 root     -51   0     0    0    0 S  10.1  0.0   6:47.89 irq/75-dwc_otg_                                                                                                                                                                                                                                                 
   64 root     -51   0     0    0    0 S   9.1  0.0   6:20.24 irq/75-dwc_otg                                                                                                                                                                                                                                                  
   65 root     -51   0     0    0    0 S   7.5  0.0   4:33.28 irq/75-dwc_otg_

… and indeed, this handles around 8000 interrupts/second:

$ (cat /proc/interrupts ; sleep 5; cat /proc/interrupts) | awk 'BEGIN{m=0} /dwc_otg/ {t[m]=systime(); i[m]=$2; m++;} END { dt=t[1]-t[0]; printf("%d secs; %.2f int/s\n", dt, (i[1]-i[0])/dt);}'
5 secs; 8084.40 int/s

This interrupt should be handling USB and ethernet AFAIK, but there is nothing connected to any USB port, and ethernet port just communicates low-speed via an SSH connection.

Do others see that as well ? It is pretty unusual, and could probably indicate a problem in this kernel.

2

Yes, I have noticed this too. Also, I found CPU to spike even higher and things become sluggish when USB is heavily loaded, eg USB/Serial or USB cam. USB WIFI adaptor is OK though.

3

We are using non-FIQ USB driver which has higher CPU load and interrupt rate. We don’t use default USB driver because it relies on FIQ and it conflicts with Full RT preemption mode.

4

Is there a way to change this behaviour while not using the USB port?

I just installed emlid-raspberrypi2-raspbian-rt-20150401.img on a Raspberry Pi B (new out of the box, but the PCB design is from 2011). Everything is sluggish (unpacking small packages with dpkg takes a few minutes per package), and I also see three irq/75-dwc_otg_ processes hogging the CPU.

5

Meet the same problem, however, old Nov image deleted from download. Emlid, where can i find Nov .img?
This issue make my system unusable, 2nd time after “joke” with Navio 2 RTK support.

6

Hi,

I have noticed the same problem, I have 3 irq/75-dwc_otg_ processes that take up to ~30% CPU on an idling RPi B+. Is there any workaround ?

7

Hi,

We are slowing down our support of old Raspberries (A/B/A+/B+) as they are low performance and upgrade to B2 or Pi3 is only 35$. Thess processes are due to using an older USB driver, which has better stability than the new one. For Navio2/Pi2 we have reworked the software architecture which allows use of new driver that lowers the USB load.

8

Thanks for the answer.

Do you mean that the newest version of RT kernel for RPi 2 would avoid the problem ? The description of the kernel image says that FIQ driver is not compatible with RT so I guess not… Does the difference come from navio2 capabilities then ?

9

With Navio+ we have been using DMA decoding, which required rt priority to run. Navio2 does it on the MCU, so there is no need in rt process. FIQ+RT_Preempt+rt process results in a kernel crash. Also, as RT features are merged into the mainline there is less need in RT and PREEMPT is sufficient. Our next images will very likely be PREEMPT with FIQ enabled. We are still studying stability and performance of different solutions.

10

Pi2 have the same problem with 8k iops, it’s just not so visible as pi 2 is more powerful.
However, adding dwg_otg.speed=1 in cmdline solve the problem… well, not in my case, i need to use usb drive.