RPi 2 FIQ USB Driver Problems

Hi guys. I’m using NAVIO+ on Raspberry Pi2 and a Logitech C920. We have had problems with corrupted USB video that we didnt know the reason until we stumbled upon this post where we found other users with the same video corruption problem. In the post it was suggested that the FIQ drivers corrected the problem, and indeed it worked. The problem is that now NAVIO cannot stay online much more than a few seconds, hard crashing.

I took a look at /var/log/syslog and found that the stack trace went trhough scheduling code paths, which seems to indicate it is a RT kernel problem.

This problem is a showstopper for our application. Do you have any suggestions to enable FIQ or to stop the video corruption?
Can you confirm if this particular problem is also present in RPi 1 B+ setup?

Best regards
Pedro Alves

@Pedro_Alves1

Hi Pedro!

The problem is relevant only to RPi2. We’ve experienced the same issue on RPi2 as quoted below:

That’s why we’re still using an old driver which also had had some issues with RT patch.

@mikhail.avkhimenia has fixed the issue with the old driver we’re stuck using for now, though.

But unfortunately, the issue with the FIQ driver has not been solved yet. We aspire to have this issue go away as well. It’ll be awesome if someone with greater insight into the USB driver internals would take a look.

Indeed we have experienced the same problem in RPi 1 but with even lower response from the RPi due to hardware limitations.

We have made this video in order to demonstrate our setup’s behavior with RPi2

Our gstreamer code:

gst-launch-1.0 v4l2src device=/dev/video0 ! video/x-h264,width=1920,height=1080,framerate=30/1 ! h264parse ! rtph264pay ! udpsink host=10.42.0.1 port=6000

I work with Pedro Alves and I would like to summarize the movie: Basically with increasing load USB packets get dropped, just like the behavior that originated the need for a rewritten USB driver stack.

In the movie what we can see is that with no intensive tasks running the video is only slightly corrupted, but as the processing load increases (in this case with ArduCopter-quad launched) the corruption gets significantly worse to the point where is it completely unusable.

The scenario we setup is indeed a worse case scenario because as you know, the ethernet port is also connected internally through USB, which makes an USB video capture redirected through ethernet a very demanding operation to the USB stack. The problem seems to be with isochronous transfers, although I have still to confirm this for myself. This makes the camera not work acceptably in any resolution.

We gave up on capturing USB video with a webcam for now.

@pneves

Have you tried to adjust parameters as stated in our docs?

The FIQ-based USB driver is indeed might be a holy grail for increasing throughput as the old driver could be a bottleneck. As soon as we acquire any results on the issue we update the image.

The docs you posted are for the raspvid camera which has its own camera interface, plus the camera is not 1920x1080@30fps, which is what we have. We have offloaded the camera duties to an ODroid connected through the UART.

Just to summarize this, FIQ driver (fast interrupts) was introduced in Raspberry Pi2 and standard USB driver, which was used in RPi1 did not work for RPi2. Unfortunately new FIQ driver crashes RT kernel, which was a very difficult bug to find. It was one of the main reasons for a delay with real-time Raspbian for Raspberry Pi 2. Mikhail was able to fix the standard driver to work with RPi2 and it works stable with real-time patches.

We are making effort in the direction of fixing FIQ driver, but it is a very complicated task.

1 Like

Has this been resolved in the RT Kernel for the Navio2?

We’re connecting a TeraTower with eight(8) cameras to the RPi2B and USB packets are still getting dropped.

I noticed that the emlid branch repo is 3.18 - which the Raspberry Pi Foundation hasn’t used that for about a year. Unfortunately, it seems that the dwc_otg driver has never been proven to work reliably under -rt patches though.