Large Wifi and NMEA over TCP Delay

I currently have 5 total Reach RS+ units.

1 Base Station
2 Units for 2 Rovers (accurate heading / bearing)

A Mifi Hotspot is being used as an access point that the receivers, laptop, and RPi connect to.

1 of the 4 rover units appears to occasionally operate with a large network latency.

When everything is working wifi ping times are averaging 5ms for all devices on the network.

Code running on the Pi validates that a fix from each receiver is received within a small window from each other or it throws out the fix and doesn’t calculate machine heading. These events are logged so it it obvious when this occurs.

During these high network latency events pinging the Reach RS+ unit results in a 2000ms to 12000ms ping time. Pings to all other devices are averaging 5ms. The TCP NMEA packets eventually all arrive but they are old so are ignored.

As previously mentioned this is only happening with 1 of my 4 rover receivers. After a couple minutes the ping time will slowly improve back to around 5ms.

My next anticipated steps are to try and ssh into the Reach RS+ unit and view top to see if this a CPU issue on the receiver.

I’ve verified that the settings are the same for all receivers so it is unclear why this is only happening on 1 of the 4 receivers.

Hi @kylelanman,

May I ask you to send the Full System Reports from two of your units: one with the issue and another working normally? Please, send them to me in PM.

Could it be that the system has ‘filled’ a log file, and is using system resources to do a scheduled job? e.g. RINEX conversion of UBX logs and/or zipping of the log files?

Are the logging settings identical on all receivers (including the ones in the 'settings page)?

What is the interval between the periods of delay? Hours?

Does disabling the logging avoid the problem?

The logging settings are the same on both receivers.

One has been in service for longer than the other one but they both have adequate storage space available.

During my 1.5 hour of testing when I discovered the issue the problem occurred twice. I don’t have exact times but I would say within an hour of each other.

I will disabling the logging… I don’t need it. I thought I had it disabled but it appears that a software updates wipes out some or all settings. I know I have had to reconfigured RTK correction settings and hostnames after a software update in the past. Not sure if this is the correct behavior or not.

It will be interesting to find out if the behaviour will change with the logging disabled.

Hi @kylelanman,

Thanks for the logs!

We’ll look into them and I’ll get back to you once there’s news.

Hi @kylelanman,

May I ask you to reflash the device and see if it helps?

Also, it’d be great if you could run the same tests on our new v2.22.0 firmware version.

I will upgrade the receivers later today hopefully. Is installing the new FW through the ReachView web app equivalent to the re-flashing or is there another procedure for that?

Since having disabled logging we’ve ran about 2 hours of non continuous tests and haven’t seen the problem reoccur so that may have largely been contributing to it.

Hi @kylelanman,

You can find out here how to reflash your Reach RS+. It is not equal to the updating as it resets the unit’s settings.

Please download all of the important data from your unit before you reflash it as it’s going to be deleted during the process.

Hi @kylelanman,

Just wanted to check if you had the opportunity to check your setup with the latest firmware version. Would you mind updating us on whether the issue persists?

Between disabling the logging and updating to 2.22.0 the problem has not reoccured. Note, I haven’t reflashed the device. If the problem happens again I will attempt that. The receivers aren’t easily physically accessible so I was holding off on flashing them given it requires connecting to them over USB.

@kylelanman,

Please keep us posted on the progress.