M2 RTK solution: ReachView LLH vs RTKNAVI DEMO5 data analisys

[fabio_lobo]

The reachview on M2 can be improved to get RTK solution… This is my point.

The challenged reception environment is present for both solutions - reachview and DEMO5.

The same test (the same location… the same base) with M+ and a single frequency antenna, shows that this behavior is exclusive on M2.

On M+, LLH data and POS data are very close to each other, and there is no tendency to one get a better solution, such as happens with M2 - all LLH data shows a worst solution.

The assumption is that this happens even with fix solutions…

On my setup, I’ve got about 35% fixed positions - lookin into it, the results shows the same behavior: reachview RMS is allways bigger then rtknavi.

[vgo195]

I agree. There is no point on testing non-fixed solutions.
What are the RMS differences of the fixed solutions between emlid and demo5?

[fabio_lobo]

Here a graphic showing that the quality position is direct influenced by the % of fix (as every body knows…) - bigger the amount of fix (blue), shorter the RMS (orange).

Looking on the stats of the data, it is not a 100% correlation, in fact, it is a 76% correlation. There are more aspects that infulences the quality - not only if it is a fix ambiguity.

[fabio_lobo]

Let’s see one occupation: one with a good amount of fix data.

The data is from M2 and GRADE antenna, RTK solution from 18km.

• The RTKNAVI point 91% of fix data.
  With all kind of position quality (FIX,FLOAT, ETC…) the 3D RMS is 0,47 m.
  Considering only FIX, it is 0,10 m

• The REACHVIEW aquired 50% FIX positions.
  With all kind of position quality (FIX,FLOAT, ETC…) the 3D RMS is 1,21 m.
  Considering only FIX, it is 0,46 m

So… with Q1, the REACHVIEW RMS solution keeps the tendency of been bigger than the RTKNAVI’s.
On this particular example, this tendency is worst: the ratio increses by near 2… going from 2,57 to 4,60.

So… all results are valid to compare the capability of REACHVIEW and RTKNAVI to get RTK solution - no metter what kind of solution! QED.

[fabio_lobo]

Hello, @dmitriy.ershov.
Have you got the solution data I’ve sent?

@fabio_lobo

M2_CEEU_EMLID.zip (1.4 MB)

I also have the original UBX data, case you need.

[Johannes_Eberenz]

This is consistent with some tests I did with RS2 and RS1:

Under challenging skyview condition, the RS2 internal RTK solution performs significantly worse than RTKLIB demo5 PPS solution, both static and kinematic.
Fewer fixes, and worse accuracy of fixes (against ground measured reference). This also occurred for much shorter baselines (< 50m).

RS1 internal RTK solutions are much closer to RTKLIB demo5 PPS solutions.

So I second @fabio_lobo: RS2 RTK could be much improved in regard to partly obstructed sky view.

@wizprod why bother? Because we’re working in areas with vegetation, and are interested in submeter, not cm. Ground measurements are much more work & more complex.

Emlid can’t do much about it directly, as RS2 uses the ublox internal RTK, while RS1 runs RTKLIB.
Emlid promised me they would try replicating these results under controlled conditions, and present that to ublox. I understand this is a lot of work, as ublox will want to see some solid data recorded with their standard evaluation board… so this might take time.

And running RTKLIB on RS2 would also mean a lot of dev and maintenance work for Emlid…
But might be worth it. After all, the real world is full of places with obstructed sky view.

[svetlana.nikolenko]

Hi Fabio,

Yes, please share the raw data logs as well. It’s interesting to check how the ambient conditions you described affect the signal. I guess it may have a point.

[fabio_lobo]

Hello, @svetlana.nikolenko.

The UBX data is on the original name and compressed format.
The ZIP file, with all archives, has 236 mb… here the link to it.

M2_RAW.zip

@svetlana.nikolenko

It’s interesting to check how the ambient conditions you described affect the signal.

The ambient conditions was not so good for both softwares (reachview and demo5)

Demo5 had got (!ALL and EVERY TIME!) a much better solution - some times, as shown, 4x better (…) with , in every case, data from the same constelation, same time and same environment…

So… my hope is that EMLID improve the M2 RTK software solution (certainly there is a good magin to do so!)

Best regards.

[fabio_lobo]

Thank’s for adding your expirience!

@Johannes_Eberenz

RS1 internal RTK solutions are much closer to RTKLIB demo5 PPS solutions.

The same ocours with the M+ and M2… showing that the EMLID’S expertise in simple frequency has not (yet) guaranteed comparable results between the two solutions (demo5 and reachview).

I expect M2’s reachview to improve a lot for the next version! I do not expect to use a computer on the field to get demo5 solutions…

[fabio_lobo]

@Johannes_Eberenz

Emlid can’t do much about it directly, as RS2 uses the ublox internal RTK, while RS1 runs RTKLIB

I didn’t know that information!
So, the RTK solution of RS2 (also M2?) Is not implemented via RTKLIB of EMLID development ?!

[fabio_lobo]

Today there is a update to M2’s reachview - 2.22.5.
Let’s see if it improve the RTK solution…

[fabio_lobo]

Hello.
I made new tests with the recent Reachview 2.22.5

Unfortunately the new version has not improved the behavior that I have been complaining about: reachview continues to show worse RTK results than rtknavi demo5 - in all data collection (!)
indicating that there is room for improvement in the RTK implementation of reachview, since this behavior is not observed in measurements made with M +.

The good news is that the new version has narrowed the gap between the two results.

The study shows that the M2 with version 2.22.4 showed a difference of 0.9m between the solutions LLH and NAVI, while with the implementation 2.22.5 (M2_5), the difference dropped to 0.49m.

The methodology was the same presented since the beginning of this topic: 7 occupations of a landmark with known coordinates, with 1 hour duration each and data collection directly from the receiver (LLH) and with the generation of the solution in demo5 (NAVI).

[sbm]

U-blox has released a firmware update for the F9P on May 25 - v1.13 that has a note about improved RTK performance among many other things. Perhaps this will help the issue. Or perhaps it is already rolled into emlid v2.22.5?
https://www.u-blox.com/en/docs/UBX-20019211

[fabio_lobo]

Good question!
Maybe @dmitriy.ershov and/or @svetlana.nikolenko can answer this point.

[Johannes_Eberenz]

Yes that would be great to know.
My recent tests with RS2 ReachView v2.23.6 DEV showed still a large difference between internal RTK and RTKLIB PPK, especially with canopy.
It’s very unfortunate that the we have to accept much accuracy in real-time versus the true potential of the RS2.

[fabio_lobo]

Thank you for joining us in this discussion.

[svetlana.nikolenko]

Hi Fabio,

We’ve thoroughly checked all the logs you sent.

In the meantime, we don’t see any issues with how Reach behaved in your test. It looks like it performs the way it should according to raw data. I’ve attached the screenshot with one of the raw logs below. Satellites’ SNR value jumps even in conditions it should be stable, and some satellites just fall out of sight.

And here’s sky plot screenshot showing how a building affects data quality. With such logs, it may be complicated to achieve consistent results.

I understand your point that RTKNavi performs better compared to Reach, however, I can’t agree the dataset is representative to draw conclusions. A test in an open area should be conducted, and this is something we may consider doing in the nearest future.

We also have plans on testing the new firmware from u-blox. We’ll post updates on this on the forum once there is news.

[fabio_lobo]

Hi @svetlana.nikolenko.

Thanks for the feedback.

I am pleased that the receiver is working as expected - it was one of my concerns.

I hope that my effort to test the reachview RTK solution against rtknavi will encourage this community to do similar tests for quality verification. If EMLID does it, even better!

I will do a sequence of tests with the occupation in an open sky spot - it implies an internet reception infrastructure that I am providing.

It certainly won’t be a robust analysis as this one, on wich I did several occupations with a wide variety of times, constellations and base distances and a partially occluded skies … ALL the results indicated that the reachview solution can be improved (because rtknavi managed best results … ALL AND EVERY TIME).

As soon as possible I will publish one more sequence of tests with open sky.

I reiterate my intention to support this community in improving the reachview RTK solution, using M2, which I consider to be excellent equipment!

[fabio_lobo]

Hello everyone.

Continuing with the tests, I did 16 occupations in a landmark that I installed on the roof of the building - 100% open sky.

I evaluated two antennas in the test - the aforementioned GRADE (Navspark model GRADE ANTENNA - Survey Grade Antenna - NavSpark Store) and the recently acquired BT170 (https: //beitian.en.made-in- china.com/product/zsLEdNUjGyWw/China-Beitian-GPS-Module-Receiver-Antenna-Rtk-Gnss-Antenna-GPS-Glonass-Bds-Galileo-High-Precision-Survey-Antenna-Bt-170-D-SMA. html) - 8 occupations with each antenna, each occupation lasting about 1 hour with a baseline of 18km.

The receiver used was the M2 with version 2.22.5 of reachview, tabulated as “M2_5” so as not to be confused with the measurements I had made with the previous reachview - tabulated as M2.

To eliminate outliers from the analysis, I followed the steps: (1) I eliminated the first 5 minutes of screening; if the data was not yet within an acceptable range; (2) I eliminated the initial 10 minutes - if the data still showed anomalous values, I considered only the data with Q1. If none of these actions corrected the data, it was removed from the analysis completely.

The figure below shows all occupations of M2_5 in WINDOW (JANELA, poor signal reception) and ROOF (TELHADO, open sky), comparing the accuracy of the LLH (obtained by REACHVIEW) and NAVI (obtained by DEMO5) results.
The red part shows the filters that were done to eliminate outliers.

From the figure it is possible to see that the LLH solution tends to be less accurate than NAVI, since the bars in BLUE are larger than the ORANGE ones - if they were the same size, the two solutions would be the same. It is also noticed that this behavior is valid for both the data of the WINDOW (JANELA) and the ROOF (TELHADO).

In the figure below I present the averages of the solutions for occupations with M +, tabulated as “MP”; M2 and M2_5 (MP solutions are highlighted in green).

The bars present the results for the different antenna models that were used.
For the M2, the aforementioned GRADE, TERSUS and BT170 gain the company of the Tallysman TW3865 antenna (https://www.tallysman.com/app/uploads/2018/09/TW3865_TW3867_Datasheet_rev3_7.pdf), tabulated as “TAL”.

For the MP, two implementations of the TW2406 (TW2406 Embedded Single Band GNSS Antenna | Tallysman) were used - a DIY with a 10 cm ground plane (tabulated as " FCL “) and an inheritance from a EMLID REACH RS receiver that no longer works (tabulated as” RS "). In addition to these two versions of the TW2406, a helical antenna was tested which does not know the make and / or model - tabulated as “HELIX_TR”.

From the figure you can see that the LLH and NAVI results of the MP receiver are very similar to each other (the BLUE and ORANGE bars are the same size) … for the M2 and M2_5 results, the trend is that the ORANGE bars are smaller - regardless of the quality of satellite reception (WINDOW - JANELA, or ROOF - TELHADO) and regardless of the type of antenna.

To which, I conclude: to compare the RTK processing capacity between reachview and rtknavi, it does not matter if the signal quality is the best possible (TELHADO - ROOF with clear skies) or with a poor reception quality (JANELA - WINDOW with partially overcast sky) .
I also conclude that there is a good margin for improvement in the LLH solution, because in all scenarios, the tendency is that the LLH result is worse than the NAVI!

I hope that my evaluation effort will serve as an inspiration for other colleagues in the community to test the equipment and, thus, we can assess whether we are getting the best RTK result that the product can generate … and if we are not, that the team EMLID can implement the appropriate adjustments in the reachview (either by RTKLIB or another solution that is being used).

[fabio_lobo]

Hello,

I could send the UBX and the LLH data.
Any of you would like to see it, @dmitriy.ershov; @svetlana.nikolenko or @tatiana.andreeva ?

There is a point that you could help the community to understand: is the M2 and RS2 RTK solution based on RTKLIB or on internal solution, provided by the ublox chip (F9T or F9P)?