Timestamp inconsistency (survey points vs logs)

I wasn’t able to find another phone to use… but I conducted the same test again and am still finding the same lapse in timestamps…

Here’s the second test data (base and rover logs, plus survey points). Hopefully we can figure this out !

Thanks again for your help !

timestamp_question.zip (8.0 MB)

Hi Zoe,

As I can see, these logs are similar to the ones you shared earlier. Could you please clarify if you wanted to share any other data?

Oops !

Here is the file I meant to share ! https://we.tl/t-MqfGWtp7dh

Cheers !

1 Like

Hello !

I carried out another test with the emlid modules.

I took a series of survey points (with different solution statuses and while connected to either the base or the rover). I simultaneously recorded the time shown on my computer (I checked my computer’s time with that of 3 other computers and 2 cell phones, all were in sync). I then calculated the difference between point times registered in the CSV and that of my computer.

When connected to the rover the time difference is about 2 minutes 39 seconds between my CSV points and the time of my non-emlid devices. The time difference is 2 minutes and 45 seconds when connected to the base.

Once again, there is a lag in the survey points compared to base and rover logs (shown by the last points occurring outside of base and rover log end times)

Hope this is helpful in solving this problem

I’ve included my excel worksheet where I did the calculations (survey points + clock observations), and base and rover logs : https://we.tl/t-rtYYJSyCBY

Hi Zoe,

Thanks for the details! I’ll check the data and get back with the news.

Hi Zoe,

Sorry it took so long to get back to you!

We looked into your data and could see that the end of the last point indeed isn’t within the time of the rover RINEX log. However, we couldn’t reproduce this issue in a test with our devices.

Since we haven’t heard about such issues before and can’t reproduce them, it may take us a while to determine what might be wrong. At the moment, we are discussing your case with our devs. Once there is any news, I’ll let you know.

Hello!

No problem for the delay. I was on vacation!

As far the this timestamp problem. It sounds like it could be a problem with our device(s), or even operator error…

Is there anything I can do from my end (further tests, confirmation of settings, etc) to help investigate the problem ?

Thanks !
-Z

Hi Zoe,

Thanks for your help!

It sounds like it could be a problem with our device(s), or even operator error…

It’s true that the issue may not be related to ReachView 3 or Reach performance. However, we’ll check all the possibilities to find out the root of the issue and provide you with our suggestions. If we need any additional info for the investigation, I’ll reach out to you.

Hi Zoe,

We conducted some additional tests but still couldn’t reproduce this issue.

May I ask you to conduct one more test? It is needed to ensure that the issue repeats every survey and see if the difference between the CSV file time and the actual time on your PC/mobile device stays the same. Also, it would be of great help if you can find another mobile device to check whether the issue persists when you work with it.

Hi,
Yes, I’ll send the results as soon I can carry out the tests !

1 Like

Hello !
Here are 2 tests that I carried out.

  • The first one is with two different phones. The results between the phones seem to be coherent. Although due to my testing methods the timestamp error seems to be resolved (stayed too long at the test points so extracted timestamp coordinates produce similar coordinates to those written in the survey file despite time lag.
  • The second test I carried out to confirm the presence of the time lag. I collected the points as soon as I reached the point I wished to survey.

I included some documentation to (hopefully!) be even more clear.

I also updated the firmware on both modules (before the tests)

Have a great week !
-Zoe

1 Like

Hi Zoe,

I’ve investigated the data you shared. The start and end of the point collection in all the tests lie within the rover’s raw data log time. So, the intervals from the CSV file seem fine. If you find the lag by checking when a minute turns, a human factor can play a part there. To see if there is indeed a lag in the recorded time, I’d recommend processing the data in Emlid Studio.

You can use the Stop & Go feature in Emlid Studio to calculate separate points. The points’ coordinates obtained from raw data will be averaged automatically using the time intervals from the CSV file.

Please check whether you can get the correct results in Emlid Studio. It’s needed to understand if the difference between the extracted coordinates and the ones from the CSV file in your test was related to the data quality or something went wrong during the processing. If you face any difficulties with Emlid Studio, don’t hesitate to contact me.

If Emlid Studio works for you, I’d suggest using it for your survey as a most straightforward way to process data. If it doesn’t suit your needs, please tell me which result you expect and how it differs from the one you can obtain with Stop & Go. Also, please elaborate on these points:

  • How do you use the CSV time intervals to extract data from solution logs?

  • How did you measure the 15 seconds lag in the latest tests?

I will try out Emlid studio. Being able to post process points automatically would be great. However, without accurate timestamps in the .csv, I fear that my problem will persist.

As stated previously, after the firmware update the lag seems to have shortened and the timestamps all appear within the time of the survey, but are still not totally correct.

To answer your questions:

Using the minute turn was just for organisational purposes. I did not base my conclusions on that. I only mentioned it because waiting for the minute the turn meant we were waited at the points we wanted to survey before starting the 15 second point collection. This was only done for the first test of the most recent 2 I sent (those of last week).

To test if there was a lag between the times encoded into the .csv and that of the solution file:

I used the timestamps in the .csv file (columns “Averaging start” and “Averaging end”) for each point. I then went to the solution file and took the coordinates collected within those times (for each point) and averaged them. The worksheets for each of the points for the last 2 tests are included in the files I submitted above previously.

I then mapped each point from the averaged coordinates based on the timestamps in the .csv and compared them to the coordinates encoded in the .csv file directly. The map of both sets of points (those written directly into the .csv by the emlid modules, and those I calculated by hand using the timestamps from the .csv and the averaged coordinates from the solution file) are on the map I also sent.

The difference between these 2 sets of points is visible along the ‘tracks’. The coordinates that I obtained using the average coordinates from the solution file occur at an earlier point along the track than the coordinates written in the .csv file. I’m sure that the timestamp in the .csv file doesn’t correspond with the coordinates listed in the .csv.

I quantified the ~15 second lag by selecting the coordinates that were grouped around the survey point coordinates in the .csv file and checking the ‘oldest’ and ‘newest’ timestamps for all close points. This is less precise…. But the coordinates of the data in the solution surrounding the ‘survey’ point (as written in the .csv) correspond with timestamps that occur roughly ~15 seconds later.

I hope that is clear… It is hard to explain all of the details in messages….

Am I correct to understand that the time lag is between your phone and RS2 receiver?

I don’t know where the timelag originates. My goal is to find out ! It is not from the phone because I’ve tried with multiple phones (android and iphone) and had the same problem.

Hi Zoe,

I used the timestamps in the .csv file (columns “Averaging start” and “Averaging end”) for each point. I then went to the solution file and took the coordinates collected within those times (for each point) and averaged them.

Could you clarify if you not only subtracted 2 hours for the time zone but also added 18 seconds to convert UTC time to GPST? The CSV file includes the time intervals in UTC, while the solution file is in GPST. In case you didn’t add 18 seconds, I suppose it might cause the lag you describe.

1 Like

Definitely subtracted the 2 hours. However, I did NOT count for the leap seconds! I bet that is where the problem lies…

I will confirm and get back to you

1 Like

Hi Zoe,

How is your project going? Curious to see if the leap seconds will do the trick!

1 Like

Yes !!!
I re-processed my data (with leap seconds) and it looks good for the tests that I did after the update.
Those from before the update are still off, so there was definitely a problem, but it seems as though it was fixed by updating the firmware !

As far as using Emlid Studio “Stop-and-Go” feature, it seems like exactly the functionality I was trying to create myself. Just a precision: using Stop-and-Go along and inputing raw-logs along with the .csv will post-process the raw-logs (PPK) and those corrected coordinates are what are written into the “corrected” survey file, yes ? Just double checking !

I’m so relieved to finally have this mystery solved !!

Thank you !
-Zoe

2 Likes

Glad to hear it’s working now! :slight_smile:

Just a precision: using Stop-and-Go along and inputing raw-logs along with the .csv will post-process the raw-logs (PPK) and those corrected coordinates are what are written into the “corrected” survey file, yes ?

That’s right. Emlid Studio calculates solutions from the base and rover raw data as it usually happens in PPK. After that, it averages coordinates related to each point and writes the precise LLH into the _corrected.csv file.