V26 Loosing log UBX after reboot M+ and M2

m.rocha · May 8, 2021, 4:32am

Hi,

After updating v26 the UBX logs that were automatically saved when interrupting the unit’s power supply, the logs are now being saved and made available in a random manner.

in the M2 unit, after the collection, I performed the registration interruption manually, I expected to be listed as available for download, in the sequence I performed the shutdown, and when connecting the unit in the office to download the data, I verified that the file is no longer available.

I accessed via SSH and I was able to vialize that the file exists but I don’t have permission to download it.

on average, I perform more than 10 operations, with M+ and M2, to turn the unit on from 30 minutes to an hour with the unit on and interrupt the power supply and proceed to the next log on another place. After the update, 2 out of 10 logs are lost.

tatiana.andreeva · May 11, 2021, 9:30am

Hi @m.rocha,

Do I get it right that the main issue is that the logs are lost if you power off the unit by disconnecting it from power?

Does the issue persist if you turn it off manually, through the app?

m.rocha · May 12, 2021, 11:39am

Hi @tatiana.andreeva,

The operation with the Emlid M + and M2 units has always been powered by USB with the use of a powerbank.

We have a button on the USB cable that turns the unit on / off. This button would be the same as inserting and removing the cable from the unit at the end of the operation.

This procedure rarely lost an event log. The unit takes a few seconds longer to initialize but nothing that impacts everyday production.

to solve the intermediary, we are at the end of the operation by accessing the unit and finalizing the registration and then downloading it.

However, we also noticed that some logs disappear even after finishing and the download zip is not generated.

Yesterday we observed that a unit created a log of more than 2h50min being that our operation hardly exceeds 50min with maximum intervals of 20min, time to remove and install the other battery in the equipment to continue the operation.

This log with these crazy times always disappears from the download option and the unit hangs for a few minutes on startup.

We are literally working with a lottery of data because we were never able to know which of the 4 M + units or the 2 M2 units had a usable log.

polina.buriak · May 13, 2021, 6:53am

A post was split to a new topic: Gaps in the raw data logs

polina.buriak · May 14, 2021, 5:25pm

Hi @m.rocha,

Thanks for the detailed explanation!

Please share the Full System Reports from all the suffering units with me. As they contain sensitive information, it’s better to share them privately with me via PM or support@emlid.com. The info in them will help us to check the internal settings of the unit to identify the reason.

It’d be also useful to check the serial numbers of the devices. They can be found on the bottom sides of the devices under the bar code.

polina.buriak · May 18, 2021, 3:31pm

Hey,

Just wanted to say that I’ve received the data from the unit. Currently, we’re checking what can trigger this behavior. In case of any news, I’ll be in touch.

m.rocha · May 20, 2021, 12:00am

Hi @polina.buriak,

Today I performed a static point registration and calculating the ntrip. when accessing the log in the period in which it was in the float status, the ubx log failed, after fixing the stabilized log of ubx stabilized.

tatiana.andreeva · May 21, 2021, 10:35am

Hi @m.rocha,

I looked through all your data and tried to reproduce the issue on Reach M2 running the 26 firmware version but failed.

I tried to reload the unit by replugging the power several times, and the logs were recorded properly for each of them, though it took some time from Reach to compress all the UBX files.

I also noticed that you shared 2 Full System reports with us: one from the unit running the v2.24.2 firmware version and the other one from the device on the 26 version.

If I get everything right, it looks like the issue you experience occurs not exclusively on 26 firmware. Since we can’t reproduce the issue ourselves, I believe it may have a lot to with your specific setup and use case. Even it started happening after the update, it’s still possible that something could go wrong with the setup at the same moment.

Still, we are here to investigate it and find out the root of the issue, and it’s vital to do it the fastest way possible so that our devices work for you the way it’s expected. Let’s check several different things - just to exclude each of them as a potential suspect.

Try changing the way you reboot the unit: would it change anything if you do it by manual replugging?
Try connecting the unit to the power using another cable or power source
Share the setup photos with us over support@emlid.com: we will examine them together with our electronics engineers

m.rocha · May 22, 2021, 4:31pm

Hi @tatiana.andreeva ,

Very strange, since all M + and M2 are in the latest FW26 version,

The M + and M2 units are disappearing all the files when looking at the logs with a longer boot time than the one connected to the unit, I took some screenshots for samples with 0.53mb and 4h of operation and after starting they were only 5 minutes away.

M+ Print Start Log
M+ Print End Log

These logs simply after being finalized, via SSH we can see and download them, but they are corrupted. To resolve this, we are collecting data with the Rinex 3.03 option enabled and downloading it in the field with the unit before shutting down via software, which is taking too long.

For NTRIP, we download the logs and disable the log, after we can use NTRIP without problems and without problems, if the Log is open, the repair time becomes extremely time-consuming and also generates failures in the UBX Log.

The operation is currently in 3 stages.

M2 = Static Base (recording Logs UBX and Rinex 3.03 (Full Rate same RTK Settings 5Hz))
M + = Rover Kinematics (Recording Logs UBX and Rinex 3.03 (Full Rate same RTK Setting 5Hz))
M2 = Ntrip recording fix point standalone, no logs recording.

When the ntrip is active and the solution is in FLOAT, this information is generated by the log.

In the M + units, mainly, we noticed that some log names are missing, with a time offset of a few minutes in the name of the logs.

Before unzip

Plot of raw_202105181441.obs

Plot of raw_202105181442.obs

In other words, we are having to unzip and trying to find out who each one is through the head of the rinex or plot of the rtklib to later process with the M2 bases.

About setup energy, the 3 units M+ follow Teokit setup, since 2018 and 2019.

And 2 M2 Bases setup since Feb.2020 (one power bank, microusb cable).

The setup has been working perfectly for years and none has had a similar log fault disappear, name errors or holes in the logs in the firmware prior to firmware 26.

And we lost a m2 unit that failed test 1 (which says nothing about what it would be) and had to be retired compusively to everyone’s sadness, and it also makes us very concerned about future investments in new emlid hardware.

polina.buriak · May 24, 2021, 1:28pm

Hi @m.rocha,

Thanks for the detailed description! All of this looks rather strange. I’ll discuss this with the team and will get back.

Meanwhile, would it be possible for you to share all of the logs with us?

m.rocha · May 24, 2021, 2:22pm

Hi,

Perfect, follow the link below

https://1drv.ms/u/s!An2p4XX_P07WoJFI1aryo-iZG6swjQ?e=tG6BQZ

m.rocha · May 26, 2021, 12:10pm

Hi @polina.buriak,

Would you have any news?

polina.buriak · May 27, 2021, 9:21am

Hi @m.rocha,

Sorry for keeping you waiting.

We’re checking the logs. As we can’t reproduce the issue, it’s hard for us to find the root of the issue quickly. That’s why troubleshooting takes time. Once we have some news, I’ll be in touch.