I am having trouble getting Applanix PosPac & YellowScan CloudStation to accept my .23O file and the .csv file with my GCPs. I read in the forums it might be because Emlid Flow doesn’t log data on even seconds so I imported my .23O file into Emlid Studio and then exported it with the “Time rounding (required for OPUS and other third party services)” option checked. Next I read about there possibly being an issue with missing leap second info? I’m not sure if this is the case or not, but if so, I need to know how to add leap second info?
In any case, it looks to me like my original .23O might have been recorded in even seconds after all?
Unfortunately, when I try to process my lidar data in Applanix PosPac and YellowScan CloudStation it says there’s no overlap between my Area of Interest and the .csv file with my GCPs. I really do need to be able to use the .23O file from my Emlid Reach RS2+ and the GCPs I recorded with my rover.
If I use data from my local CORS station instead of my Emlid Reach RS2+ and don’t incorporate my GCPs the processing completes, but with less accuracy than I need.
The thing is that your initial log was already recorded on even seconds if the receiver has the firmware newer than 30. That’s why the conversion in Emlid Studio hasn’t changed anything with the log: it doesn’t interpolate RINEX file on even seconds, only UBX.
As for the leap seconds, they are added to the navigation file 23P, not the observation file 23O.
You can send us the CSV file and the observation file 23O to suppport@emlid.com or via PM, so I can check whether they overlap or not.
Hi @kirill.pavlyuchuk, thank you for your reply. I have a couple of follow up questions: How do I add leap second information to the navigation file? How do I convert my UBX file to even seconds? Do I even need to do these things? It doesn’t appear that I can do either of these things in Emlid Studio so do I need to use RTKConv?
Just an FYI, after I “converted” my .23O in Emlid Studio I now have two .23O files with the same name, the original is 42.5MB and the one that came out of Emlid Studio is 41.6MB.
It should be already there if your log is adjusted to even seconds. And it’s adjusted because the firmware version is newer than the 30. So you don’t need to do this.
It’s expected because the converted file is created, while the original one is kept.
I’ve checked the files you sent over email and see that the reason for the issue is likely related to the CSV file. It doesn’t indicate the time of collection because it was probably exported in the CSV(PENZD) format. You can try the regular CSV to see if that helps.
Hi @kirill.pavlyuchuk, I do not believe I exported the survey project in the .CSV(PENZD) file format, but I have re-exported it and made sure it is a “regular” .CSV file just to be sure. I will let you know if I am able to successfully process the data in Applanix PosPac & YellowScan CloudStation.
One follow up question is in the .CSV file I noticed column G is Ellipsoidal Height even though I made sure to select NAVD88(GEOID18) in the vertical datum field. Column AA confirms this, so I want to make sure column D (Elevation) is in Orthometric Height?
Hi @kirill.pavlyuchuk, I tried again using a .CSV that was absolutely exported in the “regular” .CSV file format. I got the same result, which makes me certain my original .CSV file was also a “regular” .CSV file and not a .CSV(PENZD). I am now at a loss as to what the problem is.
Since YellowScan tech support was able to process the data using a feature called SmartBase instead of my .23O file I have to think the problem is caused by my .23O file. I just don’t know what I did wrong. The refresh rate was set to 1Hz and I believe all satellite constellations were selected.
On the other hand, I was able to successfully generate and apply an SBET using my .23O file and didn’t run into any problems until I imported my .CSV file with my GCPs. So the problem lies either with my .23O file or my .CSV file.
Yes, you’re right. The Elevation column represents the orthometric heights.
You can send me this CSV file, so I can at least check if it overlaps with the observation time in the RINEX 23O file. If it does, you need to check from the YellowScan side because they know the requirements for the CSV files. Probably, the software requires them with a specific column order or adapted to a certain template.
Another idea is that the software indicates the overlap of the flight trajectory coordinates and the GCPs at the site. If this is the case, this error could be caused by the difference in coordinate systems.
As for the initial file you sent, it has neither collecting time nor the header, only coordinates. That’s why I supposed that it was rejected by the software.
Hi @kirill.pavlyuchuk, I will send you the .CSV file, but I know the collection time of when I recorded my GCPs will not overlap with the time I recorded the observation file. I recorded the GCPs at a separate time using Fixed corrections over NTRIP. I was not logging data on my base at the same time I was recording the locations of my GCPs with my rover. The .23O file was (obviously) logged during my drone flight, but the GCPs were collected on an entirely different day without my base simultaneously logging data.
I did not know the .CSV file included the time of recording in the metadata.
In the future I will always set up my base and log data while I am recording my GCPs with my rover. But if I collect my GCPs on a different day or time from my drone flight I will end up with two different .23O files, one for my drone flight and one for when I record the locations of the GCPs. I hope this will not create a problem?
For incorporating GCPs YellowScan CloudStation requires a .CSV file without a header row in the P, X, Y, Z format.
Hi @kirill.pavlyuchuk, YellowScan tech support has been looking at my data and they believe the problem lies in the .23O file. They do not believe there is a problem with my .CSV file with my GCPs.
They say there is a large discrepancy in the Z value showing a significant difference in altitude between what the drone recorded and what the Reach RS2+ recorded. They sent me screenshots of what they’re seeing, but for some reason the images aren’t loading in my email. I have asked them to send the screenshots in a different format. I will forward them to you once I am able to view them.
It seems to be pretty apparent because the base is located on the ground while the drone flies.
I’ve decided to check the Z-accuracy of the base log and downloaded the log from the nearest CORS station located in Reno. The red cross shows the base position:
The U-D (m) represents the changing of the elevation of the processed log. As you can see, the elevation fluctuates in the range that doesn’t exceed 2 mm.
So I’m not sure what they meant about the discrepancy in the Z value.
I don’t think this is the root cause at all. The error shown in the screenshot made me think that there was something related to the CSV file. It’s probably just not the real cause.
If they meant that the elevation of the base didn’t match the elevation of the near GCP or the elevation of calculated DEM, this is expected. The RINEX file contains the coordinates of the last averaged epoch in ECEF coordinates, not the precise ones that can be calculated via different processing services.
Hi @kirill.pavlyuchuk, YellowScan tech support sent me the screenshots in a different format so I was finally able to see what they meant by a difference in Z values. I forwarded the email to you as well. They generated an SBET file using the .23O file from my RS2+ and another SBET using a permanent base station (maybe a CORS station) and the difference in altitude was about 14m. This is why YellowScan thinks I am getting the No Overlap error when I try to add my GCPs to my point cloud. Do you have any thoughts on why this might be happening?
I went back to the job site today and re-flew the entire mission so I was starting fresh with a completely new .23O file and new lidar data and I still got the No Overlap error.
I’m a bit confused because it sounds logical that the altitudes of the two base stations vary from each other: they’re located in different places. As I see, the SBET is a proprietary PosPac format that stands for smoothed best estimate of trajectory.
I’ve decided to process your log in three ways: Static processing in Emlid Studio, OPUS, NRCan CSRS-PPP. You can find the results below:
Emlid Studio: 39.48661033°N, -119.71121320°E, 1377.9055 m NRCan CSRS-PPP: 39° 29’ 11.79873", -119° 42’ 40.37010", 1377.928 m OPUS: 39° 29’ 11.79550, -119° 42’ 40.36709, 1377.980 m
You can see that the ellipsoidal height matches within several centimeters given the fact that NRCan and OPUS use different datums: NAD83(CSRS) vs. NAD83(2011).
They mentioned the Z-value of your base as 1400.128 m. It’s close to the orthometric heights from NRCan and OPUS, respectively:
1401.296 m, vertical datum CGVD2013 (CGG2013a) (2010.0)
1402.076 m, vertical datum NAVD88 (GEOID18).
So I don’t see why the base elevation can be the root cause of such an issue. Since the log is processed well in other services, I think there might be something from the YellowScan side. At least, I’m pretty sure this isn’t related to the base log quality.
Hi @kirill.pavlyuchuk, so the great mystery of why I was getting a No Overlap error between my .23O file and my .csv file with my GCPs has been solved. I was entering the wrong Reference Frame in CloudStation. It was defaulting to ITRF2014 when it should have been NAD83(2011). Additionally I set the Epoch to my mission date, when it should have been Jan 1, 2010.
However, this only worked when I used an OPUS corrected solution from my Emlid base station. If I used the raw Rinex file from my Emlid base station there was a (massive) 10m difference in elevation from the OPUS corrected value. I now need to find out what is causing the 10m discrepancy between the two. Do you have any ideas?
I saw you also got an elevation of 1377.9055 when you processed my observation file in Emlid Studio, so I am wondering why it became 1387.0769 when it was imported into CloudStation?
These are the coordinates of the last averaged epoch which are calculated with meter accuracy, relatively accurate. Meanwhile, OPUS outputs precise coordinates of the centimeter level - accurate and tied to the specific coordinate system.
Ok, so what you’re saying is I will always have to submit my observation file to OPUS or process it in Emlid Studio to get the most accurate base position, and not use the raw .23O file straight out of my RS2+?
Thanks, I really appreciate all your help! I now have a much better understanding of what I need to do to accurately process my lidar data!
Hi @kirill.pavlyuchuk, Is the same true for my GCPs that I record in a Survey Project in Emlid Flow? (Meaning I must process them in Emlid Studio to get the most accurate GCP positions?) And if so, is there a way to process multiple points / an entire Survey Project at the same time in Emlid Studio?
It looks like I can do this using Kinematic Processing or Stop & Go in Emlid Flow. The problem is I wasn’t logging a Rinex file when I was recording my GCPs, I was receiving Fix level corrections via NTRIP though. I think I am going to have to go back to the site and use the StakeOut function to re-record my GCPs and record a Rinex file at the same time. Hopefully this works.
I am still having trouble adding my GCPs to YellowScan CloudStation. When I use a raw uncorrected .23O file and an uncorrected .csv file with my GCPs I am able to successfully run the Strip Adjustment process. However the results are not as accurate as I need them.
When I use an OPUS corrected .23O file and an uncorrected .csv file I get a No Overlap error and Strip Adjustment fails.
So I tried “correcting” my .csv file in Emlid Studio using the Stop & Go process. I imported my .23O file from my rover, the .23O file from my base, the .23P file from my rover, and the .csv file from my survey project in Emlid Flow. For some reason only 4 of the 5 GCPs were averaged in Fix even though it shows all five were recorded in FIX in the field.
The corrected .csv returned empty values for Easting, Northing, & Elevation as well as Lat, Long, & Ellipsoidal Height for the one GCP that apparently was averaged in Float or Single.
The next thing I tried is removing the first GCP from the spreadsheet and filling in the missing Easting, Northing, and Elevation data by converting the Long & Lat in columns E & F and entering the State Plane Coordinates into columns B & C and copying the Ellipsoidal Heights in column G and pasting them into column D? I used the NCAT tool on the NOAA / NGS website to do the conversion.
Unfortunately, I got the same result, a failed Strip Adjustment due to No Overlap.
The last thing I tried is using a .23O file from my local CORS station and the same corrected .csv file with only four GCPs instead of five. This also failed.
The good news is I think once I learn how to get a “corrected” .csv file of my GCPs that matches OPUS level accuracy, I think Strip Adjustment will succeed and I won’t get any more No Overlap errors. Can you tell me how to process my survey project .csv files so they are “corrected” coordinates? I hope this can be done in Emlid Studio? If this doesn’t work, I will really be at a loss, not to mention even more frustrated than I am now.
