LandXML Surface file sizes

OK, the quick answer is your problem is up front with the contour generation. The huge number of little jiggly and dangly artifacts in the contour lines are compounding into a gazillion triangles that are blowing the file sizes out.

It’s easy fixed and best explained in pictures, this is long but bear with it…

This is your original contour file. It already looks jiggly when zoomed out but gets much worse if we zoom in:

This is the little circle:

And these are all the points in those lines, and check the scale it’s now at the decimetre level:

And as a result the TIN suffers excruciating agony, it’s extremely busy and it also gets worse if you zoom in:

To here:

Then further again to here, note we are now at the cm level and it’s over the top and not commensurate with your collection precision and accuracy:

My lights went dim while this was processing as I was watching TV. Global Mapper by default chased down every single one and proudly presented me with a 2.7GB LandXML file.

What you need to do is look for QGIS functions that Smooth, Simplify, Resample, or similar terminology while you are generating the contours and/or clean them up afterwards.

Global Mapper does all of these, and in this case for the purpose of illustration only I chose a quick and dirty simplify to 2M and you can see the effect in the black line overlayed on the same screenshot above at the decimeter level here:

And this is the overall result of the simplification here, yes it’s still ugly but the intention is to give you a quick idea at the other end of the extreme:

And in this case the final TIN visually looks like this:

As a comparison the resultant LandXML file for the simplification is now 4MB.

Your call how you balance collection vs actual terrain, detail and file size, but you can see quite clearly if you follow this process that the noisy artifacts have to go.

4 Likes

In QGIS, these tools exist and I do use them as I create contours from lidar DEMs for almost all my projects. The main tool would be the default QGIS tool called Simplify.

But it is also possible to use the GRASS processing tool called r.contour (instead of the default QGIS algorithm) to create the contours. You can’t specify a smoothing or simplification setting in there, but you can specify the minimum number of vertices a polygon/line should have. I set it to something high like 50 or 100, and this prevents all the little bumps and pebbles from creating islands in the contour layer. Then the simplification steps take less time.

3 Likes

For the purpose of my example only I chose a 0.5m grid, and primarily because it was the default for the DEM I was using to generate the xml.

FYI as it turns out Mikes example source contours above are 1m intervals, and the large slope predominantly has them around 4.4m apart so also sparse for what you are doing.

2 Likes

Hi @timd1971,

EF does not currently have the feature to create a TIN surface from points.

Could you please tell me how important this feature is? I’ll relay this to the developers. But we need to understand how this feature can be helpful, in what kind of workflow, and in what industries need the creation of TIN surfaces in the field.

Wimbo,

Thank you for chasing this down. I think there must be a song in amongst this “The lights of lay low…” :wink:

I will follow your lead on this and also the advice of @Gabriel_C about the appropriate QGIS tools to see if I can achieve the outcome.

As for the 1m contours - I kept them at that in hopes of a smaller LandXML…

1 Like

That’s Wombo

and The lights of “name at town” lay low…

1 Like

I decided to play around with this a little.

In QGIS
1m DEM TIFF, covering 1km2 (5.2MB)
Convert to 1m contours (14MB as a scratch layer)
Convert to TIN Mesh (79MB as a scratch layer)
Export TIN Mesh as LandXML 2DM (135MB)

In AutoCAD Civil 3D
Same 1m DEM TIFF (as above) as Surface
Show Triangles to confirm they are at the same 1m intervals.
Export to LandXML (177MB)
If I Simplify the Surface to a Maximim Change in Elevation of 0.1m and 50%, it only gets down to about 112MB.

I decided then to play around further in QGIS. I did manage to get the same DEM down to 6.4MB. I ran the GRASS “r.neighbours” tool (at default settings), then the “Contour” tool with interval at 1 (which on my EPSG is in Metres, make sure you get the right attribute name for the Elevation Field). I then ran the “Simplify” tool on the contours (default settings). Then ran the “TIN Mesh Creation” tool on the simplified contours, making sure to add the right “Value on Vertex” (which has the Z Value) and clicking the green plus. Finally, then ran the “Convert Meshes to LandXML Surfaces” tool.


As with any form of “compression”, you always give up a bit of detail, or in this case, accuracy.

Looks like XML’s are not really efficient at storing this amount of data. Granted 100 hectares (247 acres) is a very, very large area. Maybe Emlid should give us the option to use TIFF’s?

3 Likes

I ran a simplified version of @joelbladen strategy - I ran the Simplify tool against the existing 1m contours then → Create TIN then → Mesh to LandXML…the outcome was a 880kb file!!

However as a surface it’s not great.

More work required…there’s a middle ground somewhere.

1 Like

I’m sure if our our friend @Brent_W would have more spare time he would jump in here :blush:

@MikeH Could I get a copy of your data? I’d like to run it through both Civil 3D and Agtek to see how the results compare to the other attempts. Can you export a DXF and send me a link to download?

1 Like

G’day @michaelL you’ll find both ESRI shapefile and DXF files in this directory:

I’ll be interested to see what you come up with.

No problems Moke. :slight_smile:

Your last reduction is on the right track, at least you found the opposite extreme.

Best to build the contours cleanest to start with but they will always need some cleaning up. And in this case there are probably also a lot of tiny residual fragments in there that are also spawning excess triangles so worth trying to get rid of as a next step if you can.

I did it here and it cleaned things up a bunch better. Do you have a “select or delete lines shorter than” command?

If not maybe someone could help you build a query in QGIS similar to this convoluted process I went through in GM. Note it did find heaps of “lines” 0.00001m long that I was able to zap so the effort would be worthwhile.

Note I’m not trying to tell you to buy/use another tool, apart from being a QGIS ignoramus I’m just showing examples to hopefully help you leverage what you already have.

1 Like

@MikeH

In QGIS, how did you convert to TIN mesh then to LandXML? QGIS plugins?

Thanks.

Correct. If you search for LandXML in the Plugins, you should find it.

2 Likes

Thanks Wombo, I take all this in good spirit. I’m a QGIS newbie, but I like it. I’ll keep working on this and see if I can hit a nice middle ground. I think it will be helped also by clipping the DEM to the area of interest as that will take out a lot of peripheral junk.

1 Like

@timd1971, @joelbladen is correct there is a plugin with two functions - convert from and convert to LandXML.

2 Likes

Well, I clipped the extent of the contours to remove peripheral junk. No simplification, just standard contour extraction. It comes to 16.1mb LandXML and probably a workable surface. I can see the reason for the 25mb file size limit. Flow 360 starts grunting a bit even at this size.

Thanks to everyone for their ideas and guidance.

3 Likes

That’s heaps better, good work. Time to switch around and start sending you our files to fix up!

1 Like

What is your CRS? When you exported the DXF did you enable Z attribute? The contours I am seeing have no elevation. I have tried the GIS files yet. I do see that there are WAY too many vertices! Like 6mm? Is that what you see? I am using my photogrammetry processing rig and it has lag trying to just select a line.

2 Likes

I was VERY surprised how well and fast QGIS handled the files. Just a Surface Pro 6. I think I used the SHP GIS files that contain Z height. The CS came through automatically in QGIS. Cannot remember which, but one that ran vertically through Melbourne.