Another survey post-processing software

tobias-dahms · August 8, 2019, 8:31am

Hi,

I have written a small program to post process surveys. Other than the already existing PPK point extractor software its less beautiful and only compatible with Windows, but at the other hand it does process every single point in static mode with the help of the RTKlib command line tools and it involves less steps/work for the user. It also does UTM conversion and can generate a survey file which can be uploaded into ReachView again.

Screenshot

Workflow

select parameters for point averaging
select base UBX file
Enter base coordinates or select pos file to average base position from and enter antenna offset
Select rover UBX file and enter antenna offset
Select survey file exported as csv from Emlid Reach, antenna offset will be transferred automatically.
Select output file if you do not want the standard name generated automatically
Choose conversion options etc.
Hit execute and get a coffee, the command line tools are slower than the GUI version. I do not know why.

The program will save several files in the output directory (depending on your selection and including a log file with the information from the console window). It will overwrite existing output files. You can change the processing settings if you open the gcp.conf file in RTKpost.

Disclaimer

The script is written in Python and is provided “as is”, withoute any warranties. Please open for details.

The script is written in Python and is provided “as is”, without any warranties. I explicitly want to state that the software may contain errors and that the user must check the validity of the produced data.

IN NO EVENT SHALL THE AUTHORS OR DISTRIBUTORS BE LIABLE TO ANY PARTY FOR DIRECT, INDIRECT, SPECIAL, INCIDENTAL, OR CONSEQUENTIAL DAMAGES ARISING OUT OF THE USE OF THIS SOFTWARE, ITS DOCUMENTATION, OR ANY DERIVATIVES THEREOF, EVEN IF THE AUTHORS HAVE BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

THE AUTHORS AND DISTRIBUTORS SPECIFICALLY DISCLAIM ANY WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE, AND NON-INFRINGEMENT. THIS SOFTWARE IS PROVIDED ON AN “AS IS” BASIS, AND THE AUTHORS AND DISTRIBUTORS HAVE NO OBLIGATION TO PROVIDE MAINTENANCE, SUPPORT, UPDATES, ENHANCEMENTS, OR MODIFICATIONS.

http://www.tcl.tk/software/tcltk/license.html
https://docs.python.org/3/license.html

Warning: the programm may cause a malware warning because of the way it is compiled . Even if I cannot guarantee that there is no malware, I do guarantee that I did not intentionally introduce one and take care that my system is malware free.

Download, use, and share your benefit

You can download a demo for Windows if you agree with the conditions. It is still work in progress (a little bit slow and the help information is missing) but it is working. The download contains also RTKlib files (demo5_b31a) compiled by @rtklibexplorer and a settings file with settings adapted from @wsurvey. All the glory are belong to them.

If you benefit from the software and want to use it I would like you to share your benefit by donating annually to the Against Malara Foundation via this link: AMF . If you make a significant contribution I will consider to implement your personal wishes to the script.

Any feedback is welcome.

Download: survey processing tool I did put the demo offline again because I need to adapt the RTKlib config file.

Please help

tracy.love · August 30, 2019, 6:24pm

Hi, sounds interesting, where do i find the software?

bide · August 31, 2019, 7:41am

The idea, purpose, and effect of this charity seem wonderful, and it is great that their financials are so transparent and perfect-looking. After a quick review of their audited financial statements (2018), I have to ask why does the Against Malaria Foundation pay between $1.83 and $1.94 per mosquito net at a purchase volume of between 9 and 15 million nets per year, when anyone can see on the open web that the price for a minimum order of 1000 or 3000 same/similar insecticide-treated mosquito nets is $1 each?

To be clear, the question is about the basic cost of each mosquito net. The external shipping/distribution/monitoring/etc. costs are accounted for separately in the financials. It seems reasonable that at these purchase volumes, the economy of scale should get the price of these insecticide-treated mosquito nets into the sub $1 price range.

For a charitable organization such as this which prides itself on low overhead costs, the gross overpayment for the one single item they purchase seems to ‘the elephant in the room’.

If there is a middleman involved in the purchasing, then it might be acceptable to allow up to 10% profit margin on high-volume commodities. Is there anyone reading here that would not jump at the opportunity to resell 9 to 15 million mosquito nets a year at 10% profit margin? Essentially you get between $900k and $1.5 million US dollars a year to do the job. With your money, you would have to pay for a business phone line, and have working relationships with a banker, plus a customs broker, and an accountant.

But wait, the numbers show that whoever the AMF deals with is probably getting 46 to 49% profit margin. That works out to 8 to 12 million dollars profit per year! So if one of us stepped forward and offered to resell to them at 10% profit margin, then that could save the organization between 7 and 10.5 million dollars a year. With your new million-dollar middleman job you would also be a hero of the AMF and their donors and clients!

It makes you wonder how such a tightly-run ship could let such a massive amount of money slip through its fingers. … unless … No, I don’t dare say it. Perish the thought!

But I think you know where I’m going with this. If you took a hard look at any charity or benevolent entity operating today, my feeling is that there will be a surprising amount of ‘unnecessary leakage’ in the value by the time it gets delivered to the intended recipient.

It’s just that in this case the amount of leakage has my eyes bulging out of my head

edit: In conclusion, I should say that I wouldn’t expect one to decide to discontinue suppport the AMF (or other charity) because of something like this, but rather make them proove that both you and the intended recipient are getting value for your generous donations, and that isn’t happening, then fix the problem.

tobias-dahms · August 31, 2019, 8:10am

I do not know what you want to reach äh achieve with your comment. I think you should put more effort in reading before writing such wrong accusations.

The AMF is one of the most efficient charity organisation in the world and one of the best monitored one. They deliver the nets for 1.8x $ to the users, they do not buy them for that price. They do not make profit. If you can manage to deliver nets, educate people about it and monitor the success for less than 1.8x $ you should do that because that would be the most significant thing you can achive in youre life and I and many others would be happy to support you doing so.

Edit: @bide is right. 1.8x $ are the nets only.
The question remains if the sub 1 $ net has the same properties as the 1.8x $ net. I will ask Rob Mather who is running the AMF.

bide · August 31, 2019, 8:18am

I was making an edit to my post as you replied (see bottom). I think that addresses what I am trying to achieve with my comment.

I don’t think I made an error, but if I did, please point it out. The bottom line is that they buy nets for what appears to be an inflated price. They account for their delivery, eductaion, monitoring and the other added costs in a separate section of the financial statements, so that argument is invalid.

For everything else that is so extremely transparent, the purchasing of the nets is the most opaque thing. One would think that they would have a whole page on nets, including where they buy and the prices and public offers to buy nets, etc. There is none of that and I think that is strange.

Being probably the single largest purchaser of nets in the world, one would think they would have the best price on nets and be able offer nets for sale and ship worldwide to anyone who wants to help out.

tobias-dahms · August 31, 2019, 8:30am

I think that is about a conflict between @bide and me resulting from my earlier posts about the price and alternatives to RS2. I think we should not drag AMF into this conflict because it would affect many people needing help.

I will reply in the evening when I’m back.

Anyhow we can contact the CEO about the price oft the nets if we do not find an explanation. Are these cheaper nets WHOPE certified?

bide · August 31, 2019, 8:45am

Well really it is a great idea to donate your time to write code and get people to support a charitable organization. Most people wouldn’t think twice about it. They look at the surface, “Hey these guys do a good thing for people to prevent disease and promote health,” That is great.

I just looked took a look and it seemed to good to be true, so I looked a tiny bit deeper and that is what I found. I haven’t even donated, and I felt wronged, so I had to say something. To me it is like someone approaching you like this: “Hey, I am going to start a food bank and donate it all to people in need. If you donate $200, I will go to the grocery store and pay $200 for $100 worth of food, and hand it out to people in need.” Makes no sense to me.

You seem a pretty up front guy @tobias-dahms, so I thought bringing this to light would be welcome, but I guess not.

bide · August 31, 2019, 8:49am

Oh probably. I’m sure if I came to that conculsion, then there would be many, many more who have done the same. If there is a basis of reason for the purchase price where it is, then for AMF to dedicate a page to it on their website would only help gain them more support.

Good luck if you go digging deeper.

And sorry, I guess a PM would have been so much better on this rather than dragging your thread off-topic. I just got all worked up over it I can swap it over to a PM if you like

bide · August 31, 2019, 10:28am

I should wait and give you a chance to respond again, but I went back for another look. This one takes the wind out of my sails:

AMF currently has $100 million in the bank; They spent $50 million in the last 3 years; and they collected $100 million in the last 3 years. So they are overflowing at the bank with cash and haven’t been able to disperse it. I see some talk in that time period about lack of funds, so I’ll bet something is preventing them from getting their job done.

Anyway, my take-away from that is the money is pouring in and they can’t spend it fast enough. In the end if they pay 1.8x the going rate for nets, it doesn’t affect their ability to deliver. It is a sad fact, but the people have spoken by sending in their money and the company does what it can to disperse it.

If they only spent 1x for their nets in the last three years the result would bt $122 million in the bank instead of $100 million, but no more nets distributed.

Unfortunately, I think I just invalidated my initial argument about the purchase price being inflated.

Again, I’m happy to move these posts into a PM at your request. (I’ll check back after )

tobias-dahms · August 31, 2019, 3:03pm

Oh man, that will take some time to reply to

My first response is the following:

with your concerns you should get to know the concept of effective altruism. Similar critical and economically thinking people like you, e.g. Bill Gates are donating their money according to this principle. Investing the money you cannot spend effectively does comply with this idea.
have a look at the organisation Give Well. They look for the most efficient charities and give background information and advice where to put your money to create the highest impact (I would also give access to my scripts to people who donate to one of the top charities listed there if that feels better than AMF.). But AMF is still one of the top charities and I like it because it because it helps to save lives and helps to keep children in school because their families don’t get ill etc…
Now I have to read all your writing

tobias-dahms · August 31, 2019, 3:32pm

You are welcome to bring light into it. But I think it is very unlikely that no one else has looked at and brought up the case you are making. And because I think there is a very high degree of certainty that your accusations are wrong I think the way you promote them is inadequate.

As I wrote, I will ask Rob Mather for more detail on the price of the nets. I would expect that the cheaper ones have lower quality, other properties and that they use certified nets.

tobias-dahms · August 31, 2019, 3:44pm

I personally think that we are very lucky that we are living today and at the place we are living. Even if I have an income below the average German I have more than 85% of all people in the world (and all that have ever been living). My daughter did not die as she would have if I lived 100 years earlier. I did not have to die in a war or in a concentration camp.

We are even so lucky that we do not have to ask our self whether we would risk our lives to rescue our Jewish neighbors from the Gestapo. At the other hand, we can ask our self whether we are ready to give only 5 or 10% of our income to rescue far more lives than we could during the third Reich. And since we do not have to risk our lives and the lives of our family to do so I think there is a far better reason to feel bad if we do not than for those who did not during the third Reich.

I did not want to bring all that moral thinking up here (@emlid please open a specific category within the forum for that ), but you did ask. I also do not want to tell you that I’m better or do I think that there is something significant in the small amounts I do donate. I’m just starting as very one just can start and get better in doing the right thing.

I think that it is just ok that someone who would otherwise buy a software for 500 $ or more to do the same can donate some money for using my scripts (and those who donate get it back from their taxes anyhow…). I like to help but I started to focus on helping those who need it most.

And lastly, that one organisation is inefficient is not an argument for not donating. It is an argument for donating to another organisation or to start your own organisation if you can do better.

bide · August 31, 2019, 5:34pm

You make good points. I probably should have just posted a one-liner to open up a private discussion. The drama I started doesn’t belong in this forum anyway. If one looks hard enough with a skeptical eye, they can probably find fault in anything.

My apologies for dumping this info on you here.

tobias-dahms · August 31, 2019, 7:56pm

No problem, I like discussions anyhow. And what would life be without drama

And I think we agree that it is important to use the donated money in the most efficient way to reduce suffering and that it is good to ask some critical questions and think twice about the things we do.

I will post the answer oft Rob regarding the price of the nets as soon as he replies.

tobias-dahms · September 1, 2019, 3:35pm

Got an answer from Rob. I think they key is quality. Of course you may find sellers on different plattforms offering best quality products (I constantly get a google advertising for a GPS which has better quality than Emlid Reach and only costs 125 € somewhere in China…), unbelievable cheap, but I prefere to rely on known quality. If lifes depend on it you might choose even more carefully.

Dear Tobias,

Thanks for the questions.

Cost per net

We buy long-lasting insecticidal nets (LLINs) - as do all others involved in global malaria control - and you cannot buy these nets for $1 each. Prices range from $1.60 (or so) to $4 per net depending on the size and type of net. AMF has consistently bought nets at a lower cost than any other purchaser of nets. Perhaps the person you are in discussion with is referring to nets that are not treated with insecticide. There is a very important different and no one involved in significant malaria control would buy untreated nets. The long-lasting refers to the extended time the insecticide remains active on a net and which causes knock down and killing of mosquitoes when they land and walk on a net. The mosquitoes pick up the insecticide through their feet. The insecticide is key as it means that nets remain highly effective even when they have developed holes and rips and tears, which frequently happens given the environments in which they are needed, as the mosquitoes do not head straight for a hole and fly through it but instead land on the net and migrate (walk) to the hole picking up insecticide in the process.

AMF funds in hand

If we are operating successfully i.e. we are receiving significant funds and funding multiple-million net distributions, we will always have significant funds in hand. This is because the average time being between us receiving funds and when we spend them is about 12 months. The following provides a little more information on the sequence of steps and timing. We cannot sign a legal contract to fund nets until we have the funds in hand, so imagine we are going to fund $10m of nets. We might receive those funds over a 4 month period. The midpoint is at 2 months. We then engage in, or continue, discussions with the relevant Ministry of Health and it may take two to four further months, let’s say three, to reach an agreement (which leads to a legally binding document being signed between the MoH and AMF covering operational and data-led accountability elements). We then place an order for nets which may be produced one to nine months hence (depending on the lead time for the distribution - a longer lead time is better, and we aim for a distribution that will take place 12 months hence, as it allows for all the logistical and operational elements of the distribution to be appropriately planned, resourced and scheduled in-country - a country-wide mass distribution of nets is a significant logistical exercise), then one to two months for the production of the millions of nets, two to three months of shipping time (the start of which being when we pay the net invoices – i.e. when the nets have just sailed), and then one to three months of in-country transport to distribution zones and distribution. In summary, the average time between receipt of funds (at the ‘minus 2 months’ point) is followed by 3 months (discussion leading to an agreement) + 6 months (time until nets start being produced) + 1 month (net production time) before we pay for the nets, = 12 months.

Of the $99.1m we currently have in hand, $16.2m is legally committed and will be disbursed in a few weeks and $80m is about to be committed (against requests for funding of more than $160m).

We have a very large gap in funding for nets at the moment of many tens of millions of dollars.

Warm rgds,
Rob

agrimgalina · September 1, 2019, 3:38pm

Hi Tobias, how much is the minimum money to donate?

tobias-dahms · September 1, 2019, 3:55pm

Hi, thx,

I think if you can afford a RS2 than you can also afford a significant donation. I don’t set an minimum amount. If the software helps you to earn several thousand or ten thousand $ a year you might consider to share some percentage of the benefit - you get it back through your taxes anyhow.

Others offer lever arm correction for 500$, @jantjj donated 250 € and will again. I think it would be nice to donate on an annual basis an amount you can afford.

It should be also clear that many others here might build better softwares. Mine are working (e.g. 1mm difference to other lever arm correction tools) and have been tested by others (e.g. @wsurvey) but they are primitive tools as far as programming style is concerned. Anyhow, for those who do not program and work with Excel etc. they will provide a great benefit.

I do not sell the software but share it with those who donate and I also will share future improvements and implement ideas.

agrimgalina · September 1, 2019, 4:04pm

Tobias! I live in Argentina, it is a lot of money here, imagine that a trimble or topcom costs 22,000 dollars. An emlid for me is a saving of something I could never have. You have not read the newspapers of my country

tobias-dahms · September 1, 2019, 4:19pm

Yes, that is why a minimum amount is maybe not a good approach. And its good that everyone judges on their own what they can afford.
I don’t judge by the country, there a living more Chinese people in luxery than Germans - at the other hand the poorest German might be better off than the average Chinese. I cannot afford one or better two RS2 since I’m living in East Germany and other then in the South (CU in $tuttgart at the INTERGEO ;-)) its where hard to earn money here and the original Reach moduls still do the job good enough.

Unfortunately I cannot read other languages than English, German and French…

tobias-dahms · September 1, 2019, 5:33pm

The demo is available now. I’m happy about some feedback.