12.21.07
Posted in PhD An Acoustic Disdrometer at 11:50 am by philwinder
I have finished rewriting all the subroutines now and I have manage to increase the speed even more. When rewriting, I got rid of any grossly underperforming code and now at 100mm/h it runs through in just under a second.
But once again, the errors are still plaguing the system. At this rainfall rate there are quite a few ghost drops and therefore still quite a lot of missed drops. The one I have just done is at 9% and I can clearly see at least 2 ghost drops. Have to investigate this further.
However, I think its about time I stopped for Christmas, so I don’t think I will be doing much except maybe showing a few people what I have done over the holidays.
Merry Christmas all!
Update
Just a quick update, I continued tarting up the code, and it now reports all ghost drops and all missing drops and why they were missed (ghosting/verification/etc.). Ive just done a beefy test at 100mm/h over 1 second:
run(’..\..\Create Samples\V0_7\V0_7′); V0_9;
7966 Drops Generated
Elapsed time is 143.394716 seconds.
12.7% Error
Thats without all the warning and debug messages. As a results Ive found that the thresholding/peak detection routine does not always find all the pulses. In fact, it only finds them at very slow rain rates. As a consequence I have had to pad the data so they are all the same length. So I am definitely going to have to look into that.
The graph also now shows the errors so it makes them easier to pick out. So just for formality, here are a few screenshots and program outputs:
run(’..\..\Create Samples\V0_7\V0_7′); V0_9;
79 Drops Generated
Elapsed time is 1.486008 seconds.
Ghost Detected at: (0.2093440.1,-0.3657250.1) Due to Pulses: 3774,3492,3380,3705
Ghost Detected at: (0.3033100.1,-0.0157940.1) Due to Pulses: 3619,3407,3602,3758
Ghost Detected at: (-0.0449920.1,0.3495790.1) Due to Pulses: 3481,3749,3880,3702
Ghost Detected at: (0.0771620.1,-0.4683740.1) Due to Pulses: 4014,3790,3505,3864
Ghost Detected at: (0.3148400.1,0.0721700.1) Due to Pulses: 4199,4023,4276,4384
Drop Missed: Number 6. The data did not pass varification tests.
Drop Missed: Number 30. The data was eaten by Ghosts!
Drop Missed: Number 31. The data was eaten by Ghosts!
Drop Missed: Number 32. The data was eaten by Ghosts!
Drop Missed: Number 33. The data did not pass varification tests.
Drop Missed: Number 36. The data was eaten by Ghosts!
Drop Missed: Number 37. The data was eaten by Ghosts!
Drop Missed: Number 38. The data was eaten by Ghosts!
Drop Missed: Number 39. The data was eaten by Ghosts!
Drop Missed: Number 40. The data was eaten by Ghosts!
Drop Missed: Number 41. The data was eaten by Ghosts!
Drop Missed: Number 56. The data did not pass varification tests.
79 drops originally, we have found 67
15.2% Error
And Some output Screenshots:
Permalink
12.20.07
Posted in Future Project Ideas at 10:24 pm by philwinder
Just been looking around for some open source sound libraries to work in Qt, since Qt has no native support for low(ish) level access.
Permalink
12.19.07
Posted in PhD An Acoustic Disdrometer at 5:20 pm by philwinder
After a knock in the right direction from Kevin, I found a slight error in the Multilateration routines. When simplifying U, we chose points on the edge of the tank to solve for. The point which is at the top right of the tank was wrong. I was solving for x^2+y^2 = 1. I should have been solving for x^2 + y^2 = d^2. Fixed that, and now the results are a lot more accurate. No more inaccuracies around the outsides of the tank. Somewhere near +/-0.2mm. Too small to measure.
Still the largest problem is the number of false detects. I have started rewriting the code, but because of this silly error today, I have not had any time to continue rewriting the rest of the code.
Permalink
Posted in Future Project Ideas at 1:09 pm by philwinder
I would like to have a play with facial recognition algorithms, so I had a quick sgeggy on the net to see what I could find. I found quite a few:
Would be great if I could implement a simple algorithm in a PIC. There are a few hits, but nothing specific.
On another note, I am definatly going to do my NFX software Idea of a pedal testing system. This should give me some more good experience in Qt. Have to move this from the back to the front burner.
Permalink
12.17.07
Posted in PhD An Acoustic Disdrometer at 4:09 pm by philwinder
Well here it is. A good couple of months worth of work summed up in a single image. Says a lot doesnt it…
Permalink
Posted in PhD An Acoustic Disdrometer at 1:13 pm by philwinder
I have confirmed that the source of the errors in the newest program is definatly because there is a chance that a random combination of pulses could produce a valid output. So I think the only way to mitigate this is to also check the value and add that to the error.
Plus there are also times where the actual input data has some pulses missing off the end. This is simply because I have not made the array big enough in the original creation. A simple fix.
Permalink
12.15.07
Posted in PhD An Acoustic Disdrometer at 12:43 am by philwinder
I think I have finally chosen all the things I need to get cracking with the real software. No more of this nancy Matlab.
Ive chosen C++, mainly because it will be more beneficial to me in the future. Companies really need you to know c++, and I can learn C# on top of that if I need. Plus I will get a slight speed increase due to less overheads.
Ive chosen Qt because its free, looks pretty neat, has a good reputation and doesnt have all the nasty Microsoft related overheads/complications.
Finally Ive chosen an IDE called HaiQ because it was one of the only compilers I could get to work, and it looks pretty slick.
So now I just have to get learning! (again!)
Permalink
12.14.07
Posted in PhD An Acoustic Disdrometer at 11:52 am by philwinder
As I thought, by changing the code to copy the good records rather than trying to delete the bad ones is a million times faster. The 20 second code from yesterday is now down to 8 seconds. Now the real possible combination sorting isnt even on the profiler any more, its that fast. The slowest part of the code now is the check combination routine, and only because its called 125984 times! Cant be bothered to change any of that since its pretty good anyway. At 10mm/h its now down to about 0.3 seconds. This is all for 0.01 seconds of data mind you.
For 1 second of data it takes about 8-9 seconds. So still roughly an order of 10 behind at 10mm/h.
Ha! And at 1mm/h it takes about 0.95 seconds. So we are just real time at that point.
Permalink
12.13.07
Posted in PhD An Acoustic Disdrometer at 4:56 pm by philwinder
Found something very interesting with the profiler. In the file “Find_Best_Combination.m”, line 56, there is the code:
Possible_Combinations(Temp_Index,:) = [];
Line 56: Possible_Combinations(Temp_Index,:) = []; 220223 Calls, 7.095 s
So basically it takes for ever to delete a record that is in the middle of a set of data. Maybe instead of deleting a record, I should move correct ones to a new array, and then delete the first record. I have heard that deleting records either at the start or the end is quicker.
The only other sources of delay are the “Generate_Possible_Combinations.m” and “Check_Order.m” routines. This is purely due to the fact that they are called so many times. I think they are fairly well optimised already.
What i could try is to remove the redundant order check part of the code and let the “SolvePosition.m” routine take all the working…
Ok forget about that! I just did the above, and not only did it take 140 seconds, there was a 95% error. Probably something wrong with the error, but I’m not about to check when it took that long. It should take something around 20 seconds.
Thats about it for today, so I am waiting a reply from kevin about his thoughts on the accuracy of the “SolvePosition.m” routine, but I’m just about done. So…
Todo:
- I think I should check where these errors are actually happening to make sure that I can’t do anything about it.
- Then after that I should finish the whole thing and present a graph of the found drops and data. Then thats about it!
- Then onto the c++. I think I have decided c++ because it will give me a better stead in the long run. Companies are really looking for c++ rather than c#, but c# is only a simple step away. I could probably learn c# in my spare time if I have a good c++ background.
Permalink
Posted in PhD An Acoustic Disdrometer at 4:01 pm by philwinder
Note: What I do need to do is create some new tests to see if we are actually capturing the correct drop pulses, or just collecting combinations of pulses and pretending they are a drop. Causing not 1, but 2 gross errors in the data. I might leave this until we start writing the real program in c++. It would also be a great exercise in c++ to get me started.
Got lots done since my last post, which is good because I was lagging slightly as of last week. All the notes are in the respective files, and I should probably convert them into word format at some point. Basically what has happened is we have managed to get the simplified multilateration routines working, without any matlab specific code. This has happened mainly thanks to Kevin, who initially wrote some Sanskrit about a solution to the Multilateration routines and some code to solve for x and y. It took some time to decode those scribbles but now its done, it works at least 5 times faster than my previous attempts. There is also room for more speed increases with some close code profiling.
The code uses some simple analytical methods to obtain an initial estimate of x and y, then using some general iteration and Aitken’s Acceleration to create a very accurate solution. At the worst point, we are about 6mm out, which converts to about 4uS.
I generate an error value by recreating the ToA’s using the x,y’s calculated previously. Because we are not strictly using the ToA data, just TDOA to calculate the x,y’s then the ToA’s are independent; meaning that when we calculate it back, the answer is not necessarily what we started off with. So add the differences between the actual and the calculated, and we have an error signal!
As for timings and accuracy, here is some matlab output comparing the original routines to the new ones:
run('..\..\Create Samples\V0_5\V0_5.m')
run('..\V0_7\V0_7.m')
run('..\..\Create Samples\V0_6\V0_6.m')
V0_8
Drop_Number = 99
Elapsed time is 94.876437 seconds.
There were 99 drops originally, we have found 88
11.1% Error
Drop_Number = 103
Elapsed time is 20.169884 seconds.
There were 103 drops originally, we have found 93
9.7% Error
So the errors approximatly the same (random of course). I might look into these errors some more, because 10% is quite a lot, but it only happens at higher rain rates. I believe it is to do with the possiblility of generating some combination of a number of drops that happens to be correct. I could try turning the tolerance down a bit.
Permalink
« Previous entries
