serious number crunching

[cpc6128]

I was wondering whether someone could take a look at my code and give me some points as to how I could optimise it? It is fairly slow when it runs on my desktop so I haven't even bothered running it on my PPC yet!

Basically the program takes some initial lat/long data from a GPS (but for the purposes of testing, from 2 text boxes on the form) and compares this to a list of UK postcodes and their lat/longs read from the data file. It then works out which postcode is closest to the initial lat/long data.

I have only included a very small part of the data file as an example as I do not think I am allowed to distribute the full data set.

I am reading the text file a line at a time. Firstly, I couldn't see a way in the help to read part of a line (as in a comma delimited file) so my program has to iterate through each character in the line to check for commas. I'm sure there is a much more efficient way of doing this!

Secondly, I kept getting errors when I tried to peform maths functions on the data I read in. Ideally I need the numeric data to be stored as a Double, but since there are no functions to convert data types (that I could see) I think my program was treating the data as a String which was causing me errors. To overcome this I have used the Decimal library throughout my code which I'm sure is horribly inefficient given the number of calculations involved.

The program works as-is but as I said is incredibly slow... any help in speeding it up would be much appreciated!

[LineCutter]

Sort the file by Lat
Break it up into convenient sized chunks (index the range of lat in each file somehow)
[This saves memory space for the PPC version]

Read the Lat (& long) from your textboxes
Load the correct file (as a whole) into a database structure (table or SQL)
Use a binary search routine to home in on the correct Lat
Read "up" & "down" to find the limits of that Lat in your database
Search these to find the Long (you could sort by long, within each lat value when you create the data files,& do a binary search through these if there are lots)
Read the postcode.

[cpc6128]

Thanks, I will give that a try.

[Leginus]

I agree with linecutter that I think the speed problem comes from reading the file line by line and not the number crunching. Processing from memory is much quicker

[cpc6128]

I think the main point of linecutters post was that I could speed up the program by making the search algorithm more efficient e.g. by pre_sorting some of the data so I don't have to search the entire file every time.
However I think you also have a very good point about file access slowing things down and I agree that loading the data into memory first is sensible. I will implement both ideas, hopefully together they will make a big difference. thanks!