I think the main point of linecutters post was that I could speed up the program by making the search algorithm more efficient e.g. by pre_sorting some of the data so I don't have to search the entire file every time.
However I think you also have a very good point about file access slowing things down and I agree that loading the data into memory first is sensible. I will implement both ideas, hopefully together they will make a big difference. thanks!
|