Question

I am developing the machine learning analysis program which has to process the 27GB of text files in linux. Although my production system won’t be rebooted very often but I need to test that in my home computer or development environment.

Now I have power failure very often so I can hardly run it continuously for 3 weeks.

My programs reads the files, applies some parsing, saves the filtered data in new files in dictionary, then I apply the algorithm on those files then saves result in mysqlDB.

I am not able to find how can I save the algorithm state.

Answer 1

I everything regarding the algorithm state is saved in a class, you can serialize the class an save it to disk: http://docs.python.org/2/library/pickle.html