Whilst looking at the python doc for multiprocessing.terminate() I came across the following:
Terminate() If this method is used when the associated process is using a pipe or queue then the pipe or queue is liable to become corrupted and may become unusable by other process. Similarly, if the process has acquired a lock or semaphore etc. then terminating it is liable to cause other processes to deadlock.
Which basically says if you terminate a process which is using a queue, pipe or similar you run the risk of the structure becoming corrupt.
I have a couple of questions about this,
- What will happen to another process trying to retrieve the data from the PIPE, Queue or similar if a corruption occurs?
- How can a process check to see if there is corruption?
- Can the deadlock be resolved in any way if you know another process has been terminated?
I understand you should always try to not use terminate, but this is for that situation where you cannot do anything else but this
Okay, so it was not the nicest solution but I have firstly handled the exceptions as best as possible. I did not want to run the risk of having corruption so I now restart the application in an instance where there is the possibility of a corruption occurring to reduce the chance of possible issues.
Thanks @MarwanAlsabbagh for your suggestion.