Question

This is a situation where there is always more work to be done since there is always a queue of emails to process. I want to make sure that any exceptions produced in the parallel.foreach will be logged. The examples I’ve found all wait until the queue (in this case MailToDownload) is finished before I can handle the exceptions. In my situation, how should I handle exceptions?

Below is a basic example of how I’m writing my methods. The first task pulls a list of emails to be pulled from the database. Every time an email account gets mail, the email address is inserted into the table, so there is always data.

Sub Main()
    Dim MailToDownload As New BlockingCollection(Of myMail)

    Task.Factory.StartNew(Sub()
                            For i As Integer = 0 To 99
                                MailToDownload.Add(New myMail With {.id = i})
                                System.Threading.Thread.Sleep(1000)
                            Next
                          End Sub)

    Task.Factory.StartNew(Sub()
                            Dim options As ParallelOptions = New ParallelOptions With {.MaxDegreeOfParallelism = 16}
                            Parallel.ForEach(MailToDownload.GetConsumingPartitioner(), options, Sub(item) doSomething(item))
                          End Sub)
End Sub

I’m thinking I should just handle all exceptions by loading them into another blockingcollection and spawn a different thread to handle those.

Answer 1

I ended up putting the exceptions into a blocking collection and put a timer in another thread to periodically process the exceptions. It works quite well in a situation where the processing on the blocking collection never ends.