Question

I am trying to send 5 emails in parallel. The function scheduleEmails takes a list of list of 5 email addresses as an input. My code looks like as follows:

import multiprocessing
import smtplib

def sendMail(email):
    #sends email using smtplib
    # returns True in case of success, False in case of failure

def scheduleEmails(emailLst):
    """
    emailLst is a list of list of 5 emails
    emailLst = [[emailAddr1,emailAddr2,emailAddr3...emailAddr5],
                [emailAddr6...emailAddr10],
                [emailAddr11... emailAddr[15],...]
    """
    FREQUENCY = 5 # no. of emails to be send per second
    for i in range(len(emailLst)):
            p = multiprocessing.Pool(FREQUENCY)
            emails = emailLst[i]
            results = p.map(sendEmail,emails)

 scheduleEmails(someEmailLst)

The code works well but after some time, it hangs. Can you point out the mistake or suggest a better way to achieve this ?

Answer 1

How many subprocesses do you have spawned after ‘a while’? To be sure not to spam your system with zillions of subprocesses, explicitly close the multiprocessing pool after collecting the results. Also, I do not see any logic in you code that controls the actual frequency of sending emails. Consider you have 1000 lists of 5 emails in your emailLst. Your loop will spawn 1000 multiprocessing pools, i.e. 5000 subprocesses, as fast as it can. You need to implement some timing/scheduling code. In the simplest case this would be a time.sleep(1) in your loop.

Also, multiproecessing is not really appropriate for this problem, because sending mails is not CPU bound. It’s even not I/O bound in your case I guess. You could consider threading for the purpose of outsourcing a blocking call to a thread instead of a subprocesses.