Question

In my program, I use a single thread pool to dispatch all my task, like timer task, non-blocking socket I/O, etc. A task is actually an callback function, which will be executed when specific event received.

The architecture is :

1. The main thread calls epoll() to harvest the I/O event, then dispatch the I/O callback to the thread pool.

2. The main thread also handle timer timeout, and dispatch the timeout callback to the thread pool

3. In an I/O callback, one timer task may be cancelled, depending on I/O processing result.

4. During one I/O callback is running, the coresponding socket is not monitored for further identical event.

5. Durning one timer callback is running, that timer task will temporarily removed from the timer task queue.

Here is the problem:

1. During thread A in the pool is running a timer callback T.

2. Thread B in the pool may be running another callback(registered for an socket I/O read event), after processing request received, thread B decide to delete the timer task T, but that timer task T is being executed by thread A right now.

I can add an lock for the timer task, but where should I place the lock? I can’t place the lock object in the timer task structure, because when I decide to free the task object, I must have already acquired the lock, free and held lock, may lead to undefined behaviours:

pthread_mutex_lock(T->mutex);
 free(T);
 /*without a pthread_mutex_unlock(T->mutex);*/

And what happened if another thread is blocked on :

pthread_mutex_lock(T->mutex);

Without these problem being addressed, I can’t continue my work.Please HELP me out!

Should I use separate thread pool for task of different types in my single process? Or just use single thread?

Any suggestion is appreciated!

Answer 1

You can use a global table of timers protected by its own mutex. The table does not actually need to be global but can belong to some collection, such as whatever owns all the things you are doing I/O on.

Then use this logic:

To create a timer:

1. Lock the global table.

2. Add the timer to the global table in state “pending”.

3. Unlock the global table.

4. Schedule the timer with the thread pool.

To fire a timer:

1. Lock the global table.

2. Check the timer’s state. If it’s not “pending”, delete the timer, unlock the table, and stop.

3. Change the timer’s state to “firing”.

4. Unlock the global table.

5. Perform the timer operation.

6. Lock the global table.

7. Remove the timer from the table.

8. Unlock the global table.

To cancel a timer:

1. Lock the global table.

2. Find the timer. If it’s state is “pending”, change it to “cancelled”. Do not delete it.

3. Unlock the global table.