Question

I’m very new to pthread world.
I need to process a file with list of commands, let say the file looks like this-

Command1
Command2
Command3
.
.
CommandN

For each command, I want to create a thread. Now the problem is if there are large number of commands, I will end up creating a large number of threads which is what I want to avoid. So, I would like to limit number of threads that are executing at any given point of time. Let’s say that number is 5.

Can someone please suggest how to achieve this? I’m using following code for learning-

#include <pthread.h>
#include <stdio.h>
#include <stdlib.h>
#define NUM_THREADS 4

void *BusyWork(void *t)
{
   int i;
   long tid;
   double result=0.0;
   tid = (long)t;
   printf("Thread %ld starting...\n",tid);
   for (i=0; i<1000000; i++)
   {
      result = result + sin(i) * tan(i);
   }
   printf("Thread %ld done. Result = %e\n",tid, result);
   pthread_exit((void*) t);
}

int main (int argc, char *argv[])
{
   pthread_t thread[NUM_THREADS];
   pthread_attr_t attr;
   int rc;
   long t;
   void *status;

   /* Initialize and set thread detached attribute */
   pthread_attr_init(&attr);
   pthread_attr_setdetachstate(&attr, PTHREAD_CREATE_JOINABLE);

   for(t=0; t<NUM_THREADS; t++) {
      printf("Main: creating thread %ld\n", t);
      rc = pthread_create(&thread[t], &attr, BusyWork, (void *)t); 
      if (rc) {
         printf("ERROR; return code from pthread_create() is %d\n", rc);
         exit(-1);
         }
      }

   /* Free attribute and wait for the other threads */
   pthread_attr_destroy(&attr);
   for(t=0; t<NUM_THREADS; t++) {
      rc = pthread_join(thread[t], &status);
      if (rc) {
         printf("ERROR; return code from pthread_join() is %d\n", rc);
         exit(-1);
         }
      printf("Main: completed join with thread %ld having a status of %ld\n",t,(long)status);
      }

printf("Main: program completed. Exiting.\n");
pthread_exit(NULL);
}

Are there any good examples out there for Boss/Worker thread model?

Answer 1

Create a queue where you put the new jobs. Create 5 threads. Each thread will pick one job and process it, then it will pick the next job. Once the queue is empty the threads can exit and you can do thread_join. You’ll need synchronization for the queue (or array).