Question

The application I am currently working on is a server which will manage connection with the client using select(), each time the server receive a message, it will open a new thread in order to read the socket. During that time the file descriptor of the socket is remove from the set, and will be added at the end of the read.
Here is a sample of the code

struct s_handle {
 int sock;
 fd_set * rdfs;
};




int main(){
 ...
 fd_set rdfs;
 ...
 while(1){
 ....
  select(nb_fd,&rdfs,NULL,NULL,NULL)
  for_each(peer){
   if(FD_ISSET(peer->sock,&rdfs)){
     struct s_handle * h = malloc(sizeof(struct s_handle));
     h->sock = peer->sock;
     h->rdfs = &rdfs;
     FD_CLR(peer->sock,&rdfs);
     pthread_create(thread,NULL,handle,(void *)&h);
   }
  }
 ...
 }
 ...
 }

void* handle(void* argss){
 struct s_handle * temp = (struct s_handle *) argss;
 ...
 FD_SET(temp->sock,temp->rdfs);
}

Does FD_SET, FD_ISSET and FD_CLR are atomic operation, or do i need to lock rdfs with a mutex?

If a mutex is needed, how can I avoid deadlocks?

Answer 1

First, you shouldn’t be creating threads like that. Creating a thread is a fairly high overhead operation and should only be used when you need more threads, not just because you have to do more work.

And yes, you do need to protect the FD_* functions with a mutex. The usual solution is to have a mutex that you hold only for the split second it takes to perform the FD_* operation. Before calling select, you acquire the mutex, make a copy of the descriptor set, and then release the mutex.

In general though, it’s a bad idea to remove the socket from the read set. Putting the socket back in the read set won’t change the select that’s already occurring later on. And you’ll have the ugly mess of figuring out how to get the thread calling select out of select in order to operate on the new set.

You may want to rethink your I/O discovery method and use one of the standard ones instead of trying to roll your own. You’re forcing the ugly tradeoff of either having some sockets not being listened to for read because the were recently read and the select is still blocked or having to re-select as you finish reading from each socket. Neither solution is good.

A more common pattern is to keep the socket in the set while you’re reading on it and not go back to select until all sockets have been read from (but not necessarily had their data processed).