Question

On a listening socket I set the EPOLLIN bit however on client connections I set EPOLLIN | EPOLLOUT bits to struct epoll_event like so:

struct epoll_event ev;

ev.data.fd = fd;
ev.events = EPOLLIN | EPOLLOUT;
if (epoll_ctl(evs->epoll_fd, EPOLL_CTL_ADD, fd, &ev) < 0)
     ...

And this is how I test bits:

if ((events & EPOLLIN) == EPOLLIN)
     ...
if ((events & EPOLLOUT) == EPOLLOUT)
     ...

I’ve also tried like:

if (events & EPOLLIN)
     ...
if (events & EPOLLOUT)
     ...

Both ways are ALWAYS true!

However, whenever I call epoll_wait on my epoll fd, the active file descriptor returned ALWAYS has both bits set even though send() didn’t return EAGAIN but when I try to recv() it returns EAGAIN.

I have no idea what am I supposed to do when recv() returns EAGAIN, am I supposed to remove the EPOLLOUT flag or what?

More code as requested by @Nikolai N Fetissov:

static int get_active_fd(events *evs, int index, sstate_t *flags)
{
    uint32_t events = evs->events[index].events;
    int fd = evs->events[index].data.fd;;

    if ((events & EPOLLERR) == EPOLLERR || (events & EPOLLHUP) == EPOLLHUP) {
        close(fd);
        return -1;
    }

    if (events & EPOLLIN)
        *flags |= DATA_IN;

    return fd;
}

void sockset_add(events *evs, int fd)
{
    struct epoll_event ev;
    ...
    ev.data.fd = fd;
    ev.events = EPOLLIN;
    if (epoll_ctl(evs->epoll_fd, EPOLL_CTL_ADD, fd, &ev) < 0)
        eprintf("sockset_add(): epoll_ctl(%d) returned an error %d(%s)\n",
                fd, errno, strerror(errno));
}

Then later where I call epoll_wait():

if (flags & DATA_IN) {
       /* try to read which is impossible because this is never set.  */

Answer 1

I think you got it backwards. Don’t include EPOLLOUT unless you got EAGAIN from a write attempt, and remove it when you have successfully written bytes to a socket.

This basically means that socket is always writable as long as there’s space in socket in-kernel send buffer.

Edit 0:

You have two data streams – input and output.

You are waiting for the input by including EPOLLIN in the flags. If upon return from epoll_wait(2) that flag is not set, then either some event happened on some other socket, or this socket had some other event. Leave the flag in the events unless you get an error (meaning you are still interested in the input on the socket).

You don’t have to wait for the output (since it’s your action), you just write to the socket, but if you overflow socket send buffer, you’ll get EAGAIN from send(2) or write(2). In this case you start waiting for output to be possible (kernel draining socket send buffer thus making room for you to send more) by including EPOLLOUT. Once you get that, write your pending output bytes to the socket, and if you are successful, remove EPOLLOUT from the events.

Now EPOLLET indicates edge-triggered wait, meaning your desired event would only be signaled once per state change (like from “no input” to “there’s input”). In this mode you are supposed to read input bytes in a loop until you get EAGAIN.

Hope this helps.