Question

I am an experienced Linux socket programmer and am writing a server application which has many outgoing interfaces. Now server socket binds to a random source port in the start of the process along with INADDR_ANY.

Later at some point when submitting response to a specific node, i need to assign a fixed source ip address. The standard way to do this is calling bind. However, bind is called once for the port number, successive calls fail with invalid argument error.

Creating a new socket is not really a good choice since i will have to be doing this very often upon responding to some clients.

I have also explored SO and a lot of socket options such as IP_FREEBIND, but it doesn’t quite suite my scenario.

Perhaps using IP_PKT_INFO and setting source address might work unless it suffers the same problem i.e. not allowing a socket once bound to INADDRANY to rebind to a fixed source ip latter.

Is there a way to unbind an existing socket or an alternate way to setting source ip address in outgoing packet?

    int sock = socket(AF_INET, SOCK_DGRAM, 0);

    if(sock < 0)
        printf("Failed creating socket\n");

    struct sockaddr_in addr;
    memset(&addr, 0, sizeof(struct sockaddr_in));
    addr.sin_family = AF_INET;
    addr.sin_port = htons(1500);
    addr.sin_addr.s_addr = INADDR_ANY;

    // first bind succeeds
    if ( (status = bind(sock, (struct sockaddr *) &addr, sizeof(addr))) < 0)
        printf("bind error with port %s\n", strerror(errno));  

    struct sockaddr_in src_addr;
    memset(&src_addr, 0, sizeof(struct sockaddr_in));
    src_addr.sin_family = AF_INET;
    if (inet_aton("10.0.2.17", &(src_addr.sin_addr)) == 0)
        printf("Failed copying address\n");

    // second bind fails
    if((status = bind(sock, (struct sockaddr *)&src_addr, sizeof(src_addr))) < 0)
        printf("re bind error with ip %s\n", strerror(errno));

Any ideas in this regard will be highly appreciated. I have gone through considerable material on sockets, SO etc. but no success yet.

Answer 1

I finally found the solution myself so accepting my own answer (shameless but correct plugin), supplemented with code sample.

I originally wanted to rewrite source address of an outgoing packet without creating the socket again where the socket was already bound. Calling bind multiple times fail for this case, and (in my particular situation), i was not able to just have separate sockets for each source ip and use it.

I found some references in IP_PACKET_INFO but it was a pain to get it to work correctly. Following reference was helpful.

Setting source of udp socket

Sample Code

Here is a trivial application which creates a udp socket, binds it to a local port, then before sending a particular message, it appends the outgoing source ip address. Keeping in mind that in my case, i created a sudo interface and assigned it another ip. The send call will fail if this is not the case.

int status=-1;
int sock = socket(AF_INET, SOCK_DGRAM, 0);

if(sock < 0)
    printf("Failed creating socket\n");

int opt = 1;
setsockopt(sock, SOL_SOCKET, SO_REUSEADDR, &opt, sizeof(opt));

struct sockaddr_in bind_addr;
memset(&bind_addr, 0, sizeof(struct sockaddr_in));
bind_addr.sin_family = AF_INET;
bind_addr.sin_port = htons(44000); // locally bound port

if((status = bind(sock, (struct sockaddr *)&bind_addr, sizeof(bind_addr))) < 0)
    printf("bind error with port %s\n", strerror(errno));

// currently using addr as destination
struct sockaddr_in addr;
memset(&addr, 0, sizeof(struct sockaddr_in));
addr.sin_family = AF_INET;
addr.sin_port = htons(80); // destination port
if (inet_aton("74.125.236.35", &(addr.sin_addr)) == 0)
    printf("Failed copying remote address\n");
else
    printf("Success copying remote address\n");

struct sockaddr_in src_addr;
memset(&src_addr, 0, sizeof(struct sockaddr_in));
src_addr.sin_family = AF_INET;
if (inet_aton("10.0.2.17", &(src_addr.sin_addr)) == 0)
    printf("Failed copying src address\n");
else
    printf("Success copying src address\n");

char cmbuf[CMSG_SPACE(sizeof(struct in_pktinfo))];

char msg[10] = "hello";
int len = strlen(msg);

struct msghdr mh;
memset(&mh, 0, sizeof(mh));

struct cmsghdr *cmsg;
struct in_pktinfo *pktinfo;

struct iovec iov[1];
iov[0].iov_base = msg;
iov[0].iov_len = len;

mh.msg_name = &addr; // destination address of packet
mh.msg_namelen = sizeof(addr);
mh.msg_control = cmbuf;
mh.msg_controllen = sizeof(cmbuf);
mh.msg_flags = 0;
mh.msg_iov = iov;
mh.msg_iovlen = 1;

// after initializing msghdr & control data to 
// CMSG_SPACE(sizeof(struct in_pktinfo))
cmsg = CMSG_FIRSTHDR(&mh);
cmsg->cmsg_level = IPPROTO_IP;
cmsg->cmsg_type = IP_PKTINFO;
cmsg->cmsg_len = CMSG_LEN(sizeof(struct in_pktinfo));
pktinfo = (struct in_pktinfo*) CMSG_DATA(cmsg);

//src_interface_index 0 allows choosing interface of the source ip specified
pktinfo->ipi_ifindex = 0;
pktinfo->ipi_spec_dst = src_addr.sin_addr;

int rc = sendmsg(sock, &mh, 0);
printf("Result %d\n", rc);

The key statement is

pktinfo->ipi_spec_dst = src_addr.sin_addr;

where we are specifying the source ip address to be used. The rest of things like cmsg struct etc. are merely used in order to be able to write ipoktinfo struct ourselves