Question

This could be considered a continuation of this earlier SO question.

Ideally, I’d like to jail a process into only using a certain interface, no matter what. It will be making TCP connections, sending UDP datagrams, and listening for UDP broadcasts. Currently, what I am doing is:

1. Determine the IP of the interface to use.
2. Create an IP policy rule to route all packets coming from the interface to that IP
3. Create another IP policy rule to route all packets coming from that IP to that interface
4. Set up a default routing table for each rule

Now, this works, mostly, but the client process has to also be willing to play along. That is, it needs to bind to the specific IP of the interface it wants to use, and I think I need to set SO_BINDTODEVICE as well. (However, I keep reading conflicting information about whether SO_BINDTODEVICE actually works when using TCP or UDP.) Luckily, the client application is Python, and I can extend the socket class to do all of this transparently. But I’m not sure it is a complete solution, especially with regard to receiving broadcasts.

My questions are:

1. Does SO_BINDTODEVICE do what I want here? Or is it only effective for raw sockets? Someone commented that, ‘SO_BINDTODEVICE on a socket doesn’t guarantee the socket will only receive packets that arrived on that physical interface’s wire/antenna.’ If this is indeed true, then what does SO_BINDTODEVICE do?

2. Is there a way to do this such that the local IP doesn’t have to be unique? This wouldn’t be a problem other than the fact that the DHCP server on one interface may allocate it an IP that is in use by another interface, thus confusing the routing table.

3. How do I receive broadcasts only from a specific interface? Binding to a specific IP seems to make it ignore broadcasts, which makes sense, but isn’t quite what I’m looking for.

I’m running on Ubuntu 8.04 w/ Linux kernel 2.6.26. Being able to access the same subnet on two different networks through two different interfaces simultaneously is a non-negotiable requirement, thus making it (mostly) immune to ‘don’t do that.’ 🙂

Answer 1

After a hard-fought weekend, I’m pleased to present a solution that addresses most of what I’ve previously discussed with almost zero hassle.

There is a sysctl called net.ipv4.conf.all.rp_filter that can be set to 0 to disable source validation:

     rp_filter - INTEGER          2 - do source validation by reversed path, as specified in RFC1812              Recommended option for single homed hosts and stub network              routers. Could cause troubles for complicated (not loop free)              networks running a slow unreliable protocol (sort of RIP),              or using static routes.           1 - (DEFAULT) Weaker form of RP filtering: drop all the packets              that look as sourced at a directly connected interface, but              were input from another interface.           0 - No source validation. 

This can also be set on a per interface basis using /proc/sys/net/ipv4/conf/<interface>/rp_filter.

As one poster explained it, it makes IP routing ‘less deterministic’ in the sense that packets coming from one subnet aren’t guaranteed to always go out the same interface. In this instance, this is exactly what it is needed. Please do additional research to determine if this is really what you want.

Broadcasts are still problematic for reasons I do not understand, but I am finally satisfied with this issue and I hope it helps others.