Multihome

local copy from following public blogs

Strong_End_System_Model_/_Weak_End_System_Model

linux-source-routing-strong-end-system-model-strong-host-model

Step 1: Enable ARP filtering on all interfaces:

# sysctl -w net.ipv4.conf.all.arp_filter=1
# echo "net.ipv4.conf.all.arp_filter = 1" >> /etc/sysctl.conf

From the file networking/ip-sysctl.txt in the Linux kernel docs:

arp_filter - BOOLEAN

   1 - Allows you to have multiple network interfaces on the same
   subnet, and have the ARPs for each interface be answered
   based on whether or not the kernel would route a packet from
   the ARP'd IP out that interface (therefore you must use source
   based routing for this to work). In other words it allows control
   of which cards (usually 1) will respond to an arp request.

   0 - (default) The kernel can respond to arp requests with addresses
   from other interfaces. This may seem wrong but it usually makes
   sense, because it increases the chance of successful communication.
   IP addresses are owned by the complete host on Linux, not by
   particular interfaces. Only for more complex setups like load-
   balancing, does this behaviour cause problems.

   arp_filter for the interface will be enabled if at least one of
   conf/{all,interface}/arp_filter is set to TRUE,
   it will be disabled otherwise

Step 2: Implement source-based routing

I basically just followed directions from http://lartc.org/howto/lartc.rpdb.multiple-links.html, although that page was written with a different goal in mind (dealing with two ISPs).

Assume that the subnet is 10.0.0.0/24, the gateway is 10.0.0.1, the IP address for eth0 is 10.0.0.100, and the IP address for eth1 is 10.0.0.101.

Define two new routing tables named eth0 and eth1 in /etc/iproute2/rt_tables:

... top of file omitted ...

1    eth0
2    eth1

Define the routes for these two tables:

# ip route add default via 10.0.0.1 table eth0
# ip route add default via 10.0.0.1 table eth1
# ip route add 10.0.0.0/24 dev eth0 src 10.0.0.100 table eth0
# ip route add 10.0.0.0/24 dev eth1 src 10.0.0.101 table eth1

Define the rules for when to use the new routing tables:

# ip rule add from 10.0.0.100 table eth0
# ip rule add from 10.0.0.101 table eth1

The main routing table was already taken care of by DHCP (and it's not even clear that its strictly necessary in this case), but it basically equates to this:

# ip route add default via 10.0.0.1 dev eth0
# ip route add 130.127.48.0/23 dev eth0 src 10.0.0.100
# ip route add 130.127.48.0/23 dev eth1 src 10.0.0.101

And voila! Everything seems to work just fine. Sending pings to both IP addresses works fine. Sending pings from this system to other systems and forcing the ping to use a specific interface works fine (ping -I eth0 10.0.0.1, ping -I eth1 10.0.0.1). And most importantly, all TCP and UDP traffic to/from either IP address works as expected.