Solving Docker Networking , Let the whole network see the IP ranges of each Container Host using Cumulus Routing On Host
Containers such as Docker, LXC, LXD support different networking methods such as LinuxBridge, OVS and Linux Sockets. The most simple method which is widely used is Linux Bridge Networking. Means that the host, creates a linux bridge and attaches the containers to that bridge.
Although we have the concept of destination NAT on the container host to NAT certain ports to the container, but in reality it has many limitation. For example you can have only 1 container to as a web server on port TCP:80, since the host port 80 will be NATed to that specific container. Therefore there is a certain need to expose the container itself to the network and let the container to communicate with network directly.
Above diagram is self explanatory about the routing trouble. How the router knows where to route the container IP prefixes od 172.17.0.0/16 and 172.18.0.0/16 ?
The first thing every network administrator will do in hurry to solve this issue is to add static routes to route the specific docker0 IP prefixes to each host. It works, but static routing is not a good, manageable option.
What if we could have a method for the hosts to advertise their prefixes to the router? That could be an ideal solution.
Cumulus Routing On Host (ROH) is a Linux host routing package which can be installed on container hosts. Its an enhanced Quagga.
Using Cumulus ROH, each host can run a routing protocol to advertise its container bridge networks. You don't need to worry about IP changes, new linuxbridges, all the changes will be advertised to your network via the routing protocol you choose.
Additional benefits of using Cumulus Routing On Host :
Host can use multiple physical NICs, ROH will use ECMP to load balance on multiple NICs without need of bonding, LACP, MLAG.
You don't need to use Layer 2.
Using BGP Unnumbered will help you to not configure any IP address on NIC , yet establish the BGP connectivity.
No performance issue or additional processing by host.
Unifying the network as a end-to-end Layer 3 fabric.
It is Free
Can peer with any vendor device such as Cisco, Arista, Juniper, Dell, Huawei, Brocade, Fortinet and of course Cumulus switches.
Why using Cumulus ROH instead of standard Quagga?
Cumulus enhanced Quagga has some additional features such as Unnumbered interfaces RFC5549 and other improvement. In addition Cumulus ROH is an enterprise grade product with fully enterprise level support.
You can download Cumulus ROH from here.