Packet Flow Between Two Different Networks

We shall now discuss about packet flow between two different machines hosted behind two different routers which are acting as a gateway for the local machine in each respective region (Region 1 and Region 2).
The local machine connects to layer 2 switch first and the switch has a connection to Router. Here Router is acting as a L3 device (again in reference with OSI model).

Click on the image to have a bigger picture and more clarity and compare with our below statements when discussion starts on packet flow.

Here in the packet flow, I will stick to simple header information considering we are using a TCP protocol in this communication. Our main focus is to understand how router generally behaves when there is a route request coming in for different network.
Some basic understanding or properties of a router:

1. Router is a Layer 3 device with reference to OSI model.
2. Router take forwarding decision on the basis of routing table.
3. Router routes packets between networks. Hence every port of a router has to belong to a different network. Important point to consider.
4. No two interfaces of a router can ever belong to the same network.
5. Routes in a routing table can be populated statically by an administrator or dynamically via a routing protocol.
6. A router can forward a packet only if it has got a route to reach the network. If no route is present in the routing table for the destination network, router will drop or discard the packet.
7. Every port of a router is a member of a separate collision domain and separate broadcast domain. Hence a router is a multiple collision and broadcast domain device.

Setting the stage: Now after we are clear with the properties of a router, the packet flow will become easy. Let us skip the packet flow within a Switch (a layer 2 device with reference to OSI model) since we learned this already how it works in our previous section. If not please have a quick go through the process of packet flow between two machines using a switch. This Layer 2 switch will do its switching decision based on its CAM table.
Packet Flow: Let us consider the exchange of packets already took place between a router and a switch to learn each other MAC addresses. The switch has placed in the MAC address information of the router which is in its CAM table. Also, the router placed in the MAC address of the switch in its ARP table against the port it is connected to.

Let’s consider the CAM table will look like this

And the ARP table of the route will look like this

Note: Real time representation of the MAC address and ARP table in router and switch will look different and as you know this is shown for our understanding purpose.

If machines M1 wants to communicate with M2, in that case DNS resolution and AND process (logic gate) takes place. Once M1 has the IP address of M2 it will start forming the data packet but then again it will look into its own ARP cache details in its own MAC table. M1 will not have the MAC address of M2 (mmm). Thus, M1 holds on to the actual data packet and initiates an ARP packet. After AND process, M1 knows that M2 is not in the same network as that of M1 so it will try to reach out to its gateway first.

As M1 makes an attempt to reach to its gateway (10.1.1.1). If M1 already has the MAC address of the gateway by looking into its MAC table then the packet structure will be like picture-c and if the MAC address of the gateway/router is missing from its MAC table then it will again initiate one more ARP packet which will be destined for its own gateway. The packet will look like this.

Picture-a

The layer 2 switch receives the packet the ARP request initiated by M1 and the packet is received on Port-1 of the SW1. Immediately, the switch opens up the source MAC header, learns and compare the M1 MAC address which is aaa and updates its CAM table if the MAC is not already listed. Then the switch checks the destination MAC header and see it is a layer 2 broadcast address and knows that this request is an ARP request.

So, the switch makes multiple copies of this ARP request and broadcast out to all other ports except on the port it got the packet(port-1). The ARP packet will be dropped by all other end machines connected to the same SW1 except by the router that is connected on Port-7 on SW1. The router will receive the ARP packet and opens the broadcast request. It checks and learns the MAC address of M1 and updates in its ARP table. Also, it opens the destination MAC header and understand that it is a broadcast layer 2 address (this address does not get stored in the ARP table). It then opens the Layer 3 header which are the IP addresses. This is important, the router does not forward broadcast packets and if the ARP request does not have the IP address of its own under the destination IP header, the router will drop the packet then and there. But once the router looks into the destination header and sees its own IP address (10.1.1.1) it will then reply to the ARP request packet with its own MAC address mentioned in the response packet and forward it to M1. The packet will look like this.

Picture-b

The switch SW1 will receive the packet and again opens up the layer 2 header. It will then make sure its own CAM table is updated before forwarding the packet for both MAC address R1 and M1. Switch also knows where M1 is connected based on its CAM table. Once the switch forwards the ARP response packet to M1 and then M1 process the information by opening up the headers. M1 will update its own ARP table with R1 MAC address and then will send out the original data request which looks like below. Now the MAC address in this new packet would not be fff or layer 2 broadcast address since M1 knows that M2 is not in the same network as its own so it will send out to its default gateway for communication to happen.

Picture-c

Now this is again important point you are coming across based on the above packet detail. The destination MAC address is of the router R1 but the IP address is of destination machine M2. Why so? It is because at the end the packet that is being send by M1 is the actual data information and machines communicate with each other based on logical addressing which is IP address. So, the destination address will be 10.1.2.24 rather than the R1 IP address. The motive of M1 is to communicate with M2 using this logical address (IP address). Also, from here onwards you will see that the source and destination IP address will never change in the data packet (not talking about ARP packet) but the MAC addresses will keep on changing after each hop.

Once the data packet reaches R1 through SW1, the R1 opens the data packet once again and looks into the L2 addresses or MAC addresses of both source and destination. Looking at the source MAC address it will not make any new entry in its ARP table since it already knows, it will then look up for the destination MAC address and it knows the packet was meant to be sent to R1. Then it processes with L3 address lookup, it checks the source IP address of M1 and also understand that there is a different IP address mentioned in destination IP header which is different then its own. So, router does a route-lookup and checks its routing table if it knows how to reach the destination IP address which is 10.1.2.24. Based on the route entry which it has already it will then change or alter the data packet with its own source MAC and destination MAC in the above(picture-c) except for source IP header and destination IP header.

The source MAC header will contain the MAC address of R1 i.e., ccc, then the destination MAC header will contain the MAC address of the R2 which is ddd. Both R1 and R2 will complete the ARP process once the cable gets connected between these two interfaces. Routers are very active in learning the MAC addresses of its neighbor. Now the existing packet will form like this.

Picture-d

These are data plane traffic (just a fyi) and also routers are interested in looking into destination IP address.

Once the router R2 receives the packet from R1, it will check the Layer 2 MAC address and confirms that the packet is destined for itself based on the destination MAC header. It then opens up the Layer 3 header which is the IP address header for source and destination. It learns that the source IP 10.1.1.23 is trying to reach the destination IP 10.1.2.24 which belong to its one if its own network (10.1.2.0/24). R2 now alters the same data packet once again and it looks as below (considering the process of learning MAC address is already completed that is, R2 already knows the MAC address of the end destination with the help of DHCP request or ARP request.

The machine M2 receives the packet and learns the IP address of M1. M2 now can send a reply back the same path that M1 followed but this time the reply packet that will flow through the different network till M1 will not require any ARP packets since each hop knows about each other physical address (MAC address) and its corresponding logical address (IP Address).

Last Minute Notes:

1. Router will care less for MAC address in a data plane traffic and will be interested in IP headers.
2. We also have L3 switches in our networking world and which are more preferred than a Layer 2 switch since it has capability of doing switching with the help of ASIC and also has the capability of doing routing with route engine capability introduced in it.

Real-Time ARP packet. ARP is used to resolve a physical address MAC address) from a logical address(IP address).

I hope this is a helpful content for you readers.