Troubleshooting for Bursty Traffic

Identify an interface that has incremental output drops.

For example, you notice output drops on a 100Mb link while the average utilization of the link is only 55Mb. Here is the output of the command:

 !!!!!!!You Hit this command again and again but you are not showing output drops increment means this drops will occur due to burst traffic rate!!!!!!!

Switch#show int fa1/1 | i duplex|output drops|rate
Full-duplex, 100Mb/s, media type is 10/100BaseTX
Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 5756
5 minute input rate 55343353 bits/sec, 9677 packets/sec
5 minute output rate 55456293 bits/sec, 9878 packets/sec

Burst traffic is very difficult to capture and solve J so span required.

Configure SPAN on the switch in order to capture transmitted (TX) traffic. In order to capture this traffic, connect a PC that runs Wireshark and capture packets at the SPAN destination port.

Switch#config t
Switch(conf)#monitor session 1 source interface fa1/1 tx
Switch(conf)#monitor session 1 destination interface fa1/2

At the default scale, it appears that there is no bursty traffic. However, one second is a very large interval when you consider the rate at which buffering and packet switching happens. In a period of one second, a100 Mb/s link can accommodate 100 Mb of traffic across the interface in a neatly-shaped profile with a minimum need to buffer any packet.

 

However, if a major portion of this traffic attempts to leave the interface in a fraction of a second, the switch needs to extensively buffer packets and drop them when the buffers are full. If you make the scales more granular, you see a more accurate picture of the actual traffic profile. Change the Y axis to bits/tick because interfaces show output rates in bits/sec.

Link speed is 100 Mb/s
= 100,000,000 bits/s
= 100,000 bits/0.001 s

Recalculate the scales on the X and Y axes. Change the tick interval to X Axis=0.001 sec and the scale to Y axis=00,000 (bits/tick).

 

Scroll through the graph in order to identify bursts. In this example, you can see that there is a burst of traffic that exceeded 100,000 bits on a 0.001 second scale. This confirms that traffic is bursty at the subsecond level and is expected to get dropped by the switch when the buffers are full in order to accommodate these bursts.

Click on the traffic spike on the graph in order to view that packet in the Wireshark capture. The capture analysis is a useful way to discover what traffic constitutes the burst.

After this you need to analyze the traffic which is coming as burst.