Network management and debugging

Netstat: Get tons o' network statistics

netstat provides a wealth of information about the state of your computer's networking software, including interface statistics, routing information, and connection tables. There isn't really a unifying theme to the different sets of output, except for the fact that they all relate to the network. Every system provides netstat, but since the command is kind of a -kitchen sink," different systems understand somewhat different options.

Here, we discuss the four most common uses of netstat:

• Monitoring the status of network connections
• Inspecting interface configuration information
• Examining the routing table
• Getting operational statistics for various network protocols.

Monitoring the status of network connections
With no arguments, netstat displays the status of active TCP and UDP ports. Inactive (-listening") servers waiting for connections aren't normally shown; they can be seen with netstat -a. (Note: Connections for -UNIX domain sockets" are also shown, but since they aren't related to networking, we do not discuss them here.) The output looks like this:

% netstat -a
Active Internet connections (including servers)

The preceding example was run on the host nimi. It shows several inbound SSH connections, an outbound telnet connection, and a bunch of ports listening for other connections. Also of note are the lines showing the protocol as tcp46, which are services running on IPv6.

Addresses are shown as hostname.service, where the service is a port number. For well-known services, netstat shows the port symbolically, using the mapping defined in /etc/services. You can obtain numeric addresses with the -n option. Remember, if your DNS is broken, netstat will be painful to use without the -n flag.

Send-Q and Recv-Q show the sizes of the send and receive queues for the connection on the local host; the queue sizes on the other end of a TCP connection might be different. They should tend toward 0 and at least not be consistently nonzero. Of course, if you are running netstat over a network terminal, the send queue for your connection will probably never be 0.

The connection state has meaning only for TCP; UDP is a connectionless protocol. The most common states you'll see are ESTABLISHED for currently active connections, LISTEN for servers waiting for connections (not normally shown without -a), and TIME_WAIT for connections in the process of closing.

This display is primarily useful for debugging higher-level problems once you have determined that basic networking facilities are working correctly. It lets you verify that servers are set up correctly and facilitates the diagnosis of certain types of miscommunication, particularly with TCP. For example, a connection that stays in state SYN_SENT identifies a process that is trying to contact a nonexistent or inaccessible network server.

If netstat shows a lot of connections in the SYN_WAIT condition, your host is probably unable to handle the number of connections being requested. This inadequacy may be due to kernel tuning limitations or even to malicious flooding.

Inspecting interface configuration information
netstat -i shows the status of network interfaces. For example, here is output from netstat -i on the Solaris machine evolve:

This host has two network interfaces: one for regular traffic and a -backlan" connection called evolve-bl. Ipkts and Opkts report the number of packets that have been received and transmitted on each interface since the machine was booted. Ierrs and Oerrs show the number of input and output errors; many different types of errors are counted in these buckets, and it is normal for a few to show up.

Errors should be less than 1 percent of the associated packets. If your error rate is high, compare the rates of several neighboring machines. A large number of errors on a single machine suggests a problem with that machine's interface or connection. An error rate that is high everywhere most likely indicates a media problem.

Collisions indicate a loaded network; errors often indicate cabling problems. Although a collision is a type of error, it is counted separately by netstat. The Collis column gives the number of collisions that were experienced while packets were being sent. (Note: This field has meaning only on broadcast-based networks such as Ethernet.) Use this number to calculate the percentage of output packets (Opkts) that result in collisions. In the example above, the collision rate on interface hme0 is about 0.6percent and the collision rate on interface hme1 is 1.3percent. On a properly functioning network, collisions should be less than 5percent of output packets, and anything over 15percent indicates serious congestion problems.

netstat can also monitor a specific interface in real time, although the flags to request this behavior are different on each version of UNIX. The following commands give interface statistics at one-second intervals. The output that is shown is adapted from a FreeBSD system.

solaris% netstat -i 1
hp-ux% netstat 1
redhat% netstat -i -c
freebsd% netstat 1

In this example, the collision rate is running at 20percent-30 percent. The network is probably very slow and possibly even unusable.

netstat's continuous mode is especially useful for tracking down the source of errors. netstat -i can alert you to the existence of problems, but it can't tell you whether the errors came from a continuous, low-level problem or from a brief but catastrophic event. Observing the network over time under a variety of load conditions will give you a much better impression of what's going on. Try running ping with a large ping packet size while you watch the output of netstat.

Examining the routing table
netstat -r displays the kernel's routing table. Here is a sample from a Solaris machine with two network interfaces:

Destinations and gateways can be displayed either as hostnames or as IP addresses; the -n flag requests numeric output.

The Flags characterize the route: U means up (active), G is a gateway, and H is a host route. The D flag (not shown) indicates a route resulting from an ICMP redirect. G and H together indicate a host route that passes through an intermediate gateway. The remaining fields give statistics on the route: the current number of TCP connections using the route, the number of packets sent, and the interface used. Remember that this output varies slightly among operating systems.

Use this form of netstat to check on the health of your machine's routing table. It's particularly important to verify that the system has a default route and that it is correct. On some systems, the default route is represented by an all-0 destination address (0.0.0.0); on others, the word -default" appears instead.

Viewing operational statistics for various network protocols
netstat -s dumps the contents of counters that are scattered throughout the network code. The output has separate sections for IP, ICMP, TCP, and UDP. Below are pieces of netstat -s output from a gateway machine; they have been edited to show only the tastiest pieces of information.

ip:
2313683 total packets received
0 bad header checksums
1642600 packets for this host
25743 packets sent from this host
0 output packets dropped due to no bufs, etc.

The absence of checksum errors indicates a clean hardware connection. It is important to check that packets are not getting dropped because of lack of memory (bufs in this example, but often referred to as -mbufs"). (Note: To get more details about memory usage by network services on Solaris and FreeBSD, try using the -m flag with netstat.)

icmp:
57 calls to icmp_error
Output histogram:
echo reply: 157
destination unreachable: 57
Input histogram:
echo reply: 6
destination unreachable: 4
echo: 157
time exceeded: 14
157 message responses generated

The number of echo requests, responses generated, and echo replies all match. Note that -destination unreachable" messages can still be generated even when all packets are apparently forwardable. Bad packets can eventually reach a gateway that rejects them, and error messages are then sent back along the gateway chain.

tcp:
25087 packets sent
25499 packets received
31 connection requests
30 connection accepts
56 connections established (including accepts)
64 connections closed (including 13 drops)
4 embryonic connections dropped

It's a good idea to develop a feel for the normal ranges of these statistics so that you can recognise pathological states.

• Related stories

• Alerts

Be the first to comment on this article!

• Just In

• Most Popular

Reader Services

Popular Topics

Essentials