Troubleshooting Slow Performance: Using 'netstat' and 'ss' Effectively
Master the essential Linux networking tools `netstat` and `ss` for efficient performance troubleshooting. This guide compares the legacy `netstat` with the modern, faster `ss` utility, providing practical command examples. Learn how to filter results by connection state, identify listening services, and diagnose network bottlenecks quickly using Netlink socket statistics.
Troubleshooting Slow Performance: Using 'netstat' and 'ss' Effectively
When a Linux service feels slow, the socket table is one of the fastest places to separate "the app is overloaded" from "the network path is messy." A web server that cannot accept new connections, a worker that is stuck opening database sessions, and a host with a pile of half-open TCP handshakes all look like vague slowness to the person waiting on the other end.
netstat and ss help you answer a narrower question: what network sockets exist on this machine right now, what state are they in, and which process owns them? netstat is still useful on older systems and in old runbooks. ss is the tool I reach for first on modern Linux because it is faster on busy hosts and has better built-in filtering.
Why Monitor Network Sockets?
Network latency and sluggishness are frequently tied to connection issues rather than CPU or memory exhaustion. Monitoring sockets helps administrators answer critical questions such as:
- Which ports are actively listening for connections?
- Are there too many connections stuck in the
SYN_RECVorTIME_WAITstates? - Which process (PID) is using a specific port?
- Are there unexpected outbound connections occurring?
By examining socket statistics, you can quickly rule out network configuration problems or identify resource contention related to connection handling.
The Legacy Tool: netstat
netstat has been the standard utility for displaying network connections, routing tables, interface statistics, and masquerade connections for decades. While deprecated in favor of ss on many modern systems, it remains widely available and often familiar to long-time administrators.
Common netstat Examples
The most common flags used with netstat provide a comprehensive overview:
| Flag | Description |
|---|---|
-a |
Shows all sockets (listening and non-listening) |
-n |
Shows numerical addresses instead of trying to resolve hostnames and service names (speeds up output) |
-t |
Shows TCP connections |
-u |
Shows UDP connections |
-l |
Shows only listening sockets |
-p |
Shows the PID/Program name associated with the socket (requires root privileges) |
Example: Viewing all active TCP connections numerically
sudo netstat -ant
Example: Finding what is listening on port 80 (HTTP)
sudo netstat -tulpen | grep ':80'
Understanding Connection States (netstat)
The output of netstat often includes a State column. Key states to watch for include:
- LISTEN: Waiting for incoming connections.
- ESTABLISHED: An active, open connection.
- TIME_WAIT: A socket waiting for a short period after closing to ensure delayed packets are handled.
- SYN_RECV: Waiting for the final acknowledgement of a three-way handshake (can indicate a SYN flood attack if excessive).
Warning on
netstat:netstatoften relies on parsing/proc/net/*files, which can be slow, especially on systems with a very high volume of active connections (thousands). This is the primary reasonsswas developed.
The Modern Replacement: ss (Socket Statistics)
The ss utility is significantly faster than netstat because it retrieves socket information directly from the kernel space using Netlink sockets, bypassing slower file system lookups.
Common ss Examples
The flag structure for ss is very similar to netstat, promoting easy transition:
| Flag | Description |
|---|---|
-a |
Shows all sockets |
-n |
Shows numerical addresses |
-t |
Shows TCP sockets |
-u |
Shows UDP sockets |
-l |
Shows listening sockets |
-p |
Shows process information (PID/Program) |
Example: Viewing all active TCP connections numerically (Equivalent to netstat -ant)
ss -ant
Example: Finding what is listening on port 443 (HTTPS)
sudo ss -tulpen | grep ':443'
Advanced Filtering with ss
One of the biggest advantages of ss is its ability to perform direct filtering on connection states, which is much more efficient than piping netstat output to grep.
Filtering by Connection State
You can use the state option directly within the ss command. This is extremely useful for diagnosing connection buildup.
Finding all sockets currently in the TIME-WAIT state:
ss -tan state time-wait
Finding all sockets in the SYN-SENT state (client side waiting for server response):
ss -tan state syn-sent
Filtering by Port or Address
Filtering by destination or source address/port is straightforward:
Show established connections destined for port 22 (SSH):
ss -tn state established '( dport = :22 or sport = :22 )'
Show connections related to a specific local IP address:
ss -ant '( daddr = 192.168.1.100 or saddr = 192.168.1.100 )'
Performance Analysis: netstat vs. ss Comparison
When troubleshooting, the choice between the tools often comes down to speed and detail.
| Feature | netstat |
ss |
|---|---|---|
| Speed | Slower (Reads files) | Much Faster (Uses Netlink sockets) |
| Syntax | Mature, highly documented | Similar flags, newer specific options |
| Filtering | Requires piping to grep |
Native state and address filtering support |
| Information Depth | Good for basics | More detail on socket buffer sizes (TCP Info) |
| Availability | Nearly universal | Standard on modern Linux distributions |
Diagnosing Slow Connection Establishment
If clients report slow connections, check for sockets stuck waiting for handshakes. Using ss is the fastest way to determine this:
- Check for high
SYN-RECVcounts: This suggests the server is receiving connection requests but not completing the handshake, often due to resource exhaustion or high traffic load.ss -s | grep syn-rec - Check for high
SYN-SENTcounts: If the server itself is initiating many connections (e.g., acting as a client to databases or other APIs), this shows it is waiting for responses.ss -s | grep syn-sent
If you see sustained high numbers in either category, treat them as a lead rather than a verdict. SYN-SENT can mean a remote host is down, a route is wrong, a firewall is silently dropping traffic, or the remote service is overloaded. SYN-RECV can mean the server is under load, packets are being lost, or clients are opening connections and not completing them.
A Practical Triage Flow
When someone says "the app is slow," I usually start with a short, repeatable pass:
sudo ss -tulpen
ss -s
sudo ss -tan state established '( sport = :443 or dport = :443 )' | head
sudo ss -tan state syn-recv
sudo ss -tan state time-wait | head
The first command confirms that the expected service is actually listening and shows the owning process. The summary shows whether the host has a surprising number of TCP sockets. The filtered established command proves whether real client traffic is attached to the port. The syn-recv and time-wait checks show whether connection setup or connection churn deserves attention.
For example, imagine an Nginx reverse proxy where users complain that new requests hang for a few seconds. sudo ss -tulpen | grep ':443' confirms Nginx owns the HTTPS listener. ss -s shows a large TCP total, and sudo ss -tan state syn-recv '( sport = :443 )' keeps returning rows from the same source ranges. That does not automatically prove an attack, but it tells you to look at load balancer health checks, upstream packet loss, SYN backlog pressure, firewall logs, and possibly rate limits.
Now imagine the same proxy has very few SYN_RECV sockets but many established connections to an upstream database on port 5432. That points you away from public HTTPS and toward the database path:
sudo ss -tanp '( dport = :5432 or sport = :5432 )'
If the owning process is your application and the count keeps climbing, the next useful question is whether the application is leaking connections, waiting on slow queries, or failing to return connections to a pool. ss does not answer that application-level question, but it gets you to the right room.
Reading TIME_WAIT Without Panicking
TIME_WAIT is a normal TCP state, not an error by itself. A server that handles lots of short-lived connections will naturally show TIME_WAIT sockets. They exist so delayed packets from an old connection do not get confused with a new one.
The useful question is whether TIME_WAIT matches the workload. A batch job that opens a fresh HTTP connection for every small request may create a wave of TIME_WAIT. A service that should use keep-alive but does not may do the same. Before tuning kernel settings, check whether the application can reuse connections, enable HTTP keep-alive, or use a proper client pool.
Be careful with old advice that suggests blindly changing TCP sysctls to "fix" TIME_WAIT. Some settings are kernel-version dependent, some have been removed or discouraged over time, and some create subtle failures behind NAT or load balancers. Start by understanding why the connections are short-lived.
Checking Local Versus Remote Pressure
One detail that saves time is whether the local host is mostly accepting connections or mostly making them. A frontend proxy usually has many connections where the local port is 80 or 443. An application server that talks to databases and APIs may have many connections where the remote port is 5432, 3306, 6379, or 443.
For local listeners and inbound traffic:
sudo ss -tan '( sport = :443 )'
For outbound traffic to a dependency:
sudo ss -tan '( dport = :6379 )'
That distinction changes the next conversation. If inbound HTTPS is piling up, you may need to inspect the load balancer, TLS termination, worker limits, or client behavior. If outbound Redis connections are piling up, the local application may be creating too many client connections, waiting on Redis, or retrying too aggressively.
When you need a quick count without reading hundreds of rows, combine ss with simple shell tools:
sudo ss -tan state established '( dport = :443 )' | wc -l
sudo ss -tan state established '( dport = :5432 )' | wc -l
The count includes the header line, so it is not a perfect metric. For triage, it is still useful. If the number doubles every minute during an incident, you have a stronger signal than a single snapshot.
Containers and Network Namespaces
On containerized hosts, be careful about where you run the command. Running ss on the host shows host network namespaces and published ports, but it may not show the same view the process sees inside its container. If a service runs in a container, compare both views:
sudo ss -tulpen
docker exec <container> ss -tulpen
For Kubernetes, use the node view for host-level listeners and kubectl exec for the pod's network namespace. A port can be open inside the container while the host, service, ingress, or network policy still prevents traffic from reaching it. ss is a local truth tool, not an end-to-end connectivity test.
Best Practices for Network Troubleshooting
- Always Use
-n: When troubleshooting performance or scripting, use the numerical flag (-n) to avoid DNS resolution delays, which can make diagnostics sluggish. - Prioritize
ss: Adoptssas your default tool. Reservenetstatonly for legacy systems wheressis unavailable. - Run as Root for PID: To see which program is using a port, you generally need
sudoor root privileges when using the-pflag with both utilities. - Check Interface Stats: Don't forget interface counters. Use
ip -s link show <interface_name>to check for dropped packets or errors, which might indicate a physical layer issue rather than a socket issue. - Compare snapshots. One
ssoutput is a photograph. Two outputs taken a minute apart tell you whether the situation is growing, shrinking, or stable. - Write down the exact filter. During incidents, a saved command like
ss -tan '( dport = :5432 )'is easier to repeat and compare than a half-remembered grep pipeline.
The habit that pays off is simple: start with listeners, move to connection states, identify the owning process, then decide whether the next step belongs in the app, the network path, the firewall, or the kernel.