Chapter 3. Analyzing performance bottlenecks 81
Draft Document for Review May 4, 2007 11:35 am 4285ch03.fm
3.2 CPU bottlenecks
For servers whose primary role is that of an application or database server, the CPU is a
critical resource and can often be a source of performance bottlenecks. It is important to note
that high CPU utilization does not always mean that a CPU is busy doing work; it may, in fact,
be waiting on another subsystem. When performing proper analysis, it is very important that
you look at the system as a whole and at all subsystems because there may be a cascade
effect within the subsystems.

3.2.1 Finding CPU bottlenecks

Determining bottlenecks with the CPU can be accomplished in several ways. As discussed in
Chapter 2, “Monitoring and benchmark tools” on page39, Linux has a variety of tools to help
determine this; the question is: which tools to use?
One such tool is uptime. By analyzing the output from uptime, we can get a rough idea of
what has been happening in the system for the past 15 minutes. For a more detailed
explanation of this tool, see 2.3.3, “uptime” on page 43.
Example 3-1 uptime output from a CPU strapped system
18:03:16 up 1 day, 2:46, 6 users, load average: 182.53, 92.02, 37.95
Using KDE System Guard and the CPU sensors lets you view the current CPU workload.
Using top, you can see both CPU utilization and what processes are the biggest contributors
to the problem (Example2-1 on page 41). If you have set up sar, you are collecting a lot of
information, some of which is CPU utilization, over a period of time. Analyzing this information
can be difficult, so use isag, which can use sar output to plot a graph. Otherwise, you may
wish to parse the information through a script and use a spreadsheet to plot it to see any
trends in CPU utilization. You can also use sar from the command line by issuing sar -u or
sar -U processornumber. To gain a broader perspective of the system and current utilization
of more than just the CPU subsystem, a good tool is vmstat (2.3.2, “vmstat” on page 42).

3.2.2 SMP

SMP-based systems can present their own set of interesting problems that can be difficult to
detect. In an SMP environment, there is the concept of CPU affinity, which implies that you
bind a process to a CPU.
The main reason this is useful is CPU cache optimization, which is achieved by keeping the
same process on one CPU rather than moving between processors. When a process moves
between CPUs, the cache of the new CPU must be flushed. Therefore, a process that moves
between processors causes many cache flushes to occur, which means that an individual
process will take longer to finish. This scenario is very hard to detect because, when
Note: There is a common misconception that the CPU is the most important part of the
server. This is not always the case, and servers are often overconfigured with CPU and
underconfigured with disks, memory, and network subsystems. Only specific applications
that are truly CPU-intensive can take advantage of today’s high-end processors.
Tip: Be careful not to add to CPU problems by running too many tools at one time. You
may find that using a lot of different monitoring tools at one time may be contributing to the
high CPU load.