An overburdened CPU is another obvious place to look for performance problems
on your system. The vmstat command, shown earlier, can produce basic statistics
relating to CPU usage (user activity, system activity, idle time, I/O wait time, and
time stolen from a virtual machine). The iostat command (from the sysstat package),
however, can generate more detailed reports of CPU utilization.
Here are two examples of using iostat to display a CPU utilization report:
$ iostat -c 3 CPU stats every 3 seconds (starting apps)
Linux 2.6.21-1.3194.fc7 (davinci) 08/10/2007
avg-cpu: %user %nice %system %iowait %steal %idle
0.50 0.00 0.00 0.00 0.00 99.50
avg-cpu: %user %nice %system %iowait %steal %idle
28.71 0.00 5.45 18.32 0.00 47.52
avg-cpu: %user %nice %system %iowait %steal %idle
98.99 0.00 1.01 0.00 0.00 0.00
avg-cpu: %user %nice %system %iowait %steal %idle
99.50 0.00 0.50 0.00 0.00 0.00
$ iostat -c 3 CPU stats every 3 seconds (copying files)
Linux 2.6.21-1.3194.fc7 (davinci) 08/10/2007
avg-cpu: %user %nice %system %iowait %steal %idle
0.50 0.00 0.00 0.00 0.00 0.00
avg-cpu: %user %nice %system %iowait %steal %idle
0.50 0.00 24.88 74.63 0.00 0.00
avg-cpu: %user %nice %system %iowait %steal %idle
0.50 0.00 10.00 89.50 0.00 0.00
avg-cpu: %user %nice %system %iowait %steal %idle
0.50 0.00 17.41 82.09 0.00 0.00
avg-cpu: %user %nice %system %iowait %steal %idle
0.00 0.00 14.65 85.35 0.00 0.00
The first iostat example above starts with a quiet system, then several applications
started up. You can see that most of the processing to start the applications is being done in user space. The second iostat example shows a case where several large
files are copied from one hard disk to another. The result is a high percentage of time being spent at the system level, also known as kernel space (in this case, reading from and writing to disk partitions). Note that the file copies also result in a higher amount of time waiting for I/O requests to complete (%iowait).
Here are examples using iostat to print CPU utilization reports with timestamps:
$ iostat -c -t Print time stamp with CPU report
Linux 2.6.21-1.3194.fc7 (davinci) 08/10/2007
Time: 9:28:03 AM
avg-cpu: %user %nice %system %iowait %steal %idle
0.50 0.00 0.00 0.00 0.00 99.50
$ iostat -c -t 2 10 Repeat every 2 seconds for 10 times
The dstat command (dstat package) is available as an alternative to iostat for viewing information about your CPU usage (as well as other performance-related items). One advantage of dstat over other tools is that it more precisely shows the units of measurement it is displaying (such as kilobytes or megabytes) and also uses colors to differentiate the data. Here is an example of dstat for displaying CPU information:
$ dstat -t -c 3 View CPU usage continuously with time stamps
---time--- ----total-cpu-usage----
__epoch___|usr sys idl wai hiq siq
1189727284| 0 0 100 0 0 0
1189727287| 1 0 99 0 0 0
1189727290| 3 0 97 0 0 0
1189727293| 0 0 100 0 0 0
1189727296| 5 0 95 0 0 0
1189727299| 1 0 99 0 0 0
1189727302| 3 0 97 0 0 0
1189727305| 0 0 100 0 0 0
1189727308| 3 0 96 0 1 0
1189727311| 1 0 99 0 0 0
1189727314| 0 0 100 0 0 0
1189727317| 0 0 100 0 0 0
1189727320| 1 0 99 0 0 0
1189727323| 5 0 95 0 0 0
1189727326| 3 0 97 0 0 0
1189727329| 3 0 97 0 0 0
1189727332| 2 0 98 0 0 0
1189727335| 5 0 95 0 0 0
In this example, the output includes a date/time values based on the start of the
epoch (-t) for the CPU report (-c) that is produced every three seconds (3). This
report runs continuously until you stop it (Ctrl+c).