If your sites have low CPU utilization when achieving maximum throughput, the bottleneck is likely either:
database limited (if database output is maxed out); see Checking for Database Bottlenecks
disk I/O limited (if I/O output is maxed out); see Checking for Disk I/O Bottlenecks
network I/O limited (if I/O output is maxed out); see Checking for Network-Limited Problems
database or I/O activity in a synchronized method (if database or I/O output is not maxed out); see System Resource Bottlenecks
If your site is in this situation, CPU profiling tools are not that useful. Thread dumps taken while the system is under load can give you better information. If you take a few of these, you can get a quick idea of which parts of your application are the slowest. That may help you direct your efforts to the right part of your application. You should be able to tell, for example, whether threads are waiting for a response from the database, a write to the client, or a read from a local file system. If many threads are waiting for the same resource, this is an indication of a potential bottleneck on that resource. Here is some information on what to do about resource bottlenecks for various resources:
