Sun Java System Web Server's responsiveness depends greatly on the performance of the disk subsystem. Use the iostat utility to monitor how busy the disks are and how rapidly they complete I/O requests (the %b and svc_t columns, respectively). Service times are unimportant for disks that are less than about 30% busy, but for busier disks service times should not exceed about 20 milliseconds. If your busy disks have slower service times, improving disk performance may help Sun Java System Web Server performance substantially.
Your first step should be to balance the load: if some disks are busy while others are lightly loaded, move some files off of the busy disks and onto the idle disks. If there is an imbalance, correcting it will usually give a far greater payoff than trying to tune the overloaded disks.