Previous     Contents     Index     Next     
iPlanet Application Server Performance and Tuning Guide



Chapter 10   Frequently Asked Questions


Over the years, iPlanet™ system engineers, professional services consultants and customers have experimented with procedures to optimize the iPlanet Application Server production environment. The resulting set of thumb rules have been compiled and reproduced in this chapter.

Frequently Asked Questions (FAQ) related to peformance have been categorized in the following sections:

We welcome your feedback as we continue to refine and expand this FAQ.



Environment Setup


This section deals with issues related to setting up the iPlanet Application Server environment.

  1. How much RAM do I need per iPlanet Application Server CPU, in a typical production environment?

    You will need 1GB of RAM per installed CPU of iPlanet Application Server.

  2. How many CPUs are needed for a team of developers, in a typical developer sandbox installation on Solaris.

    A single iPlanet Application Server CPU can support 3-5 developers. If usage is intense, about 2-3 developers per CPU.

  3. How much disk space do I need to install iPlanet Application Server?

    The distribution is about 150 MB and you should count on three times that space for installation and operation. You will need about 450 MB of free disk space initially. After installation, iPlanet Application Server will need about 256 MB of disk for operation.

  4. How many processors can a single iPlanet Application Server instance be expected to utilize efficiently?

    As a general rule, each instance should run no more than 8 to 12 processors. A single iPlanet Application Server instance has only one KXS, but can have many KJS processes. If the application seems to be constrained by KJS rather than KXS, you can scale into the 12 processor range. However, if you make heavy use of a KXS-based service like distributed sesson management (DSync) and servlet result set caching, KXS will limit scalability to the 8 processor range. To scale beyond this 8 to 12-processor range, consider installing multiple iPlanet Application Server instances on the machine.

  5. iPlanet Application Server distributed session management (DSync) facility can have a high overhead. What key decision can I make at installation time to help lessen the load, distributed session management (DSync) has on the KXS process?

    Elect to have only one DSync backup. This will reduce the amount of DSync work required to keep the backup's in-memory session store synchronized with the master so that it is available to take over in the event of the primary failing. One DSync backup should be sufficient to assure ongoing session availability.

  6. What can a Java servlet/JSP programmer do to help iPlanet Application Server's session management facility to be as efficient as possible?

    Look at your use of sessions. The DSync facility, housed in the KXS process, can put quite a load on your environment. Here are some considerations:

    • Limit your cluster size to no more than 4 instances. Beyond this size, the overhead of keeping the distributed session store in synch typically becomes a performance limiter.

    • Use sticky load balancing so subsequent processing for a user during a session always returns to the same KJS where session information is available locally. This is particularly important for a stateful session.

  7. How can an administrator help KXS handle session management housekeeping chores more efficiently?

    As sessions timeout, or are invalidated, they need to be promptly removed from the in-memory session store. This removal can be done with greater concurrency by increasing the number of threads dedicated to session management. Adjust the following property in iPlanet Registry to make session node management more efficient:

SOFTWARE/iPlanet/Application Server/6.0/Clusters/<machine-name>-NoDsync/SyncTimeoutThreadCount



System Tuning


  1. What constitutes a well-tuned iPlanet Application Server system?

    A well-tuned system should show:

    • Uniform CPU time across all servers.

    • Even usage of CPU time across all processors in each server.

    • Comparable CPU time across all KJS processes.

    • A system time of (0-25%) if workflow is computationally intensive.

    • Utilization of all processors allocated to the KXS process.

  2. There is a temptation to install iPlanet Application Server a number of times on a multiple-CPU machine. Is that the best way to scale?

    Install multiple iPlanet Application Server instances on a single machine as a last resort. This usually complicates the ongoing management. Instead, try tuning a single iPlanet Application Server instance first by adding additional KJS (Java VM) processes and tweaking the number of threads available in iPlanet Application Server process thread pools.

  3. How many threads should a KXS process have?

    32 threads per KXS process is the default setting. However, you can increase the number of threads. Increasing the numbering of threads for KXS, however, does not ensure a dramatic improvement in performance.

    A well-tuned system should show:

    • Uniform CPU time across all servers

    • Even usage of CPU time across all processors in each server

    • Comparable CPU time across all KJS processes

    • A system time of 0-25%) if workflow is computationally intensive

    It is important that all processors allocated to the KXS proces pool are utilized. When compared with KJS threads, KXS threads don't perform a lot of work, so you don't need as many KXS threads as KJS threads. To increase KXS performance, you can consider binding (pbind) KXS to a process or a processor set.

  4. When should you consider binding KXS to a processor?

    There are benefits especially in a Solaris environment in binding iPlanet Application Server processes to specific processors or to a processor set. On a multi-processor machine, always bind KXS to a processor. This will significantly improve throughput due to overhead associated with using mutex locks on multiprocessors.

  5. When should you consider binding KXS to a processor set (multiple processes)?

    If you still see that KXS is queuing requests, create a a processor set containing two processors. Bind the KXS to this processor set. Note, going beyond two processors in a process set for KXS typically doesn't lead to substantial improvements in throughput.

  6. What should make you want to drop a processor from a process set bound to KXS?

    If KXS is not fulling utilizing the processor set and you see that there are KJS processes that are starved for power (threads are queuing), take away a processor from KXS and make it available for a new KJS process that you can add to your iPlanet Application Server configuration.

  7. How many threads should a KJS have?

    Try starting at 32 and increasing from there as processor load gets high. 48 seems to be about as large a thread pool as you will want to set for KJS. Going much beyond that will simply increase context switching resulting in wasted cycles.

  8. When should you consider binding KJS to a processor?

    Generally, you shouldn't bind KJS to processors or processor sets since doing so has shown to provide no more than a 5% performance increase. In JDK 1.2.2 and and beyond, the VM is already optimized to work well in multi-processor environments.

  9. The KJS process is a "home" for the Java VM. How can I tailor things like heap size for the JVM that KJS hosts?

    Arguments that can be supplied to the Solaris JDK may be set via the JAVA_ARGS shell variable in the iasenv.ksh shell script. Settings with JVM flags, particularly -Xms and -Xmx flags, specify the starting and maximum heap size used by each KJS engine. Heap size decisions should be based on how much memory is available on the system. Set these as large as possible without starving other applications running on the same server. The default starting heap size is 8MB.

    The heap grows automatically as needed. Starting with a large heap size avoids frequent garbage collection during growth. To cap the heap growth size, use the -Xmx flag. The JDK documentation provides more information on these and other flags.

  10. Your application seems to be behaving badly because it takes a long time between hitting the submit button and seeing the result in my browser. How can you troubleshoot where the slowdowns may be occurring?

    Think about using "clocks" at key points in the request processing cycle:

    • Clock 1 timestamps at the client (browser or load generation tool like LoadRunner).

    • Clock 2 timestamps on the iWS web server at the point where a front-end thread receives a request.

    • Clock 3 timestamps on the iWS web server at the point where the back-end worker thread gets the request.

    • Clock 4 timestamps when the iPlanet Application Server web connector actually responds to the request from the back-end worker thread.

    • Clock 5 timestamps when a request is received by KXS to be sent to a KJS for processing.

    • Clock 6 timestamps when KXS receives the request back from KJS.

  11. If there is a long delay between Clock 1 and Clock 2, what might you suspect?

    The delay could be because of one of the following reasons:

    • Network congestion.

    • CPU or NIC too busy on the client side.

    • TCP stack queuing is occurring at the web server.

    • Requests may be queuing at an intermediary, such as a load balancer or a firewall.

    • Misconfiguration of your load generation client.

  12. What if you see a delay of greater than two seconds between Clock 2 and Clock 3?

    It probably means that all the back-end worker threads are busy and requests are being queued. You can either increase the number of back-end threads or decrease the number of front-end threads.

  13. If there is a delay of greater than 30 milliseconds between Clock 3 and Clock 4, what will you have learned?

    Well, not much from a corrective action point of view. This indicates that the web connector isn't performing as fast as anticipated. Unfortunately, there aren't any tuning opportunities here.

  14. What if you see elapsed times of greater thand 3 seconds between Clock 4 and 5?

    Suspect network buffering at the web server NIC or queuing at the iPlanet Application Server NIC. You might also check for an under-resourced firewall. Also, general network congestion can be the problem.

  15. What if the Clock 5 and Clock 6 timestamps reveal slower turnaround than anticipated?

    It tells you that the KXS and/or KJS processes need some tuning. Check the thread pools for KXS and KJS. Also, consider adding additional KJS processes. Beyond that, you need to consider binding iPlanet Application Server processes to processors and maybe even to processor sets.



Application Tuning

  1. Is their some tweaking you can do as an EJB programmer to help performance of iPlanet Application Server EJB container?

    • The default passivation timeout is 60 seconds. Increase this value if the bean instance creation rate is very low and the bean size is large. You may see a slight boost in performance as passivation processing is reduced.

    • Make Meta Data Cache Size as large as the number of different types of beans that exist in your application. The default, 30, may not be enough to cache all the home handles.

    • Set Implementation Cache Size based upon the number of concurrent user sessions you expect. For example, if each 200 concurrent user sessions will have a need for one stateful session EJB, set the Implementation Cache Size to at least 200.

  2. What can I check in my Java code that might help me make iPlanet Application Server perform better?

    iPlanet Application Server KJS processes provide the J2EE containers inside which your Java code runs. The KJS processes host independent Java VMs. Therefore, you need to follow good Java guidelines to help the VMs perform as well as possible. Here are some things to check:

    • Avoid serializing/deserializing. These are expensive operations.

    • Avoid using lots of arrays since there is overhead to do what Java does for arrays, namely - initializing them for you and preventing you from accessing out of range.

    • Explicitly dereference variables by setting them to null to make garbage collection more efficient

    • Don't use class variables (static members) in servlet classes since they impose synchronization in the server. By default, all users share a single copy of the servlet code per web container (JVM).

    • Avoid using sychronized methods or synchronized blocks in your code.

    • Carefully consider EJB use. EJBs put a load on a server.

  3. For performance reasons, you should use EJBs judiciously in your application; so, what's wrong with EJBs?

    EJBs are wonderful. After all, they are the component model for server-side Java. However, all good things come with a price. This is true in any vendor's EJB server. There is quite a lot of overhead required to provide access to EJBs and to host the services that EJB containers provide, such as, security and transaction management. Here are some thoughts on minimizing performance degradation when using EJBs:

    • Cache EJB references at the servlet so you won't need to do a JNDI lookup for every request.

    • Use sticky load balancing so subsequent requests during a session always return to the same KJS (VM) for processing. The EJB resource can be local there and expensive calls across machines to get EJB access can be avoided.

    • Use stateless session EJBs as they are comparable to servlets in performance.

    • Stateful session and entity beans are much more expensive than stateless session EJB beans. Entity EJBs are most performance intensive with BMP being more expensive than CMP for persistence management of entity EJBs.

  4. How much can servlet HTML result caching help the performance of your application?

    Servlet caching can almost double performance for a 1-CPU installation, but tests have shown only 4% enhancement on a 2-CPU iPlanet Application Server instance. At 4 CPUs and beyond, the cache doesnt scale as well and we don't recommend that you use servlet result caching. Use JSP result caching instead.

    JSP result caching is KJS-based rather than KXS-based. You can have multiple KJS processes per iPlanet Application Server instance so this approach scales better.

  5. How can you make servlet HTML result caching most effective?

    You can enhance the possibility for cache hits by increasing the amount of memory available for storing the cache. This is done in the deployment descriptor for servlet caching and in the Administration Tool for JSPs.

    HTML result caching is handled by KXS for servlets, but that's one more thing to ask a probably already overworked KXS to do. HTML result caching can really enhance throughput.

  6. *******? and I want to use it.

    Use JSP result caching instead. This cache is handled by KJS. Unlike KXS processes, you can have multiple KJS process per iPlanet Application Server instance. So, you can offload KXS and bring to bear more result caching power by going the JSP result caching route.


Previous     Contents     Index     Next     
Copyright © 2002 Sun Microsystems, Inc. All rights reserved.

Last Updated March 06, 2002