This chapter describes the tools, methods, and information sources available for troubleshooting Sun GlassFish Enterprise Server v3 Prelude. Guidelines for evaluating and investigating a problem are included.
This chapter contains the following sections:
Application servers are typically deployed in complex and sophisticated operating environments and involve many technologies, products, and tools. Understanding and diagnosing problems in a large piece of software with so many components performing so many tasks can be challenging. This section gets you started, and includes the following topics:
Sometimes the most obvious solutions are overlooked. As you begin your investigation, try the following steps first.
Problems are often caused by configuration issues (system, JVMTM, server, or application-level) and by resource issues (such as when a system runs out of memory). Refer to the Sun GlassFish Enterprise Server v3 Prelude Release Notes for the latest information regarding system requirements dependencies.
Your problem might be related to a known issue for the release. Refer to the Sun GlassFish Enterprise Server v3 Prelude Release Notes for the latest information regarding known issues and possible workarounds. Also search the GlassFish issue tracker at https://glassfish.dev.java.net/servlets/ProjectIssues.
Enterprise Server includes complete documentation. Search the documentation to see if your problem is addressed. The documentation set is available at http://docs.sun.com/coll/1343.7. See Enterprise Server Documentation Set for a list of books in the documentation set and a description of their contents.
Lists and forums are extremely helpful resources, and are accessed as follows:
GlassFish mailing lists (start with email@example.com): http://glassfish.dev.java.net/servlets/ProjectMailingListList
GlassFish user forum: http://forums.java.net/jive/forum.jspa?forumID=56
Other GlassFish forums: http://forums.java.net/jive/category.jspa?categoryID=58
As the name suggests, the release is a prelude to Sun GlassFish Enterprise Server v3. It is not a full Java EE 5 application server, and not everything that was implemented in Sun Java System Application Server 9.1 (GlassFish v2) is implemented and functional in Enterprise Server v3 Prelude. You might be trying to do something that is not possible with the existing feature set. For more information about Enterprise Server v3 Prelude and the features it contains, consult the following resources:
Feature list and comparison: https://glassfish.dev.java.net/public/comparing_v2_and_v3.html
Download page: https://glassfish.dev.java.net/downloads/v3-prelude.html
Some resources refer to GlassFish v3 Prelude. GlassFish v3 Prelude is the community version of the commercially supported offering, Sun GlassFish Enterprise Server v3 Prelude. The software is the same; the difference is in support.
SunSolveSM is Sun's informational and patch database service with a browseable directory of all publicly available Sun Microsystems support documents. SunSolve is available at http://sunsolve.sun.com. Additional resources are also available for customers with support contracts.
Troubleshooting and analysis require information. The more information that you have, the easier it is to classify a problem and search for its solution. Detailed information will also be necessary should you need to contact others for support, either through a community mailing list or through more formal Sun support channels.
For general information about gathering data, also refer to Sun Gathering Debug Data for Sun Java System Application Server. The document pertains to previous versions but much of the information is generic enough that it might be helpful for Enterprise Server v3 Prelude.
As you continue your investigation, consider the following questions.
What do the logs show? What kind of error message are you getting? For more information about logs, see Examining Log Files.
What are you trying to do when the problem occurs, and how are you doing it? What is the sequence of events?
Does the problem occur every time you perform the same type of operation, or does it seem random? Can you reproduce the issue?
Have other events or problems occurred that could possibly be related, such as web sessions closing early, exceptions being thrown in your own application logic, components not being correctly deployed or undeployed?
What version of Enterprise Server are you using? What operating system and version? What JDK version?
What resources does your system have, such as memory, disk, and swap space? If system memory is an issue, look for ways to optimize your configuration and consider taking other actions such as adding memory or decreasing system load.
Have any patches been applied? If so, what are the product and operating system patch numbers?
What other products and technologies are installed and being used in the installation?
How many application servers, web servers, and directory servers are installed? What are the products and versions?
How is the web server connected to Enterprise Server? On the same system?
How is Enterprise Server connected to the directory server?
What JDBC driver is being used to access the database?
What are your settings?
On which port is Enterprise Server v3 Prelude configured? The default or something else?
What defaults were changed during installation and what are the values?
What other settings were changed from the defaults and what are their values?
What are the JVM heap, stack, and garbage collection-related parameters set to?
What are the JVM options?
What is the permgen size? OutOfMemoryError:PermGen space errors are common and indicate that you need to increase the permanent generation space available.
Is SSL enabled?
What are your network settings (proxy, firewall, and so on)? What happens if you disable your firewall and attempt the task?
What is different that could have provoked the problem or triggered the event? Was something new added or changed? Have any new applications been deployed? If changes have been made recently, consider backing them out and seeing what happens — does the problem still occur?
Was the feature or functionality working correctly at one time? If so, what changed or happened between then and now?
Is this working on another system? If so, what is different about that environment?
Logging is one of your most important troubleshooting tools. It is the process by which Enterprise Server captures data about events that occur during server operation. This data is recorded in a log file, and is usually the first source of information when Enterprise Server problems occur. The primary purpose of log files is to provide troubleshooting information.
By default, all Enterprise Server log information is captured in the server.log file, typically located in domain-dir/logs. Log Viewer is not available in Enterprise Server v3 Prelude. You cannot view the server's log file or configure logging using the Administration Console. To view log information, open the server.log file in a text editor. You can also use command-line file viewing commands such as tail, grep, or more. Logging is configured by editing the logging.properties file, located by default in the same directory as the domain.xml file, typically domain-dir/config.
Log levels such as SEVERE, WARNING, INFO, CONFIG, and others can be set to provide different types and amounts of information. Each Enterprise Server module has its own logger, and each logger has its own namespace. Log levels can be set globally for all loggers, or individually for module-specific loggers.
See Chapter 9, Administering Logging, in Sun GlassFish Enterprise Server v3 Prelude Administration Guide for complete details about logging. Also see the Sun GlassFish Enterprise Server v3 Prelude Release Notes for any known issues related to logging.
Monitoring is another helpful tool. It is the process of reviewing the statistics of a system to improve performance or solve problems. By monitoring the state of various components and services deployed in Enterprise Server, you can identify performance bottlenecks, predict failures, perform root cause analysis, and ensure that everything is functioning as expected. For more information about monitoring, see Chapter 10, Monitoring the Enterprise Server, in Sun GlassFish Enterprise Server v3 Prelude Administration Guide.
Several tools are available that can be used to collect information for troubleshooting purposes. This section provides basic information about the following:
The ps -ef command provides helpful information about processes that are running, including their process identification numbers (PIDs).
A stack trace is a user-friendly snapshot of the threads and monitors in a Virtual Machine for the Java platform (Java Virtual Machine or JVM machine). A thread dump shows what every thread in a JVM is doing at a given time and is useful in debugging. When the application server freezes, hangs, or becomes sluggish for no apparent reason, you should generate and analyze a thread dump.
This section explains how to obtain a thread dump for Enterprise Server. More information about analyzing the information contained in a thread dump can be found in “An Introduction to Java Stack Traces” (http://java.sun.com/developer/technicalArticles/Programming/Stacktrace). By default, the server dumps a core file and restarts with the -Xrs java-option flag in the server.xml file.
VisualVM is a Java troubleshooting tool that uses various technologies such as jvmstat, JMXTM, and Attach API to access monitored applications. VisualVM is a tool for visualizing data sources and by default visualizes the following types: applications, hosts, snapshots, core dumps, heap dumps, and thread dumps. These data sources are visualized in VisualVM so that they can be monitored for the purposes of analysis, management, and troubleshooting. VisualVM is commonly used to detect memory leaks.
VisualVM has a GlassFish plugin that enhances monitoring of hosted applications by adding specialized overview, a tab for monitoring the HTTP Service, and the ability to visually select and monitor any of the deployed web applications. You can experiment with VisualVM troubleshooting capabilities, but note that various features depend on the Java versions used in the client and server. Depending on your configuration, you might only get parts of the VisualVM features. For more information about VisualVM, see https://visualvm.dev.java.net.
Comparison between Sun Java System Application Server 9.1 (GlassFish v2) and Enterprise Server v3 Prelude: https://glassfish.dev.java.net/public/comparing_v2_and_v3.html
Support and services: http://www.sun.com/software/products/glassfishv3_prelude/support.xml
SunSolve knowledgebase: http://sunsolve.sun.com
Resources page (points to articles, blogs, videos, screencasts, FAQs, and many other resources related to Enterprise Server v3 Prelude): http://wiki.glassfish.java.net/Wiki.jsp?page=GlassFishV3PreludeResources
Mailing lists and forums: See Search the GlassFish Mailing Lists and Forums for details about accessing lists and forums.
Technical Note: Sun Gathering Debug Data for Sun Java System Application Server
GlassFish community site: http://glassfish.org
GlassFish wiki: http://wiki.glassfish.java.net/Wiki.jsp?page=PlanForGlassFishV3