As CAMM collects measurements, it processes incoming data and stores resulting information in the CAMM embedded database. The CAMM performance analytics feature then queries against this database to create statistical models and perform mathematical calculations.
This chapter includes the following performance analytics features:
For Portal metrics, you can use CAMM Entity Performance Ranking to accelerate bottleneck isolation.
Entity Performance Ranking becomes available when a node contains multiple children entities. For example, if the Labels node has multiple children entities, clicking on the Analysis tab of the node displays the Entity Performance Ranking.
CAMM provides Entity Performance Ranking on Nodes with Children Entities.
Performance Characterization is a set of performance analytics provided by CAMM. Use the Performance Characterization analytics to visualize actual performance of a monitored entity in a given time frame. The following is a list of Performance Characterization analytics provided by CAMM:
A statistically significant amount of data is required before these graphs will appear. You will need at least 1 hour of continuos load for any graphs to appear and at least 5 hours of continuos load for all of the graphs to appear. Analysis requires that load metrics actually be present and zero points do not count for the purposes of this analysis. It may be helpful to increase your time frame manually until you encompass enough data points.
CAMM automatically fits a multi-point line through the data set of performance and load measurements collected during a specific time frame. This feature allows you to quickly characterize the performance of an entity for a certain load level in real time.
CAMM performs multi-point regression and fits a 3-point line through the data set.
As a performance analytical tool, the Multi-Point Performance/Load Regression is extremely useful. It enables you to characterize the performance of a system or a component for a certain load range. You can use this analysis to capture the performance baseline of a system with a specific infrastructure configuration such as deployed applications, application server configuration, network topology and hardware capacity and load their characteristics. This analysis can also be used for capacity planning. Since the regression illustrates the actual application performance for a specific configuration, this information is useful to determine how to expand computing capacity to best meet estimated demand.
This feature automatically plots the interceptions between arrival rate and response time for a specific monitored entity into a scattergram. This feature gives you a visual interpretation of actual performance (response time) under a range of loads (arrival rate) in real time.
CAMM plots intersections of arrivals and response time metrics in a scattergram.
This feature automatically plots the performance (response time) of a specific monitored entity during a time frame specified by you. The X axis is response time and Y axis is time on the clock. This plot gives you a visual representation of a monitored entity's performance over time.
Time-Based Performance Distribution can help you identify abnormal performance patterns.
This feature automatically creates a histogram with ten bins (performance ranges) and plots the occurrences of response time that falls in each bin. The X axis is the number of occurrences (frequency) and Y axis is the response time ranges. You can use this histogram to see the performance distribution for a specific monitored entity.
Performance Histogram can help you visualize performance distribution.
This feature automatically plots a trend line associated with the performance (response time) of a specific monitored entity during a time frame specified by you. The X axis is response time and Y axis is time on the clock. This trend line gives you a visual representation of a monitored entity's performance over time.
Time-based Performance Trend allows you to quickly identify abnormal performance patterns during a certain time frame.
CAMM has a built-in analytic to detect performance memory leak.
To access Memory Leak Detection, select the JVM node under Resources. When the JVM node is selected, CAMM analyzes memory usage patterns during the time frame specified. Two graphs, Memory Leak Rate and Memory Leak Status, display in the Main Display Window.
You can access Memory Leak Detection by clicking on the JVM node.
The Memory Leak Rate graph enables you to perform time-based trend analysis. The graph depicts the JVM heap memory growth rate in KB/minute for a given period of time. The Memory Leak Status graph enables you to quickly identify the presence of persistent memory leak.
Tip:
Persistent JVM heap growth can result in a memory leak. CAMM evaluates the memory growth rate continuously to determine leak status. The Memory Leak Status graph has two values (0 and 1). The value 0 means there is no leak. The value 1 means there may be a leak. When there is a severe leak, the value of the Memory Leak Status graph remains at 1.Finding performance bottlenecks using CAMM is easy. Often, CAMM guides you to the exact location of the bottleneck. There are some basic ways for performing drill down operations in CAMM.
| Operation | Description |
|---|---|
|
Double-Clicking |
Available on many views in CAMM. Double-clicking triggers CAMM to bring up information one logical level lower than what is currently displayed. |
|
Right-Click Menu |
Available on most views in CAMM. From most views, selecting the Display Architecture View option initiates a diagnostic session. From the Architecture View, use the drill down option to go one level deeper. |
Whenever possible, use the built-in Delay Analysis to isolate performance bottlenecks. Delay Analysis is available under the Process node as well as in the Architecture View. CAMM Delay Analysis shows the delay contribution associated with a specific component in a process or an active call path to the overall delay.
You can also use Entity Performance Ranking to identify abnormally behaving components and performance bottlenecks.
A red light on the Operational Dashboard indicates that the performance threshold for an application has been violated.
Double-click on the application row in the Operational Dashboard to bring up the detailed performance data pop-up window.
Use the detailed performance data to further isolate the performance problem. This pop-up window contains the following information:
Servlet Performance [Table]
Servlet Response Time (ms) [Graph]
Servlet Invocation Count [Graph]
EJB Performance [Table]
Top five EJB Response Time (ms) [Graph]
Top five EJB Invocation Count [Graph]
Double-clicking on a specific row in this pop-up window brings up the Architecture View associated with the selected row.
Double-clicking on the slow performing component brings up the Architecture View with delay analysis.
Tip:
Right-click and check the Hide other arrows option to show only arrows on the component which you have clicked on. Follow the slowest Inbound calls to the slowest next level component. Repeat this method to find the bottleneck.Look at the Fan Out table first to see if any outbound calls are performing poorly. If a specific call is performing poorly, follow the call to the destination component and evaluate further.
Evaluate all the Fan Out calls and follow the slowest Fan Out call.
If Fan Out calls are not performing poorly, it is very likely the performance bottleneck exists within the currently selected component. Use the Fan In table to isolate the poorly performing calls. Double-clicking on a specific inbound call in the Fan In table brings up a pop-up window with detailed performance data associated with this inbound call.
Double-click on the slowest Fan In call to get detailed performance data.
CAMM guides you from the Operational Dashboard to the performance bottleneck that exists at the method level.
Tip:
Compare current delay contribution breakdown to baseline delay contribution breakdown to identify any abnormal behavior. This can be done using Comparative Views.After performance bottleneck is identified, you can use the CAMM unique drill out ability to determine the impact of the bottleneck. The impact analysis can be done in the Architecture View.
To start the drill out process:
Select the entity that has been identified as the bottleneck in the Architecture View
Right-click and select Drill Out. This brings the Architecture View one logical level higher.
Use a combination of Drill Out and Show in Context navigation techniques to determine the scope of impact.
To determine the impact of the performance bottleneck, refocus the Architecture View and the module that contains the class in context (white background color).
On this window you can:
Check the Hide other arrows option in the right-click menu to show only arrows on the component you have clicked on and improve visual navigation.
Use the Architecture View or the Fan In table to identify the upstream components that make inbound calls into the class. The impact analysis would follow these inbound calls to their origins to further determine the impact of the performance bottleneck.
For example, the blue background color of a class indicates this entity belongs to a module that is different from current view context.