You can create custom graphs on the Reports tab for post-run analysis for sessions where data has been saved.
The Reports tab is where you can view session data for post-run analysis. Use the Active Session tab to view reports and graphs while a load test session is still running.
The following options are available on the Reports tab:
Session - lets you select the session for which you want to view the session report.
Filter - opens a dialog box for specifying custom date and time range filter options. The filter options specify a specific date and time range to include in the report. When no date and time filter is specified, the report displays the session report for the entire session duration.
Delete - deletes the selected report template.
Generate - generates the report based on the session selected.
Export to CSV - generates the report data into a comma separated value format. You can open the data into a spreadsheet application or save the data to a file.
Print/Save HTML - opens the session report in a separate browser window. From there you can either print the report or save it in HTML format.
The Reports tab has the following sub-tabs after you generate a session report:
Session Overview - shows the start time, end time, and duration, and overview of the number of virtual users, transactions, performance totals and performance by profile.
Graphs - this sub-tab is used to create a custom graph reports.
Virtual User Logs - this sub-tab shows the virtual user information saved to the log file.
The Session Overview sub-tab shows the generated Session Performance Report. You can view the Performance Statistics using the Session Overview sub-tab on the Reports tab for sessions for which data has been saved.
Once you generate a session report, the statistics show the values for the following performance categories:
<session name> - shows the name of the session used to generate the report.
Start Time - shows the date and tie th session was started.
End Time - shows the date and time the session ended.
Duration - shows how long the session ran.
Virtual Users - shows the number of Virtual User that were run during the session and the number with errors.
Transactions - show the total number of transactions and the Average number of transactions per second.
Overview - shows a summary of the performance statistics:
Active Virtual Users - the number of virtual users currently running.
Virtual Users with Errors - the number of virtual users with errors.
Transactions Per Second - the number of times the virtual user played back the script per second.
Pages Per Second - the number of pages returned by the server per second. A "page" consists of all of the resources (i.e. page HTML, all images, and all frames) that make up a Web page.
Hits Per Second - the number of resource requests to the server per second. Each request for a page, individual images, and individual frames is counted as a "hit" by Oracle Load Testing. If Oracle Load Testing does not request images from the server (as specified in the Download Manager), images are not included in the hit count. The Hits Per Second and Pages Per Second counts will be the same if images are not requested and there are no frames in the page.
Kilobytes Per Second - the number of kilobytes transferred between the server and browser client per second.
Totals - show the totals of the performance statistics:
Transactions - the total number of times the virtual user played back the virtual user profile.
Transactions with Errors - the total number of virtual user profile iterations that had errors.
Pages - the total number of number of OpenScript step groups in the script.
Hits - the total number of resource requests to the server.
Kilobytes - the total number of kilobytes transferred between the server and browser client.
Performance by Profile - shows the performance statistics for the Virtual User profile(s) run during the load test.
<Profile Name> - the latest, minimum, maximum, and average performance for the virtual user profile in seconds.
<Timer Name> - the latest, minimum, maximum, and average performance for the server response timers in seconds. Server Response timers are added to scripts using Oracle OpenScript.
Min - the minimum performance for the virtual user profile or server response timer in seconds.
Max - the maximum performance for the virtual user profile or server response timer in seconds.
Avg - the average performance for the virtual user profile or server response timer in seconds.
Std Dev - the number of seconds that the performance of virtual user profiles or server response timers deviated from the mean (average value).
90th % - shows the number of seconds that the performance of ninety percent of the virtual users or server response timers was at or below.
Pass - the number that passed.
Warning - the number with warnings.
Fail - the number that failed.
The Graphs sub-tab is where you can generate graph reports from the counter data saved during the session.
Initially the Graphs tab displays a blank dashboard. Click Add Graph to open the graph options. Expand the Session tree to view the available default graphs and counters.
<session tree> - the Session tree lists the available graphs and counters for the selected scenario.
Filter Counters - selects the filter to use for the session data counters.
Show Average Performance Only - when selected, the counters tree shows only average performance counters.
Show All Counters - when selected, the counters tree shows all counters.
The Session tree has the following available graphs and counters for the selected session:
Default Graphs - lists the built-in graphs that can be used for reporting:
Performance Vs. Time - shows the average virtual user run time over a period of time. If you have multiple profiles in the running Scenario the Performance Vs. Time graph shows separate plot lines for each scenario profile.
Trans Failed/sec Vs. Time - shows the number of failed transactions-per-second for virtual user over a period of time. If you have multiple profiles in the running Scenario the Errors Vs. Time graph shows separate plot lines for each scenario profile. The failed transactions-per-second depends upon the number of virtual users being run by individual profiles and the number of server requests (page, image, frame, etc.) that fail during the Oracle Load Testing session.
Statistics Vs. Time - shows averages for virtual user hits, pages, transactions, and Kilobytes per second over time. The Average Statistic values are a graphical representation of the Average Value data from the Performance Statistics over time. The plot points are updated while the session runs the virtual users.
Performance Vs. Users - shows the average run time of the script for each running virtual user in each profile. If you have multiple profiles in the running Scenario, the Performance Vs. Users graph shows separate bars for each scenario profile running. For example, if you run 100 VUs and ramp 10 at a time, you will see the average response time plotted at 10 VUs, 20 VUs, 30 VUs, etc. up to 100 VUs.
Trans/Failed Vs. Users - the number of failed transactions-per-second for each virtual user profile. If you have multiple profiles in the running Scenario the Errors Vs. Users graph shows separate plot lines for each scenario profile running. The failed transactions-per-second depends upon the number of virtual users being run by individual profiles and the number server requests (page, image, frame, etc.) that fail during the Oracle Load Testing session.
The graph adds plot points based upon the number of running virtual users as the session ramps up to the total number of virtual users for all scenario profiles.
Users Vs. Time - shows the relative time when the virtual users for each profile started running. The graph represents the Autopilot ramp up times and the number of virtual users ramped up for each profile.
Performance, Users vs. Time - this graph combines the Performance vs. Time and the Users vs. Time graphs.
Hits/sec, Users vs. Time - this graph combines the Hits per second and the Users vs. Time graphs.
KBs/sec, Users vs. Time - this graph combines the Kilobytes per second and the Users vs. Time graphs.
Pages/sec, Users vs. Time - this graph combines the Kilobytes per second and the Users vs. Time graphs.
Trans/sec, Users vs. Time - this graph combines the transactions per second statistic from the Statistics vs. Time graph and the Users vs. Time graph.
All Trans/sec, Users vs. Time - this graph combines the transactions per second statistic from the Statistics vs. Time graph and the Users vs. Time graph.
Note:
For OpenScript scripts, steps run in the Initialize and Finish sections do not contribute to the overall script execution time reported in Oracle Load Testing reports. The time taken for the Initialize and Finish sections are not included in the script times so as not to skew the average time when the script runs for multiple iterations.Session Counters - lists the built-in session counters that can be used for reporting:
Kb Rcvd/sec - this counter graphs Kilobytes received per second.
Number of VUs - this counter graphs Kilobytes received per second.
Pages Rcvd/sec - this counter graphs Kilobytes received per second.
Trans Failed/sec - this counter graphs the number of transactions failed per second.
Trans sec - this counter graphs the number of transactions failed per second.
VUs with Errors - this counter graphs the number Virtual Users with errors.
Profile Counters - lists the counters from the Virtual User profile that can be used for reporting:
Profile Nodes - list the counters for virtual user profiles available to include in the report graphs. The profile is the name of the virtual user profile(s) that were run in the session. Double-click the counter name to add counters(s) to the Selected Data Series list.
Timer Nodes - list the page timers available to include in the report graphs. Timers correspond to the individual pages of the script(s) run by the virtual user profile. They allow you to view the performance of a specific page download within a test run, allowing you to identify bottlenecks. The timer is the name of the virtual user profile(s) that were run by the session. The timers consist of the virtual user profile name/Script name and the page/Step group number defined the script. The timers are added to the session data when you select the Generate timers for resources check box in Options from the User menu then selecting Default Properties. Double-click with the mouse to add timer(s) to the Selected Data Series list.
ServerStats Counters - lists the counters from the metrics included in the ServerStats counter configuration. The ServerStats counters are based upon the data source(s) and counters that were included in the ServerStats Data Source configuration when the Oracle Load Testing session was run. Double-click with the mouse to add counter(s) to the Selected Data Series list.
Oracle Database Diagnostics - groups a set of Oracle Database profile metrics on the same graph. These metrics appear only if an Oracle Database ServerStats configuration is used with the session run. The graphs and metrics are as follows:
Average Active Sessions - shows Average Active Sessions metrics.
Host PGA Utilization - shows Host "Program Global Area (PGA)" Utilization metrics.
DB Throughput - groups "Physical I/O per second" and "Physical I/O requests total" metrics.
Graphs have the following toolbar buttons:
Load Query - opens a dialog box for opening a saved graph data query.
Save Query - opens a dialog box for saving the current graph data query.
Export - opens options for saving the current graph to a file.
PNG - opens options for saving the current graph to a PNG format image file.
SVG - opens options for saving the current graph to a SVG format image file.
CSV - opens options for saving the current graph data to a comma-separated-value file.
Excel HTML - opens options for saving the current graph data to an Excel spreadsheet file.
Toggle Legend - displays or hides the names and plot line color of each data series in the graph.
Toggle Stats - displays or hides the chart statistics table view. The chart statistics show the Data Series and values for each series.
Refresh Graph - refreshes the data in the graph from the database.
Toggle Zoom - toggles the zoom tool on and off. When in zoom mode, a magnifier appears in the graph area. Click the magnifier and select the zoom options from the popup menu.
Options - opens the options for customizing graph and x/y axis options. The options are the same as graphs created on the Active Session Runtime Graphs sub- tab. See the options in Section 10.4.2, "Custom Graphs".
The Virual Usre Logs sub-tab is where you can view log messages generated during the session.
To view Virtual User Logs:
Specify Scenarios on the Composer tab.
Click Run Now or schedule the scenario to run at a later time using the Schedule.
Select the Virtual User Logs sub-tab from the Reports tab for post session analysis.
The Virtual User Log shows all messages as they are stored without grouping. You can use the filter to see only messages for a particular VU and its chronological path flow. Use this view to debug problems in depth by viewing message contents. The All Logs view shows the following information:
Show - Specifies the message level. The level can be Info, Debug, Warning, or Error.
Info - show only Info level messages.
Debug - show only Debug level messages.
Warning - show only Warning level messages.
Error - show only Error level messages.
Content - show all Info level messages.
Filter - toggles the Filter bar for specifying the filter method to use.
VU-ID - Specifies the virtual user number.
Level - Indicates the message level. The level can be Info, Debug, Warning, or Error.
Message - Shows the message.
Type - Specifies the type of message. Some examples of valid types are: Action, BeginPage, Cached Data, Customized Log, DatabankRecord, FoundResource, ParameterSubstitution, RequestHeader, ResponseContent, ResponseHeader, ScriptError, Sync Point, ThinkTime, Verification.
Script - Shows the name of the script that generated the message.
Step - Specifies the step number in the script.
Iteration - Shows the iteration number.
Timestamp - Shows the date and time the message occurred.
To set the Oracle Load Testing Report options:
Select Options from the User menu.
Select Reporting to set additional options.
Intervals - the Intervals options are as follows:
Data collection interval - specifies how often the aggregate Oracle Load Testing agent performance data is collected for reporting to the Oracle Load Testing database.
Active Session UI refresh interval - specifies how often to refresh the user interface including the VU grid, ServerStats Display, and Sync Point Status.
Graph refresh interval - specifies how often to redisplay graphs in the View Run Graphs tab.
VU Status Grid - the VU Grid options are as follows:
Default rows per page - specifies the maximum number of rows to display on a page on the Watch VU Grid tab. The value must be between 10 and 100, inclusive.
VU Logs Grid - the VU Logs options are as follows:
Deafult rows per page - specifies the maximum number of rows per page to display in the VU logs The value must be between 10 and 100, inclusive.
Performance Reporting - specifies the default setting for Performance Statistics and Session Reports, as follows:
Enter the how often you want data to be collected in the Data collection interval field.
Enter the how often you want to refresh the user interface in the Active Session UI refresh interval field. This includes the VU grid, ServerStats Status Display, and the Sync Point Status display.
Enter the how often you want to refresh the report display in the Graph refresh interval field.
Specify the maximum number of data series to include per graph.
Click OK.
Start an Oracle Load Testing session.
Oracle Load Testing statistics and performance reports and graphs can be viewed for sessions for which data has been saved.
To create custom reports:
Click the Reports tab.
Select the session and click Generate to generate a Performance report.
To create graphs, click Add Graph on the Graphs sub-tab.
Double-click data counters you want to view in the tree view.
Repeat for all of the data series that you want to graph.
Click the Options button display the Graph Options for naming the graph and selecting x/y axis options.
Click OK.
Click Back to Graphs to go back to the thumbnail view. Each custom report is displayed under a separate thumbnail.
The graph export options let you specify export options for the current graph.
To export graph image or data:
Create a graph report.
Select the appropriate export type from the Export menu on the top of the graph.
The following sections provide specific steps and information.
When you export generated data from a report, Oracle Load Testing creates a comma separated value (.csv) file containing the raw data used to generate reports. The first line of the file contains the field names separated by commas. Each additional line contains the data values separated by commas. The following is an example of a .csv file from a Performance vs. Time report:
Time,Entire Job,tutor1,tutor4, 04-09-2001 15:59:48,,.6,, 04-09-2001 16:00:00,,,.13, 04-11-2001 17:08:04,,1.15,, 04-11-2001 17:08:29,1.39,,.24, 04-11-2001 17:09:19,,.98,, 04-11-2001 17:09:25,,,.12,
This example includes the following fields:
Time Entire Job tutor1 tutor4
The additional lines of the .csv file list the actual data (the time values followed by the number of seconds).
The .csv file for each type of report will contain the type of data specific to the report type.
You can export the raw data used to generate reports to a comma separated value (.csv) file. To export generated report data:
Click the Reports tab
Create a session report.
Select the Graphs sub-tab.
Click Add Graph.
Double-click data counters you want to view in the tree view.
Repeat for all of the data series that you want to graph.
Select CSV from the Export menu on the top of the graph.
Select the save or open option from the browser file popup.
To export as file that can be used with Excel:
Click the Reports tab
Create a session report.
Select the Graphs sub-tab.
Click Add Graph.
Double-click data counters you want to view in the tree view.
Repeat for all of the data series that you want to graph.
Select Excel HTML from the Export menu on the top of the graph.
Select the save or open option from the browser file popup.
You can export reports to a PNG file. To export to a PNG file:
Click the Reports tab
Create a session report.
Select the Graphs sub-tab.
Click Add Graph.
Double-click data counters you want to view in the tree view.
Repeat for all of the data series that you want to graph.
Select PNG from the Export menu on the top of the graph.
Select the save or open option from the browser file popup.
A graph query consists of the sessions, data series, and X and Y-axis settings. Once you select this information you can save it and reuse it later in the following ways:
save a graph of a single session and apply it to a new session to create a new graph.
save a graph of a baseline session and a run session and apply it to a new run session to create a new graph with the original baseline.
save a graph of a single session and apply it to a new session and add the counters to the current graph definition.
To save a graph query:
Select the sessions and data series that you want to graph.
Click the Save Query button on the top of the graph.
Enter a name for the query.
<query list> - lists the existing saved queries.
Name - enter a name for the new query or select an existing query to overwrite it.
Click OK.
To run a saved query:
Click the Load Query button on the top of the graph to open the Load Query dialog box.
<Query List> - lists the existing queries. Select the query that you want to run.
Session - select the session against which you want to run the query.
Select the saved query.
Select the session against which you want to run the query.
Click OK.
A warning message is displayed if some of the data series cannot be applied to the selected session. If this happens, click OK to continue.
The new graph is generated.
To edit a query:
Select Options from the User menu.
Select Graphs in the Manage group.
Edit - displays the Edit Graph Query dialog box.
Delete - deletes the selected query.
Query Name - lists the existing queries.
Select the query you want to change and click Edit.
Make and changes and click OK.
Click Close to exit the Graph Query Manager dialog box.
The filter options specify a specific date and time range to include in the report. When no date and time filter is specified, the report displays the session report for the entire session duration.
To specify a custom date and time range filter:
Click the Reports tab.
Select the session and click Generate.
After the report loads, click the Filter icon.
Set the Start and End date and time values to the time range to include in the report.
Click Apply to generate a new report that include only the data between the Start and End date and time.
This section explains how to use the raw data that can be generated by a load test session. The raw data provides the ability to view non-aggregated data collected from all Virtual Users running in a load test across all Load Agent machines for reporting.
To enable raw data collection by default:
Caution:
The raw data files generated during a load session can increase in size rapidly causing file size and disk space issues on the agent machine. A brief 15 minute load test session can produce files of approximately 40MB. When running load test sessions with the Persist Raw Data setting enabled, be sure to plan for file size and disk space issues.Select Options from the User menu.
Select Default Properties in the Profiles group.
Set the Persist Raw Data setting to True in the Agent Preferences section of the default settings.
Be sure to set the Persist Raw Data setting to False after completing the raw data collection load test session.
To enable raw data collection for a specific session:
Add the script to a Scenario.
Select the Configure all parameters button next to the first script/profile.
Set the Persist Raw Data setting to True in the Agent Preferences section of the default settings.
Note:
The options for the first script in the session govern the settings for same options for the other scripts in the same session (even though user interface shows different values for the options in different scripts).When raw data collection is enabled, Oracle Load Testing saves every single measured data point to the database.
To view raw data in reports:
After running a load session with raw data collection, click the Create Reports tab.
Select the session in the Session(s) list.
Click Filter Counters.
Select Show All Counters. The Raw Data node is added to the counters tree.
Expand the Raw Data node and select the counters to add to the report.
This section lists the raw data database tables and counters. Advanced users can use customized database queries to extract additional reports from the raw data collected during a load test.
COUNTERHDR: Contains metadata for an individual counter.
COUNTERHDRID: unique counter ID.
SESSIONRUNID: ID of the session for which this counter was created (index into SESSIONRUN table).
SOURCE: Indicates the type of counter (1=ServerStats Counter, 4=OLT Aggregated Counter, 8=OLT Raw Counter).
COUNTERNAME: Name of the counter.
OBJNAME: The name of the script or ServerStats profile to which this counter applies, if applicable. Null indicates a session-level counter.
INSTNAME: The OpenScript-generated UUID of the step group to which the counter applies, if applicable.
NICENAME: The user-readable name of the step group to which this counter applies, or a collection of UUIDs indicating additional information, if applicable. This data is generated by OpenScript.
COUNTERRUN: Contains counter values.
COUNTERHDRID: ID of the counter to which this data applies (index into COUNTERHDR table).
COUNTERVAL: Numeric counter value.
COUNTERTS: Timestamp that the counter value was measured.
ITERATION: Iteration of the script for which the value was measured (aggregated counters only).
NUMBERVUS: Number of VUs that were running when the counter was measured (aggregated counters only).
SESSIONRUN: Contains metadata about load sessions.
SESSIONRUNID: Unique Session ID.
SESSIONNAME: Unique Session Name.
MACHINE: Weblogic URL of the server which started this session (ie. t3://hostname:port).
START_TS: Timestamp of the start of session.
FINISH_TS: Timestamp of the end of session. Null if the session is still running.
INFOSTRING: Indicates status of the load test. Possible values are "end of session" or "running".
This section provides example SQL queries for generating custom reports from the raw data saved to the database during a load test session.
List all session names in reverse chronological order:
SELECT sessionname FROM sessionrun ORDER BY sessionrunid DESC;
List raw performance data for VU 5 in session named "MySession1":
SELECT dat.counterts as "Timestamp", dat.counterval as "Avg Response Time (sec)" FROM counterrun dat, counterhdr hdr WHERE hdr.counterhdrid = dat.counterhdrid AND hdr.countername = 'Avg Performance (sec) (Raw, VU=5)' AND hdr.source=8 AND hdr.sessionrunid=(SELECT sessionrunid FROM sessionrun s WHERE s.sessionname='MySession1') ORDER BY dat.counterts;
List average of raw page performance counters ordered by page name (similar to OLT session report) for session named "MySession1":
SELECT hdr.nicename AS "Name", AVG(dat.counterval) AS "Avg Response Time (sec)" FROM counterrun dat, counterhdr hdr WHERE hdr.counterhdrid = dat.counterhdrid AND hdr.countername LIKE 'Avg Performance (sec) (Raw, VU=%)' AND hdr.source=8 AND hdr.sessionrunid=(SELECT sessionrunid FROM sessionrun s WHERE s.sessionname='MySession1') GROUP BY hdr.nicename ORDER BY hdr.nicename;
Calculate 75th percentile for the step group named "[1] Landing page" in session named "MySession1":
SELECT PERCENTILE_DISC(0.75) WITHIN GROUP (ORDER BY dat.counterval) AS "75th Percentile" FROM counterrun dat, counterhdr hdr WHERE hdr.counterhdrid = dat.counterhdrid AND hdr.nicename='[1] Landing Page' AND hdr.countername LIKE 'Avg Performance (sec) (Raw, VU=%)' AND hdr.source=8 AND hdr.sessionrunid=(SELECT sessionrunid FROM sessionrun s WHERE s.sessionname='MySession1');