Oracle Load Testing lets you evaluate the performance of your Web application in real time using Performance Statistics and graphs on the View Run Graphs tab. You can create custom graphs on the Create Reports tab for post-run analysis for sessions where data has been saved. The following default reports and graphs are available on the View Run Graphs tab:
Performance Statistics - shows a summary of performance statistics data generated by running virtual users.
Performance Vs. Users - shows the average run time of the script for each running virtual user in each profile. If you have multiple profiles in the running Scenario, the Performance Vs. Users graph shows separate bars for each scenario profile running in the Autopilot. For example, if you run 100 VUs and ramp 10 at a time, you will see the average response time plotted at 10 VUs, 20 VUs, 30 VUs, etc. up to 100 VUs.
Users Vs. Time - shows the relative time when the virtual users for each profile started running. The graph represents the Autopilot ramp up times and the number of virtual users ramped up for each profile.
Performance Vs. Time - shows the average virtual user run time over a period of time. If you have multiple profiles in the running Scenario the Performance Vs. Time graph shows separate plot lines for each scenario profile running in the Autopilot.
Statistics Vs. Time - shows averages for virtual user hits, pages, transactions, and Kilobytes per second over time. The Average Statistic values are a graphical representation of the Average Value data from the Performance Statistics over time. The plot points are updated while the Autopilot runs the virtual users.
Errors Vs. Time - shows the number of failed transactions-per-second for virtual user over a period of time. If you have multiple profiles in the running Scenario the Errors Vs. Time graph shows separate plot lines for each scenario profile running in the Autopilot. The failed transactions-per-second depends upon the number of virtual users being run by individual profiles and the number of server requests (page, image, frame, etc.) that fail during the Oracle Load Testing session.
Errors Vs. Users - the number of failed transactions-per-second for each virtual user profile. If you have multiple profiles in the running Scenario the Errors Vs. Users graph shows separate plot lines for each scenario profile running in the Autopilot. The failed transactions-per-second depends upon the number of virtual users being run by individual profiles and the number server requests (page, image, frame, etc.) that fail during the Oracle Load Testing session.
The graph adds plot points based upon the number of running virtual users as the Autopilot ramps up to the total number of virtual users for all scenario profiles.
Performance, Users vs. Time - this graph combines the Performance vs. Time and the Users vs. Time graphs.
Trans/sec, Users vs. Time - this graph combines the transactions per second statistic from the Statistics vs. Time graph and the Users vs. Time graph.
Note:
For OpenScript scripts, steps run in the Initialize and Finish sections do not contribute to the overall script execution time reported in Oracle Load Testing reports. The time taken for the Initialize and Finish sections are not included in the script times so as not to skew the average time when the script runs for multiple iterations.You can view the graphs while the virtual users are running or after they have been stopped or have finished. The performance data is reported at the conclusion of each iteration. When the running virtual users complete the first iteration, the first set of performance data is captured. The graphs and performance statistics that you see are updated in real time based on the Graph refresh interval setting in Reports Options to give you a snapshot of the load test results. However, there is a small amount of processing time required to generate the graphs/statistics and print them to the screen. The default refresh is 15 seconds. If you run long or over night tests, you should increase the refresh interval to 60 seconds or longer.
If you run a very quick Autopilot session, the View Run Graphs tab may not have enough time to update the screen with the real time data. However, if you use the Session Start/Stop options to save the data for reporting, the data is stored in the database.
In addition to real-time data shown, Oracle Load Testing can generate a summary report and save data to the database for post-testing analysis. The Latest Load Session reports let you generate a summary of performance and error statistics data for the virtual user session. You can use the Create Reports tab to generate a variety of reports and graphs from the data stored in the database.
To set the Oracle Load Testing Report options:
Select Options from the Tools menu.
Select Reporting to set additional options.
Intervals - the Intervals options are as follows:
Reporting data collection interval - specifies how often the aggregate Oracle Load Testing agent performance data is collected for reporting to the Oracle Load Testing database.
User Interface refresh interval - specifies how often to refresh the user interface including the VU grid, ServerStats Display, and Sync Point Status.
Graph refresh interval - specifies how often to redisplay graphs in the View Run Graphs tab.
VU Logs auto display interval - specifies how often to refresh the virtual user logs when Auto mode is being used.
VU Grid - the VU Grid options are as follows:
Maximum number of rows per page - specifies the maximum number of rows to display on a page on the Watch VU Grid tab. The value must be between 10 and 100, inclusive.
VU Logs - the VU Logs options are as follows:
Maximum number of rows per page - specifies the maximum number of rows per page to display in the VU logs The value must be between 10 and 100, inclusive.
Performance Statistics and Session Report - specifies the default setting for Performance Statistics and Session Reports, as follows:
Show Server Times Only - specifies that only server response time is included in performance statistics and session reports.
Show End-to-End Times (include think times) - specifies that think time is included in performance statistics and session reports.
Enter the how often you want data to be collected in the Reporting data collection interval field.
Enter the how often you want to refresh the user interface in the User interface refresh interval field. This includes the VU grid, ServerStats Status Display, and the Sync Point Status display.
Enter the how often you want to refresh the report display in the Graph refresh interval field.
Specify how you want timer names to be displayed.
Specify whether to include think time in the profile time.
Specify the maximum number of data series to include per graph.
Click OK.
Start an Oracle Load Testing Autopilot session.
The View Run Graphs tab is where you can view runtime reports. These graphs are only available while the load test is running. Use the Create Reports tab to view reports and graphs after the load test has finished running.
The View Run Graphs tab is refreshed according to what is set in the Graph Refresh Interval setting in the reporting options.
To stop the display from being refreshed, click the Pause button.
To resume refreshing the display, click the Resume button. Note that exiting the tab and returning to the tab will also resume refreshing the display.
Reports & Graphs - displays options for creating a new graph and viewing default graphs.
New Graph - displays a new tab with filters for customizing the graph. The tab label is the name you enter in the Graph Name field in the Filters section.
Overview - shows a thumbnail view of all graphs. Click on a thumbnail to enlarge it. The Overview tab includes the Default Graphs selector list and the Show Chart Statistics button, as follows:
Default Graphs - select a graph and click Add Data Series to view a default graph.
Show Chart Statistics - toggles the chart statistics table view on and off. The chart statistics show the Data Series and values for each series.
Reports - displays the performance statistics for the session.
Custom graph tabs have the following options:
Remove Graph - the (X) button on the tab deletes the selected graph.
Session - displays the session that is running.
Available Data Series - shows a list of session data generated by Oracle Load Testing and ServerStats. The data series listed is a subset of all of the available data series. Click Show All to display all available data series.
Default Graphs - select a graph and click Add Data Series to view a default graph.
Oracle Database Diagnostics - groups a set of Oracle Database profile metrics on the same graph. These metrics appear only if an Oracle Database ServerStats configuration is used with the session run. The graphs and metrics are as follows:
PGA/SGA - groups "Program Global Area (PGA)", "Shared Global Area (SGA)" and "Total Global Area" metrics.
CPU vs. DB Sessions - groups "Average active sessions on CPU", "CPU recursive usage", and "CPU parse time usage" metrics.
Transaction Overview - groups "User login rates per second" and "User Transaction Commit Rate" metrics.
DB Throughput - groups "Physical I/O per second" and "Physical I/O requests total" metrics.
Session Counters - the counters that are available for the overall session.
<profiles> - the counters that are available for each Oracle Load Testing virtual user profile.
Profile Nodes - list the counters for virtual user profiles available to include in the report graphs. The profile is the name of the virtual user profile(s) that were run in the Autopilot session. Select the counter(s) and click the Add Data Source button or double-click to add counters to the Selected Data Series list.
Timer Nodes - list the page timers available to include in the report graphs. Timers correspond to the individual pages of the script(s) run by the virtual user profile. They allow you to view the performance of a specific page download within a test run, allowing you to identify bottlenecks. The timer is the name of the virtual user profile(s) that were run by the Autopilot session. The timers consist of the virtual user profile name/Script name and the page number of the script. The timers are added to the session data when you select the Auto generate timers for all resources check box in Options from the Tools menu then selecting Scenario Defaults. Select the timer(s) and click the arrow button or double-click with the mouse to add timer(s) to the Selected Data Series list.
ServerStats Nodes - list the Oracle Load Testing ServerStats counters available to include in the report graph. The ServerStats counters are based upon the data source(s) and counters that were included in the ServerStats Data Source configuration when the Oracle Load Testing session was run. Select the counter(s) and click the Add Data Series button or double-click with the mouse to add counter(s) to the Selected Data Series list.
ServerStats Monitors - shows the ServerStats counter configuration.
[>] Add Data Series - adds the selected data series to the graph. You can move the mouse over a data series line in the graph to view details about specific data points in the graph.
Show Legend - opens a view of the names and plot line color of each data series in the graph. click a data series in the legend popup to show or hide plot lines in the graph. Double click a data series in the legend popup to show/hide the y-axis scale on the right side of the graph.
Show Chart Statistics - toggles the chart statistics table view on and off. The chart statistics show the Data Series and values for each series.
Show Graph Options - opens the options for customizing, exporting, opening saved queries, and saving queries.
Clear Graph - removes all data series from the graph.
Zoom - toggles the zoom tool on and off.
zoom in - click and drag a bounding box over the area of the graph. The zoom indicator shows on the graph when zoomed in from full size.
zoom out - press and hold the Ctrl key and click the mouse button to zoom out one step to the previous zoomed size. Press and hold the Ctrl key and double-click the mouse button to zoom out to the full size.
You can view the Performance Statistics report by selecting the View Run Graphs - Reports tab or the Create Reports tab for sessions for which data has been saved, when you start an Oracle Load Testing session.
View - specifies which performance times to show in the report.
Show Server Times Only - when selected, only server response time is included in performance statistics and session reports.
Show End-to-End Times (include think times) - when selected, think time is included with server times in performance statistics and session reports.
Detail Level - lets you specify the amount of detail shown in the report. This field is only displayed for the Performance Statistics report when viewed in the View Run Graphs tab.
The statistics show the values for the following performance categories:
Active Virtual Users - the number of virtual users currently running in the Autopilot. Clicking the link opens a Users vs. Time graph in a new sub-tab.
Virtual Users with Errors - the number of virtual users with errors. Clicking the link opens a Errors vs. Users graph in a new sub-tab.
Transactions Per Second - the number of times the virtual user played back the script per second. Clicking the link opens a Trans/sec, Users vs. Time graph in a new sub-tab.
Pages Per Second - the number of pages returned by the server per second. A "page" consists of all of the resources (i.e. page HTML, all images, and all frames) that make up a Web page. Clicking the link opens a Pages/sec, Users vs. Time graph in a new sub-tab.
Hits Per Second - the number of resource requests to the server per second. Each request for a page, individual images, and individual frames is counted as a "hit" by Oracle Load Testing. If Oracle Load Testing does not request images from the server (as specified in the Download Manager), images are not included in the hit count. The Hits Per Second and Pages Per Second counts will be the same if images are not requested and there are no frames in the page. Clicking the link opens a Hits/sec, Users vs. Time graph in a new sub-tab.
Kilobytes Per Second - the number of kilobytes transferred between the server and browser client per second. Clicking the link opens a KB/sec, Users vs. Time graph in a new sub-tab.
<Session Name> Totals
Transactions - the total number of times the virtual user played back the virtual user profile.
Transactions with Errors - the total number of virtual user profile iterations that had errors.
Pages - the total number of number of OpenScript step groups in the script.
Hits - the total number of resource requests to the server.
Kilobytes - the total number of kilobytes transferred between the server and browser client.
Performance by Profile and Timer - this section appears when the Detail Level is set to Summary + Profile Timers or Full View.
<Profile Name> - the latest, minimum, maximum, and average performance for the virtual user profile in seconds.
<Timer Name> - the latest, minimum, maximum, and average performance for the server response timers in seconds. Server Response timers are added to scripts using Oracle OpenScript.
Min - the minimum performance for the virtual user profile or server response timer in seconds.
Max - the maximum performance for the virtual user profile or server response timer in seconds.
Avg - the average performance for the virtual user profile or server response timer in seconds.
Pass - the number that passed.
Fail - the number that failed.
Std Dev - the number of seconds that the performance of virtual user profiles or server response timers deviated from the mean (average value).
90th % - shows the number of seconds that the performance of ninety percent of the virtual users or server response timers was at or below.
The Load graphs lets you view real-time performance, statistical, and error information about the virtual users being run by the Autopilot. The default Oracle Load Testing graphs automatically start and consist of the following types of runtime graphs:
Performance Vs. Users
Errors Vs. Users
Performance Vs. Time
Errors Vs. Time
Users Vs. Time
Statistics Vs. Time
To view runtime statistics:
Click the View Run Graphs tab. The Overview tab is displayed showing a small view of the default graphs. This is useful for getting an at-a-glance view of all of the run graphs.
Click the graph you want to view to enlarge it and display it in the Reports tab. You can switch between Load graphs using the Default Reports selector list at the bottom of the Overview tab.
To create a custom graph:
Click Graph.
In the Available Data Series section, select the data you want to view. Use Ctrl-click and Shift-click to select multiple data series. If you do not see the data series you are looking for, click Show All to display all available data series.
Click the [>] Add Data Series button to add the data series to the graph.
Click the Show Legend button to open a view of the names and plot line color of each data series in the graph.
Click a data series in the legend popup to show or hide plot lines in the graph.
Double-click a data series in the legend popup to show/hide the y-axis scale on the right side of the graph.
Click the Show Legend button again to close the legend view.
Click the Show Chart Statistics button to toggle the chart statistics table view on and off. The chart statistics show the Data Series and values for each series, as follows:
Minimum - shows the maximum value for the data series.
Maximum - shows the minimum value for the data series.
Average - shows the average value for the data series.
Click the Show Graph Options button to open the options for customizing, exporting, opening saved queries, and saving queries.
Customize options - these options let you customize the current graph.
Graph Tab
Graph Title - enter the name of the graph. This name is displayed in the tab in the Reports & graphs section.
Data Series - lists the data series contained in the graph. The (x) button marks the Data Series for removal. Click the Apply button to Refresh the graph and remove the selected Data Series.
Data Series Name - displays the name of the data series that is displayed when you place the mouse over the data series in the graph.
Data Series Label - displays the label displayed in the legend.
X-Axis Tab
Scale - specifies the x-axis data range format. Changing the scale causes the graph to be refreshed from the server.
Absolute Time - when selected, the generated report tables and graphs are based upon the actual start and end date and times of the profile/timer.
Relative Time - when selected, the generated report tables and graphs are based upon relative time. Profiles/Timers are graphed from a shared zero point. This option can be used to compare two or more metrics from different sessions in the same scale.
Users - when selected, the generated report tables and graphs are based upon users.
Range Filter - specifies a filter for data coming from the server for a particular session. For example, if a session has two hours of data and filter is set to get a one hour time slice, the graph will only show the one hour time slice. Alternatively, you can use the Graph Zoom feature to avoid having to reload the data from the server. The Filter icon appears on the graph when the Range filter is set.
Manual - when selected, the Range Filter minimum and maximum values are enabled. Edit the minimum and maximum values to specify the data points slice to include in the new graph. Click Apply to generate a new graph that includes only the data points between the minimum and maximum values.
Format - shows the Range format to use to filter the graph data. The format is based upon the Scale setting.
Minimum - enter the minimum X-axis value. This field is only available when Manual is selected.
Maximum - enter the maximum X-axis value. This field is only available when Manual is selected.
Y-Axis Tab
Scale - specifies the y-axis data range format. Changing the scale causes the graph to be refreshed from the server.
Threshold Line - displays a threshold line on the graph at the configured location.
Displays - when selected the threshold line will be included in the graph.
Value - enter the value of the threshold line, that is, the point on the Y-axis at which you want the line to be displayed.
Label - enter the label to display in the legend.
Enter a name for the graph.
Select how to plot the graph in the X-Axis and Y-Axis tabs.
Click Apply. Each custom report is displayed under a separate tab so that you can select the one you want to view. Click Remove Graph to delete the graph and its tab.
This graph plots the average run time for the number of active virtual users.
If you have multiple profiles in the running Scenario the Performance Vs. Users graph shows separate plot lines for each scenario profile running in the Autopilot.
The Average Performance values are a graphical representation of the Average Value data for the Elapsed time entries shown in the Performance Statistics view. The graph adds plot points based upon the number of running virtual users as the Autopilot ramps up to the total number of virtual users for all scenarios.
The plot points are added in the same manner as the Elapsed time entries in the Performance Statistics report.
This graph plots the number of failed transactions-per-second for each Virtual User profile.
If you have multiple profiles in the running Scenario the Errors Vs. Users graph shows separate plot lines for each scenario profile running in the Autopilot.
The failed transactions-per-second depends upon the number of virtual users being run by individual profiles and the number server requests (page, image, frame, etc.) that fail during the Oracle Load Testing session.
This graph plots the average run time for virtual users over a period of time.
If you have multiple profiles in the running Scenario the Performance Vs. Time graph shows separate plot lines for each scenario profile running in the Autopilot.
The Average Run Time values are a graphical representation of the Average Value data for the Elapsed time entries shown in the Performance Statistics report that are plotted over time.
This graph plots the number of failed transactions per second for virtual users profiles over time.
If you have multiple profiles in the running Scenario the Errors Vs. Time graph shows separate plot lines for each scenario profile running in the Autopilot.
The failed transactions-per-second depends upon the number of virtual users being run by individual profiles and the number server requests (page, image, frame, etc.) that fail during the Oracle Load Testing session.
This graph plots the number of active virtual users over a period of time.
If you have multiple profiles in the running Scenario the Users Vs. Time graph shows separate plot lines for each scenario profile running in the Autopilot.
This graph plots the average hits, transactions and kilobytes per second over a period of time.
The Average Statistic values are a graphical representation of the Average Value data from the Performance Statistics tab over time. The plot points are updated while the Autopilot runs the virtual users.
Oracle Load Testing automatically generates a set of counters for Oracle Load Testing virtual user data that can be selected in the Available Data Series field in the Filters section. The Oracle Load Testing data includes overall data for the session at the top of the list followed by a folder for each Virtual User profile included in the Configure parameters of the scenario list of the Build Scenarios tab.
Oracle Load Testing also generates counters for each ServerStats monitor in the selected ServerStats configuration. You can create a new graph to monitor any of the counters in the list during the Load Test to view both VU performance and ServerStats monitor performance.
Note:
Sessions with Oracle DB Metrics and Enterprise Manager URL that were run using an Oracle Load Testing version prior to version 9.30 will not automatically show the links for ADDM & AWR reports. To show the ADDM & AWR links under Enterprise Manager Diagnostics, generate the ADDM & AWR reports using the Reports sub-tab of the Create Reports tab.The Virtual User profile folders contain the data counters and page timer folders for the specific profile in the load testing Scenario.
You can create a new ServerStats chart to monitor any of the counters in the list during the Load Test.
The page timer folders contain the performance data for the individual pages of scripts used as a Virtual User profile, similar to the following:
You can create a new ServerStats chart to monitor any of the timers in the list during the Load Test.
The Create Reports tab is where you can view historical reports and graphs for sessions for which you have saved data for reporting. This tab has the following options:
Reports & Graphs - displays default and custom graphs, and gives options for exporting graphs.
New Graph - opens a blank tab and displays the filter options for creating a new graph.
Overview - displays a thumbnail view of each created graph.
Reports - displays the Session Performance and Hardware Estimation reports. These reports can be exported to CSV or printed.
Graph <#> - displays custom graphs that you have created. These graphs can be exported in CSV, Excel, PNG, or JPG format.
The Reports tab has the following options for Session Reports:
Session - lets you select the session for which you want to view the session report.
Report - if you have created custom report templates, they are listed here so that you can select them and display a report using the template. Refer to Creating Report Templates later in this chapter for instructions on how to create a report template. The default is the session report.
Filter - opens a dialog box for specifying custom date and time range filter options. The filter options specify a specific date and time range to include in the report. When no date and time filter is specified, the report displays the session report for the entire session duration.
Delete - deletes the selected template.
Generate - generates the report based on the session and report template selected.
Show Server Times Only - when selected, only server response time is included in performance statistics and session reports.
Show End-to-End Times (include think times) - when selected, think time is included with server times in performance statistics and session reports.
Export to CSV - opens the data in a separate browser window in comma separated value format.
Print/Save HTML - opens the session report in a separate browser window. From there you can either print the report or save it in HTML format.
The Reports tab has the following options for (hardware) Estimation Reports:
Repository - lets you select the repository containing the saved scenario file to use for the hardware estimation report.
Folder - lets you select the workspace folder containing the saved scenario file to use for the hardware estimation report.
File - lets you select the saved scenario file to use for the hardware estimation report.
Generate - generates the report based on the scenario file selected.
Export to CSV - opens the data in a separate browser window in comma separated value format.
Print/Save HTML - opens the hardware estimation report in a separate browser window. From there you can either print the report or save it in HTML format.
Custom graph tabs have the following options:
Remove Graph - deletes the selected graph.
Session - lets you select the session for which you want to view data series.
Available Data Series - shows a list of session data generated by Oracle Load Testing and ServerStats. The data series listed is a subset of all of the available data series. Click Show All to display all available data series.
Default Graphs - select a graph and click Add Data Series to view a default graph.
Oracle Database Diagnostics - groups a set of Oracle Database profile metrics on the same graph. These metrics appear only if an Oracle Database ServerStats configuration is used with the session run. The graphs and metrics are as follows:
PGA/SGA - groups "Program Global Area (PGA)", "Shared Global Area (SGA)" and "Total Global Area" metrics.
CPU vs. DB Sessions - groups "Average active sessions on CPU", name="Oracle: CPU recursive usage", and "CPU parse time usage" metrics.
Transaction Overview - groups "User login rates per second" and "User Transaction Commit Rate" metrics.
DB Throughput - groups "Physical I/O per second" and "Physical I/O requests total" metrics.
Session Counters - the counters that are available for the overall session.
<profiles> - the counters that are available for each Oracle Load Testing virtual user profile.
Profile Nodes - list the counters for virtual user profiles available to include in the report graphs. The profile is the name of the virtual user profile(s) that were run in the Autopilot session. Select the counter(s) and click the Add Data Source button or double-click to add counters(s) to the Selected Data Series list.
Timer Nodes - list the page timers available to include in the report graphs. Timers correspond to the individual pages of the script(s) run by the virtual user profile. They allow you to view the performance of a specific page download within a test run, allowing you to identify bottlenecks. The timer is the name of the virtual user profile(s) that were run by the Autopilot session. The timers consist of the virtual user profile name/Script name and the page number of the script. The timers are added to the session data when you select the Auto generate timers for all resources check box in Options from the Tools menu then selecting Scenario Defaults. Select the timer(s) and click the arrow button or double-click with the mouse to add timer(s) to the Selected Data Series list.
ServerStats Nodes - list the ServerStats counters available to include in the report graph. The ServerStats counters are based upon the data source(s) and counters that were included in the ServerStats Data Source configuration when the Oracle Load Testing session was run. Select the counter(s) and click the Add Data Series button or double-click with the mouse to add counter(s) to the Selected Data Series list.
ServerStats Monitors - shows the ServerStats counter configuration.
[>] Add Data Series - the [>] button adds the selected data series to the graph. You can move the mouse over a data series line in the graph to view details about specific data points in the graph.
Show Legend - opens a view of the names and plot line color of each data series in the graph. click a data series in the legend popup to show or hide plot lines in the graph. Double click a data series in the legend popup to show/hide the y-axis scale on the right side of the graph.
Show Chart Statistics - toggles the chart statistics table view on and off. The chart statistics show the Data Series and values for each series.
Show Graph Options - opens the options for customizing, exporting, opening saved queries, and saving queries.
Zoom - toggles the zoom tool on and off.
zoom in - click and drag a bounding box over the area of the graph. The zoom indicator shows on the graph when zoomed in from full size.
zoom out - press and hold the Ctrl key and click the mouse button to zoom out one step to the previous zoomed size. Press and hold the Ctrl key and double-click the mouse button to zoom out to the full size.
You can quickly view session data by creating custom graphs.
Click the Create Reports tab. A blank tab is displayed for creating a new graph.
Select the session you want to graph.
Select the data you want to view in the Available Data Series field. Use Ctrl-click and Alt-Click to select multiple data series.
Click Add Data Series.
Repeat for all of the data series you want to graph.
Click the Show Graph Options button to display the Graph Options for naming the graph and selecting how to plot the graph.
Click OK.
Each custom report is displayed under a separate tab so that you can select the one you want to view. Click the Remove Graph button located on the graph's tab to delete the graph and its tab. Click the Show Graph Options button and use the export options to save a graph or the graph data to a file.
You can save this query as described later in this chapter for later use against another session.
The graph export options let you specify export options for the current graph.
To export graph image or data:
Create a graph report.
Click the Show Graph Options button on the right of the graph.
Select Export. The Export options are as follows:
Format - specifies the file format to export.
PNG - when selected, the exported graph image is saved in Portable Network Graphics file format. Specify the size of the image file.
JPG - when selected, the exported graph image is saved in Joint Photographic Experts Group compression file format. Specify the size and quality level of the image file.
CSV - when selected, the exported graph data is saved in Comma Separated Value file format. You can open the file in Microsoft Excel or save it to a file.
XLS - when selected, the exported graph data is saved in Microsoft Excel file format. You can open the file in Microsoft Excel or save it to a file.
Size - specifies the pixel size of exported PNG and JPG image files.
Quality - specifies the quality level of exported JPG image files. The quality level dictates the ratio between file size and image clarity. A lower quality level produces a smaller file size but the image may lose clarity. A higher quality level produces a higher clarity image but with a larger file size. Experimentation may be the best way to determine the best acceptable quality level for your image file size and clarity requirements if the default quality level is insufficient.
When you export generated data from a report, Oracle Load Testing creates a comma separated value (.csv) file containing the raw data used to generate reports. The first line of the file contains the field names separated by commas. Each additional line contains the data values separated by commas. The following is an example of a .csv file from a Performance vs. Time report:
Time,Entire Job,tutor1,tutor4, 04-09-2001 15:59:48,,.6,, 04-09-2001 16:00:00,,,.13, 04-11-2001 17:08:04,,1.15,, 04-11-2001 17:08:29,1.39,,.24, 04-11-2001 17:09:19,,.98,, 04-11-2001 17:09:25,,,.12,
This example includes the following fields:
Time Entire Job tutor1 tutor4
The additional lines of the .csv file list the actual data (the time values followed by the number of seconds).
The .csv file for each type of report will contain the type of data specific to the report type.
You can export the raw data used to generate reports to a comma separated value (.csv) file. To export generated report data:
Create a report.
Click the Show Graph Options button on the right of the graph.
Select Export.
Select CSV format and click Export. The File Download dialog box is displayed.
Click Open to open the file in Excel in CSV format. Click Save to save the file in CSV format.
If you chose to save the file, the Save As dialog box is displayed. Enter a new name for the file or accept the default, select the location to save to, and click Save.
To export as an Excel Workbook file:
Note:
The Oracle Load Testing Server must have Microsoft Excel installed to export to Excel format.Create a report.
Click the Show Graph Options button on the right of the graph.
Select Export.
Select XLS format and click Export. The File Download dialog box is displayed.
Click Open to open the chart in Excel. Click Save to save the chart in Excel format.
If you choose to save the file, the Save As dialog box is displayed. Enter a name for the file or accept the default, select the location to save the file, and click Save.
You can export reports to a PNG file. To export to a PNG file:
Create a report.
Click the Show Graph Options button on the right of the graph.
Select Export.
Select PNG format and set the size.
Click Export.
Enter a name for the file and click Save.
You can export reports to a JPG file. To export to a JPG file:
Generate a report.
Click the Show Graph Options button on the right of the graph.
Select Export.
Select JPEG format and set the size and quality level.
Click Export.
Enter a name for the file and click Save.
A graph query consists of the sessions, data series, and X and Y-axis settings. Once you select this information you can save it and reuse it later in the following ways:
save a graph of a single session and apply it to a new session to create a new graph.
save a graph of a baseline session and a run session and apply it to a new run session to create a new graph with the original baseline.
save a graph of a single session and apply it to a new session and add the counters to the current graph definition.
To save a graph query:
Select the sessions and data series that you want to graph.
Click the Show Graph Options button on the right of the graph.
Select Save Query and enter a name for the query.
<query list> - lists the existing saved queries.
Name - enter a name for the new query or select an existing query to overwrite it.
Click Save.
To create a new query, enter a name for the query and click Save. To overwrite an existing query, select the query and click Save, then click Yes when asked to confirm.
To run a saved query:
Click the Show Graph Options button on the right of the graph.
Click Open Query.
<Query List> - lists the existing queries. Select the query that you want to run.
Session - select the session against which you want to run the query.
Add to Current Graph - when selected, the new session data is added to the graph. When deselected, the graph is cleared and the new session data is displayed.
Select the query you want to use.
Select the session against which you want to run the query.
Select whether or not to add this session data to the current graph.
Click OK.
A warning message is displayed if some of the data series cannot be applied to the selected session. If this happens, click OK to continue.
The new graph is generated.
To edit a query:
Click Graphs from the Manage menu.
Edit - displays the Edit Graph Query dialog box.
Delete - deletes the selected query.
Clone - displays the Edit Graph Query dialog box with the configuration of the selected query. This lets you copy the selected query and edit it to create a new query.
Query Name - lists the existing queries.
Select the query you want to change and click Edit.
Make and changes and click OK.
Click Close to exit the Graph Query Manager dialog box.
You can create custom report templates that you can import to Oracle Load Testing. These templates are shown on the Reports tab in the Create Reports tab. Once imported you can select a session, select the report template, and generate the report.
Report templates are created using the Oracle Load Testing Report Generator tool in Microsoft Word. This tool can be installed on any Windows machine that has Microsoft Word. Oracle Application Testing Suite does not have to be installed to use the tool.
Once installed, report templates are created in Microsoft Word and then imported to Oracle Load Testing. Then generated reports are displayed in Microsoft Word.
To install the Report Generator tool:
Close Microsoft Word.
Run the OLTOfficeToolbarInstaller.exe program located in the <installdir>\tools\ms-office\ folder in the Oracle Application Testing Suite installation directory.
The Insert OLT Data button is added to the toolbar in Microsoft Word.
You can now create report templates. Report templates can contain text, graphics, Oracle Load Testing graphs and session data. In addition, you can include custom Java code. To create a report template:
Click the Insert OLT Data button to display the following dialog box.
This dialog box has the following options:
Graphs - lists the default Oracle Load Testing graphs or a custom list of graphs if you have created a template.properties file.
Optional Settings - leave these settings blank to use the defaults.
Title - enter a title for the graph.
Height - enter a height for the graph.
Width - enter a width for the graph.
Session data - select the session data that you want to display, session name, overview, page data, or profile data. The generated data is comma-separated so that you can easily convert it to a table by selecting Convert from the Table menu then select Text to Table.
Custom code - this section can be used to create Java code using the methods that are available in the Report Generator API. Documentation for this API can be accessed from the online help table of contents.
Note:
To move to the next line use CTRL-Enter not Enter. Enter exits the dialog box.Select the graph or session data that you want to display or enter custom code.
Click Insert. A field code is inserted into the document and it is displayed as a comment. To edit the field code, select it, right-click on it and select Edit Field.
In the template, press Enter to move to the next line.
When you are finished creating the template, click Close to exit the Insert Oracle Load Testing Reference dialog box.
Click Save from the File menu. Enter a name for the file and save it as file type .rtf. The file can be saved to any location.
You can now import the file to Oracle Load Testing and start generating reports. To import the file:
In Oracle Load Testing, select Import from the Tools menu.
In the Type field, select Report Template.
Click Browse.
Select the report template and click Open.
Click Import.
Click Close. The template will be listed on the Reports tab in the report list.
To generate a report, select the session, select the template, and click Generate. The report is generated and displayed in Word.
You can change the list of graphs and session data that is available in the Insert OLT Reference dialog box by creating a template.properties file in the installdir/data/olt/reports folder. If this file exists, the graphs and session data listed in it are used. If it does not exist, the defaults are used. This file lists the graphs and session data that you want to make available. For example:
[GRAPH]
Performance vs. Time@<%out.write(session.getGraph('Performance vs. Time'))%>
Errors vs. Time@<%out.write(session.getGraph('Errors vs. Time'))%>
Statistics vs. Time@<%out.write(session.getGraph('Statistics vs. Time'))%>
[SESSION]
Session Name@<%=session.getName()%>
Overview@<%=session.getOverview()%>
You can generate summary reports containing statistical information about the last data collection and reporting session and the Scenario Profile settings.
On the Create Reports tab, select the Reports tab and then select the session for which you want to view reports and click Generate. The Session Performance report is displayed followed by the Oracle Load Testing Scenario report.
Scenario Report - displays a summary of Scenario settings and options that were used to run virtual users.
Session Report - displays a summary of performance and error statistics data for the virtual user session.
Estimation Report - displays a hardware estimation report for the selected saved scenario file.
The following sections explain more about the summary reports.
The generated report file is the same name as the Scenario in Oracle Load Testing (i.e. the Scenario Description field of the Scenario tab).
To save the report, click Print /Save: HTML. The report is displayed in a separate browser window. Select Save As from the File menu to save the report in HTML or text format.
You can print the report by clicking Print/Save HTML. The report is displayed in a new browser window. Select Print from the File menu. A print dialog box opens for setting the printer options. The available options are specific to the type of printer. Set the options and click the appropriate OK or Print button.
To save the Scenario as a different name:
Select Save As from the Scenario menu.
Enter a new Name in the Name field of the Save Scenario dialog box.
Select the workspace to save the report in and click OK.
To save the report, click Print/Save: HMTL. The report is displayed in a separate browser window. Select Save As from the File menu to save the report in HTML or text format.
You can print the report by clicking Print Friendly. Select Print from the File menu. A print dialog box opens for setting the printer options. The available options are specific to the type of printer. Set the options and click the appropriate OK or Print button.
On the Create Reports tab, select the Reports tab and then select the scenario file for which you want to view reports and click Generate.
The Hardware Estimation Report provides the following:
Hardware configuration for each agent, including CPU information, memory size, and network bandwidth limitations.
Process Runtime Configuration for each agent, including Operating System information, Max Virtual Users per Process setting, and Max heap size setting.
Estimated consumption for each script running on the corresponding agent, including CPU usage, heap size usage, and network bandwidth usage.
Suggested Configuration, including idle resources monitored before start estimation, estimated number of required machines, and suggested configurations for better performance.
See Section 4.3.7.2, "Generating Hardware Estimation Reports" for additional information.
The Automatic Database Diagnostic Monitor (ADDM) analyzes data in the Automatic Workload Repository (AWR) to identify potential performance bottlenecks. For each of the identified issues, it locates the root cause and provides recommendations for correcting the problem.
To create an ADDM Report:
Create a load testing script in OpenScript (for example, "script1") against an application using an Oracle Database.
Select Systems from the Manage menu and add a new Monitored System in Oracle Load Testing (for example, "db1") corresponding to your application's database server, and set the Database data source properties to that of the database.
Create a ServerStats configuration (for example, "config1") and apply the Oracle Database profile to it using "db1".
Add "script1" to an Oracle Load Testing scenario.
Apply "config1" to the scenario in the ServerStats Configuration of the Set up Autopilot tab.
Click the Run button and select "Yes" to perform a baseline for oracle DB. After the load test finishes, the "ADDM Report" selection will appear in the selection list of the Create Reports - Reports subtab.
Click the Create Reports tab.
Click the Reports tab.
Select the session for which you want to view the report from the Session option list.
Select ADDM Report from the Report option list.
Click Generate.
For applications under test using Oracle Database and using Enterprise Manager to monitor the Oracle database performance, Enterprise Manager provides data on various tiers in the application, including Automatic Workload Repository (AWR). Oracle Load Testing provides integration points with Enterprise Manager that can drill down to AWR reports.
You will need to know the snapshot id and the "target" domains to get to the correct location within Enterprise Manager to view reports for the duration of load test. Statistics are collected continuously during database operation and show statistics in the form of a report called AWR Report for performance analysis. The AWR report provides a detailed report of the statistics collected between two snapshots. The snapshots do not have to be consecutive. Any period of time bordered by AWR snapshots can be analyzed.
To create an AWR Report:
Create a load testing script in OpenScript (for example, "script1") against an application using an Oracle Database.
Select Systems from the Manage menu and add a new Monitored System in Oracle Load Testing (for example, "db1") corresponding to your application's database server, and set the Database data source properties to that of the database.
Create a ServerStats configuration (for example, "config1") and apply the Oracle Database profile to it using "db1".
Add "script1" to an Oracle Load Testing scenario.
Apply "config1" to the scenario in the ServerStats Configuration of the Set up Autopilot tab.
Click the Run button and select "Yes" to perform a baseline for oracle DB. After the load test finishes, the "AWR Report" selection will appear in the selection list of the Create Reports - Reports subtab.
Click the Create Reports tab.
Click the Reports tab.
Select the session for which you want to view the report from the Session option list.
Select AWR Report from the Report option list.
Click Generate.
The filter options specify a specific date and time range to include in the report. When no date and time filter is specified, the report displays the session report for the entire session duration.
To specify a custom date and time range filter:
Click the Create Reports tab, then click the Reports tab.
Select the session and click Generate.
After the report loads, click the Filter icon. The following options are available:
Time Range - specifies the date and time range to use to filter the report.
Custom (M/d/yyyy HH:mm:ss) - when selected, the minimum and maximum date and time ranges can be set for the filter.
Minimum - specifies the start date and time for the data to include in the report. The specified minimum must be within the Session Start Time and the Session End time.
Maximum - specifies the end date and time for the data to include in the report. The specified maximum must be within the Session Start Time and the Session End time.
Session Details - shows the date and time range details for he session data.
Session Start Time - shows the date and time when the session was started.
Session End Time - shows the date and time when the session ended.
Duration - shows the length of time the session was running.
Select the Custom option and set the minimum and maximum date and time values to the time range to include in the report.
Click OK to generate a new report that include only the data between the minimum and maximum date and times.
To delete session data from the database or edit your session file names and descriptions:
Select Sessions from the Manage menu to display the Session Manager.
This dialog box lets you manage session data in the database.
Edit - displays the Edit Session dialog box for changing the name and description of the session.
Delete - deletes the entire selected session data from the database. To select more than one session, hold down the CTRL key.
Delete Virtual User Logs - deletes only the VU log data for the selected sessions. To select more than one session, hold down the CTRL key.
Session Name - the names of the virtual user sessions run by the Oracle Load Testing Autopilot. Oracle Load Testing lets you specify a name for the session data when you start the Autopilot to play back virtual users.
Start Time - shows the date and time of day the virtual user session was started.
End Time - shows the date and time of day the virtual user session ended.
Description - shows the status of the session. One of the following status entries may appear:
[Blank] - the session data was stored to the database successfully.
start of session - the session database is currently being accessed by an Oracle Load Testing session.
end of session - the data for the session has been committed to the database and is ready for reporting.
saving data - session data is being saved.
VU Logs - indicates whether VU log data was saved for this session.
ADDM/AWR Data - indicates whether the session included Oracle Database metrics and enabled baseline in order to generate Automatic Database Diagnostic Monitor (ADDM) and Automatic Workload Repository (AWR) reports. This will indicate No if the session is still running even if the session includes Oracle Database metrics and enabled baseline because the ADDM/AWR reports cannot be generated while a session is running.
Select the Session Name to delete. To select more than one session, hold down the CTRL key.
Click Delete to delete the session or click Delete Virtual User Logs to delete the VU log only.
This section explains how to use the raw data files that can be generated by a load test session. The raw data provides the ability to upload non-aggregated data collected from all Virtual Users running in a load test across all Load Agent machines for reporting.
A database administrator can then create tables in an Oracle database (Oracle RDBMS only) and import the CSV files into the tables. Database administrators and end users can execute SQL queries against the raw data tables to generate reports.
To enable raw data collection by default:
Caution:
The raw data files generated during a load session can increase in size rapidly causing file size and disk space issues on the agent machine. A brief 15 minute load test session can produce files of approximately 40MB. When running load test sessions with the Persist Raw Data setting enabled, be sure to plan for file size and disk space issues.Select Options from the Tools menu.
Select Scenario Defaults.
Set the Persist Raw Data setting to True in the Java Client Preferences section of the default settings.
Be sure to set the Persist Raw Data setting to False after completing the raw data collection load test session.
To enable raw data collection for a specific session:
Add the script to a Scenario.
Select the Configure all parameters button next to the first script/profile.
Set the Persist Raw Data setting to True in the Java Client Preferences section of the default settings.
Note:
The options for the first script in the session govern the settings for same options for the other scripts in the same session (even though user interface shows different values for the options in different scripts).When raw data collection is enabled, Oracle Load Testing saves every single measured data point in a set of CSV files. However, ServerStats data is not included in the CSV files as the data is not present on the agent. The files are saved locally on the agent machines in directories specified as follows:
<oats_install>/agent/rawdata/<controller-identifier>/<session_name>/<agent-id>/<YYYY-MM-DD HH:mm:ss>
All raw data files in the agent machine directories remain there and are not delivered to the Oracle Load Testing controller, reconciled between the agents of the same session, or manipulated in any way after the capture is complete. Management of the raw data files must be performed manually.
The files are as follows:
counter_definitions.csv - for every session where the Persist Raw Data option is enabled, for every agent where the session is running, every counter definition will be saved in this CSV file. This file has the following fields:
ID - a unique identifier for each counter definition record in the file.
SOURCE_ID - the ID of the virtual user. For example, if a session is running 10 virtual users, there will be source IDs numbered 1 through 10.
SESSION_NAME - the name of the load test session.
PROFILE_NAME - the name of the profile run in the load test session.
TIMER_NAME - the name of the Oracle Load Testing timer.
COUNTER_NAME - the name of the Oracle Load Testing counter.
AGGREGATION_TYPE_NAME - the aggregation type for each counter as follows:
0, Current - the counter is a current value counter.
1, Cumulative - the counter is a cumulative value counter.
2, Average - the counter is an average value counter.
3, Min - the counter is a minimum value counter.
4, Max - the counter is a maximum value counter.
5, No data - the counter has no data.
6 Last - the counter is a last value counter.
CONTROLLER_TIME_DELTA_MS - this is a time stamp offset value for multiple agent machines. It is used to indicate the difference between time stamps (AGENT_TME_MS) on different agent machines and the Oracle Load Testing controller. For example if the controller has a time stamp of "1" and an agent machine has a time stamp of 2, the delta would be 1. if another agent machine has a timestamp of 3 the delta would be 2. Typically in milliseconds.
counter_data.csv - for every session where the Persist Raw Data option is enabled, for every agent where the session is running, every counter value for every counter will be saved in this CSV file.
There are some conditions in which all of the data is not captured for the particular session. How quickly a session is shutdown may affect the raw data capture. This file has the following fields:
ID - corresponds to the Counter Definition ID in the counter_definitions.csv file.
AGENT_TIME_MS - a timestamp indicating the first time an agent starts reporting. It indicates the time, in EPOCH time, when a session request arrived on an agent.
VALUE - the value for the counter. The AGGREGATION_TYPE_NAME values in the counter_definitions.csv file specifies the type of value.
Once you have generated the raw data files by running an Oracle Load Testing scenario with the Persist Raw Data setting enabled, you can crate tables in an Oracle Database for the counter definitions and counter data CSV files. This section explains the basic procedure for creating tables. Section 6.9.5, "Example SQL Command File" provides an example of the SQL commands required to create the tables in the database.
Create a SQL command file or use the RawTables.sql command file provided with the Oracle Application Testing Suite installation.
Copy your command file or the RawTables.sql file to Oracle Home\BIN.
Connect to the Oracle Load Testing database schema (e.g. olt/password) using SQL Plus.
Enter following command in SQLPlus (use the appropriate path for where your SQL command file is located):
@C:\OracleATS\oxe\app\oracle\product\10.2.0\server\BIN\RawTables.sql
Once the raw data tables have been created in the database, you can import the counter definitions and counter data CSV files into the database. This section explains the basic procedure for importing the CSV files into the database.
Create a control file to import the counter definitions CSV file or use the RawDataDef.ctl command file provided with the Oracle Application Testing Suite installation. Section 6.9.6, "Example Counter Definition Control File" provides an example of the control file required to import the counter definitions CSV file into the database.
Create a control file to import the counter data CSV file or use the RawData.ctl command file provided with the Oracle Application Testing Suite installation. Section 6.9.7, "Example Counter Data Control File" provides an example of the control file required to import the counter data CSV file into the database.
Copy your control files or the RawDataDef.sql and RawData.ctl files to Oracle Home\BIN.
Execute the following commands:
sqlldr olt/yourpassword control=RawDataDef.ctl sqlldr olt/yourpassword control=RawData.ctl
This section provides an example of the SQL commands used to create tables in an Oracle database.
CREATE TABLE counterRun_rawdata (
counterHdrID int NULL,
counterVal float NULL,
counterTS timestamp(3) NULL,
iteration int NULL,
numberVus int NULL,
errFlag int NULL
)
;
CREATE TABLE counterHdr (
counterHdrID int NOT NULL,
sessionRunID int NULL,
source int NULL,
machineName nvarchar2 (255),
counterName nvarchar2 (255),
objName nvarchar2 (255),
instName nvarchar2 (255),
niceName nvarchar2 (255),
aggregationType int NULL,
CONSTRAINT counterHdr_PK PRIMARY KEY (counterHdrId)
)
;
create or replace FUNCTION GETSESSIONID
( SessionName1 IN VARCHAR2
) RETURN NUMBER AS
ID number(38):=0;
BEGIN
--dbms_output.put_line('before 1' || length(SessionName1)|| ' ' || SessionName1);
select sessionrunid into ID from sessionrun where sessionname =trim(SessionName1);
--dbms_output.put_line('end');
RETURN ID;
EXCEPTION
WHEN OTHERS THEN
-- DBMS_OUTPUT.put_line(sqlerrm);
-- dbms_output.put_line(dbms_utility.format_error_backtrace);
RETURN 0;
END GETSESSIONID;
/
This section provides an example of a control file used to input a counter definition CSV file into an Oracle Database (use the appropriate path for where your control file is located).
Options (Skip = 1)
Load Data
infile 'C:\OracleATS\oxe\app\oracle\product\10.2.0\server\BIN\counter_definitions.csv'
replace into table COUNTERHDR
fields terminated by "," OPTIONALLY ENCLOSED BY '"'
(
COUNTERHDRID,
source, sessionrunid "getSessionID(:sessionrunid)", objname,
INSTNAME,
COUNTERNAME,
AGGREGATIONTYPE "decode(trim(:AGGREGATIONTYPE),'Average',2,'Cumulative', 1,'Current',0,'Min',3,'Max' ,4,'No data',5,'Last' ,6,-1)" ,
NICENAME "decode(:COUNTERNAME, null, :COUNTERNAME, :COUNTERNAME)"
)
This section provides an example of a control file used to input a counter data CSV into an Oracle Database (use the appropriate path for where your control file is located).
Options (Skip = 1)
infile 'C:\OracleATS\oxe\app\oracle\product\10.2.0\server\BIN\counter_data.csv'
replace into table COUNTERRUN
fields terminated by "," OPTIONALLY ENCLOSED BY '"'
(
COUNTERHDRID,
COUNTERTS "new_time( to_date('01011970', 'ddmmyyyy') + 1/24/60/60 * :COUNTERTS/1000, 'GMT', 'GMT')",
COUNTERVAL
)