Performing Custom Collections

Use the custom collection options to change the diagnostic collections from the default.

Adjusting the Diagnostic Data Collection Period

By default, Oracle Trace File Analyzer (TFA) trims and collects important log files that were updated during the previous one hour.

If you need diagnostic information for a shorter period, you can specify a custom collection window. A shorter collection period can reduce the size of the diagnostic collection and the time required to complete it.

You can specify the collection period in any of the following ways:

CommandDescription
tfactl diagcollect -last <n>h or tfactl diagcollect -last <n>dCollects diagnostic data from the previous <n> hours or days. The number of days must be 7 or less, and the number of hours must be 168 or less.
tfactl diagcollect -from "<yyyy-mm-dd>"Collects diagnostic data starting from the specified date and, optionally, time.
tfactl diagcollect -from "<yyyy-mm-dd>" -to "<yyyy-mm-dd>"Collects diagnostic data between the specified start and end dates and, optionally, times.
tfactl diagcollect -for "<yyyy-mm-dd>"Collects diagnostic data for the specified date.

For the -from and -to options, you can specify the date and time using any of the following formats:

"Mon/dd/yyyy hh:mm:ss"
"yyyy-mm-dd hh:mm:ss"
"yyyy-mm-ddThh:mm:ss"
"yyyy-mm-dd"

For the -for option, use one of the following date formats:

"Mon/dd/yyyy"
"yyyy-mm-dd"

Collecting for Specific Events

You can use tfactl diagcollect to perform a diagnostic collection for a specific recent event or to perform a default collection for a time range.

When you run tfactl diagcollect without additional event-specific options, TFA displays a list of recent incidents and provides the following choices:

  • Collect diagnostics for a specific recent event.
  • Perform the default diagnostic collection for the configured time range.

Running a Default Diagnostic Collection

Run:

tfactl diagcollect

TFA displays a list of recent events and a Default diagnostic collection option.

For example:

$ tfactl diagcollect

Choose the event you want to perform a diagnostic collection for:

1. Mar/12/2019 16:08:20 [ db.orcl.orcl ] ORA-04030: out of process memory when trying to allocate
2. Mar/12/2019 16:08:18 [ db.orcl.orcl ] ORA-04031: unable to allocate 8 bytes of shared memory
3. Mar/12/2019 16:08:16 [ db.orcl.orcl ] ORA-00494: enqueue held for too long more than seconds by osid
4. Mar/12/2019 16:08:14 [ db.orcl.orcl ] ORA-29709: Communication failure with Cluster Synchronization
5. Mar/12/2019 16:08:04 [ db.orcl.orcl ] ORA-29702: error occurred in Cluster Group Service operation
6. Mar/12/2019 16:07:59 [ db.orcl.orcl ] ORA-32701: Possible hangs up to hang ID= detected
7. Mar/12/2019 16:07:51 [ db.orcl.orcl ] ORA-07445: exception encountered: core dump [] [] [] [] [] []
8. Mar/12/2019 16:07:49 [ db.orcl.orcl ] ORA-00700: soft internal error, arguments: [700], [], [],[]
9. Mar/11/2019 22:02:19 [ db.oradb.oradb ] DIA0 Critical Database Process Blocked: Hang ID 1 blocks 5 sessions
10. Default diagnostic collection, for no specific event

Please choose the event : 1-10 [] 10

By default, TFA collects diagnostics for the last 12 hours. This can result in a large collection.

For a more targeted collection, enter the time of the incident when prompted. Press Enter to use the default 12-hour collection period.

Example output:

By default TFA will collect diagnostics for the last 12 hours.
This can result in large collections.

For more targeted collections enter the time of the incident, otherwise hit <RETURN> to collect for the last 12 hours
[YYYY-MM-DD HH24:MI:SS,<RETURN>=Collect for last 12 hours] :

Collecting data for the last 12 hours for all components...
Collecting data for all nodes

Collection Id : 20190312163846node1

Detailed Logging at :
/scratch/app/product/19c/tfa/repository/collection_Tue_Mar_12_16_38_47_PDT_2019_node_all/diagcollect_20190312163846_node1.log

2019/03/12 16:38:50 PDT : NOTE : Any file or directory name containing the string .com will be renamed to replace .com with dotcom
2019/03/12 16:38:50 PDT : Collection Name : tfa_Tue_Mar_12_16_38_47_PDT_2019.zip
2019/03/12 16:38:50 PDT : Collecting diagnostics from hosts : [node1]
2019/03/12 16:38:50 PDT : Scanning of files for Collection in progress...
2019/03/12 16:38:50 PDT : Collecting additional diagnostic information...
2019/03/12 16:38:55 PDT : Getting list of files satisfying time range [03/12/2019 04:38:50 PDT, 03/12/2019 16:38:55 PDT]
2019/03/12 16:39:02 PDT : Collecting ADR incident files...
2019/03/12 16:39:06 PDT : Completed collection of additional diagnostic information...
2019/03/12 16:39:07 PDT : Completed Local Collection

.------------------------------------.
|         Collection Summary         |
+----------+-----------+------+------+
| Host     | Status    | Size | Time |
+----------+-----------+------+------+
| node1    | Completed | 21MB |  17s |
'----------+-----------+------+------'

Logs are being collected to:

/scratch/app/product/19c/tfa/repository/collection_Tue_Mar_12_16_38_47_PDT_2019_node_all

/scratch/app/product/19c/tfa/repository/collection_Tue_Mar_12_16_38_47_PDT_2019_node_all/node1.tfa_Tue_Mar_12_16_38_47_PDT_2019.zip

Collecting Diagnostics for a Specific Event Without an SRDC

You can select a recent event that does not have an associated Service Request Data Collection (SRDC). TFA collects diagnostic information for the event using the applicable time range.

Run:

tfactl diagcollect

Select the required event when prompted.

For example:

$ tfactl diagcollect

Choose the event you want to perform a diagnostic collection for:

1. Mar/12/2019 16:08:20 [ db.orcl.orcl ] ORA-04030: out of process memory when trying to allocate
2. Mar/12/2019 16:08:18 [ db.orcl.orcl ] ORA-04031: unable to allocate 8 bytes of shared memory
3. Mar/12/2019 16:08:16 [ db.orcl.orcl ] ORA-00494: enqueue held for too long more than seconds by osid
4. Mar/12/2019 16:08:14 [ db.orcl.orcl ] ORA-29709: Communication failure with Cluster Synchronization
5. Mar/12/2019 16:08:04 [ db.orcl.orcl ] ORA-29702: error occurred in Cluster Group Service operation
6. Mar/12/2019 16:07:59 [ db.orcl.orcl ] ORA-32701: Possible hangs up to hang ID= detected
7. Mar/12/2019 16:07:51 [ db.orcl.orcl ] ORA-07445: exception encountered: core dump [] [] [] [] [] []
8. Mar/12/2019 16:07:49 [ db.orcl.orcl ] ORA-00700: soft internal error, arguments: [700], [], [],[]
9. Mar/11/2019 22:02:19 [ db.oradb.oradb ] DIA0 Critical Database Process Blocked: Hang ID 1 blocks 5 sessions
10. Default diagnostic collection, for no specific event

Please choose the event : 1-10 [] 9

Collecting data for all nodes
Scanning files from Mar/11/2019 18:02:19 to Mar/11/2019 23:02:19

Collection Id : 20190312162708node1

Detailed Logging at :
/scratch/app/product/19c/tfa/repository/collection_Tue_Mar_12_16_27_09_PDT_2019_node_all/diagcollect_20190312162708_node1.log

2019/03/12 16:27:12 PDT : NOTE : Any file or directory name containing the string .com will be renamed to replace .com with dotcom
2019/03/12 16:27:12 PDT : Collection Name : tfa_Tue_Mar_12_16_27_09_PDT_2019.zip
2019/03/12 16:27:12 PDT : Collecting diagnostics from hosts : [node1]
2019/03/12 16:27:12 PDT : Scanning of files for Collection in progress...
2019/03/12 16:27:12 PDT : Collecting additional diagnostic information...
2019/03/12 16:27:17 PDT : Getting list of files satisfying time range [03/11/2019 18:02:19 PDT, 03/11/2019 23:02:19 PDT]
2019/03/12 16:27:23 PDT : Collecting ADR incident files...
2019/03/12 16:27:28 PDT : Completed collection of additional diagnostic information...
2019/03/12 16:27:33 PDT : Completed Local Collection

.------------------------------------.
|         Collection Summary         |
+----------+-----------+------+------+
| Host     | Status    | Size | Time |
+----------+-----------+------+------+
| node1    | Completed | 10MB |  21s |
'----------+-----------+------+------'

Logs are being collected to:

/scratch/app/product/19c/tfa/repository/collection_Tue_Mar_12_16_27_09_PDT_2019_node_all

/scratch/app/product/19c/tfa/repository/collection_Tue_Mar_12_16_27_09_PDT_2019_node_all/node1.tfa_Tue_Mar_12_16_27_09_PDT_2019.zip

Collecting Diagnostics for a Specific Event with an SRDC

If the selected event has an associated SRDC, TFA runs the corresponding SRDC and collects the diagnostic information required for that problem.

Note:

When selecting an SRDC, the user running the collection must belong to the dba group of the database selected for the event.

Run:

tfactl diagcollect

Select the event associated with the required SRDC.

For example, selecting an ORA-04030 event can start the corresponding ora4030 SRDC:

$ tfactl diagcollect

Choose the event you want to perform a diagnostic collection for:

1. Mar/12/2019 16:08:20 [ db.orcl.orcl ] ORA-04030: out of process memory when trying to allocate
2. Mar/12/2019 16:08:18 [ db.orcl.orcl ] ORA-04031: unable to allocate 8 bytes of shared memory
3. Mar/12/2019 16:08:16 [ db.orcl.orcl ] ORA-00494: enqueue held for too long more than seconds by osid
4. Mar/12/2019 16:08:14 [ db.orcl.orcl ] ORA-29709: Communication failure with Cluster Synchronization
5. Mar/12/2019 16:08:04 [ db.orcl.orcl ] ORA-29702: error occurred in Cluster Group Service operation
6. Mar/12/2019 16:07:59 [ db.orcl.orcl ] ORA-32701: Possible hangs up to hang ID= detected
7. Mar/12/2019 16:07:51 [ db.orcl.orcl ] ORA-07445: exception encountered: core dump [] [] [] [] [] []
8. Mar/12/2019 16:07:49 [ db.orcl.orcl ] ORA-00700: soft internal error, arguments: [700], [], [],[]
9. Mar/11/2019 22:02:19 [ db.oradb.oradb ] DIA0 Critical Database Process Blocked: Hang ID 1 blocks 5 sessions
10. Default diagnostic collection, for no specific event

Please choose the event : 1-10 [] 1

Scripts to be run by this srdc:
srdc_db_sid_memorysizes_10glower.sql
srdc_db_sid_memorysizes_11gplus.sql
ipspack

Components included in this srdc:
OS DATABASE CHMOS

Collecting data for local node(s)
Scanning files from Mar/12/2019 14:08:20 to Mar/12/2019 18:08:20

WARNING: End time entered is after the current system time.

Collection Id : 20190312163524node1

Detailed Logging at :
/scratch/app/product/19c/tfa/repository/srdc_ora4030_collection_Tue_Mar_12_16_35_25_PDT_2019_node_local/diagcollect_20190312163524_node1.log

2019/03/12 16:35:30 PDT : NOTE : Any file or directory name containing the string .com will be renamed to replace .com with dotcom
2019/03/12 16:35:30 PDT : Collection Name : tfa_srdc_ora4030_Tue_Mar_12_16_35_25_PDT_2019.zip
2019/03/12 16:35:30 PDT : Scanning of files for Collection in progress...
2019/03/12 16:35:30 PDT : Collecting additional diagnostic information...
2019/03/12 16:35:35 PDT : Getting list of files satisfying time range [03/12/2019 14:08:20 PDT, 03/12/2019 16:35:30 PDT]
2019/03/12 16:35:49 PDT : Collecting ADR incident files...
2019/03/12 16:35:52 PDT : Completed collection of additional diagnostic information...
2019/03/12 16:35:54 PDT : Completed Local Collection

.-------------------------------------.
|          Collection Summary         |
+----------+-----------+-------+------+
| Host     | Status    | Size  | Time |
+----------+-----------+-------+------+
| node1    | Completed | 2.9MB | 24s |
'----------+-----------+-------+------'

Logs are being collected to:

/scratch/app/product/19c/tfa/repository/srdc_ora4030_collection_Tue_Mar_12_16_35_25_PDT_2019_node_local

/scratch/app/product/19c/tfa/repository/srdc_ora4030_collection_Tue_Mar_12_16_35_25_PDT_2019_node_local/node1.tfa_srdc_ora4030_Tue_Mar_12_16_35_25_PDT_2019.zip

If the user does not have the required permissions to run the SRDC, TFA reports an error.

For example:

User root does not have permissions to run SRDC 'ora4030' for database 'orcl'.

Collecting for a Specific Time Range

For more targeted diagnostic collections, specify the required time range using the collection-period options.

For example:

tfactl diagcollect -last 6h

You can also use the -from, -to, and -for options to define the collection period.

See Adjusting the Diagnostic Data Collection Period for supported syntax and date formats.

Excluding Large Files from Diagnostic Collection

Prevent large files from delaying or stalling diagnostic collections by configuring the maximum file size that Oracle Trace File Analyzer (TFA) collects.

When a file exceeds the configured maxfilecollectionsize value, TFA collects only the last 200 KB of the file.

  1. Set the maximum file size in MB:

    tfactl set maxfilecollectionsize=<size_in_MB>
  2. Run the diagnostic collection:

    tfactl diagcollect

Collecting from Specific Nodes

You can limit a diagnostic collection to one or more specific nodes in the cluster.

Use the following syntax:

tfactl diagcollect -node <list_of_nodes>

For example, to collect diagnostic data from myserver65 for the previous day:

tfactl diagcollect -last 1d -node myserver65

Collecting from Specific Components

You can limit a diagnostic collection to one or more specific components.

Use the following syntax:

tfactl diagcollect <component>

For example, to trim and collect files from the hrdb and fdb databases that were updated during the previous day:

tfactl diagcollect -database hrdb,fdb -last 1d

To trim and collect Oracle Clusterware files, operating system logs, and CHMOS/OSW data from node1 and node2 that were updated during the previous six hours:

tfactl diagcollect -crs -os -node node1,node2 -last 6h

To trim and collect Oracle ASM logs from node1 that were updated during a specified time range:

tfactl diagcollect -asm -node node1 -from "2016-08-15" -to "2016-08-17"

The following component options are available:

Component OptionDescription
-chaCollects Oracle Cluster Health Advisor logs.
-ipsCollects Incident Packaging Service (IPS) logs.
-database <database_names>Collects database logs from the databases specified in a comma-separated list.
-asmCollects Oracle ASM logs.
-crsclientCollects client logs under GIBASE/diag/clients.
-dbclientCollects client logs under DB ORABASE/diag/clients.
-dbwlmCollects Database Workload Management (DBWLM) logs.
-tnsCollects TNS logs.
-rhpCollects Rapid Home Provisioning (RHP) logs.
-procinfoCollects process stack and file descriptor (fd) information from /proc for all processes.
-afdCollects Oracle ASM Filter Driver (AFD) logs.
-crsCollects Oracle Clusterware logs.
-wlsCollects Oracle WebLogic Server (WLS) logs.
-emagentCollects Oracle Enterprise Manager Agent (EMAGENT) logs.
-omsCollects Oracle Management Service (OMS) logs.
-ocmCollects Oracle Configuration Manager (OCM) logs.
-empluginsCollects Oracle Enterprise Manager Plug-in (EMPLUGINS) logs.
-emCollects Oracle Enterprise Manager (EM) logs.
-acfsCollects Oracle Advanced Cluster File System (Oracle ACFS) diagnostic information.
-installCollects Oracle installation-related files.
-cfgtoolsCollects configuration tools logs.
-osCollects operating system files, such as /var/log/messages. On Exadata systems, this option also collects /var/log/exadatatmp/*netdiag* files.
-ashhtmlCollects an Active Session History (ASH) HTML report.
-ashtextCollects an Active Session History (ASH) text report.
-awrhtmlCollects an Automatic Workload Repository (AWR) HTML report.
-awrtextCollects an Automatic Workload Repository (AWR) text report.
-avsCollects Audit Vault Server logs.

Collecting from Specific Directories

Oracle Trace File Analyzer (TFA) automatically discovers Oracle diagnostic files and collects relevant files based on their file type and last modification time.

You can also specify additional directories when you need to collect files that are outside the standard diagnostic locations. By default, TFA collects only files updated during the relevant collection period, which is one hour unless you specify a different period.

You can configure a directory to collect all files regardless of when they were last updated by using the -collectall option.

To collect diagnostic data from specific directories:

  1. Collect files from additional directories that were updated during the previous one hour:

    tfactl diagcollect -collectdir <dir1,dir2,...,dirn>

    For example, to collect Oracle Clusterware files updated during the previous one hour together with files from /tmp_dir1 and /tmp_dir2 on the initiating node:

    tfactl diagcollect -crs -collectdir /tmp_dir1,/tmpdir_2
  2. Configure a directory to collect all files regardless of their last modification time:

    tfactl add <dir> -collectall

    Alternatively, modify an existing directory configuration:

    tfactl modify <dir> -collectall
  3. Start a diagnostic collection using the -collectalldirs option:

    tfactl diagcollect -collectalldirs

    Note:

    If -collectalldirs is not specified, the normal file type, file name, and time-range restrictions continue to apply.

Changing the Collection Name

Oracle Trace File Analyzer (TFA) packages diagnostic collections as ZIP files and stores them in the repository using the following naming format:

<repository>/collection_<date_time>/node_all/<node>.tfa_<date_time>.zip

Use the TFA collection options to customize the collection or ZIP file name. Do not manually rename generated ZIP files because doing so can prevent the collections from being used with Oracle Support self-service tools.

To customize the collection name:

  1. Use the -tag option to organize a collection under a custom directory name:
    -tag <tagname>

    TFA stores the collected files in the <tagname> directory within the repository.

  2. Use the -z option to specify a custom name for the ZIP file:

    -z <zip_name>

Preventing Copying ZIP Files and Trimming Files

By default, Oracle Trace File Analyzer (TFA) Collector:

  • Copies ZIP files from remote nodes to the node where the diagnostic collection was initiated.
  • Trims files to include data around the relevant collection time.

You can disable either behavior when required.

  1. To prevent ZIP files from being copied from remote nodes to the initiating node, use the -nocopy option:

    -nocopy

    For example:

    tfactl diagcollect -last 1d -nocopy
  2. To collect files without trimming them, use the -notrim option:

    -notrim

    For example:

    tfactl diagcollect -last 1d -notrim

Performing Silent Collection

Use the -silent option to run a diagnostic collection as a background process.

To initiate a silent collection:

-silent

For example, to collect diagnostic data for the previous day in silent mode:

tfactl diagcollect -last 1d -silent

Collecting Core Files

Use the -cores option to include core files in a diagnostic collection.

To collect core files:

-cores

For example, to collect diagnostic data and core files from the previous day:

tfactl diagcollect -last 1d -cores

Collecting Incident Packaging Service (IPS) Packages

Incident Packaging Service (IPS) packages diagnostic information about Oracle Database problems stored in the Automatic Diagnostic Repository (ADR) for subsequent analysis.

Oracle Trace File Analyzer (TFA) uses IPS to query and collect these diagnostic packages.

Syntax

tfactl ips <option>

The following commands and options are available:

CommandDescription
tfactl ipsRuns IPS.
tfactl ips show incidentsDisplays all IPS incidents.
tfactl ips show problemsDisplays all IPS problems.
tfactl ips show packageDisplays all IPS packages.
tfactl diagcollect -ips -hDisplays all available IPS options for diagcollect.
tfactl diagcollect -ipsPerforms an IPS collection and prompts for the required information. You can use standard diagcollect options to limit the scope of the IPS collection.
tfactl diagcollect -ips -adrbasepath <adr_base> -adrhomepath <adr_home>Performs an IPS collection in silent mode for the specified ADR base and ADR home.
tfactl diagcollect -ips -incident <incident_id>Collects ADR diagnostic information for a specific incident ID.
tfactl diagcollect -ips -problem <problem_id>Collects ADR diagnostic information for a specific problem ID.

You can modify the contents of an IPS package before completing the diagnostic collection.

To modify an IPS package:

  1. Start the IPS collection and suspend it using the -manageips option.

    For example:

    tfactl diagcollect -ips -incident <incident_id> -manageips -node local
  2. Find the suspended IPS collection:

    tfactl print suspendedips
  3. Modify the IPS package as required.
  4. Resume the suspended collection using the -resumeips option:

    tfactl diagcollect -resumeips <collection_id>