Set Up Continuous Log Collection From Your Hosts
To continuously collect log data from your entities, install the Management Agent on your host. Before that, ensure that you have completed the prerequisite tasks for using the Management Agents.
Topics:
Additionally:
For end-to-end steps to set up continuous log collection for the supported sources, see:
Allow Continuous Log Collection Using Management Agents
When you perform the prerequisites for deploying Management Agents in the step Install Management Agents, you will create the required compartment, user group for Logging Analytics users, and create IAM policies to install the Management Agents. As part of the prerequisites, ensure that the following policies are created for your user group:
ALLOW GROUP Logging-Analytics-User-Group TO MANAGE management-agents IN COMPARTMENT <compartment_name>
ALLOW GROUP Logging-Analytics-User-Group to MANAGE management-agent-install-keys IN TENANCY
ALLOW GROUP Logging-Analytics-User-Group TO READ METRICS IN COMPARTMENT <compartment_name>
ALLOW GROUP Logging-Analytics-User-Group TO READ USERS IN TENANCY
In the above example policy statements, Logging-Analytics-User-Group
is
an example user group.
Also, create a dynamic group for the Management Agents if it already doesn't
exist, for example Management-Agent-Dynamic-Group
:
ALL {resource.type='managementagent', resource.compartment.id='<management_agent_compartment_OCID>'}
Create IAM policies for Management-Agent-Dynamic-Group
to
enable log collection and metrics generation:
ALLOW DYNAMIC-GROUP Management-Agent-Dynamic-Group TO USE METRICS IN TENANCY
ALLOW DYNAMIC-GROUP Management-Agent-Dynamic-Group TO {LOG_ANALYTICS_LOG_GROUP_UPLOAD_LOGS} IN TENANCY
Grant READ Access of the Logs to the Agent User on Your Host
While deploying the management agents for using Oracle Logging Analytics on UNIX-based hosts, ensure that the management agent has the correct privileges to read the log files from where data has to be collected.
On Unix-based hosts, the user that installs management agent is mgmt_agent for the manually installed management agent, and oracle-cloud-agent when the management agent is a plugin enabled with Oracle Cloud Agent.
Check the file permissions for the log files with the management agent user:
sudo -u <agentuser> /bin/bash -c "cat <log file with complete path>"
If the management agent user cannot read the log files, then use one of the following ways (in the order of best practice) to make the log files readable to the management agent:
-
Use Access Control Lists (ACLs) to enable the cloud agent user to read the log file path and log files. An ACL provides a flexible permission mechanism for file systems. Ensure that the full path to the log files is readable through the ACL.
To set up an ACL in a UNIX-based host:
Determine whether the system that contains the log files has the
acl
package:rpm -q acl
If the system contains the
acl
package, then the previous command should return:acl-2.2.39-8.el5
If the system doesn’t have the
acl
package, then download and install the package. -
-
Grant the management agent user READ access to the required log file:
setfacl -m u:<agentuser>:r <path to the log file/log file name>
-
Grant the READ and EXECUTE permissions to each folder in the log file path:
//set read, execute permissions on folders other than parent folder setfacl -m u:<agentuser>:rx <path to the folder> //set read, execute permissions with recursive options on parent folder setfacl -R -m u:<agentuser>:rx <path to the folder> //set read, execute permissions with default option to allow all future log files created under this folder to be readable. setfacl -d -m u:<agentuser>:rx <path to the folder>
For example, the following commands are needed for the path
/scratch/logs/*.log
for the management agent usermgmt_agent
:setfacl -m u:mgmt_agent:rx /scratch setfacl -R -m u:mgmt_agent:rx /scratch/logs setfacl -d -m u:mgmt_agent:rx /scratch/logs
For nfs mount, it may not be possible to give READ and EXECUTE permission to the agent user to read the log files or folders. In such cases, add the agent user to the log file group:
usermod -a -G <group of log file> <agentuser>
Restart the management agent after running the above command.
-
-
Place the management agent and the product that generates the logs in the same user group, and make the files readable to the entire group. Restart the agent.
-
Make the log files readable to all users. For example,
chmod o+r <file>
.You may have to give executable permission to the parent folders. For example,
chmod o+rx <parent folder>
.
Install Management Agents
See Oracle Management Agents Documentation to complete the following tasks:
-
Perform prerequisites for deploying Management Agents
-
Install Management Agent
After you install the Management Agent, complete the following Logging Analytics specific tasks to start the log collection:
-
Map your entities to your agent: Create your entities and select the Management Agent that was installed to associate the agent with this entity. See Create an Entity to Represent Your Log-Emitting Resource. You can also edit an existing entity and add the agent.
-
Configure source-entity association. You can use the Add Data wizard to perform this task. For step-by-step help to complete the task, see OCI Logging Analytics: Set Up Continuous Log Collection (
Tutorial ).
The management agent connects to the following endpoints for Oracle Logging Analytics operations:
- Upload the logs and log collection warning:
https://loganalytics.<region>.oci.oraclecloud.com/<additional_part_pertaining_to_the_operation>
- Metrics:
https://telemetry-ingestion.<region>.oraclecloud.com/<additional_part_pertaining_to_the_operation>
In the above endpoints, region
is the identifier
for your region, for example, us-ashburn-1
.
Ingest Application, Infrastructure, Database and Other Generic Logs
Create the File type of log source to collect logs from your applications, infrastructure, databases, or most other type types of logs.
Oracle Logging Analytics provides a large set of Oracle-defined log sources of the source type File. You can view them in the sources listing page by filtering the sources of creation type Oracle-defined, and source type File.
Overall Flow for Collecting
Logs for File
Source Type
The following are the high-level tasks for collecting log information from your host:
-
Install Management Agents on your log-emitting hosts. See Set Up Continuous Log Collection From Your Hosts.
-
Create the entity. See Create an Entity to Represent Your Log-Emitting Resource.
- Identify a log source from the existing set of sources, both Oracle-defined and user-defined. If the existing source is not suitable for your requirement, then create a source. See Create a Log Source of Type: File.
-
Associate the entities with the source that you created earlier. See Configure New Source-Entity Association.
After the association is complete, the logs start flowing into Oracle Logging Analytics.
-
View log data in the Log Explorer by selecting the Windows Event source that you created earlier. See Filter Logs by Source Attributes.
Set Up Syslog Monitoring
Syslog is a commonly used standard for logging the system event messages. The destination of these messages can include the system console, files, remote syslog servers, or relays.
Overview
Oracle Logging Analytics allows you to collect and analyze syslog data from various sources. You just need to configure the syslog output ports in the syslog servers. Oracle Logging Analytics monitors those output ports, accesses the remote syslog contents, and performs the analysis.
Syslog monitoring in Oracle Logging Analytics lets you listen to multiple hosts and ports. The protocols supported are TCP and UDP.
Overall Flow for Collecting Syslog Logs
The following are the high-level tasks for collecting log information from your host:
-
Install Management Agents on your syslog listener. See Set Up Continuous Log Collection From Your Hosts.
The syslog listener is configured to receive the syslog logs from instances that might not be running on the same host. However, the agent that's installed on the syslog listener host collects those logs for which the listener is configured to collect.
-
Create the syslog entity. See Create an Entity to Represent Your Log-Emitting Resource.
-
Associate the syslog entity with the source. See Configure New Source-Entity Association.
Create the Syslog Source
Oracle Logging Analytics already provides several Oracle-defined log sources for syslog collection. Check if you can one of the available Oracle-defined syslog sources and Oracle-defined parsers. If not, use the following steps to create a new log source:
-
Open the navigation menu and click Observability & Management. Under Logging Analytics, click Administration. The Administration Overview page opens.
The administration resources are listed in the left hand navigation pane under Resources. Click Sources.
-
The Sources page opens. Click Create Source.
This displays the Create Source dialog box.
-
In the Name field, enter the name for the log source.
-
From the Source Type list, select Syslog Listener.
-
Click Entity Type and select one of the variants of Host such as
Host (Linux)
,Host (Windows)
,Host (AIX)
, orHost (Solaris)
as your entity type. This is the host on which the agent is running and collecting the logs. The syslog listener is configured to receive the syslog logs from instances that might not be running on the same host. However, the agent that's installed on the syslog listener host collects those logs for which the listener is configured to collect.Note
-
It is recommended that a maximum of 50 senders are sent to a single management agent or syslog. To have more senders, use more management agents.
-
You must have at least 50 file handles configured per sender in the operating system to handle all the possible incoming connections that the senders may open. This is in addition to the file handles needed on the operating system for other purposes.
-
-
Click Parser and select a suitable parser.
Typically, one of the variant parsers such as
Syslog Standard Format
orSyslog RFC5424 Format
is used. You can also select from the Oracle-defined syslog parsers for specific network devices. -
In the Listener Port tab, click Add to specify the details of the listener to which Oracle Logging Analytics will listen to collect the logs.
Enter the listener port that you specified as the output port in the syslog configuration file in the syslog server, and select either UDP or TCP (recommended for heavy traffic) as the required protocol. Ensure that the Enabled check box is selected.
Repeat this step for adding multiple listener ports.
The following listener ports are used in the Oracle-defined Syslog log sources:
Oracle-defined Syslog Source Listener Port Palo Alto Syslog Logs
8500
Symantec Endpoint Protection Syslog Listener Logs
8501
Symantec DLP Syslog Listener Logs
8502
Cisco Syslog Listener Source
8503
QRadar LEEF Syslog Listener Source
8504
F5 Big IP Logs
8505
Juniper SRX Syslog Logs
8506
Citrix NetScaler Logs
8507
NetApp Syslog Logs
8508
Fortinet Syslog Logs
8509
ArcSight CEF Syslog Source
8510
Check Point Firewall LEA Syslog Logs
8511
Palo Alto Syslog CEF Logs
8512
TrendMicro Syslog Common Event Format Logs
8513
Symantec Endpoint Protection System Syslog Logs
8514
F5 Big IP ASM WAF Syslog CEF Logs
8516
CyberArk Syslog Common Event Format Logs
8517
Squid Proxy Syslog Listener Source
8518
-
Click Create Source.
View Syslog Data
You can use the Log Source field in the Fields panel of the Log Explorer in Oracle Logging Analytics to view syslog data.
- In the Oracle Logging Analytics Log Explorer, click Source in the Fields panel.
- In the Filter by Source dialog box, select name of the syslog source that you created, and click Apply.
Set Up Database Instance Monitoring
Oracle Logging Analytics can extract database instance records based on the SQL query that you provide in the log source configuration.
Overall Flow for Collecting Database Logs
The following are the high-level tasks for collecting log information stored in a database:
-
Install Management Agent on your database instance. See Set Up Continuous Log Collection From Your Hosts.
-
Associate the database entity with the source. See Configure New Source-Entity Association.
Oracle Database
Oracle Database includes
- Pluggable Database (PDB), Multitenant Container Database (CDB), and Application Container
- Oracle Database Instance
- Oracle Autonomous Database
- Autonomous Data Warehouse (ADW)
- Autonomous Transaction Processing (ATP)
For an example of how to collect logs from tables or views in Oracle Autonomous Database, see Collect Logs from Tables or Views in Oracle Autonomous Database (
Tutorial ).
Oracle Logging Analytics provides a large set of Oracle-defined log sources of the type Database for Oracle Database:
Log Source | Entity Type |
---|---|
AVDF Alert in Oracle Database |
Oracle Database Instance |
AVDF Event in Oracle Database |
Oracle Database Instance |
Identity and Access Management Audit Database |
Oracle Database Instance |
Oracle DB Audit Log Source Stored in Database |
Oracle Database Instance |
Oracle EBS Transaction Logs |
Oracle Pluggable Database, Oracle Database Instance |
Symantec DLP System Events |
Oracle Database Instance |
Oracle Unified DB Audit Log Source Stored in Database 12.1 |
Oracle Pluggable Database, Oracle Database Instance |
Oracle Unified DB Audit Log Source Stored in Database 12.2 |
Oracle Pluggable Database, Autonomous Data Warehouse, Oracle Database Instance, Autonomous Transaction Processing |
Additionally, more oracle-defined log sources of the type File are available for Oracle Database such as Database Alert Logs, Database Audit Logs, Database Audit XML Logs, Database Incident Dump Files, Database Listener Alert Logs, Database Listener Trace Logs, Database Trace Logs, and Database XML Alert Logs.
Microsoft SQL Server Database Instance
- For successful log collection from Microsoft SQL Server Database source, ensure that Management Agent version is 210403.1350 or later.
- Monitoring of Microsoft SQL Server Database Instance is supported only with the installation of standalone Management Agent. It is not supported with Management Agent plugin in Oracle Cloud Agent.
The following Oracle-defined log sources of the type Database are available for monitoring Microsoft SQL Server Database Instance:
- McAfee Data Loss Prevention Endpoint
- McAfee ePolicy Orchestrator
Additionally, more oracle-defined log sources of the type File are available for Microsoft SQL Server Database Instance such as Microsoft SQL Server Agent Error Log and Microsoft SQL Server Error Log Sources.
MySQL Database Instance
- For successful log collection from MySQL Database source, ensure that Management Agent version is 210205.0202 or later.
- Monitoring of MySQL Database Instance is supported only with the installation of standalone Management Agent. It is not supported with Management Agent plugin in Oracle Cloud Agent.
The following Oracle-defined log sources of the type Database are available for monitoring MySQL Database Instance:
- MySQL Error Logs Stored in Database
- MySQL General Log Source Stored in Database
- MySQL Slow Query Logs Stored in Database
Additionally, more oracle-defined log sources of the type File are available for MySQL Database Instance such as MySQL Database Audit XML Logs, MySQL Error Logs, MySQL General Query Logs, and MySQL Slow Query Logs.
To perform remote collection for a MySQL database instance, the following configuration must be done at the database instance:
-
To allow access from a specific host where the management agent is installed:
-
Create the new account authenticated by the specified password:
CREATE USER '<mysql_user>'@'<host_name>' IDENTIFIED BY '<password>';
-
Assign READ privileges for all the databases to the
mysql_user
user on hosthost_name
:GRANT SELECT ON *.* TO '<mysql_user>'@'<host_name>' WITH GRANT OPTION;
-
Save the updates to the user privileges by issuing the command:
FLUSH PRIVILEGES;
-
-
To allow access to a specific database from any host:
-
Grant READ privileges to
mysql_user
from any valid host:GRANT SELECT ON <database_name>.* TO '<mysql_user>'@'%' WITH GRANT OPTION;
-
Save the updates to the user privileges by issuing the command:
FLUSH PRIVILEGES;
-
Create the Database Entity
Create the database entity to reference your database instance and to enable log collection from it. If you are using management agent to collect logs, then after you install the management agent, you must come back here to configure the agent monitoring for the entity.
-
Open the navigation menu and click Observability & Management. Under Logging Analytics, click Administration. The Administration Overview page opens.
-
The administration resources are listed in the left hand navigation pane under Resources. Click Entities.
-
Ensure that your compartment selector on the left indicates that you are in the desired compartment for this new entity.
Click Create.
-
Select an Entity Type that suits your database instance, for example Oracle Database Instance.
Provide a Name for the entity.
-
Select Management Agent Compartment in which the agent is installed and select the Management Agent to associate with the database entity so that the logs can be collected.
Alternatively, you can create the entity first, edit it later and provide the management agent OCID after the agent is installed.
Note
-
Monitoring of MySQL Database Instance and Microsoft SQL Server Database is supported only with the installation of standalone Management Agent. It is not supported with Management Agent plugin in Oracle Cloud Agent.
-
Use Management Agent version 210403.1350 or later to install on your database host to ensure Microsoft SQL Server Database support.
-
For successful log collection from MySQL Database Instance source, ensure that Management Agent version is 210205.0202 or later.
-
-
To ingest SQL, provide the following properties in case of Oracle Database Instance or Oracle Pluggable Database:
port
hostname
sid
orservice_name
If you provide both the values, then Logging Analytics uses
service_name
to ingest SQL.
For log collection from Microsoft SQL Server Database Instance and MySQL Database source, provide the following properties:
database_name
host_name
port
If you intend to use Oracle-defined log sources to collect logs from management agents, it is recommended that you provide any parameter values that may already be defined for the chosen entity type. If the parameter values are not provided, then when you try to associate the source to this entity, it will fail because of the missing parameter values.
Click Save.
Create the Database Source
-
Open the navigation menu and click Observability & Management. Under Logging Analytics, click Administration. The Administration Overview page opens.
-
The administration resources are listed in the left hand navigation pane under Resources. Click Sources.
-
In the Sources page, click Create Source.
This displays the Create Source dialog box.
-
In the Source field, enter the name for the source.
-
From the Source Type list, select Database.
-
Click Entity Type and select the required entity type. For example, Oracle Database Instance, Oracle Pluggable Database, Microsoft SQL Server Database Instance, or MySQL Database Instance.
-
In the Database Queries tab, click Add to specify the details of the SQL query, based on which Oracle Logging Analytics instance collects database instance logs.
See SQL Query Guidelines.
-
Click Configure to display the Configure Column Mapping dialog box.
-
In the Configure Column Mapping dialog box, map the SQL fields with the field names that would be displayed in the actual log records. To create a new field for mapping, click the
icon.
Specify a Sequence Column. The value of this field must determine the sequence of the records inserted into the table. It must have unique incremental value. If you don't want the fields to determine the sequence of the records, then you can select SQL query collection time to use the collection time as the log entry time. In that case, all the log records are re-collected in every collection cycle.
Note
The first mapped field with a data type of
Timestamp
is used as the time stamp of the log record. If no such field is present, then the collection time is used as the time of the log record.When the logs are collected for the first time after you created the log source (historic log collection):
-
If any field in the SQL query is mapped to the
Time
field , then the value of that field is used as reference to upload the log records from previous 30 days. -
If none of the fields in the SQL query are mapped to the
Time
field, then a maximum of 10,000,000 records are uploaded.
Click Done.
-
-
Repeat Step 6 through Step 8 for adding multiple SQL queries.
-
Select Enabled for each of the SQL queries and then click Save.
Provide the Database Entity Credentials
-
Log in to the host on which the management agent is installed.
-
Create the
DBCreds
type credentials JSON input file. For exampleagent_dbcreds.json
:cat agent_dbcreds.json { "source": "lacollector.la_database_sql", "name": "LCAgentDBCreds.<entity_name>", "type": "DBCreds", "usage": "LOGANALYTICS", "disabled": "false", "properties": [ { "name": "DBUserName", "value": "CLEAR[username]" }, { "name": "DBPassword", "value": "CLEAR[password]" }, { "name": "DBRole", "value": "CLEAR[normal]" } ] }
The following properties must be provided in the input file as in the above example
agent_dbcreds.json
:- source : "lacollector.la_database_sql"
- name :
"LCAgentDBCreds.<entity_name>"
entity_name
is the value of the Name field that you entered while creating the entity. - type : "DBCreds"
- usage : "LOGANALYTICS"
- properties : user name, password and role. Role is optional.
-
Use the
credential_mgmt.sh
script with theupsertCredentials
operation to add the credentials to the agent's credential store:Syntax:
$cat <input_file> | sudo -u mgmt_agent /opt/oracle/mgmt_agent/agent_inst/bin/credential_mgmt.sh -o upsertCredentials -s <service_name>
In the above command:
- Input file: The input JSON file with the credential parameters, for
example,
agent_dbcreds.json
. - Service name: Use
logan
as the name of the Oracle Logging Analytics plug-in deployed on the agent.
By using the example values of the two parameters, the command would be:
$cat agent_dbcreds.json | sudo -u mgmt_agent /opt/oracle/mgmt_agent/agent_inst/bin/credential_mgmt.sh -o upsertCredentials -s logan
After the credentials are successfully added, you can delete the input JSON file.
For more information about managing credentials on the management agent credential store, see Management Agent Source Credentials in Management Agent Documentation.
- Input file: The input JSON file with the credential parameters, for
example,
View Your Database Entity in Database Management Service
If your database is enabled for Database Management and has a cloud resource OCID associated with it, then Logging Analytics enables you to view it in Database Management with the help of the option available in the Log Explorer.
To enable Database Management for your database, see OCI Documentation: Enable Database Management.
-
Open the navigation menu and click Observability & Management. Under Logging Analytics, click Log Explorer.
-
Search for your logs by entity type which must be one of the database types. In the Fields panel, under Pinned section, click Entity Type. In the Entity Type dialog box, select the required entity types, for example,
Oracle Database Instance
, and click Apply. -
From the Visualize panel, select one of the visualization options that display the records table, for example, Records with Histogram.
Then the logs are filtered by the entity type and displayed in the Records with Histogram visualization. In the records table, under each log record, the information about the entity name, log source, and entity type are displayed.
-
Click the name of the entity. From the menu, click View in Database Management.
A new tab with the Database Management service console in the context of your database is displayed.
Set Up Windows Event Monitoring
Windows event log is generated by Windows operating system to record the events related to OS operations, file access, user access, and applications running on it. These event logs can provide insights about security and application performance and issues.
The types of events logged in the Windows Event logs are broadly classified as below:
-
Application: Errors and events related to the application installed on the Windows instance.
-
Security: File and user access events. These are recorded through Windows auditing.
-
Setup: Installation related events.
-
System: Record of events related to Windows OS system and its components.
Oracle Logging Analytics provides Oracle-defined log sources to match the Windows event classification to be able to process all kinds of collected data:
-
Windows Application Events
-
Windows Security Events
-
Windows Setup Events
-
Windows System Events
Oracle Logging Analytics can collect all historic Windows Event Log entries and supports Windows as well as other custom event channels.
Overall Flow for Collecting Windows Event Logs
The following are the high-level tasks for collecting log information from your host:
-
Install Management Agents on your Windows hosts. See Set Up Continuous Log Collection From Your Hosts.
-
Create the Windows entity. See Create an Entity to Represent Your Log-Emitting Resource.
- Identify a log source from the existing set of sources, both Oracle-defined and user-defined. If the existing source is not suitable for your requirement, then create a source. See Create a Windows Event Source.
-
Associate the entities with the source that you created earlier. See Configure New Source-Entity Association.
After the association is complete, the logs start flowing into Oracle Logging Analytics.
-
View log data in the Log Explorer by selecting the Windows Event source that you created earlier. See Filter Logs by Source Attributes.
Create a Windows Event Source
Oracle Logging Analytics already provides several Oracle-defined log sources for Windows Event collection.
Oracle Logging Analytics already provides several Oracle-defined log sources for syslog collection. Check if you can use one of the available Oracle-defined or user-defined sources. If not, use the following steps to create a new log source:
-
Open the navigation menu and click Observability & Management. Under Logging Analytics, click Administration. The Administration Overview page opens.
The administration resources are listed in the left hand navigation pane under Resources. Click Sources.
The Sources page opens. Click Create Source.
-
In the Name field, enter the name of the source.
Optionally, add a description.
-
From the Source Type list, select Microsoft Windows. With this option, all historic Windows Event Log entries as well as records from custom event channels can be collected.
This source type does not require the field Log Parser. Also, the default entity type
Host (Windows)
is automatically selected, and cannot be changed. -
Specify an event service channel name. The channel name must match with the name of the Windows event so that the agent can form the association to pick up logs.
-
To filter the Windows events with specific event IDs, add Data Filters. See Create Data Filters for Windows Event System Source.
-
Click Create Source.
Create Data Filters for Windows Event System Source
To filter the Windows events with specific event IDs for Windows
Event System
source, follow these steps:
-
On the host where you installed the Management Agent, go to the location
agent_inst\state\laStorage\os_wineventsystem\config
. If the directory or path does not exist, create it. -
Create a user properties file
dataFilter.properties
. To this file, add all the Windows event IDs in the following format:<channel>.dropEvent.eventID=<id1> <channel>.dropEvent.eventID=<id2> <channel>.dropEvent.eventID=<id3>
For example,
Security.dropEvent.eventID=4624 Security.dropEvent.eventID=4672 Security.dropEvent.eventID=4673
-
Restart the Management Agent.
Ingest Logs of Oracle Diagnostic Logging (ODL) Format
Oracle Diagnostic Logging (ODL) is an industry-wide accepted format for writing diagnostic messages to log files. The log file can be in ODL text format or ODL XML format. Most Oracle Fusion Middleware components, Oracle Enterprise Performance Management System products, and other Oracle applications write diagnostic log files in the ODL format.
Oracle Logging Analytics provides Oracle-defined log sources to match the ODL format to be able to support several Oracle applications:
Oracle-defined Source | Entity type |
---|---|
FMW OHS Diagnostic Logs (V11) |
Oracle HTTP Server |
FMW OHS Diagnostic Logs (V12) |
|
FMW OID Directory Control Logs |
Oracle Internet Directory |
FMW OID Directory Dispatcher Server Logs |
|
FMW OID Directory Replication Server Logs |
|
FMW OID Directory Server Logs |
|
FMW OID Monitor Logs |
|
Fusion Apps Diagnostic Logs |
WebLogic Server |
FMW BI Publisher Logs |
|
FMW BI JBIPS Logs |
|
FMW WLS Server Diagnostic Logs |
|
Oracle VM Manager Diagnostic Logs |
Oracle VM Manager |
Overall Flow for Collecting ODL Logs
The following are the high-level tasks for collecting log information from your host:
-
Install Management Agents on your log-emitting hosts. See Set Up Continuous Log Collection From Your Hosts.
-
Create the entity. See Create an Entity to Represent Your Log-Emitting Resource.
- Identify a log source from the existing set of sources, both Oracle-defined and user-defined. If the existing source is not suitable for your requirement, then create a source. See Create a Source for Diagnostic Logs in ODL Format.
-
Associate the entities with the source that you created earlier. See Configure New Source-Entity Association.
After the association is complete, the logs start flowing into Oracle Logging Analytics.
-
View log data in the Log Explorer by selecting the Windows Event source that you created earlier. See Filter Logs by Source Attributes.
View Agent Collection Warnings
Oracle Logging Analytics lets you view the warning messages generated during log collection using the management agent. This helps you to diagnose problems with the sources or entities and to take corrective action.
Following are the types of collection warning messages that are displayed:
-
Agent configured to monitor too many files
-
Authorization failed
-
Cannot open port
-
Cannot read file
-
Configuration issue
-
Connection identifier is empty
-
Credential can not be accessed
-
Credential corrupted
-
Credential is not enabled
-
Credential not found
-
Credential store not found
-
Database connection can not be established
-
File not found
-
Invalid sequence column
-
Large directory handling not enabled
-
Log upload request failed
-
Missing file permission
-
Too many historic files
-
SQL query execution error
Topics:
View Agent Collection Warnings Details
-
Open the navigation menu and click Observability & Management. Under Logging Analytics, click Administration. The Administration Overview page opens.
-
The administration resources are listed in the left hand navigation pane under Resources. Click Agent Collection Warnings.
The Agent Collection Warnings page opens. This displays the list of warnings generated while collecting logs on the agent side.
Use the multiple filters available in the left pane like Start Date, End Date, Entity Type, Source, Warning Message, and Warning State to narrow down your search for the warning messages.
The Start Date and End Date filters use the First Reported information of the warning message to help in filtering.
The Source Pattern that is displayed adjacent to the warning message is the one which is associated with the problem among the multiple patterns defined for that source.
Hover the cursor on the warning message to view more details about the warning.
-
Optionally, you can hide a warning if you want to temporarily ignore it and address it at a later point in time. Click Actions
icon > click Hide.
Alternatively, you can select multiple warnings and hide them using the Hide Warnings button. Use the Warning State filter in the left navigation pane to view the hidden warning messages. You can move the hidden warnings back to the active list by using the Actions
icon and clicking Unhide.
-
Additionally, in response to a warning, if you want to remove the association between its entity and source, then click Actions
icon > click Remove Association. The management agent stops collecting logs from that source and entity after removing the association. Then, the warning gets automatically cleared.
View Agent Collection Warnings in Entity Detail or Source Detail Page
-
Open the navigation menu and click Observability & Management. Under Logging Analytics, click Administration. The Administration Overview page opens.
-
The administration resources are listed in the left hand navigation pane under Resources.
Click the name of the <resource> whose warning information you want to view. The <resource> can be Entities or a Sources.
-
In case of Sources, the Sources page is displayed. Click the name of source whose warnings summary you want to view. The Source Detail page is displayed.
In case of Entities, the Entities page is displayed. Click the name of entity whose warnings summary you want to view. The Entity Detail page is displayed.
-
Click Agent Collection Warnings in the Resources section.
The warnings summary is displayed. If you are viewing the warnings for the source, then you can see the associated entity and the entity type in the warnings summary. If you are viewing the warnings for the entity, then you can see the associated source in the summary.
As in the case of Agent Collection Warnings page, you can hide or unhide the warnings, and remove association between the source and entity. For more information, see View Agent Collection Warnings Details.
Use the filters in the left navigation pane to narrow down your search for the warning messages.
Monitor Your Continuous Log Collection
After you complete the set up for continuous log collection, the Management Agent installed on your host emits information about the size of log data that it is uploading to Logging Analytics and errors encountered, if any.
This data is displayed for each log source with the following agent log collection metrics:
-
Agent Data Upload Size (
logCollectionUploadDataSize
): The size of the log data collected through the Management Agent for each log source. -
Agent Data Upload Errors (
logCollectionUploadFailureCount
): The count of errors occurred for each log source during the log collection and the type of errors.
To access the Agent Data Upload Size and Agent Data Upload Errors metrics, see Monitor Logging Analytics Using Service Metrics.
To modify the filters applied on the metrics data, you can view the metrics in the metrics explorer and change the metrics dimensions:
-
Click the Options menu on the top right corner of the agent log collection metric, and select View in Metric Explorer.
The metric is now displayed in the Metrics Explorer. Here, you can view the chart in finer detail.
-
Click Edit Queries and select Dimension Name and Dimension Value for the metric. For example, if you want to view the upload data size for a specific host
host123
, then select the metric namelogCollectionUploadDataSize
, dimension name asagentHostName
and the dimension value ashost123
.Click Update Chart to refresh the chart visualization. The chart will now display only the upload data size for the specified host.
Similarly, if you want to view the number of upload errors encountered of the type
LogGroupPolicyError
, then select the metric namelogCollectionUploadFailureCount
, dimension name aserrorCode
and the dimension value asLogGroupPolicyError
.Click Update Chart to refresh the chart visualization. The chart will now display the count of upload errors of the specified type for the specified period.
You can switch to the Data Table view for a tabular representation of the data points in the metrics.
Following are the dimensions available to filter the metric data:
Dimension | Metrics | Details |
---|---|---|
agentHostName |
logCollectionUploadDataSize ,
logCollectionUploadFailureCount |
The name of the host on which Management Agent is installed |
logGroup |
logCollectionUploadDataSize ,
logCollectionUploadFailureCount |
The log group in which the log collection happens |
logSourceType |
logCollectionUploadDataSize ,
logCollectionUploadFailureCount |
The log source type, which can be
|
resourceId |
logCollectionUploadDataSize ,
logCollectionUploadFailureCount |
The OCID of the Management Agent |
errorCode |
logCollectionUploadFailureCount |
The error reported by the Management Agent |
Following are the various types of errors reported by the Management Agent in
the logCollectionUploadFailureCount
metric for the dimension
errorCode
:
Error Type | Description | Recommended Fix |
---|---|---|
|
Occurs due to authorization failure during log upload. This is caused by incorrect IAM policies. HTTP status code: 404 |
Check the IAM policies you created for enabling continuous log collection and verify that the required permissions are given. See Allow Continuous Log Collection Using Management Agents. |
|
Occurs when the Management Agent sends request with incorrect parameters. HTTP status code: 400 |
Contact Oracle Support with the Error Type information. |
|
Occurs when the Management Agent sends request with incorrect signature. HTTP status code: 401 |
|
|
Occurs when the Management Agent sends request with a payload which is larger than expected. HTTP status code: 413 |
|
TooManyRequests |
Occurs when the Management Agent sends requests which are more in number than what is defined in the endpoint configuration. HTTP status code: 429 |
|
InternalError |
Occurs when an unexpected exception crops up in the Management Agent. HTTP status code: 500 |
|
HTTP Error Code <error code> |
All other unexpected error codes returned on the log upload endpoint. |
For the actions that you can perform with each metric, see Actions for Service Metrics.