Oracle Cloud Infrastructure Documentation

Managing Logs and Log Groups

This topic describes how to manage logs and log groups.

Overview of Logs and Log Groups

Logs contain critical diagnostic information that tells you how your resources are performing and being accessed. You can enable logging on supported resources. To see a list of supported resources grouped by service, see Supported Services.

Log groups are logical containers for organizing logs. Logs must always be inside log groups. You must create a log group to enable a log.

Use log groups to limit access to sensitive logs with IAM policy. With log groups, you don't have to rely on complex compartment hierarchies to secure your logs. For example, say the default log group in a single compartment is where you store logs for the entire tenancy. You grant access to the compartment for log administrators with IAM policy as you normally would. However, let's say some projects contain personally identifiable information (PII) and those logs can only be viewed by a select group of log administrators. Log groups allow you to put logs that contain PII into a separate log group, and then use IAM policy to restrict access so that only a selected set of log administrators have the elevated access.

Log groups IAM policy

Required IAM Policy

To use Oracle Cloud Infrastructure, you must be granted security access in a policy  by an administrator. This access is required whether you're using the Console or the REST API with an SDK, CLI, or other tool. If you get a message that you don’t have permission or are unauthorized, verify with your administrator what type of access you have and which compartment  to work in.

Administrators: for policy samples specific to logs and log groups, see Required Permissions for Working with Logs and Log Groups.

If you're new to policies, see Getting Started with Policies and Common Policies. If you want to know more about writing policies for Logging, see Details for Logging.

Required Permissions for Working with Logs and Log Groups

To enable service logs in a resource, a user must be granted manage access on the log group and access to the resource. In general, inspect access on the resource is enough, but check for specific resources. Inspect access provides permission to update the resource and permission for the log group that contains the log.

Logs and log groups use the log-group resource-type, but to search the contents of logs you must use a different resource-type.

Managing log groups and log objects

To manage groups or objects, use verbs for log-groups:

Allow group A to use log-groups in compartment C
Allow group B to manage log-groups in compartment C
Allow group D to read log-groups in compartment C

This allows users in group A to create, update, or delete log groups and log objects in compartment C.

To provision agent configurations

Three different types of access are needed:

  1. Access to operate on configurations.
  2. Access to operate on log groups.
  3. Inspect capabilities on dynamic groups or groups.
To have access to configurations, the policy must be:
Allow group B to use unified-configuration in compartment X
To create, update, or delete custom logs used as a destination in a configuration

This policy allows users in compartment B to create, update, or delete configurations in compartment X.

To provide a destination for the logs incoming from the configuration, you need log-groups access:
Allow group B to use log-groups in compartment X
To assign a configuration to a set of instances
To assign a configuration to a set of instances, you need inspect access to the dynamic group or group that identifies the instances:
Allow group B {IDENTITY_DYNAMIC_GROUP_INSPECT} in tenancy  / Allow group B {IDENTITY_GROUP_INSPECT} in tenancy
Enable instances to push logs into the Logging service
To allow instances to push logs, the instances need to have access to get a configuration and push the logs. log-content controls this permission (where X is the compartment where the configurations are located):
Allow dynamic-group production-fleet to use log-content in compartment X
To view logs
To view logs in the Console (Search), the following is required:
Allow group Searchers to read log-content in compartment X
Example log search policy

To allow a group to read the contents of indexed logs:

allow group GroupA to read log-groups in tenancy
allow group GroupA to read log-content in tenancy
Example policies for logs and log groups

In these examples, policy statements use GroupA as the name of the group.

To allow a group to view the log groups in the tenancy (or in a compartment), requires inspect access: 

allow group GroupA to inspect log-groups in tenancy

To allow a group to read metadata for logs or log groups, requires read access: 

allow group GroupA to read log-groups in tenancy

To allow a group to update log groups, or the logs in them, requires use access: 

allow group GroupA to use log-groups in tenancy

To enable a log on a resource (or to create and delete log groups and the logs in them), requires manage access: 

allow group GroupA to manage log-groups in tenancy

To allow usage of a specific log group or groups , use a where clause with the target.loggroup.id variable. For example:

Allow group GroupA to manage loggroups in tenancy where 
target.loggroup.id='ocid1.loggroup.oc1.phx.<uniqueID>'

To specify multiple log groups:

Allow group GroupA to manage log-groups in tenancy where any {target.loggroup.id='ocid1.loggroup'}
Custom logs

For custom logs the following is required. This policy is needed to allow the user to search logs through the Console Search page:

allow group userGroup1 to read log-content in compartment c
Note

Even though this policy is described for usage with custom logs, the policy is also true for all logs. LOG_CONTENT_READ allows reading logs from both custom and OCI service logs. It is identical in behavior to this policy:
allow group GroupA to read log-content in tenancy

The following is needed for the agent that uses the instance principal on the virtual machine to send logs:

allow dynamic-group dynamicgroup1 to use log-content in compartment c
Note

If a user group is being used instead of a dynamic group for pushing custom logs, replace the dynamic group name with the user group name in these policies.

For custom logs, if you use allow dynamic-group dynamicGroup1 to use log-content in compartment c, the instances in that dynamic group get permission to download the configuration, send logs, and can search logs.

IAM Policy Requirements for Resources

In addition to the permissions to work with the log group, to add service logs to a resource you must have the update permission for the resource. For many resources, the update permission is granted with the use verb. For example, users who can use buckets in CompartmentA, can also enable logging on a bucket in CompartmentA.

However, some resources don't include permission to update a resource with the use verb. For example, to update a rule for the Events service, you must have the full manage permission. To enable a log on an Events rule (or any other resource that doesn't include the update permission with the use verb), you must have the manage permission.

To allow a group to enable logging for these resources, without granting the full permissions of manage, you can add a policy statement to grant only the <RESOURCE>_UPDATE permission (or, in the case of the Events service, <RESOURCE>_MODIFY) from the manage verb. For example, to allow a group EventUsers to enable logs on Events rules in CompartmentA, you could write a policy like the following:

Allow group EventUsers to read cloudevents-rules in compartment CompartmentA
Allow group EventUsers to manage cloudevents-rules in compartment CompartmentA 
 where request.permission='EVENTRULE_MODIFY'

For information about resource permissions, see Policy Reference.

VCN Flow Logs IAM Policy

In addition to Required Permissions for Working with Logs and Log Groups, subnet read and update permissions are required for managing VCN Flow Logs.

To provide subnet permissions, use one of the following policies, listed in order from broader to narrowed privileges:

Allow group FlowLogsEnablers to manage virtual-network-family in tenancy

Or:

Allow group FlowLogsEnablers to manage subnets in tenancy

Or:

Allow group FlowLogsEnablers to {SUBNET_READ, SUBNET_UPDATE} in tenancy

This group is similar to what is described for EventUsers in IAM Policy Requirements for Resources.

Example Scenario

Your company has an Operations department. Within the Operations department are several costs centers. You want to be able to tag resources that belong to the Operations department with the appropriate cost center.

  1. Create a log group called "confidential". Avoid entering confidential information.
  2. Add logs with sensitive data to the "confidential" log group.

An employee named Alice already belongs to the group BucketManagers. Alice can manage buckets in CompartmentA. You want Alice and other members of BucketManagers group to be able to enable logs on buckets in CompartmentA.

To grant the BucketManagers group access to the sensitive data log group (and only the sensitive data log group), add the following statements to the BucketManagers policy: 

Allow group BucketManagers to manage log-groups in compartment CompartmentA where 
target.loggroups.id='ocid1.lumloggroup.oc1.phx.<uniqueID>'

Alice can now enable logs to bucket resources in CompartmentA.

Move Log Groups to a Different Compartment

You can move log groups from one compartment to another. When you move a log group to a new compartment, all the logs in the log group move with the log group to the new compartment. After you move the log group to the new compartment, the policies in the new compartment apply immediately, and affect access to the log group and any logs the log group contains.

For more information, see Moving Resources to a Different Compartment.

Log and Log Group Names

For log group names, the first character must start with a letter. Otherwise, the following guidelines apply to both log and log group names:

  • Use from 1 to 256 characters.
  • Valid characters are letters (upper or lower case), numbers, hyphens, underscores, and periods.
  • Log and log group names are case-sensitive. Logging handles write-log and WRITE-log as separate logs.
  • Avoid entering confidential information.

Legacy Archival

Legacy automatic log archival was previously available before general release. This is a toggle configurable for every log. If enabled, it automatically creates a bucket in your compartment, and places a copy of your log there. New and improved functionality is now available in Service Connector Hub.

Using the Console

The Logs table lists both custom logs and service logs (indicated by the Log type field). The table is organized in terms of the following fields:

  • Log name
  • Log type
  • Status
  • Details
  • Created

From this page you can click the Log name entry to go to the Log details page, or click the linked resource in Details to go directly to the resource. For example, if the log is for the Load Balancer service, clicking the link opens the Load Balancer Details page. From the action menu, you can edit the log, disable logging, change the log group, view tags, or delete the log.

To view the contents of logs
  1. Open the navigation menu and click Observability & Management. Under Logging, click Logs.
  2. Under List scope, Compartment, choose a compartment you have permission to work in.
  3. Under Log name, click the name of the log you want to view. The log detail page opens. This page displays the following on the Log Information tab:
    • OCID
    • Compartment
    • Log Group (click to view the log group contents)
    • Created date and time in UTC format
    • Retention Period
      Note

      Retention period can be set in 30-day increments, up to a maximum of 180 days.
    • Legacy Archival mode
    • Status (Creating, Active, Updating, Inactive, Deleting, Deleted)
    • The Tags tab shows associated tags for this log.
    • Under Log Details, following is displayed: the Log Type (whether Service or Custom), Service (for service logs), Log Category, and Resource (click to view the linked resource).

      In the Explore Log resource, log data is displayed in a similar manner as the Log Data on the Search page. You can apply some simple filters, such as sorting by newest or oldest from the Sort field, or filtering by time from the corresponding Filter by Time field.

  4. Click Explore with Log Search, which allows you to view this log on the Search page directly. After clicking this link, the Search page opens with the Select Logs to Search field populated with the log in the filter settings. At this point, you can perform more analysis and investigation related to this log directly on the Search page. For more information, see Searching Logs.

    In addition to these functions in the Explore Log resource, you can also click Metrics to view interactive charts for either a chosen time period (Start Time and End Time), or pre-selected ranges from Quick Selects. The charts display the Bytes Ingested (total bytes of log entries ingested) and the Search Success (number of successful search queries issued by the user).

    Clicking anywhere in a chart displays a larger version of the chart. You can perform several chart actions from the Options menu (both in the Metrics resource and in the zoomed-in view):
    • View Query in Metrics Explorer: Opens the chart in the Monitoring Metrics Explorer. See Exploring a Default Metric Chart for more information.
    • Copy Chart URL
    • Copy Query (MQL)
    • Create an Alarm on this Query: Opens the Monitoring Create Alarm page. See Creating an Alarm for more information.
    • Table View: Displays a tabular summary of the chart data. Select Chart View to switch back to the chart.

    For more information, see Logging Metrics.

To edit the name of a log
  1. Open the navigation menu and click Observability & Management. Under Logging, click Logs.
  2. Under List scope, Compartment, choose a compartment you have permission to work in.
  3. Under Log name, click the name of the log you want to view. The log detail page opens.
  4. Click Edit. The Edit Log panel is displayed. This panel indicates the resource, and allows you to change the log name in the associated field under Configure Log. You can also enable or disable the Enable Legacy Archival Logs setting (see Legacy Archival). Avoid entering confidential information.

    From the main Logs page, for the log you want to edit, you can also click the Actions menu, and then click Edit to access the Edit Log panel.

  5. Make your changes and click Save Changes.
To delete a log
  1. Open the navigation menu and click Observability & Management. Under Logging, click Logs.
  2. Under List scope, Compartment, choose a compartment you have permission to work in.
  3. Under Log name, click the name of the log you want to view. The log detail page opens.
  4. Click Delete. A confirmation dialog is displayed regarding the delete operation.
  5. Confirm by clicking Delete.

    From the main Logs page, for the log you want to delete, you can also click the Actions menu, and then click Delete.

To move a log to a new log group
  1. Open the navigation menu and click Observability & Management. Under Logging, click Logs.
  2. Under List scope, Compartment, choose a compartment you have permission to work in.
  3. Under Log name, click the name of the log you want to view. The log detail page opens.
  4. Click Change Log Group. The Move Log to a Different Log Group dialog is displayed.
  5. From the Choose new group list, select the new group from the list, and click Change Log Group. The new log group membership is reflected on the Log Information tab's Log Group field.

    From the main Logs page, for the log you want to move to a new group, you can also click the Actions menu, and then click Change Log Group.

To enable or disable an existing log
  1. Open the navigation menu and click Observability & Management. Under Logging, click Logs.
  2. Under List scope, Compartment, choose a compartment you have permission to work in.
  3. Under Log name, click the name of the log you want to view. The log detail page opens.
  4. Click Disable Log/Enable Log. A confirmation dialog is displayed regarding the disabling or enabling of the log.
  5. Confirm by clicking Disable Log/Enable Log. The log detail page changes its status and displays Inactive (for a disabled log) or Active (for an enabled log) in the status field, both on the log detail page and the Logs page.

    From the main Logs page, for the log you want to enable or disable, you can also click the Actions menu, and then click Disable Logging/Enable Logging.

To create a log group
  1. Open the navigation menu and click Observability & Management. Under Logging, click Log Groups.
  2. Choose a compartment you have permission to work in and click Create Log Group. The Create Log Group panel is displayed.
  3. Enter the following:
    • Compartment: The compartment in which you want to create the log group. This field is pre-filled based on your compartment choice.

    • Name: A name for this log group. The first character of a log group name must be a letter. For more, see Log and Log Group Names. Avoid entering confidential information.

    • Description: A friendly description.
    • Optionally, enter tagging information.
  4. Click Create. The log group detail page is then displayed. From this page you can:
    • Edit the group
    • Move resources
    • Add tags
    • Delete the log group
    • View log group information and tags
    • View log group resources (explore the log group, view the logs included in the log group, create custom or service logs, and view metrics)

      The Metrics resource in a log group detail page functions the same as in a log detail page. See To view the contents of logs for more information.

To edit a log group
  1. Open the navigation menu and click Observability & Management. Under Logging, click Log Groups.
  2. Choose a compartment you have permission to work in and click the linked log group name under Log Group in the table. The log group detail page is displayed.
  3. Click Edit. The Edit Log Group panel is displayed. From here, you can change the log group name and its description in the associated fields. Avoid entering confidential information. See Log and Log Group Names for more information on naming.

    From the main Log Groups page, for the log group you want to edit, you can also click the Actions menu, and then click Edit.

  4. Make your changes and click Update.
Note

You cannot move, edit, or delete the default log group.
To delete a log group
  1. Open the navigation menu and click Observability & Management. Under Logging, click Log Groups.
  2. Choose a compartment you have permission to work in and click the linked log group name under Log Group in the table. The log group detail page is displayed.
    Tip

    You cannot delete a log group that contains logs.
  3. Click Delete. A confirmation dialog is displayed regarding the delete operation.
  4. Confirm by clicking Delete. The log group is removed from the Log Groups page.

    From the main Log Groups page, for the log group you want to delete, you can also click the Actions menu, and then click Delete.

Note

You cannot move, edit, or delete the default log group.
To move a log group to a different compartment
  1. Open the navigation menu and click Observability & Management. Under Logging, click Log Groups.
  2. Choose a compartment you have permission to work in and click the linked log group name under Log Group in the table. The log group detail page is displayed.
  3. Click Move Resource. The Move Resource to a Different Compartment dialog is displayed.
  4. Choose the new compartment and then click Move Resource.

    From the main Log Groups page, for the log group you want to move to a new compartment, you can also click the Actions menu, and then click Move Resource.

Note

You cannot move, edit, or delete the default log group.
To list the logs in a log group
  1. Open the navigation menu and click Observability & Management. Under Logging, click Log Groups.
  2. Choose a compartment you have permission to work in.
  3. For the log group you want to inspect for logs, click the name of the log group under Log Group in the table. The log group detail page is displayed.
  4. In Resources, click Logs to display a list of all the logs contained in the log group. This resource table of logs functions in the same manner as the main Logs page.