4.4 Monitoring with Log Files

The SQLcl daemon includes a built-in reporting system for job execution events, scheduler lifecycle events, and configuration issues. It records key events and errors in log files to help you debug and monitor scheduled jobs.

Learn about log formats, log file locations, and their descriptions.

• Log Formats
• Log File Locations
• Log File Descriptions

4.4.1 Log Formats

The log files have a semi-structured format, where each log line contains tokens enclosed in `[]` and separated by ` - ` (a hyphen surrounded by spaces) for better readability and parsing.

Logs are generated in two primary formats, one for informational entries and another for error entries. However, key events like job scheduling or job completion have unique, dedicated formats.

• Informational messages: This format is used for general messages, such as job started, scheduler started, and so on.

  [timestamp] - [STATUS] - [MESSAGE: <message>]

• Error reports: This format is used for error messages, such as job failed, scheduler start failed, and so on.

  [timestamp] - [STATUS] - [ERROR: <error-message>]

  In the above formats, the STATUS field classifies the type of log event, and the MESSAGE or ERROR field provides the details.

• Special formats for key events.
  • Job scheduling (SCHEDULE_JOB): This format is used to indicate that a job has been successfully scheduled for execution.

    [timestamp] - [SCHEDULE_JOB] - [JOB: <job-name>] - [JOB_LOG: <full-log-file-path>]

  • Job completion (COMPLETED): This format is used to indicate that a job has successfully completed execution.

    [timestamp] - [COMPLETED] - [JOB_RESULT: <output returned by running the job>]

    If the output spans multiple lines, it's wrapped in the following way:

    [timestamp] - [COMPLETED] - [JOB_RESULT:
        line 1 of output
        line 2 of output
    ]

  • Schedule file events: This format is used to indicate the changes that can occur to the schedule file (scheduler.yaml), such as SCHEDULE_FILE_LOADED, SCHEDULE_FILE_VALIDATED, and SCHEDULE_FILE_ERROR.

    [timestamp] - [STATUS] - [SCHEDULE_FILE: <schedule-file-path>]

4.4.2 Log File Locations

The Database Tools (dbtools) home directory contains the log files.

The log files path is:

${HOME}/.dbtools/schedules/

The SQLcl daemon uses the following directory structure for log files:

~/.dbtools/schedules
├── logs
│   ├── job-<job-name>.log
│   └── jobs.log
├── pid
├── scheduler.log
└── scheduler.yaml

4.4.3 Log File Descriptions

Each log file of the SQLcl daemon has a distinct purpose. For example, one file records daemon lifecycle logs while another captures scheduling logs of jobs, and so on.

The daemon contains the following log files:

• Daemon Life Cycle Logs (scheduler.log)
• Shared Job Scheduling Logs (jobs.log)
• Per-Job Execution Logs (job-<job-name>.log)

Note:

Log rotation is not supported, so these log files may grow over time.

4.4.3.1 Daemon Life Cycle Logs (scheduler.log)

The scheduler.log file records high-level events related to the daemon's life cycle, such as startup, shutdown, and failures.

Table 4-1 Statuses reported in life cycle logs

Status	Description	Example
STARTED	Indicates the scheduler has started successfully and is ready to begin scheduling jobs.	[2025-05-06 09:00:00] - [STARTED] - [MESSAGE: SQLcl daemon started]
STOPPED	Indicates the scheduler has been stopped and the job scheduling has ceased.	[2025-05-06 09:00:00] - [STOPPED] - [MESSAGE: SQLcl daemon stopped]
STARTUP_FAILED	Indicates the scheduler has failed to start. The message provides the reason.	[2025-05-06 09:00:00] - [STARTUP_FAILED] - [ERROR: <error-message>]
SHUTDOWN_FAILED	Indicates the scheduler has failed to shut down cleanly. The message provides the reason.	[2025-05-06 18:35:00] - [SHUTDOWN_FAILED] - [ERROR: <error-message>]

4.4.3.2 Shared Job Scheduling Logs (jobs.log)

The jobs.log file logs events that affect all jobs, such as updates to the schedule file, validations, and job scheduling actions.

Table 4-2 Statuses reported in shared-job scheduling logs

Status	Description	Example
SCHEDULE_FILE_LOADED	Indicates the schedule file was successfully loaded and serialized, and jobs are available for scheduling.	[2025-05-06 10:00:00] - [SCHEDULE_FILE_LOADED] - [SCHEDULE_FILE: /Users/<username>/.dbtools/schedules/scheduler.yaml]
SCHEDULE_FILE_ERROR	Indicates the schedule file is invalid due to missing fields or malformed content.	[2025-05-06 10:00:00] - [SCHEDULE_FILE_ERROR] - [ERROR: <error-message>]
SCHEDULE_FILE_VALIDATED	Indicates the schedule file is valid and the system can proceed to schedule jobs.	[2025-05-06 10:00:00] - [SCHEDULE_FILE_VALIDATED] - [SCHEDULE_FILE: /Users/<username>/.dbtools/schedules/scheduler.yaml]
SCHEDULE_JOB	Indicates a job has been successfully scheduled. The log shows the job's name and its dedicated log file path.	[2025-05-06 18:35:00] - [SCHEDULE_JOB] - [JOB: test] - [JOB_LOG: /Users/<username>/.dbtools/schedules/logs/job-test.log]
JOB_SCHEDULE_FAILED	Indicates a job failed to be scheduled due to an invalid cron expression or internal error.	[2025-05-06 18:35:00] - [JOB_SCHEDULE_FAILED] - [ERROR: <error-message>]

4.4.3.3 Per-Job Execution Logs (job-<job-name>.log)

Each job has its own dedicated log file, recording only that job's activity. This facilitates isolating and debugging job-specific behavior.

Table 4-3 Statuses reported in individual job logs

Status	Description	Example
RUNNING	Indicates the job has started executing. This confirms that the job's scheduled time has arrived and a resource has been allocated to run it.	[2025-05-06 10:00:00] - [RUNNING] - [MESSAGE: Job execution started]
COMPLETED	Indicates the job has completed successfully. The job's output is included in the log.	[2025-05-06 10:00:00] - [COMPLETED] - [RESULT: Name Null? Type ------------ -------- ------------ COUNTRY_ID NOT NULL CHAR(2) COUNTRY_NAME VARCHAR2(40) REGION_ID NUMBER ]
FAILED	Indicates the job failed to complete due to errors like a missing connection or script failure. The reason is included in the log.	[2025-05-06 10:00:00] - [FAILED] - [ERROR: Unknown connection test]