32312 3000000000001000 - Server Disk Space Shortage Error

Alarm Type: TPD

Description: This alarm indicates that one of the following conditions has occurred:

A filesystem has exceeded a failure threshold, which means that more than 90% of the available disk storage has been used on the filesystem.
More than 90% of the total number of available files have been allocated on the filesystem.
A filesystem has a different number of blocks than it had when installed.

Severity: Major

OID: 1.3.6.1.4.1.323.5.3.18.3.1.2.13

Alarm ID: TKSPLATMA133000000000001000

Recovery

Run syscheck.
Examine the syscheck output to determine if the file system /var/TKLC/epap/free/var/TKLC/elap/free is low on space. If it is, continue to the next step. Otherwise, go to 4
If possible, recover space on the free partition by deleting unnecessary files:
1. Log in to the EPAP GUI.
2. Select Debug>Manage Logs & Backups.
  
  A screen similar to Figure 4-1 is displayed. This screen displays the information about the total amount of space allocated for and currently used by logs and backups. The display includes logs and backup files which might be selected for deletion to recover additional disk space.
  Figure 4-1 Manage Logs and Backups
3. Click the checkbox of each file that you want to delete and then click Delete Selected File(s).
If the file system mounted on /var/TKLC/epap/logs/var/TKLC/elap/logs is the file system that syscheck is reporting to be low on space, execute the following steps:
1. Log into the server generating the alarm as the admusr:
```
Login:  admusr
Password:<Enter admusr password>
```
2. Change to the /var/TKLC/epap/logs directory:
```
$ cd /var/TKLC/epap/logs
```
3. Confirm that you are in the /var/TKLC/epap/logs directory:
```
$ pwd
/var/TKLC/epap/logs
```
4. When the pwd command is executed, if /var/TKLC/epap/logs is not output, go back to sub-step b.
5. Look for files that you want to delete and execute an rm command for each:
```
$ sudo rm <filename>
```
  where <filename> is replaced by the name of the file to be deleted.
6. Re-run syscheck.
  - If the alarm is cleared, the problem is solved.
  - If the alarm is not cleared, go to the next step.
If syscheck has determined inodes have been depleted or a file system has a different number of blocks, skip to 11.
Execute the following steps to collect and remove any core files from the server.
Core files can occupy a large amount of disk space and may be the cause of this alarm:
1. Log into the server generating the alarm as the admusr:
```
Login:  admusr
Password:<Enter admusr password>
```
2. To list core files on the server, execute the following command, where <mountpoint> is the file system’s mount point:
```
$ sudo find <mountpoint> -name core.[0-9]\* -print -exec gzip -9 {} \;
```
  Note:
  The find command shown above will list any core files found and then compress and rename the file adding a “.gz” extension.
  If any core files are found, transfer them off of the system and save them aside for examination by Oracle. Once a copy of a compressed file has been saved it is safe to delete it from the server.
3. Re-run syscheck.
  - If the alarm has been cleared, the problem is resolved.
  - If the alarm has not been cleared, proceed to 7.
Execute the following steps if the file system reported by syscheck is /tmp, otherwise skip to 11.
1. Log into the server generating the alarm as the admusr:
```
Login:  admusr
Password:<Enter admusr password>
```
2. Change to the /tmp directory:
```
$ cd /tmp
```
3. Confirm that you are in the /tmp directory:
```
$ pwd
/tmp
```
4. When the pwd command is executed, if /tmp is not output, go back to 5.
5. Look for possible candidates for deletion:
```
$ ls *.iso *.bz2 *.gz *.tar *.tgz *.zip
```
6. If any files that can be deleted exist, the output of the ls will show them. For each of the files listed, execute the rm command to delete the file:
```
$ sudo rm <filename>
```
7. Run syscheck.
  - If the alarm is cleared, the problem is solved.
  - If the alarm is not cleared, go to the next step.
8. Upon a reboot the system will clean the /tmp directory.
  To reboot the system, issue the following command:
```
$ sudo shutdown -r now
```
9. Re-run syscheck.
  - If the alarm has been cleared, the problem is resolved.
  - If the alarm has not been cleared, proceed to the next step.
Execute the following steps if the file system reported by syscheck is /var, otherwise skip to 11.
1. Log into the server generating the alarm as the admusr:
```
Login:  admusr
Password:<Enter admusr password>
```
2. Change to the /var/tmp directory:
```
$ cd /var/tmp
```
3. Confirm that you are in the /var/tmp directory:
```
$ pwd
/var/tmp
```
4. When the pwd command is executed, if /var/tmp is not output, go back to 6.
5. Since all files in this directory can be safely deleted, execute the rm * command to delete all files from the directory:
  $ sudo rm -i *
6. Re-run syscheck.
  - If the alarm is cleared, the problem is solved.
  - If the alarm is not cleared, go to 11.
Execute the following steps if the file system reported by syscheck is /var/TKLC, otherwise skip to 11.
1. Log into the server generating the alarm as the admusr:
```
Login:  admusr
Password:<Enter admusr password>
```
2. Change to the /var/TKLC/upgrade directory:
```
$ cd /var/TKLC/upgrade
```
3. Confirm that you are in the /var/TKLC/upgrade directory:
```
$ pwd
/var/TKLC/upgrade
```
4. When the pwd command is executed, if /var/TKLC/upgrade is not output, go back to 6.
5. Since all files in this directory can be safely deleted, execute the rm * command to delete all files from the directory:
```
$ sudo rm -i *
```
6. Run syscheck.
  - If the alarm is cleared, the problem is solved.
  - If the alarm is not cleared, go to 11.
For any other file system, execute the following command, where <mountpoint> is the file system’s mount point:
```
$ sudo find <mountpoint> -type f -exec du -k {} \; | sort -nr > /tmp/file_sizes.txt
```
This will produce a list of files in the given file system sorted by file size in the file /tmp/file_sizes.txt.
Note:
The find command noted above could possibly take a few minutes to complete if the given mountpoint contains many files.
Do not delete any file unless you know for certain that it is not needed. Continue to 11 .
Run savelogs to gather all application logs (see Saving Logs Using the EPAP GUI).
Run savelogs_plat to gather system information for further troubleshooting, (see Saving Logs Using the EPAP GUI), and contact My Oracle Support (MOS).
Run syscheck in Verbose mode.
Contact My Oracle Support (MOS).