This chapter describes DFM administration, operations, and monitoring, and includes the following information:
You can start, stop, and restart the DFM service using the operating system services facilities, or from the DFM command line utility as follows:
dfm {start|stop|restart}
The command line options are as follows:
start: tells the DFM service to start
stop: tells the DFM service to stop
restart: tells the DFM service to stop, and then start again
When the DFM service starts, or restarts, DFM loads and validates the configuration file. If DFM detects any configuration issues, the process terminates and runs a diagnostics routine.
For successful starting of DFM, you must remove the .ini extension from the dfm.trace.ini file.
If the configuration validation is successful, DFM scans all of the configured Drop Folders, then checks the status of all objects initialized before DFM was last shutdown, and then updates the internal database with the current status of the objects.
After all of these steps have completed, the DFM status will be Running.
Caution:
The last file to be created in the File Set Drop Folder must be the metadata file (.mdf), which then triggers the archive operation. If the metadata file is transferred before the actual data files, content loss can occur!When DFM locates files in a configured Drop Folder, it updates the internal database and requests DIVArchive to archive all files found as new objects. In order to avoid repeated archive requests, DFM continuously updates the archive operations status in the internal database.
DFM logs information about the incomplete files and calls the DFM File Manager module to move them to the Trash folder.
If DIVArchive requests fail, the DIVArchive Status module informs the internal database of the failure. If the number of unsuccessful request attempts reaches a preconfigured number, the object status is changed to Could not be archived and the object is marked as incomplete.
If the request completes successfully, the internal database is updated by the DIVArchive Status module. DFM removes the MDF file, and the File Set folder, for a File Set object.
Executing the shutdown script will terminate the DFM service. All internal process are stopped before all archive operations are completed. After all of the modules are stopped, all internal statuses are saved to disk in the internal database before the DFM completes shut down.
The following is an overview of the administrative functions available in DFM. There is no special licensing required to use DFM. See Appendix A for Oracle DIVArchive options and licensing information.
The DFM configuration file is %DIVA_HOME%\Program\conf\dfm\dfm.conf.
You use the standard DIVArchive upgrade procedure to upgrade DFM.
DFM module is installed with DIVArchive in the same installation package.
The command line syntax to install the service is %DIVA_HOME%\Program\InterLink\dfm\bin\dfm.bat install.
The command line syntax to uninstall the service is %DIVA_HOME%\Program\InterLink\dfm\bin\dfm.bat uninstall.
You can execute %DIVA_HOME%\Program\InterLink\dfm\bin\dfm.bat start to start the service from the command line, or you can start it from the Windows Services panel.
You can execute %DIVA_HOME%\Program\InterLink\dfm\bin\dfm.bat stop to stop the service from the command line, or you can stop it from the Windows Services panel.
The log folder is %DIVA_HOME%\Program\log\dfm\.
The logging configuration is %DIVA_HOME%\Program\conf\dfm\dfm.trace.
You can use the DFM logs, located in %DIVA_HOME%\Program\InterLink\log\dfm\, to obtain the current status.
You can initiate commands using the operating system command-line. The syntax and options are as follows:
dfm {install|uninstall|start|stop|restart|status|version|help} [-conf|-f]
The commands are as follows:
installThis command installs the module as a system service.
uninstallThis command removes the executable as a system service.
startThis command starts the service.
stopThis command stops the service if it is currently running.
restartThis command stops, and then subsequently starts the service.
statusThis command displays the current status of the service.
versionThis command displays the module release information, and then exits
helpThis command displays the help information, and then exits.
The options are as follows:
-confThis option identifies the configuration file to load the DFM settings from.
-fThis option is an alternative to using the -conf option and performs the same function.
DFM constantly monitors the configured Drop Folders. When a file, or file set, is found in one of the configured Drop Folders, it is entered into the internal database (the internal database is an XML file) with the file size, date, and time when the file was originally found. After waiting for a configured period, DFM will again check the size, time, and date of the file (or file set) to see if any changes have occurred.
If the file in a Drop Folder configured in Single File Mode satisfies the applicable criteria in the DFM folder configuration (that is, it was not modified within the previous 60 seconds), DFM marks the file as complete and sends an Archive request to the DIVArchive system. If the file has not fulfilled the criteria for a complete file after a configured allowable period, then it is marked as incomplete and set for deletion.
Caution:
The last file to be created in the File Set Drop Folder must be the metadata file (.mdf), which then triggers the archive operation. If the metadata file is transferred before the actual data files, content loss can occur!In File Set Folders, after the metadata file (.mdf) is located in the Drop Folder, DFM marks all of the files in the File Set as complete, and initiates the Archive request to DIVArchive. If the metadata file does not exist after a configured period, then all files are marked incomplete and set for deletion.
By default, DFM will try to archive any files located in the configured and monitored Drop Folders. If the files cannot be archived according to the DFM Folder configuration, they are marked as incomplete files. The different reasons for files and file sets being either archived, marked incomplete, or deleted are identified in the following sections.
The time interval for how often the Drop Folder is scanned is specified in the fileReloadingThreshold parameter in the configuration file for that specific folder. Any files or file sets that are not complete, archived, or both, within the specified time period are flagged as an unsuccessful archive, an incomplete object, or deleted.
The DFM internal database is stored as an XML file and updated after these different tasks take place. Each time DFM checks the configured Drop Folders, it compares the file date, time, and size to what was previously written to the internal database.
Source files cannot be deleted from a Drop Folder after an archive completes unless the appropriate Source/Destination is configured using the -allow_delete_on_source option.
See Chapter 3 for details on configuration options, features, and examples.
For Single File Mode folders, DFM uses a series of checks to identify if a file is complete and ready for processing, already processed, or incomplete.
To begin the process, DFM checks if the file size is equal to zero, or if the file has been marked as incomplete. If the file size does equal zero, or the file was already marked incomplete, then the file is skipped.
Next, DFM checks the date of the last modification to the file, and the size of the file. If the date of the last modification to the file, or the file size, has changed then the file is skipped. The lastModificationTime value used in this check is the local DFM time when the last modification of size, or date, was captured by DFM.
DFM checks if the difference between the current time (local DFM time) and the lastModificationTime is greater than the fileReadyThreshold parameter setting in the configuration file, whether the file is locked for processing, and whether the disk has free space available.
If the difference is greater than the fileReadyThreshold parameter setting, the file is unknown to DFM, not locked for processing, and disk space is available, then the file will become known to DFM, and DFM tells DIVArchive to archive the file.
If the difference is greater than the fileReadyThreshold parameter setting, the file is already known to DFM, not locked for processing, and should be deleted as incomplete, then DFM will delete the file.
If the difference is greater than the fileReadyThreshold parameter setting, DFM knows about the file for longer than the incompleteThreshold parameter setting, and it is not locked for processing, then DFM will mark the file as incomplete.
Note:
An unknown status indicates that all required file information was not added to DFM. DFM stores the date and time information for each file, but if it does not have all of the required information, the file will remain unknown to DFM. It will only become known after all file information has been added to DFM for processing.The following is an example workflow:
The file test.mpg was found by DFM, and the date and size was stored.
The file test.mpg was found by DFM again (during the next cycle of folders investigation), the date and size was changed from the previous check. Therefore, the file will not be added as known to DFM.
The file test.mpg was found by DFM again (during the next cycle of folders investigation), the date and size was not changed from the previous check, the difference is greater than the fileReadyThreshold parameter setting, the file is unknown to DFM, not locked for processing, and disk space is available. Therefore, the file was added as known to DFM.
The Archive request was sent to DIVArchive.
The archive was complete.
The following is an example workflow:
The file test.mpg was found by DFM, and the date and size was stored.
The file test.mpg was found by DFM again (during the next cycle of folders investigation), the date and size was changed from last check. Therefore, the file will not be added as known to DFM.
The file test.mpg was found by DFM again (during the next cycle of folders investigation), the date and size was not changed from the previous check, the difference is greater than the fileReadyThreshold parameter setting, the file is unknown to DFM, not locked for processing, and disk space is available. Therefore, the file was added as known to DFM.
The Archive request was sent to DIVArchive.
The archive was not completed. Therefore, it was terminated by DIVArchive for some reason.
DIVArchive attempts to archive again after <repeatRequestDIVAThreshold> seconds, but the request was rejected again.
If the difference is greater than the fileReadyThreshold parameter setting, DFM knows about the file for longer than the incompleteThreshold parameter setting, and it is not locked for processing, then DFM will mark the file as incomplete. Otherwise, the system will continue to retry starting with Step 4.
For File Set Mode folders, DFM uses a series of checks to identify if a file is complete and ready for processing, already processed, or if it is incomplete.
To begin the process, DFM first checks to find out if the file size is equal to zero (0) or if the file has been marked as incomplete. If the file size does equal zero, or the file was already marked incomplete, then the file is skipped.
DFM next checks the date of the last modification to the file and the size of the file. If the date of the last modification to the file or the file size has changed, then the file is skipped. The lastModificationTime used in this check is the local DFM time when the last modification of size or date was captured by DFM.
DFM checks if the difference between the current time (local DFM time) and the lastModificationTime is greater than the fileReadyThreshold parameter setting in the configuration file, if the file is locked for processing or not, and if the disk has free space available.
If the difference is greater than the fileReadyThreshold parameter setting, the file is unknown to DFM and not locked for processing, then DFM tells DIVArchive to archive the file.
If the difference is greater than the fileReadyThreshold parameter setting, the file is known to DFM, not locked for processing, the file archived successfully, and the difference between the current time (local DFM time) and the archive time is greater than 5 seconds, or the file should be deleted as incomplete, then DFM will delete the file.
If the difference is greater than the fileReadyThreshold parameter setting, DFM knows about the file for longer than the incompleteThreshold parameter setting, and it is not locked for processing, then DFM will mark the file as incomplete.
Note:
An unknown status indicates that all required file information was not added to DFM. DFM stores the date and time information for each file, but if it does not have all of the required information, the file will remain unknown to DFM. It will only become known after all file information has been added to DFM for processing.The following is an example workflow:
The file test.mdf was found by DFM, and the date and size was stored.
The file test.mdf was found by DFM again (during the next cycle of folders investigation), and the date and size was changed from the previous check. Therefore, the file will not be added as known to DFM.
The file test.mdf was found by DFM again (during the next cycle of folders investigation), the date and size was not changed from the previous check, the difference is greater than the fileReadyThreshold parameter setting, the file was unknown to DFM. Therefore, the file was added as known to DFM.
The Archive request is sent to DIVArchive.
The archive was completed.
The file test.mdf was found by DFM again (during the next cycle of folders investigation), the date and size was not changed from the previous check, the difference is greater than the fileReadyThreshold parameter setting, the file is known to DFM, not locked for processing, the file archived successfully, and the difference between the current time (local DFM time) and the archive time is greater than 5000. Therefore, the file was deleted by DFM.
The following is an example workflow:
The file test.mdf was found by DFM, and the date and size was stored.
The file test.mdf was found by DFM again (during the next cycle of folders investigation), the date and size was changed from the previous check. Therefore, the file will not be added as known to DFM.
The file test.mdf was found by DFM again (during the next cycle of folders investigation), the date and size was not changed from the previous check, the difference is greater than the fileReadyThreshold parameter setting, the file is unknown to DFM. Therefore, the file was added as known to DFM.
The Archive request was sent to DIVArchive.
The archive was not completed, and terminated by DIVArchive for some reason.
DIVArchive attempts to archive again after <repeatRequestDIVAThreshold> seconds, but the request was rejected again.
If the difference is greater than the fileReadyThreshold parameter setting, DFM knows about the file for longer than the incompleteThreshold parameter setting, and it is not locked for processing, then DFM will mark the file as incomplete. Otherwise the system will continue to retry starting with Step 4.
The following sections describe different methods for monitoring the DFM service.
The DIVArchive Control GUI is used for monitoring the requests initiated by DFM. See Appendix A for Oracle DIVArchive options and licensing information.
You execute %DIVA_HOME%\Program\GUI\bin\gui.bat to start the DIVArchive Control GUI.
The DIVArchive Control GUI enables viewing the status of the running requests, Actors, and disks. The Objects and Requests panels also offer search functionality. The Control GUI Manager screen displays the information in the following columns:
Object ID
Errors
Type (of request)
Status (of request)
Step (of DFM processing request)
Object (name)
Category
Comments
Tape
Progress (of request)
Throughput (Mb/s)
Data Scanned (KB)
Third Party ID
Instance
Use the Control GUI Requests panel to query completed DFM archive requests. You can search based on the source/destination. To make querying DFM archive requests easier, assign a dedicated source/destination exclusively for DFM (possibly with the same parameters as another source/destination in the system). However, note this technique will not work for DIVA delete requests issued by DFM.
You use the DFM log files for status monitoring of DFM. The files are located in the %DIVA_HOME%\Program\InterLink\log\dfm\ folder. The log files contain detailed information about files and folders encountered by DFM, information about requests submitted to DIVArchive, files/folders marked as incomplete, and errors encountered in processing.
See Sample of the Full Trace Log File for detailed information about this file.