User's Guide

Get
Adobe Reader

System Backup and Restoration

The following sections describe how to backup and restore WebLogic Network Gatekeeper installations:

About system backup

The following sections describe two types of system backups. The first type, the full system backup, makes a backup copy of all BEA WebLogic Network Gatekeeper related software, installed applications, configuration files and database tables. A full system backup should be performed after the system has been successfully verified and after major re-configurations, upgrades and when patches have been installed in the system.

The second type, the system data backup, is a smaller backup that makes a backup copy of the configuration files, user certificates and database tables. This backup has to be performed on a regular basis and when new applications have been installed.

System backup procedures may vary for different systems and configurations, depending on system type, additional equipment connected to the system, and so on. This chapter gives a recommendation on how to perform backup and restore, and also gives information the tools to support these procedures.

Setting up backup directories

The database and system backup files generated by the backup scripts are named as follows:

Static MySQL files are backed up to system_<date>.zip.
MySQL database files are backed up to data_<date>.zip

Where <date> represents the creation date and time.

The files are zipped together and requires an unzip-utility, like jar, in order to restore them.

The backup-files are created in a directory as defined in the
xml-tag <BACKUP directory="<dir>"/> in the file slee_properties.xml. Replace <dir> with the full path to the desired directory, preferably on a separate disk.

To apply the changes, the script SLEEConfig.sh -px must be executed. The script is located in the bin directory of the installation.

For example, edit the file slee_properties.xml and change the lines to (the given path is just an example of a path to a separate disk):

<BACKUP directory="/backup"/>

Save and close the file, and run the script SLEEConfig.sh -px to apply the changes.

Enable binary logging

To enable data backups on a single database system, make sure binary logging is enabled.

Activate binary logging

Open the MySQL configuration file /usr/local/mysql/data/my.cnf

Add the following rows, if they are not present:

server-id=1

log-bin=/usr/local/mysql/data/query-bin

If the rows were not present, proceed with the next step. If the rows are present, binary logging is enabled and no further action is needed.
Restart MySQL
Since MySQL is stopped during this procedure, make sure that all SLEEs using the database are in state shutdown.

Open a command tool.

Change directory to the installation of MySQL. Enter command:

cd /usr/local/mysql/

Stop MySQL. Enter command:

./bin/mysqladmin -u root -p shutdown

When asked for, enter the password for the MySQL root user.

Execute the restart-script. Enter command:

./bin/safe_mysqld &

Performing a full system backup

Follow the instruction below to perform a full system backup. The backup saves all software and data related to the BEA WebLogic Network Gatekeeper system, including third party products and database tables.In order to reduce the size of the database backup, it is recommended to dump the charging data to a text-file, see Chapter 9, Charging Data Export before performing the backup. It is also recommended to remove old alarms, events, and statistics from the database. See Deleting alarm entries from the database on page 10-4, Deleting event entries from the database on page 10-7, and Deleting statistics data from the database on page 8-7.

Copy files and directories

Copy the following on each server in the system to the backup directory:

All scripts for autostart of the database and the SLEE, as cofigured during the installation.
/usr/local/slee/ (SLEE, SLEE services, and applications)

It is recommended to use a compression utility, for example jar, in order to minimize the diskspace required for the backup.

Backup single database system

If using a single database system, follow the instructions below to perform a backup. If using a replicated database system, perform the procedure as described in Backup replicated database system on page 18-6.

Note: No updates can be made on the database during the full system backup, causing the system to halt during the procedure. It is strongly recommended to shutdown all SLEE instances that use the database during the full system backup.

/usr/local/mysql/bin/mysql -u root -p

When prompted for, enter the password for the database root user.

The prompt mysql> is displayed.

Caution: If the MySQL client is closed during the execution of Step 4 to Step 9, the result will be corrupt data in the backup.

Flush tables and obtain table locks. Enter command:

FLUSH TABLES WITH READ LOCK;

Flush binary logs, that is switch current log file. Enter command:

FLUSH LOGS;

List currently used binary log-file. The name of the log will be used in a later step. Enter command:

SHOW MASTER STATUS;

A list is displayed: +--------+----------+--------------+------------------+ | File | Position | Binlog_do_db | Binlog_ignore_db | +--------+----------+--------------+------------------+ | <file> | <pos> | | | +--------+----------+--------------+------------------+ 1 row in set (0.00 sec)
<file> is the name of the old binary log.

Remove old binary logs. Enter command:

PURGE MASTER LOGS TO '<file>'

<file> is replaced by a file name as output in Step 6 on page 5. The name is for example query-bin.114

Copy the MySQL directory /usr/local/mysql/ to a backup directory without exiting the MySQL client.

Note: This step must be performed from another command window, or using a file manager. Do not exit the MySQL client.

Unlock all tables. Enter command:

UNLOCK TABLES;

Exit from the MySQL client. Enter command:

exit

The backup is complete and the tables are unlocked.

Backup replicated database system

follow the instructions below to perform a backup.

If using a single database configuration, see Backup single database system on page 18-4.

Note: The standby database will be shutdown during the backup. If the currently active database crashes during the backup, there will be no database available for BEA WebLogic Network Gatekeeper, causing it to fail.

Note: It is important to perform the procedure on both computers hosting the database in order to create a complete backup.

Repeat Step 2 to Step 3 on both computers hosting the database.

Execute the dual database system backup script. Enter command:

/usr/local/slee/bin/db_runDualSystemBackup.sh

If the script was run on the standby database, it will create a backup. It will also produce a message informing where to find the files. The files system_<date>.zip and data_<date>.zip, where <date> represents the current date and time, are created. Static MySQL files are backed up to system_<date>.zip. MySQL Database files are backed up to data_<date>.zip
If the script was run on the currently active database only the configuration file, my.cnf, and MySQL static files are copied. It will also produce a message informing where to find the files.
In both cases, the files are created in the directory as configured. See Setting up backup directories on page 18-1.

Performing a system data backup

Follow the instruction below to perform a system data backup. The backup saves all BEA WebLogic Network Gatekeeper system data from the database, but not BEA WebLogic Network Gatekeeper system software.

Copy files and directories

Copy the following directories, including subdirectories, on each server in the system.

/usr/local/slee/bin/ (directories with jar files for SLEE services, auto started services and installed applications)
/usr/local/slee/certificates/ (user certificates)

It is recommended to use a compression utility, for example jar, in order to minimize the diskspace required for the backup.

Single database system

The system data backup on a single database system does not interfere with the system operation. The script can be a scheduled job to be executed periodically. The backed up files should be moved from the backup directory and saved on tape after each execution to avoid full disk.

Note: The files backed up are the ones generated since the last full system backup. In order to reduce the size of the files, and thereby shorten the time it takes to perform a restoration, it is recommended to perform a full system backup on a regular basis when using a single database system. It is important to save all system data backup files since the last full system backup.

Execute the single database system data backup script. Enter command:

/usr/local/slee/bin/db_runSingleDataBackup.sh

When the script completes, it has created a backup. It also produces a message informing where to find the files.
The file data_<date>.zip, where <date> represents the current date, is created. The file is created in the directory as configured. See Setting up backup directories on page 18-1.

Replicated database system

The script can be a scheduled job to be executed periodically. It should be scheduled to run at the same time on both database nodes, since it will only copy files on the standby database. The backed up files should be moved from the backup directory and saved on tape after each execution to avoid full disk.

If using a replicated database system, follow the instructions below to perform a backup.

Note: It is important to perform the procedure on both computers hosting the database in order to create a complete backup.

Repeat Step 3 to Step 4 on both computers hosting the database.

Execute the dual database system backup script. Enter command:

/usr/local/slee/bin/db_runDualDataBackup.sh
If the script was run on the standby database, it will create a backup. The script displays a message informing where to find the backed up files. The file data_<date>.zip, where <date> represents the current date and time, is created. Database tables are backed up to data_<date>.zip
If the script was run on the currently active database only the configuration file, no files are copied. The script displays a recommendation to run the script on the inactive database.
In both cases, the files are created in the directory as configured. See Setting up backup directories on page 18-1.

About system restoration

System restoration includes both restoring the full BEA WebLogic Network Gatekeeper system and restoring the system data. When restoring the full system, all BEA WebLogic Network Gatekeeper software (including third party software) and all system data is restored. In case of a system data restoration, user certificates and database tables are restored.

For a successful restoration, the backup must have been performed according to the sections Performing a full system backup on page 18-3 and Performing a system data backup on page 18-6 respectively.

Also, it is important that the same directory structure is used on the restored system as on the backed up system.

If one of the databases in the system has failed it is possible to create a snapshot of the still running database host and use that snapshot for restoring the failed database. This is called a database snapshot restoration.

Performing a full system restoration

Follow the instruction below to perform a full system restoration. The restoration requires:

the UNIX installation package
BEA WebLogic Network Gatekeeper CD-ROM or downloaded installation files.
a backup copy made according to the section Performing a full system backup on page 18-3.
any patches installed after the backup was made

Install OS

Install the UNIX operating system. See the installation instructions included in the UNIX installation package.

Install resent patches

Install BEA WebLogic Network Gatekeeper patches received since the backup was performed. See the instructions provided with the patches. There is no need to install the patches included prior to the latest full system backup.

Copy backup files

Copy the full system backup copy, as set up in Copy files and directories to the newly installed BEA WebLogic Network Gatekeeper.

Restore the database software

Install the MySQL software according to the installation guide.

Make sure the MySQL server is shut down. Enter command:

/usr/local/mysql/bin/mysqladmin -u root -p shutdown

When prompted for, enter the password for the root database user.

Overwrite the entire MySQL directory with the files backed up during the last full system backup. Perform the following commands:

cd /usr/local/mysql/

jar xf <system_backup_file>

Where <system_backup_file> is the name of the back-up file.

Perform a system data restoration from the latest system data backup, as described in Performing a system data restoration on page 18-11.

Start SLEE process

Start the SLEE process by rebooting the server.

When the server reboots, the SLEE agent will be automatically start up the SLEE process. When the process is started, the SLEE will be in the same state as it was at the time of the backup.

Performing a system data restoration

Caution: Follow the instruction below to perform a restoration of BEA WebLogic Network Gatekeeper system data. After the restoration, all data that has been written to the database after the latest backup will be lost.

It is necessary to run system data backups frequently to avoid to large data loss in case of a database crash

Note: The restoration requires a system data backup copy made according to the section Performing a system data backup on page 18-6.

The MySQL root user is used in all MySQL commands.

Restore files

Copy the directories from the backup copy to following directories:

/usr/local/slee/bin/
usr/local/slee/certificates/
/usr/local/slee/servlet_engine/webapps/

Caution: For single database system data restoration you also need the last full system backup together with all system data backup files since the latest full system backup. The different backup files needs to be synchronized; never add system data backup files older than the full system backup used.

Make sure the MySQL server is shut down. Enter command:

/usr/local/mysql/bin/mysqladmin -u root -p shutdown

When prompted for, enter the password for the root database user.

Copy the latest full system backup data files to the MySQL data directory /usr/local/mysql/data

Start MySQL server. Enter commands:

cd /usr/local/mysql/

./bin/safe_mysqld &

Extract the relevant system data backup files to a directory.

Note: Relevant system data backup files are the ones taken since last full system backup.

Import the system data backups made after the latest full system backup. Change to the directory holding the system data backup and database backup files backed up since last full system backup. Enter command:

/usr/local/mysql/bin/mysqlbinlog <backup dir>/query-bin.[0-9].* | /usr/local/mysql/bin/mysql -u root -p

Replace <backup dir> with the directory used in Step 5.

When prompted for, enter the password for the root database user.

All query-bin files are imported and the embedded SQL statements are executed. This procedure may take a few minutes.
Now the MySQL server should be in the same state as when the latest system data backup was done.

Restore a replicated database system

For dual (replicated) database system data restoration you need the latest performed system data backup.

Note: After the restoration, all data written to the database after the latest backup will be lost. Run backups frequently to avoid to large data loss in case of a database crash.

In a replicated database system, if one database crashes, all information should be present in the still working database. It is possible to save this data when restoring this node, and import this data in a separate MySQL installation to be able to, for instance, dump charging data.

The following references are used below.

Host A: The computer hosting database A.

Database A: The database that will replace the previously malfunctioning database (or standby database if the restoration is a rollback to a previous configuration).

Host B: The computer hosting database B.

Database B: The currently active database used by the system.

In Host B:

When prompted for, enter the password for the database root user.

The prompt mysql> is displayed.

Stop the (replication) slave thread. Enter command:.

SLAVE STOP;

In Host A:

Make sure the MySQL server to be restored is shut down. Enter command:

/usr/local/mysql/bin/mysqladmin -u root -p shutdown

When prompted for, enter the root password.

Extract the full system backup files from the backup directory to the home directory of mySQL. The backup directory is configured according to Setting up backup directories on page 18-1.

cd /usr/local/mysql/
jar xf <system_backup_file>

Where <system_backup_file> is the name of the system back-up file.

Clear (remove all files and directories in) the MySQL data directory /usr/local/mysql/data

Unzip and copy the latest system data backup data files to the MySQL data directory /usr/local/mysql/data. Check both database A and database B to get the latest backup copy.

cd /usr/local/mysql/data
jar xf <data_backup_file>
chown -R root:mysql /usr/local/mysql
chown -R mysql /usr/local/mysql/data

Where <data_backup_file> is the name of the data back-up file.

Open the file /usr/local/mysql/data/my.cnf

In the file, transform each row starting with the keyword master into comments, by inserting a hash (#) character in the beginning of the row.

Start Database A. Enter commands:

cd /usr/local/mysql/
./bin/safe_mysqld &

In Host B:

Shut down Database B. Enter command:

/usr/local/mysql/bin/mysqladmin -u root -p shutdown

When prompted for, enter the root password.

This will cause BEA WebLogic Network Gatekeeper to switch to Database A.

Optional step:
Make a copy of the data directory of the mySQL installation.

This copy contains changes made since the latest backup.
This copy can be used to copy to a separate MySQL installation in order to, for example dump charging data or viewing alarms since latest backup. For information on how to import table data files in a MySQL installation, see MySQL Reference Manual.

Clear (remove all files and directories in) the MySQL data directory /usr/local/mysql/data

Unzip and copy the latest system data backup files to the MySQL data directory /usr/local/mysql/data. Use the same backup copy as in Step 2 on page 13, sub-step f.

Verify that the parameter master-host in my.cnf in host A is the IP address of Host B and vice versa.

In Host B:

Start Database B. Enter commands:

cd /usr/local/mysql/
./bin/safe_mysqld &

Database B will start replicating data updated in Database A since Database B was shut down.

In Host A:

Open the file /usr/local/mysql/data/my.cnf

Remove the comment characters, as added in Step 2 on page 13, sub-step h.

cd /usr/local/mysql/
./bin/mysql -u root -p

When prompted for, enter the password for the database root user.

The prompt mysql> is displayed.

Change database master to database B. Enter the following commands:

CHANGE MASTER TO
   MASTER_HOST='<database B IP address>',
   MASTER_USER='<replication user>',
   MASTER_PASSWORD='<replication password>',
   MASTER_PORT=<database B port>;

Where <database B IP address>, <replication user>, <replication password>, <database B port> are replaced with the corresponding values, as defined in my.cnf.

In Database A enter command:


START SLAVE;

Database A starts replicating changes made in Database B.
Verification of restoration of replicated database system

In both Database A and Database B, verify that the database replication is working correctly. Enter commands:

SHOW MASTER STATUS;

SHOW SLAVE STATUS;

Cross compare the output values.

If corresponding, the restoration procedure was successful.
If the values did not correspond, a corrupt database may the reason. For actions to take, see Handling corrupt database tables on page 18-18 or Contact MySQL support.

Database shapshot restoration

Follow the instruction below to perform a backup and restore procedure that can be used when one of the databases in the system has failed and needs to be restored. The procedure assumes that there is still one functional and active database in the system.

When performing a snapshot of the functional database it must be locked for some time in order guarantee a stable copy. During the lock time all write requests towards the database will be blocked. In order to reduce the lock duration it is possible to specify a list of database tables that can be temporary renamed and copied separately in order to reduce the lock duration.

Snapshot

The tables to be separately copied can me managed with the addMasterSnapshotSpecialCopyTable, removeMasterSnapshotSpecialCopyTable and listMasterSnapshotSpecialCopyTable OAM methods in the SLEE_backup service.

The data in the specified tables will become unavailable for the duration of the snapshot process but will become available again after completion of the snapshot process. This may cause outstanding requests to be lost.

The following are some examples of tables suitable for separate copy:

ic_p_gms_message
ic_slee_alarm
ic_slee_charging
ic_slee_statistics_data
ic_slee_event

The backup snapshot is created by executing the createMasterSnapshotForRestore method in the SLEE_backup OAM interface.

Note that during creation of the snapshot the database slave replication thread at the functional database will be stopped. Once the other database host has been restored it must be started again using the restartSlaveThreadAfterSnapshotRestore OAM method.

Alarms will be generated to indicate when the snapshot procedure has ended and where it is located.

Pack the backup snapshot files together That is, perform the following commands:

cd /usr/local/slee/backup/<snapshot_timestamp> tar cvf db_snapshot_backup.tar mysql slee_db

FTP the backup to the machine with the failed database.

Restoration

Restore the failed database host:

Re-install mysql if required.

Delete all files under /usr/local/mysql/data using:

rm -r /usr/local/mysql/data/*

Unpack the backup files under /usr/local/mysql/data so that the mysql and slee_db directories are created under /usr/local/mysql/data

Changed to correct ownership using:

chown -R mysql /usr/local/mysql/data

chgrp -R mysql /usr/local/mysql/data

Execute postconfig.sh in /usr/local/slee/bin in order to create a proper my.cnf file. Executing postconfig.sh will also restart the database.

Restart the slave replication thread on the original functional active master database host by executing the restartSlaveThreadAfterSnapshotRestore OAM method in the SLEE_backup service.

Check the replication status by executing the getDatabaseReplicationStatus OAM method in the SLEE_backup service. It should report "OK".

Handling corrupt database tables

If corrupt database tables are discovered during a restoration, they might be recovered. Follow the instructions given in section "Disaster Prevention and Recovery "in MySQL Reference Manual.