1.4 Files that Are Backed Up

DBA and development work typically involves logical structures such as tables, rows, columns, the data dictionary, and so on. For backups, you must understand the physical details of how these structures are represented by files.

Table 1.1 Files in a MySQL Enterprise Backup Output Directory

File Name, Pattern, or Extension	Relation to Original Data Files	Notes
`ibdata*`	The InnoDB system tablespace, containing multiple InnoDB tables and associated indexes.	Because the original files might change while the backup is in progress, the `apply-log` step applies the same changes to the corresponding backup files.
`*.ibd`	InnoDB file-per-table tablespaces, each containing a single InnoDB table and associated indexes.	Used for tables created using the `innodb_file_per_table` option. Because the original files might change while the backup is in progress, the apply-log step applies the same changes to the corresponding backup files.
`*.ibz`	Compressed form of InnoDB data files from the MySQL data directory.	Produced instead of `.ibd` files in a compressed backup. The `ibdata*` files representing the InnoDB system tablespace also receive this extension in a compressed backup. The `.ibz` files are uncompressed during the `apply-log`, `copy-back`, or `copy-back-and-apply-log` step.
`*.frm`	Hold metadata about all MySQL tables.	The database is put into a read-only state while these files are copied. These files are copied without changes.
`*.MYD`	MyISAM table data.	The database is put into a read-only state while these files are copied. These files are copied without changes.
`*.MYI`	MyISAM index data.	The database is put into a read-only state while these files are copied. These files are copied without changes.
`*.CSM`	Metadata for CSV tables.	These files are copied without changes. The `backup_history` and `backup_progress` tables created by mysqlbackup use the CSV format, so the backup always includes some files with this extension.
`*.CSV`	Data for CSV tables.	These files are copied without changes. The `backup_history` and `backup_progress` tables created by mysqlbackup use the CSV format, so the backup always includes some files with this extension.
`*.MRG`	MERGE storage engine references to other tables.	The database is put into a read-only state while these files are copied. These files are copied without changes.
`*.TRG`	Trigger parameters.	The database is put into a read-only state while these files are copied. These files are copied without changes.
`*.TRN`	Trigger namespace information.	The database is put into a read-only state while these files are copied. These files are copied without changes.
`*.opt`	Database configuration information.	The database is put into a read-only state while these files are copied. These files are copied without changes.
`*.par`	Definitions for partitioned tables.	The database is put into a read-only state while these files are copied. These files are copied without changes.
`*.ARM`	ARCHIVE storage engine table metadata.	The database is put into a read-only state while these files are copied. These files are copied without changes.
`*.ARZ`	ARCHIVE storage engine table data.	The database is put into a read-only state while these files are copied. These files are copied without changes.
`backup-my.cnf`	Records the configuration parameters that specify the layout and other important information about the MySQL data files.	The file is created during a backup, and it contains crucial parameters describing the backed-up data like `innodb_data_file_path`, `innodb_log_file_size`, `innodb_log_files_in_group`, and so on. It might also contain other InnoDB parameters like `innodb_data_home_dir` and `innodb_undo_directory` if some of the backup repository options were used during the backup. mysqlbackup uses the parameters stored in this file to understand the structure of the backup and to perform various operations. You might need to supply some of these parameters to mysqlbackup during a restore and to mysqld when you start the target server if the target server and the backup are configured differently. See the discussion in Section 4.2.4, “Restoring a Database” for details.
`ibbackup_ibd_files`	Records names of the `.ibd` files and their space IDs during an incremental backup.	This file is created during an incremental backup. During a restore, the information in the file is used to delete the tables from the full backup that has been removed between the time of the full backup and the time of the incremental backup.
`ibbackup_logfile`	A condensed version of the `ib_logfile*` files from the MySQL data directory.	The InnoDB log files (`ib_logfile`) are fixed-size files that are continuously updated during the database's operation. For backup purposes, only the changes that are committed while the backup is in progress are needed. These changes are recorded in `ibbackup_logfile`, and used to re-create the `ib_logfile` files during the apply-log phase.
`ibbackup_redo_log_only`	Created instead of the `ibbackup_logfile` for incremental backups taken with the `--incremental-with-redo-log-only` option.
`ib_logfile*`	Created in the backup directory by mysqlbackup during the `apply-log` phase after the initial backup.	These files are not copied from the original data directory, but rather re-created in the backup directory during the `apply-log` phase after the initial backup, using the changes recorded in the `ibbackup_logfile` file.
`*.bl`	Renamed version of each .isl file from the backed-up server.	A `.isl` file is created when you specify the location of an `InnoDB` table using the syntax `CREATE TABLE ... DATA DIRECTORY = ...`, to act like a symbolic link pointing to the tablespace file. (See Creating Tables Externally for details.) The `.bl` files might or might not be turned back into `.isl` files during the `copy-back` operation. If the specified directory does not exist on the server where the backup is restored, mysqlbackup attempts to create it. If the directory cannot be created, the restore operation fails. Thus, if you would like to use the `DATA DIRECTORY` clause to put tables at different locations or to restore to a server with a different file structure where the corresponding directories cannot be created, edit the `.bl` files before restoring to point to directories that do exist on the destination server. If the directory on the target server pointed to by a `.bl` file already contains .ibd files, the `--force` option is required when you restore the backup.
Timestamped directory, such as `2011-05-26_13-42-02`	Created by the `--with-timestamp` option. All the backup files go inside this subdirectory.	Use the `--with-timestamp` option to easily keep more than one set of backup data under the same main backup directory.
`datadir` directory	A subdirectory that stores the data files and database subdirectories from the original MySQL instance.	Created under the backup directory by mysqlbackup.
binary log files	Binary log files from the server, which are included in a backup by default (except when the backup is created with the `--use-tts` option). They allow a snapshot of the server to be taken, so a server can be cloned to its exact state. Using a full backup as a basis, the binary log files that are included with an incremental backup can be used for a point-in-time recovery (PITR), which restores a database to its state at a certain point in time after the last full backup. See Section 5.3, “Point-in-Time Recovery” for details.	Saved under the `datadir` directory under the backup directory. Use the `--skip-binlog` option to exclude binary logs in the backup. For MySQL 5.5 and earlier, as well as all offline backups, use the `--log-bin-index` option to specify the absolute path of the index file on the MySQL server that lists all the used binary log files, if it is different from the default value of the option, for mysqlbackup to find the binary log files and include them in the backups. The index file itself, with the locations of the binary log files properly updated to point to the files' locations in the backup directory, is included into the backup under the `datadir` directory. The binary log files are compressed and saved with the `.bz` extension when being included in a compressed backup. The binary log files and the index file, when included in a backup, are always copied into the restored server's data directory during a restore operation. Use the `--skip-binlog` option to skip the restoring of the binary log. Notes If any binary log files are missing on the server you are backing up, you should use the `--skip-binlog` option to avoid mysqlbackup throwing an error for the missing files. No binary log files are copied into the incremental backup if the `--use-tts` option or the `--start-lsn` option is used. To include binary log files for the period covered by the incremental backup, do not use the `--use-tts` option and, instead of `--start-lsn`, use the `--incremental-base` option, which provides the necessary information for mysqlbackup to ensure that no gap exists between binary log data included in the previous backup and the current incremental backup.
relay log files	Relay log files from a replica server, which are included in a backup of a replica server by default (except when the backup is created with the `--use-tts` option). Their inclusion saves the time and resources required for fetching the relay logs from the source when the replica is being restored.	Saved under the `datadir` directory under the backup directory. Use the `--skip-relaylog` option to exclude relay logs in the backup. For offline backup, use the `--relay-log-index` option to specify the absolute path of the index file on the MySQL server that lists all the used relay log files, if it is different from the default value of the option, for mysqlbackup to find the relay log files and include them in the backups. The index file itself, with the locations of the relay log files properly updated to point to the files' locations in the backup directory, is included into the backup under the `datadir` directory. The relay log files are compressed and saved with the `.bz` extension when being included in a compressed backup. The relay log files and the index file, when included in a backup, are always copied into the restored server's data directory during a restore operation. Use the `--skip-relaylog` option to skip the restoring of the relay log.
*.bz	Compressed binary log or relay log files.	The binary log and relay log files are compressed and saved with the `.bz` extension when being included in a compressed backup. They are decompressed during a restore.
replication metadata repository files	Usually named `master.info` and `relay-log.info`, they are included by default in a backup of a replica database in a replication setup. See Replication Metadata Repositories, for details.	Saved under the `datadir` directory under the backup directory. For an offline backup, use the`--master-info-file` and `--relaylog-info-file` options to specify the absolute paths of the information files, if they are different from the default values of the options, for mysqlbackup to find those files and include them in the backups. The copying of these files are skipped during a backup or a restore when the `--skip-relay-log` option is used.
Backup image file	A single-file backup produced by the `backup-to-image` command, with a name specified by the `--backup-image` option.	If your backup data directory consists only of zero-byte files, with a single giant data file in the top-level directory, you have a single-file backup. You can move the image file without losing or damaging the contents inside it, then unpack it with mysqlbackup using the `extract` option and specifying the same image name with the `--backup-image` option. Although some extra files such as `backup-my.cnf` and the `meta` subdirectory are present in the backup directory, these files are also included in the image file and do not need to be moved along with it.
Any other files in subdirectories under the `datadir` directory (that is, under `backup-dir/datadir/subdir`)	Copied from the database subdirectories under the MySQL data directory.	By default, any unrecognized files in subdirectories under the MySQL data directory are copied to the backup. To omit such files, specify the `--only-known-file-types` option. Note Some limitations apply to this behavior. See the discussion here in Appendix B, Limitations of MySQL Enterprise Backup.
`meta` directory	A subdirectory that stores files with metadata about the backup.	Created under the backup directory by mysqlbackup. All files listed below go inside the `meta` subdirectory.
`backup_variables.txt`	Holds important information about the backup. For use by mysqlbackup only.	mysqlbackup consults and possibly updates this file during operations after the initial backup, such as the apply-log phase or the restore phase.
`image_files.xml`	Contains the list of all the files (except itself) that are present in the single-file backup produced by the `backup-to-image` or `backup-dir-to-image` options. For details about this file, see Section 11.4, “Using the MySQL Enterprise Backup Manifest”.	This file is not modified at any stage once generated.
`backup_create.xml`	Lists the command line arguments and environment in which the backup was created. For details about this file, see Section 11.4, “Using the MySQL Enterprise Backup Manifest”.	This file is not modified once it is created. You can prevent this file from being generated by specifying the `--disable-manifest` option.
`backup_content.xml`	Essential metadata for the files and database definitions of the backup data. It also contains details of all the plugins defined on the backed-up server, by which users should make sure the same plugins are defined in the same manner on the target server for restoration. For details about this file, see Section 11.4, “Using the MySQL Enterprise Backup Manifest”.	This file is not modified once created. You can prevent this file from being generated by specifying the `--disable-manifest` option.
`comments.txt`	Produced by the `--comments` or `--comments-file` option.	The comments are specified by you to document the purpose or special considerations for this backup job.
`backup_gtid_executed.sql`	Signifies the backup came from a server with GTIDs enabled.	GTIDs are a replication feature in MySQL 5.6 and higher. See Replication with Global Transaction Identifiers for details. When you back up a server with GTIDs enabled using mysqlbackup, the file named `backup_gtid_executed.sql` is created in the `meta` folder under the backup directory. Edit and execute this file after restoring the backup data on a replica server; see Section 6.1, “Setting Up a New Replica” for details.
`server-my.cnf`	Contains values of the backed-up server's global variables that are set to non-default values. Use this file or `server-all.cnf` to start the target server for restoration.	During a `copy-back` or `copy-back-and-apply-log` operation, the server repository options values (e.g., `--datadir`, `--innodb_data_home_dir`, etc.) in the file are modified if the command makes changes to them through the command options. However, during an `apply-incremental-backup` operation, the values already saved in the file take precedence and they are not modified by the option values supplied through the command. Warning When using the file to restart the target server, change parameters like `--tmpdir`, `--general-log`, etc., and any global variable that uses an absolute filepath to avoid the accidental usage of the wrong file locations by the target server.
`server-all.cnf`	Contains values of all the global variables of the backed-up server. Use this file or `server-my.cnf` to start the target server for restoration.	During a `copy-back` or `copy-back-and-apply-log` operation, the server repository options values (e.g., `--datadir`, `--innodb_data_home_dir`, etc.) in the file are modified if the command makes changes to them through the command options. However, during an `apply-incremental-backup` operation, the values already saved in the file take precedence and they are not modified by the option values supplied through the command. Warning When using the file to restart the target server, change parameters like `--tmpdir`, `--general-log`, etc., and any global variable that uses an absolute filepath to avoid the accidental usage of the wrong file locations by the target server.

InnoDB Data

Data managed by the InnoDB storage engine is always backed up. The primary InnoDB-related data files that are backed up include the ibdata* files (which represent the system tablespace and possibly the data for some user tables), any .ibd files (which contains data from user tables created with the file-per-table setting enabled), and the data extracted from the ib_logfile* files (the redo log information representing changes that occur while the backup is running), which is stored in a new backup file ibbackup_logfile.

If you use the compressed backup feature, the .ibd files are renamed in their compressed form to .ibz files.

The files, as they are originally copied, form a raw backup that requires further processing before it is ready to be restored. You then run the apply step, which updates the backup files based on the changes recorded in the ibbackup_logfile file, producing a prepared backup. At this point, the backup data corresponds to a single point in time. The files are now ready to be restored to their original location, or for some other use, such as testing, reporting, or deployment as a replica.

To restore InnoDB tables to their original state, you must also have the corresponding .frm files along with the backup data. Otherwise, the table definitions could be missing or outdated if someone has run ALTER TABLE or DROP TABLE statements since the backup. By default, mysqlbackup automatically copies the .frm files during a backup operation and restores the files during a restore operation.

Data from MyISAM and Other Storage Engines

mysqlbackup also backs up the .MYD files, .MYI files, and the .frm files associated with the MyISAM tables. Files with other extensions that are backed up are shown in this list.

Note

While MySQL Enterprise Backup can back up non-InnoDB data (like MYISAM tables), the MySQL server to be backed up must support InnoDB (i.e., the backup process will fail if the server was started up with the --innodb=OFF or --skip-innodb option), and the server must contain at least one InnoDB table.

MyISAM tables and these other types of files cannot be backed up in the same non-blocking way as InnoDB tables can. They are backed up using the warm backup technique: changes to these tables are prevented while they are being backed up, possibly making the database unresponsive for a time, but no shutdown is required during the backup.

Note

To avoid concurrency issues during backups of busy databases, you can use the --only-innodb or --only-innodb-with-frm option to back up only InnoDB tables and associated data.

Generated Files Included in the Backup

The backup data includes some new files that are produced during the backup process. These files are used to control later tasks such as verifying and restoring the backup data. The files generated during the backup process include:

meta/backup_create.xml: Lists the command line arguments and environment in which the backup was created.
meta/backup_content.xml: Essential metadata for the files and database definitions of the backup data.
backup-my.cnf: Records the crucial configuration parameters that apply to the backup. These configuration parameters are read by mysqlbackup during operations like apply-log to determine how the backup data is structured. These parameters are also checked during a restore operation for their compatibility with your target server's configuration.
server-my.cnf: Contains values of the backed-up server's global variables that are set to non-default values.
server-all.cnf: Contains values of all the global variables of the backed-up server.

For details about all the files in the backup directory, see Table 1.1, “Files in a MySQL Enterprise Backup Output Directory”.

Single-File Backups

Depending on your workflow, you might want to perform a single-file backup rather than a directory backup, which produces a separate file for every file on the original server instance. A single-file backup is easier to transfer to a different system, to compress, and to uncompress; it also helps to prevent the situation in which individual files that form parts of a backup are deleted by mistake. It is just as fast as a multi-file backup to do a full restore; restoring individual files can be slower than in a multi-file backup. For instructions, see Section 4.3.5, “Making a Single-File Backup”.