How TSM server determines the eligibility of files during different types of backup

Tivoli Storage Manager offers different types of backup methods for efficient use of tape resources and for faster backup window timings. In this post we will see how IBM Tivoli Storage Manager selects files  and determine the eligibility of client files during client backups and archive methods.

Which files are eligible during different types of Backup ?

Backup-archive clients can choose to back up their files using full or partial incremental backup. A full incremental backup ensures that clients' backed-up files are always managed according to policies. Clients should use full incremental backup whenever possible.

If the amount of time for backup is limited, clients may sometimes need to use partial incremental backup. A partial incremental backup should complete more quickly and require less memory. When a client uses partial incremental backup, only files that have changed since the last incremental backup are backed up. Attributes in the management class that would cause a file to be backed up when doing a full incremental backup are ignored. For example, unchanged files are not backed up even when they are assigned to a management class that specifies absolute mode and the minimum days between backups (frequency) has passed.

The server also does less processing for a partial incremental backup. For example, the server does not expire files or rebind management classes to files during a partial incremental backup.

Also Read: Full vs Differential vs Incremental vs Progressive Incremental Backups 

If clients must use partial incremental backups, they should periodically perform full incremental backups to ensure that complete backups are done and backup files are stored according to policies. For example, clients can do partial incremental backups every night during the week, and a full incremental backup on the weekend.

Performing full incremental backups is important if clients want the ability to restore files to a specific time. Only a full incremental backup can detect whether files have been deleted since the last backup. If full incremental backup is not done often enough, clients who restore to a specific time may find that many files that had actually been deleted from the workstation get restored. As a result, a client's file system may run out of space during a restore process. See Setting Policy to Enable Point-in-Time Restore for Clients for more information.

Which files get backup during Full Incremental Backup

When a user requests a full incremental backup, IBM Tivoli Storage Manager performs the following steps to determine eligibility:

Checks each file against the user's include-exclude list

• Files that are excluded are not eligible for backup.

• If files are not excluded and a management class is specified with the INCLUDE option, IBM Tivoli Storage Manager uses that management class.

• If files are not excluded but a management class is not specified with the INCLUDE option, IBM Tivoli Storage Manager uses the default management class.

• If no include-exclude list exists, all files in the client domain are eligible for backup, and IBM Tivoli Storage Manager uses the default management class.

Also Read: Different types of Incremental Backups

Checks the management class of each included file

• If there is a backup copy group, the process continues with checking mode and frequency settings.

• If there is no backup copy group, the file is not eligible for backup.

Checks the mode, frequency, and serialization defined in the backup copy group

• Mode - Specifies whether the file is backed up only if it has changed since the last backup (modified) or whenever a backup is requested (absolute).

• Frequency - Specifies the minimum number of days that must elapse between backups. For windows this attribute is ignored during a journal-based backup.

• Serialization - Specifies how files are handled if they are modified while being backed up and what happens if modification occurs. 

If the mode is modified and the minimum number of days have elapsed since the file was last backed up, IBM Tivoli Storage Manager determines if the file has been changed since it was last backed up

• If the file has been changed and the serialization requirement is met, the file is backed up.

• If the file has not been changed, it is not backed up.

• If the mode is modified and the minimum number of days have not elapsed since the file was last backed up, the file is not eligible for backup.

• If the mode is absolute, the minimum number of days have elapsed since the file was last backed up, and the serialization requirement is met, the file is backed up.

• If the mode is absolute and the minimum number of days have not elapsed since the file was last backed up, the file is not eligible for backup.

Which files get backup during Partial Incremental Backup

When a user requests a partial incremental backup, IBM Tivoli Storage Manager performs the following steps to determine eligibility:

Checks each file against the user's include-exclude list

• Files that are excluded are not eligible for backup.

• If files are not excluded and a management class is specified with the INCLUDE option, the server uses that management class.

• If files are not excluded but a management class is not specified with the INCLUDE option, the server uses the default management class.

• If no include-exclude list exists, all files in the client domain are eligible for backup, and the server uses the default management class.

Also Read: Different types of Image Backups

Checks the management class of each included file

• If there is a backup copy group, the process continues with next step.

• If there is no backup copy group, the file is not eligible for backup.

Checks the date and time of the last incremental backup by the client, and the serialization requirement defined in the backup copy group

• Serialization specifies how files are handled if they are modified while being backed up and what happens if modification occurs.

• If the file has not changed since the last incremental backup, the file is not backed up.

• If the file has changed since the last incremental backup and the serialization requirement is met, the file is backed up.

Which files get backup during Selective Backup

When a user requests a selective backup, IBM Tivoli Storage Manager performs the following steps to determine eligibility:

Checks the file against any include or exclude statements contained in the user include-exclude list

• Files that are not excluded are eligible for backup. If a management class is specified with the INCLUDE option, IBM Tivoli Storage Manager uses that management class.

• If no include-exclude list exists, the files selected are eligible for backup, and IBM Tivoli Storage Manager uses the default management class.

Checks the management class of each included file

• If the management class contains a backup copy group and the serialization requirement is met, the file is backed up. Serialization specifies how files are handled if they are modified while being backed up and what happens if modification occurs.

• If the management class does not contain a backup copy group, the file is not eligible for backup.

Also Read: Understanding Management Class Binding and Management Class Rebinding ?

An important characteristic of selective backup is that a file is backed up without regard for whether the file has changed. This result may not always be what you want. For example, suppose a management class specifies to keep three backup versions of a file. If the client uses incremental backup, the file is backed up only when it changes, and the three versions in storage will be at different levels. If the client uses selective backup, the file is backed up regardless of whether it has changed. If the client uses selective backup on the file three times without changing the file, the three versions of the file in server storage are identical. Earlier, different versions are lost.

Which files get backup during Logical Volume Backup

When a user requests a logical volume backup, IBM Tivoli Storage Manager performs the following steps to determine eligibility:

Checks the specification of the logical volume against any include or exclude statements contained in the user include-exclude list

• If no include-exclude list exists, the logical volumes selected are eligible for backup, and IBM Tivoli Storage Manager uses the default management class.

• Logical volumes that are not excluded are eligible for backup. If the include-exclude list has an INCLUDE option for the volume with a management class specified, IBM Tivoli Storage Manager uses that management class. Otherwise, the default management class is used.

Checks the management class of each included logical volume

• If the management class contains a backup copy group and the logical volume meets the serialization requirement, the logical volume is backed up. Serialization specifies how logical volumes are handled if they are modified while being backed up and what happens if modification occurs.

• If the management class does not contain a backup copy group, the logical volume is not eligible for backup.

Which files get archived during Archive

When a user requests the archiving of a file or a group of files, IBM Tivoli Storage Manager performs the following steps to determine eligibility:

Checks the files against the user's include-exclude list to see if any management classes are specified

• IBM Tivoli Storage Manager uses the default management class for files that are not bound to a management class.

• If no include-exclude list exists, IBM Tivoli Storage Manager uses the default management class unless the user specifies another management class. See the user's guide for the appropriate client for details.

Also Read: TSM Server Performance Tuning Parameters

Checks the management class for each file to be archived

• If the management class contains an archive copy group and the serialization requirement is met, the file is archived. Serialization specifies how files are handled if they are modified while being archived and what happens if modification occurs.

• If the management class does not contain an archive copy group, the file is not archived.

If you need to frequently create archives for the same data, consider using instant archive (backup sets) instead. Frequent archive operations can create a large amount of metadata in the server database resulting in increased database growth and decreased performance for server operations such as expiration. Frequently, you can achieve the same objectives with incremental backup or backup sets. Although the archive function is a powerful way to store inactive data with fixed retention, it should not be used on a frequent and large scale basis as the primary backup method.