CV SYS Admin Curse-R02

System Administration Training Guide

ii Copyright ©1999-2007 CommVault Systems, Inc. All rights reserved. CommVault, the “CV” logo, CommVault Systems, Solving Forward, SIM, Singular Information Management, Simpana, CommVault Galaxy, Unified Data Management, QiNetix, Quick Recovery, QR, GridStor, Vault Tracker, QuickSnap, QSnap, Recovery Director, CommServe, CommCell, and InnerVault are trademarks or registered trademarks of CommVault Systems, Inc. All other third party brands, products, service names, trademarks, or registered service marks are the property of and used to identify the products or services of their respective owners. All specifications are subject to change without notice.

CommVault System Administration Course – R02

iii

TABLE OF CONTENTS User Administration Module Objectives:................................................................................... 8 CommCell Security Basics ......................................................................................................... 9 CommCell Console Access Options ......................................................................................... 10 User Identification .................................................................................................................... 12 Users and User Groups ............................................................................................................. 14 Understanding Users/User Groups ........................................................................................... 15 Understanding Capabilities & Managed Objects...................................................................... 16 Command Line Access ............................................................................................................. 18 Increased Security Options ....................................................................................................... 20 Using Computer Client Groups ................................................................................................ 22 CommCell Administration Options .......................................................................................... 24 E-mail and IIS Configuration.................................................................................................... 25 System Thresholds .................................................................................................................... 26 User Preferences ....................................................................................................................... 27 Automatic Update Configuration.............................................................................................. 29 DR Backup Settings.................................................................................................................. 31 Summary ................................................................................................................................... 33 Library Management Module Objectives: ............................................................................... 36 Understanding Libraries............................................................................................................ 37 Media ........................................................................................................................................ 38 Connection ................................................................................................................................ 40 NDMP Libraries........................................................................................................................ 43 PnP Libraries............................................................................................................................. 45 Content Addressable Storage .................................................................................................... 46 Single Instanced ........................................................................................................................ 47 Configuring Media Agent Options............................................................................................ 49 Using Shared Index Cache........................................................................................................ 51 Configuring Magnetic Libraries ............................................................................................... 53 Mount Path Usage Options ....................................................................................................... 55 Mount Paths Settings ................................................................................................................ 57 Configuring Removable Media Libraries ................................................................................. 59 Library Settings......................................................................................................................... 60 Drive Settings............................................................................................................................ 63 Summary ................................................................................................................................... 65 Media Management Module Objectives:.................................................................................. 68 Understanding Media Management.......................................................................................... 69 Types of Media ......................................................................................................................... 70 How Data is Stored ................................................................................................................... 72 Removable Media States........................................................................................................... 73 Removable Media Pools ........................................................................................................... 75 Managing Media Content ......................................................................................................... 77 Job State on Media.................................................................................................................... 79 Removable Media Usage .......................................................................................................... 82

CommVault System Administration Course – R02

iv Media Labeling ......................................................................................................................... 84 Threshold Settings .................................................................................................................... 86 Managing Internal and External Media .................................................................................... 87 Exporting Media ....................................................................................................................... 89 Export using Vault Tracker ....................................................................................................... 91 Using Virtual Mail Slots ........................................................................................................... 92 Using Containers....................................................................................................................... 93 Vault Tracker Jobs..................................................................................................................... 94 Importing Media ....................................................................................................................... 96 Discovering new Media ............................................................................................................ 98 Verifying Existing Media........................................................................................................ 100 Summary ................................................................................................................................. 101 Storage Policies Module Objectives: ....................................................................................... 104 Understanding Storage Policies .............................................................................................. 105 Design of Storage Policies ...................................................................................................... 106 Types of Storage Policy Copies .............................................................................................. 108 Updating Secondary Copies.................................................................................................... 109 Managing Data Streams .......................................................................................................... 111 Job vs. Device Streams ........................................................................................................... 112 Parallel vs. Multiplexing......................................................................................................... 113 Combine to Streams ................................................................................................................ 115 Data Retention, Management, and Aging ............................................................................... 116 Determining Data Retention Requirements ............................................................................ 118 Managing Backup and Archive Retention .............................................................................. 120 Understanding Cycles and Days ............................................................................................. 121 Data Aging .............................................................................................................................. 122 Retention Variants ................................................................................................................... 124 Roles of Verification, Encryption and Content Indexing........................................................ 127 Inline vs. Copy-based Encryption........................................................................................... 128 Content Indexing..................................................................................................................... 130 Content Indexing and Search .................................................................................................. 131 Data Verification ..................................................................................................................... 133 Summary ................................................................................................................................. 134 Clients Module Objectives: ...................................................................................................... 136 What are Clients?.................................................................................................................... 137 Understanding the role of Agents ........................................................................................... 138 Backup and Archive Sets ........................................................................................................ 140 Understanding Backup and Archive sets ................................................................................ 141 Using On Demand BackupSets............................................................................................... 143 Data Classification for Archive and Backup........................................................................... 145 Configuring Subclients ........................................................................................................... 147 Defining Content and Content Options................................................................................... 148 Using Filters............................................................................................................................ 150 Using Pre/Post Process Commands ........................................................................................ 152 Setting Storage Options .......................................................................................................... 154 Using Subclient Policy............................................................................................................ 156

CommVault System Administration Course – R02

v Summary ................................................................................................................................. 157 Job Management Module Objectives: .................................................................................... 160 Configuring Data Protection Tasks ......................................................................................... 161 Types of Data Protection......................................................................................................... 162 Understanding Synthetic Full ................................................................................................. 164 Archive Concept – HSM......................................................................................................... 165 Executing Data Protection Tasks ............................................................................................ 167 Job Initiation Options.............................................................................................................. 168 Using Backup/Archive Options .............................................................................................. 170 Creating a Schedule Policy ..................................................................................................... 172 Using Command Line ............................................................................................................. 173 Using Backup/Archive Options .............................................................................................. 175 Activity Controls..................................................................................................................... 178 Operation Windows ................................................................................................................ 180 Job Management Control........................................................................................................ 182 Job Controller Window ........................................................................................................... 184 Summary ................................................................................................................................. 186 Restore Module Objectives: ..................................................................................................... 188 Image/No-Image Browse ........................................................................................................ 190 List Media & List Media (Precise) ......................................................................................... 192 Exact Index ............................................................................................................................. 194 Find ......................................................................................................................................... 195 Understanding Restore and Recovery..................................................................................... 197 Restore/Recall Options ........................................................................................................... 198 Restore by Job......................................................................................................................... 199 Mapped Restores..................................................................................................................... 201 Recall Archived Objects ......................................................................................................... 203 Copy Precedence..................................................................................................................... 205 Restore From Anywhere ......................................................................................................... 207 Out-of-Place Restores ............................................................................................................. 208 Restores to Non-client hosts ................................................................................................... 209 Working with stub files ........................................................................................................... 210 Full System Restore ................................................................................................................ 212 Summary ................................................................................................................................. 214 Monitoring Module Objectives:............................................................................................... 216 CommCell Explorer ................................................................................................................ 218 CommNet................................................................................................................................ 220 Job Controller.......................................................................................................................... 223 Resource View ........................................................................................................................ 225 Event Viewer........................................................................................................................... 226 Understanding Alerts .............................................................................................................. 229 Alert Output Options............................................................................................................... 230 Recommended Alerts .............................................................................................................. 231 Available Reports.................................................................................................................... 234 Recommended Reports ........................................................................................................... 235 Summary ................................................................................................................................. 238

CommVault System Administration Course – R02

vi Appendix.................................................................................................................................... 239 Vault Tracker vs. Vault Tracker Enterprise............................................................................ 240 CommCell ............................................................................................................................... 241 Training Environment............................................................................................................. 242

CommVault System Administration Course – R02

User Administration - 7

User Administration

www.commvault.com/training

CommVault System Administration Course – R02

8 – User Administration

¡ Overview CommCell Security Basics „ Users and User Groups „ Increased Security Options „ Using Client Computer Groups „ CommCell Administration Options „

User Administration Module Objectives: • • • • •

List and describe the prescribed means to secure and access the CommCell Management Interface. Understand the purpose for CommCell User Group and how they are created and managed. Create a CommCell User group with capabilities and associate it to an Object for administration. Understand how external Active Directory Users and User Groups are integrated into a CommCell. List and describe common CommCell Management Interface settings and options for basic administration tasks.

CommVault System Administration Course – R02

User Administration - 9

¡ CommCell Security Basics „ „

CommCell Console Access Options User Identification

CommCell Security Basics CommVault Security supplements the host OS Security. To be independent and applicable to all operating systems, a CommCell User has no association with users and security of the host operating system. CommCell Users have access to the resources and features of the CommCell based on the following: • • •

CommCell User Group Membership CommCell User Group capabilities CommCell User Group resource associations

CommVault System Administration Course – R02

10 – User Administration

CommCell Console Access Options „

Local CommCell Console z

„

Remote CommCell Console z

„

Installed by Default Windows, Unix, Macintosh

Web-Based Console z

JAVA-enabled Browser

CommCell Console Access Options Local and Remote The CommCell Console is a SUN Java Runtime Environment based graphical user interface that allows you to control and manage a CommCell. The CommCell Console is automatically selected for installation when installing the CommServe component and can be installed as a separate component on Windows, UNIX, or Macintosh hosts. The installed CommCell Console can be run in two ways: •

As a stand-alone Java application installed directly onto your computer.

•

Using Java Web Start, which allows access to the CommCell console remotely via any computer running a supported platform with a java-enabled platform. This allows remote CommCell Console sessions to be active, so multiple users can access the CommServe simultaneously.

Web-based During the installation of the CommServe component - if the Internet Information Server (IIS) is installed and running on the same host - you are offered an option to configure the CommServe for web administration. Note that CommServe web administration is only possible with IIS. If the CommServe computer does not have IIS installed, or was not configured for web administration during installation, you can at anytime enable local web administration or

CommVault System Administration Course – R02

User Administration - 11 configure web administration via an alternate IIS host. Note that whatever IIS host is used it must have the CommCell Console installed. If using an IIS host different than the CommServe the CommCell Console’s e-mail and IIS Server applet must be configured to use the alternate IIS host. Once configured for web administration, any java enabled browser can be used to access the Java applet and perform remote administration. The web-based version of the CommCell Console has the same appearance and functionality as the installable CommCell Console Java application. Note that all login sessions where a password is required are done using Secure Socket Layer (SSL) protection.

CommVault System Administration Course – R02

12 – User Administration

User Identification „

CommCell User z z

„

External User z z

„

SMTP Address (for Alerts & Notifications) Password Aging Associated Active Directory Group Single Sign On capable

Outlook User z

Advanced message recovery

User Identification CommCell User A CommCell User has access to the CommCell Console only. There are no host or network privileges assigned. A CommCell user’s ability to view CommCell Console objects and perform tasks is based on CommCell User Group membership. Each CommCell User account should have a SMTP e-mail address which is used for receiving reports and alerts. Passwords assigned to the CommCell User account can be aged to provide for periodic password changes to prevent long-term brute force hacks. External user An External user is a member of a Windows Active Directory domain group that has been associated with a CommCell User Group that has, at a minimum, Browse capability. External users must enter their name in the CommCell Login prompt window in the following format: \. When a username is entered with a domain name, the CommServe Server automatically recognizes that the password information must be authenticated by the external domain server.

CommVault System Administration Course – R02

User Administration - 13

When enabled for an associated AD domain. External users can use the Single Sign On (SSO) feature to automatically login to the CommServe using their Active Directory user-account credentials. To bypass Single Sign On in order to login as a different user, click Cancel at the CommCell Login prompt window and enter the alternate username and password. Outlook User CommCell authentication is required for end-users using the DataArchiver Outlook Add-In to perform advanced message recovery operations such as find recoveries and browse recoveries from Outlook. The Single Sign On (SSO) feature allows Exchange administrators to establish a CommCell User Group for Outlook Add-In end-users to perform these functions using their existing Windows user accounts and passwords residing in the Active Directory domain. To configure this feature: Create and enable the nUseCommonSSOUser registry key. Once the key is enabled, refresh the CommCell Console. This will automatically create a user group called Common Outlook Add-In Default Group and a user account called Default Outlook User. This group is automatically created with associations to all CommCell objects. If necessary, change this configuration in the User Group Properties (General) dialog box. Once Single Sign-On has been configured, then Outlook users may perform find and browse recoveries of archived messages without the need to enter CommCell authentication credentials. When users select the Outlook Add-In option to Find and Recover Messages, their Windows user accounts are automatically granted rights to access the CommServe to perform this function as part of a CommCell User Group.

CommVault System Administration Course – R02

14 – User Administration

¡ Users and User Groups „ „

„ „ „

Understanding User and User Groups Understanding Capabilities & Managed Objects Command Line Access Adding CommCell Users and Groups Enabling External Users/Groups

Users and User Groups Users have access to the resources and features of the CommCell based on the following: • • • •

CommCell user accounts CommCell user groups User group capabilities User group object associations

Using this approach, a CommCell administrator can provide users with the exact capabilities they are required. These requirements can vary, depending on the tasks each user needs to perform. A CommCell administrator can also restrict the CommCell objects that a user can view, by restricting the CommCell objects that a user's member user group has an association with.

CommVault System Administration Course – R02

User Administration - 15

Understanding Users and User Groups

Understanding Users/User Groups The focus point of CommCell Security is the CommCell User Group. Each group can be assigned one or more capabilities necessary to perform needed tasks. The group is then associated with objects on which these tasks may then be performed. Users derive their ability to perform tasks on objects by membership in a group. CommCell Users are created/managed within the CommCell and assigned membership to one or more groups as required. External Users are members of an External Domain Group that has been enabled and associated with a CommCell User group. Multiple associations are possible. As such, both an External and CommCell user can be members of multiple CommCell User groups with different capabilities and managed object association. Because of its widespread acceptance and level of integration, Microsoft’s Active Directory was selected as the integration path for enabling External User access into the CommCell. As such, Novell Directory Service and LDAP administrators can use their Active Directory integration tools to extend their user management/authorization into Active Directory and from there into the CommCell.

CommVault System Administration Course – R02

16 – User Administration

Understanding Capabilities & Managed Objects Must be Member of

Performs

Has To Perform

TASK

To/with

Associated to

Understanding Capabilities & Managed Objects Each CommCell User group must be assigned capabilities and objects so that its member users can perform functions within the CommCell. A CommCell User group can be assigned all capabilities and/or all associations, or individual associations and capabilities. Capabilities are privileges that allow users to perform a variety of functions within a CommCell. These functions include performing data protection, data recovery, and administration operations, such as license administration and administering user accounts. When assigning capabilities to a CommCell User group, the capabilities you assign should match the functions you want the users of that CommCell User group to perform within the CommCell. CommCell Objects are levels in the CommCell that a user group can be associated with. CommCell User groups must be given permissions to these objects. CommCell object associations enable members of a group to perform operations on a specific object. The nature of those operations depends on the capabilities assigned to the group. If an object, such as a client computer or higher level object is not associated with a given CommCell User group, then the users of that group cannot perform any operations involving that client computer. The following objects can be associated with a CommCell User group: •

The CommServe

CommVault System Administration Course – R02

User Administration - 17 • • • • • • • •

Client Computer Group Client Computer Agent Backup set Subclient Media Agent Library Storage policy

Each of these objects support specific functions within the CommCell. Two default and two permanent groups exist. These are the Master groups which have All Capabilities for All Associations, and the View All group whose members have All Association level access in the CommCell Console for viewing privileges only. Users who are not members of the View All group can only see those objects, jobs, and events to which their CommCell User Group(s) have been associated. By default, new users are automatically added to the View All group. This default characteristic can be changed in the Security tab of the CommCell level properties page.

CommVault System Administration Course – R02

18 – User Administration

Command Line Access „

„ „

Supported on any CommCell component host Single Login/Logout required Encrypted Password required

Command Line Access Command line tools can be used to perform tasks within the CommCell. CommVault’s QCommands provide access to many basic functions through the command line which can be integrated into your own scripts and scheduling programs. CommCell tasks can be configured using the CommCell Console’s graphical interface and saved as scripts or batch files on the CommServe. These scripts/batch files can be copied to and executed from any CommCell component host. Configuration No special configuration is required to use the command line interface. The commands are integrated with the software, and are therefore available on all computers which have any CommServe, Media Agent, or Agent software installed. In order for the commands to function, the Galaxy Commands Manager service should be up and running on the CommServe. This service is installed by default with the CommServe and is responsible for handling command line requests. Log in Sessions Using the Qlogin command you can start a single user login session on a component host. This removes the need to log in for every command window or shell instance. Once logged in, the authorized session remains valid until you explicitly log out using the Qlogout command.

CommVault System Administration Course – R02

User Administration - 19 Encrypted password Starting a command line session requires an encrypted password. The -p argument of the qlogin command provides for this purpose. You can obtain this encrypted value by saving any supported operation (i.e., a backup or restore job) as a script through the CommCell Console. This creates the qlogin string and encrypted password for the user that is currently logged on to the CommCell Console. You can then copy and reuse the encrypted password from that script in other scripts. Be sure you restrict read/execute privileges to scripts containing the qlogin password.

CommVault System Administration Course – R02

20 – User Administration

¡ Increased Security Options „

Restrict access to login process z

„

Restrict access to Web administration z

„

..\Galaxy\GUI directory Require Domain login

Restrict outside port access z

Limit 8401 port address

Increased Security Options CommCell security is a totally independent supplement to host OS access control. In addition to physical access, CommServe Host Administrators should restrict logical access to CommCell executables and its SQL Server metadata through normal OS security groups and permissions. Since the CommServe does allow remote administration via the Web, IIS security must also be managed to prevent unauthorized access. (Note: the CommServe uses a Secure Socket Layer (SSL) connection for its graphical interface.) Restricting access to login process The CommVault install directory structure contains a GUI folder (..\Galaxy\GUI) that contains the Java files and executables for the CommCell Console. Host administrators should restrict executable access to this directory to minimize local brute force attacks against the CommCell using the CommCell Console login prompt. Restricting access to Web Administration While having web administration capability provides a degree of freedom for the administrators and users to log in from any java-enabled browser, it could expose the CommCell to unwanted login attempts. IIS Administrators need to apply appropriate security controls such as requiring domain access first or restricting what systems can access the web administration site. Restrict port access The CommCell Console requires port 8401 be enabled in order to open a CommCell Console session. CommCell Console access from outside a firewall can be limited by restricting this port

CommVault System Administration Course – R02

User Administration - 21 and/or even changing the port to a lesser known address. It can be changed in the services file of the Windows/system32/drivers/etc directory. Once access to CommServe is allowed, CommCell security limits task execution and resource visibility of validated CommCell Users via CommCell User Groups with assigned functional capabilities and associated object management.

CommVault System Administration Course – R02

22 – User Administration

¡ Using Client Computer Groups „

Scalable Object Management Tool Enables consistent object association z Supports multi-group membership z Best Used for: z

¾ Security/Management ¾ Alerts/Reports

Using Computer Client Groups A client computer group is a logical grouping of client computers that serves as a single CommCell object in which selected options can apply to all member clients. Hence, the need to configure options for individual clients is minimized once those clients are members of the group. Once created, client computer groups can be used to perform the following tasks: • • • • •

View job history details Install updates Create new schedule policies and reports Set the activity control for data protection and/or data recovery operations within the CommCell Set the security parameters for users within the CommCell

A task configured for a client computer group will affect all the clients within the client computer group, e.g., the View Job History task will display the details for all of the clients within the client computer group. Client Computer Groups allow an administrator to scale down the CommCell Browser navigational requirements and manage multiple clients at the same time. A client can be a member of multiple groups. For example, an Exchange server may be a member of the Chicago site group and also a member of the Exchange Server groups.

CommVault System Administration Course – R02

User Administration - 23

On the security side, CommCell User Groups can be associated with a Client Computer Group for management. All clients within the CommCell User group and any clients added to the group can be managed by member users of the CommCell User Group. CommCell User Groups also add another level of Activity Control. Administrators can control multiple clients in a logical grouping with a single activity control. The greatest advantage to Client Computer Groups lies in the area of reporting and alerts. Scheduled reports and alerts can be created for each client group. As clients are added or removed from the CommCell or group, there is no need to revise the report or alert.

CommVault System Administration Course – R02

24 – User Administration

¡ CommCell Administration Options E-mail & IIS Configuration „ System Thresholds „ User Preferences „ Automatic Update Configuration „ DR Backup Settings „

CommCell Administration Options All command and control functions of the CommServe and other components are defined within the CommCell.

CommVault System Administration Course – R02

User Administration - 25

E-mail and IIS Configuration „

E-mail Server z

„

SMTP Capable

IIS Alternate for Reports z Alternate for Books Online z

E-mail and IIS Configuration If reports or alerts are configured to use e-mail as the means of notification, then an SMTP capable Mail Server is required. The SMTP capable hostname or IP address and port number can be configured in the control panel’s E-Mail and IIS Configuration applet. The standard Mail Server Port is 25 but has been know to change for security reasons in some companies. An Internet Information Service (IIS) host is required if you wish to: • • •

Access the CommCell Console as a Java Applet Access and generate reports remotely Access Books Online remotely

The support for separate IIS hosts for Books Online and Reports is based upon expressed customer security requirements for distinct access ability and separation. Web servers other than Microsoft’s IIS are not supported. Alternate JAVA capable web browsers for viewing reports are supported.

CommVault System Administration Course – R02

26 – User Administration

System Thresholds „

Event Viewer z

„

GUI Timeout z

„

Default -10,000 Events retained Improve response time

Database Checks z

Warning on disk space availability

System Thresholds Event Viewer The event viewer provides a real-time display of all relevant events in the CommCell. By default, 10,000 events are logged and the most recent 200 are displayed. A max of 1000 events can be configured for display in the User Preference applet. However, the more events selected for display, the longer the CommCell Console takes to initiate, and the more memory is required to maintain the CommCell Console session. Note that during Event Log search action, all 10,000 events are included regardless of how many events are initially displayed. GUI Timeout To improve individual response time of each CommCell Console you should keep the amount of CommCell Consoles that are concurrently opened to a minimum. Obviously, updating many CommCell Consoles displays can impact performance. To assist you in this effort, you can enable the GUI timeout feature in the Control Panel’s System control options to disconnect when a console session is inactive for a certain period of time. Database Checks The Database Space Check thresholds generate events and alerts if there is insufficient disk space for the CommServe Database to grow. While the database growth rate is low, the impact of having insufficient disk space can be severe. Events are generated and sent to the Event viewer when the database disk space is below the defined Information, Minor, Major, and Critical thresholds. The Disk Space Low alert, if configured, is only generated when the CommServe Database disk space falls below the defined Critical threshold. Changes to the Database Space Check Interval take effect immediately.

CommVault System Administration Course – R02

User Administration - 27

User Preferences Default Browser for Reports „ GUI Look & Feel „ Job/Event Filters „ Window Settings „

User Preferences User preferences are configurable for each user login session and are retained for subsequent sessions. Reports Microsoft Internet Explorer is the default browser used to view reports. You are able to change the browser to any other preferred browser (i.e., Mozilla, Firefox). GUI Look & Feel You can configure the CommCell Console display to use one of the following styles: •The Metal style, resembling a Java application environment •The CDE/Motif style, resembling a UNIX environment •The Windows style, resembling the Microsoft Windows environment Job Filter When a job completes, the Job Controller window continues to show the job and its final status for a configurable number of seconds (300 by default). In an environment where most jobs are performed off hours and alerts or reports are used for monitoring the results, reducing the display time can improve GUI performance.

CommVault System Administration Course – R02

28 – User Administration Event Filter Events can be filtered to display by their severity levels. In an active environment, limiting the displayed events to major and critical, or just critical can clear the display of unnecessary information and improve GUI response times. Window Settings (7.0) Within the CommCell Console you are able to make adjustments to the size, shape, and content of any of the task windows. The configuration of the three primary windows – CommCell Browser, Job Controller, and Event Viewer can be saved for subsequent sessions.

CommVault System Administration Course – R02

User Administration - 29

Automatic Update Configuration „

Updates z

„

Update Download Site z

„

FTP or HTTP Proxy

Update Cache/Share z z

„

Service Packs & Certified Patches

Local – Supports all OS types Shared – CIFS/NFS locations

Install Updates z

Able to Schedule

Automatic Update Configuration The Automatic Update feature allows for quick and easy installation of updates in your CommCell, ensuring that the software is up-to-date. This feature includes a cache for holding update packages. Update packages can either be downloaded automatically to this cache via FTP or manually added to the cache using the CopyToCache tool. This tool is available in the base directory of the CommVault installation folder. Once update packages are available in the cache, the system provides several options to seamlessly install the updates to all the computers in the CommCell. CommCell Components can be selected for updating individually, as a group, run on demand, or on a scheduled basis. If an update requires a reboot of the system, this can be denied or allowed as required. Baseline The updates found in the CommServe cache directory are considered the baseline for the updates that should be distributed throughout the CommCell environment. Maintaining update consistency will prevent potential software conflicts. If an update is applied to a component that is not communicating with the CommServe, the reported update status of the component may be

CommVault System Administration Course – R02

30 – User Administration inaccurate. A CommCell Summary report that includes Update status will query all components and resolve and update status inconsistencies. FTP Download The FTP download operation downloads the updates from the FTP source site to the CommServe cache directory. This server name can be changed if your FTP site for downloading updates differs from the default location or if you would like to reroute the FTP transfer through an FTP proxy. If necessary you can also configure an HTTP proxy server to connect to the FTP server to download the updates. The FTP download can be scheduled or run on-demand. The software also provides you with the option to schedule FTP download operations during the install of the CommServe software. FTP download operation will download updates based on the operating system of the clients currently available in the CommCell. The Cache Directory The CommServe cache directory is configured to serve as a holding area for update packages. This can either be a local drive on the CommServe or a shared network directory. If the CommServe cache area resides on a local drive, the appropriate CommServe service is used to copy and then install updates locally on the appropriate CommCell computers. If the CommServe cache directory resides on a share, this must be a shared location with permissions set to write to the directory. Client computers that are not in the same domain as the domain in which the CommServe cache is located, must have bidirectional trust in place. Windows computers that have permissions to access the share can install updates directly from it, without the need to first copy them. UNIX computers can use this directory for installation after copying the updates to a local directory. However, if you wish to configure UNIX computers to install updates directly from a cache, a separate UNIX cache (i.e., an NFS Mount Path) must be created. If a UNIX share is configured, and an automatic download of UNIX updates from the FTP site has occurred, a copy of each UNIX update is put in the CommServe cache and then in the UNIX cache.

CommVault System Administration Course – R02

User Administration - 31

DR Backup Settings „

Protects z z z z

„

CommServe Database CommServe Registry Firewall settings Log Files

Destination z z

Backup – Magnetic Export – Removable Media

DR Backup Settings The software stores all information for the CommCell in a SQL database, and in the Windows 2000 and Windows 2003 registries. It is critical to be able to retrieve this information in the case of a disaster or system failure. This metadata and Windows registry data are backed up during a Disaster Recovery (DR) Backup. This data can be browsed and then restored using the CommServe Recovery Tool. Determine the Number of Disaster Recovery Backups to Retain on Disk When the metadata is backed up and archived, it remains valid (available for restoration) for a period of time determined by the number of backup sets to retain. The retention rule is determined by the number of successful full backup cycles to be maintained. The number of Disaster Recovery backup sets is preset to five full backups during installation. This can be changed from the DR Backup Settings dialog box. This retention count refers to the number of backup sets created during the Export phase of the Disaster Recovery Backup. Retention time for copies written to media drives during the Backup phase are determined by the characteristics of the Disaster Recovery Backup storage policy. A DR Backup has two possible phases. Phase I should be to a magnetic share as far away as practical from the CommServe Host. This is your primary source of recovery and it should be located where a disaster that befalls the CommServe does not affect the Phase I backup copy. A Phase II DR Backup is assigned to a DR Storage Policy or a Standard Storage Policy and should be written to a removable media library. As with Phase I, the intent is to place a copy of the CommServe Metadata offsite for disaster recovery.

CommVault System Administration Course – R02

32 – User Administration

DR Backups can be configured to run at user defined times. Right click on the CommServe in the CommCell browser; select all tasks and Disaster Recovery backup. CommServe DR jobs can be scheduled or run on-demand. A best practice for managing your CommCell environment would be to perform a DR backup before making any major modifications to the backup environment. This would include modifying or deleting storage policies, redefining subclient content, and adding/removing libraries. Disaster Recovery Backups Content Disaster Recovery backs up the following: Metadata - Metadata includes the Microsoft SQL Server database that holds information about all CommCell database components (including clients and media configuration) Windows registry - The Windows registry is a central resource from which the Windows operating system obtains many of the system’s operating parameters. Firewall Configuration Files – The firewall configuration files (FwPeers.txt, FwHosts.txt and FwPorts.txt) are also included in the Disaster Recovery backup. If necessary, the entries associated with Clients/Media Agents on the other side of the firewall from the CommServe can be restored in the event of a CommServe re-build. Note that a restore of the disaster recovery backup does not automatically restore the firewall files. To restore these files, manually select these files for restore from the CommserveDisasterRecoveryGUITool.

CommVault System Administration Course – R02

User Administration - 33

Module Summary Key points to remember

Summary •

Creation of Users & User Groups (External Security (AD integration, SSO), Users with just SMTP address for alerts and reports)

•

Creating Client Computer Groups (consolidate security, reporting, activity control, updates)

•

What are Capabilities? (where to apply levels –CC, MA, CL, DA, SP,SUB, LIB, Client Comp Group, BackupSet)

•

Where can I go to access CommCell Configuration Options? (e-mail, IIS, DR backup settings)

•

Configuring Auto Updates (D/L from FTP, Cache area, client install options, MA update options)

CommVault System Administration Course – R02

34 – User Administration

CommVault System Administration Course – R02

Library Management Module - 35

Library Management

www.commvault.com/training

CommVault System Administration Course – R02

36 – Library Management Module

¡ Overview Understanding Libraries „ Configuring Media Agents „ Configuring Magnetic Libraries „ Configuring Removable Media Libraries „

Library Management Module Objectives: • • • • • •

List and describe the types of Media Agents and Libraries configurations available. Describe the role and configuration options of the index and index cache in data protection, recovery, storage, and content. Configure and understand the common settings and options for use of a magnetic library and associated media. Configure and understand the common settings and options for use of a robotic tape library and associated media. Configure and understand the common settings and options for use of a Standalone or PnP Library and associated media. Understand the use and implementation of NDMP or IP-Based libraries.

CommVault System Administration Course – R02

Library Management Module - 37

¡ Understanding Libraries Consistent term for all protected storage devices „ Types of Libraries „

Media z Connection z Special z

Understanding Libraries Libraries is a consistent term used by the software to refer to any managed storage device and its associated attributes. Even standalone drives are referred to as libraries. Libraries are managed by Media Agents. In some cases a single Media Agent may manage multiple libraries and multiple Media Agents may have access to the same library. All data written to, or read from a library must transit through a Media Agent.

CommVault System Administration Course – R02

38 – Library Management Module

Media „

Magnetic Disk z z

„

Read/Write WORM

Removable Media z z z

Tape Magnetic Optical USB Storage Device

Media Magnetic Disk Magnetic disk - as a protected storage library is supported on any OS supported file system. Disks, disk partitions, or logical volumes are addressed as mounted file systems or by network protocol (CIFS/NFS). Each addressable storage unit is referred to as a Mount Path. A library can have any number of Mount Path. Access to data stored on Mount Paths can be shared with other Media Agents. The primary advantage of Magnetic Disk is it’s ability to conduct multiple simultaneous read/write operations. The second advantage is write speed. Normally, magnetic disk is used as read/write media. However, in archive situations, Write Once Read Many (WORM) media may be required and is supported. Removable Media Removable Media is supported in three forms – Tape, Magnetic Optical (R/W and WORM), and USB storage devices. The most common of the three is tape. Tape media has its advantages in being a low cost storage media that is easily transportable for offsite protection and disaster recovery. Tape media has its disadvantages in that it is a sequential access medium with a potential for wasting capacity if the tape becomes unavailable or removed before data has been written to the end of the tape. In some cases, a tape is deliberately not filled in order to meet data protection (e.g. removal) or operational needs (e.g. business boundaries).

CommVault System Administration Course – R02

Library Management Module - 39 Magnetic Optical media’s advantage is its durability and reliability. It’s primary disadvantages are cost and capacity. USB Storage Devices provide a highly compatible and portable storage device at about the same cost as tape. However, USB Storage Device support is restricted to the equivalent of a standalone tape library (i.e. no robotics or internal storage device for multiple drives)

CommVault System Administration Course – R02

40 – Library Management Module

Connection „ „

Direct Attached Shared z z

Media Changer

MediaAgent

Static Dynamic MediaAgent

„

IP-based MediaAgent

MediaAgent

Connection Direct Attached Direct attached Libraries are dedicated to a single Media Agent. This is the simplest form of library configuration. Shared A Shared library is one where its media devices are accessible by two or more Media Agents. In its basic form, each media device is accessible by only one Media Agent. For example: a Tape library with 4 tapes drives and exposed SCSI connectors and share its drives by having each Media Agent connect independently to a drive. Media Changer/control is managed by a Media Agent. A Static library is a magnetic library using Shared Disk Devices. These devices can mounted on the Media Agent and/or shared via network protocol or shared file system (e.g. GFS Sistina, PolyServe). Devices can be enable for read/write or read-only. Additionally, a mount path can be cloned via hardware or software and made available for read-only. This is called a Replica Static library. A Dynamic Library is a library where the read/write devices are available to two or more Media Agents. Dynamic libraries required a SAN environment. Devices are dynamically assigned/released from a Media Agent using SCSI-2 or SCSI-3 commands via Fibre/iSCSI protocol. Dynamic libraries are usually removable media libraries, but can be magnetic disks. In

CommVault System Administration Course – R02

Library Management Module - 41 either case, only the controlling Media Agent can read/write from the device at any one time. Media Changer/control is managed by a Media Agent. IP-Based An IP-Based library is one where the Media Changer/controller is managed by 3rd party software. CommVault currently support StroageTek Libraries controlled by ACSLS (Solaris) or Library Manager (Windows) software and ADIC library controlled by the Scalar Distributed Library Controller (SDLC)/Windows. Drives within the library can be direct-attached, shared independently or dynamically shared.

CommVault System Administration Course – R02

42 – Library Management Module

Special „ „ „ „

NDMP Libraries PnP Libraries Content Addressable Storage Single Instanced

CommVault System Administration Course – R02

Library Management Module - 43

NDMP Libraries NAS Filer

NAS Filer Full Library Control And DDS or Standalone NDMP Drives SAN or Directattached

NDMP Data Path Load Balancing

NDMP Remote Server Media Agent

Full Library Control And/Or DDS Drives

NDMP Remote Server

NON-NDMP Data Path

Media Agent

Media Agent NDMP Restore Enabler Other Clients

NDMP Libraries Network Data Management Protocol (NDMP) is an open standard protocol for enterprisewide backup of heterogeneous network-attached storage. NDMP is used by Network Attached Storage (NAS) devices where the proprietary nature of the operating system precludes installation of an Agent. If a library is attached to a NAS filer, a NDMP backup can be sent directly to the library. Other NAS filers can also send NDMP backups to the NAS filer with the direct attached library. A SAN based NAS filer library and drives can be shared with other NAS Filers and Media Agents. The first configured host will have library control. Drives can be dynamically or independently shared. Non-NDMP data can use a NAS filer library even if the library is only attached to the NAS filer. However, non-NDMP data must pass through a Media Agent in route to the NAS Filer/library. NDMP Remote Server (NRS) is an optional software component from CommVault that allows data from a NAS filer to be backed up to a magnetic library or a NDMP drive pool on a tape library attached to a Media Agent. Installation requires that an appropriate license be available on the CommServe. Encryption and multiplexing of NDMP data is supported on NRS capable Media Agents. Auxiliary copy from any Media Agent managed library to a NAS filer library is not supported. NAS NDMP load balancing is supported with multiple NRS capable Media Agents.

CommVault System Administration Course – R02

44 – Library Management Module

NDMP data can be restored back to a NAS filer or to a Windows File System Agent with NDMP Restore Enabler software component installed. Direct Access Restore (DAR) optimizes the restore operation by allowing the NDMP client to directly access backed up data anywhere in a tape set without having to traverse the tape set sequentially. In a normal restore operation, a large portion of the data from the backup that included the file must be read. With DAR, only the portion of the tape which contains the data to be restored is read. This option is available for each of the NAS NDMP restore types and can save significant time in the restore operation.

CommVault System Administration Course – R02

Library Management Module - 45

PnP Libraries „

„

„

„

One PnP Library per Windows Media Agent Uses removable USB device as media Similar functionality to a Standalone Tape Library USB Device must be detected with drive letter assigned

PnP Libraries PnP (Plug and Play) Libraries can be added for any Windows Media Agent. Each PnP library has a “Drive holder” that acquires and can use a USB storage device as removable media. USB media is detected and an On Media Label written to identify it. A minimum size can be assigned to prevent detection use of small USB Drives. Additionally, a restriction that only blank USB drives. Once configured, PnP Disk Libraries are similar to a stand-alone drive and all the features, including the following are supported by these libraries: • •

Automatic Labeling Schemes Manual Stamping of Media

USB Media is handled similar to Magnetic Optical media. When all jobs on a USB media are aged, the media will be assigned to the scratch media pool, the CV_MAGNETIC folder will be erased and the disk re-used. In the 7.0 release of the software, only one PnP Library can be added/configured per Media Agent. While multiple PnP Drives can be added/configured, only one data stream should be used to a PnP Library. With multiple USB drives, a job can spill over from one drive to the next. Multiple streams and additional capabilities are planned.

CommVault System Administration Course – R02

46 – Library Management Module

Content Addressable Storage „ „

„

Magnetic Library Data is written in uniquely addressable blocks Duplicate blocks are logged but not written

Content Addressable Storage Content Addressable Storage is a scalable solution for large fixed data volumes. CommVault supports EMC’s Centera and HDS Data Retention Utility (DRU) as content addressable storage. Data is broken into blocks and a unique HASH identity is assigned. Duplicate blocks with the same HASH identity are logged by not written.

CommVault System Administration Course – R02

Library Management Module - 47

Single Instanced „

CommVault z z

„

Software Based Object level

Others z z

Hardware Based Block level

Single Instanced The need for storage has been growing steadily for businesses and are based on a number of needs, such as data retention, compliance requirements, data-intensive applications, etc. This explosion necessitates the need for a smarter way of storing data. Single Instance Storage addresses this problem by identifying the duplicate items in a data protection operation and maintaining references for the duplicate items. When a data protection operation is performed for the first time all the data is stored; if the same data is subsequently identified in another data protection operation, reference counters are incremented and the data itself is physically not stored in the storage media. Single Instance Storage is currently only supported for magnetic libraries. Note that there are appliances that support single instancing of data (e.g., Centera, Data Domain, HDS DRU). Such hardware can be configured as a magnetic library with the option to write the data in a single-instance format enabled. Single Instance Storage is supported by both backup and data archival products, specifically for the file system and email attachments, in order to provide optimization in storage when copies of the same data is backed up and stored. This feature requires a Feature License to be available in the CommServe and needs to be enabled on the magnetic library.

CommVault System Administration Course – R02

48 – Library Management Module Single Instance Single Instance Storage addresses the problem of storing duplicate items in a data protection operation and maintaining references for the duplicate items. When a data protection operation is performed for the first time, all the data is stored. On subsequent operations, if the same data is encountered a reference is made to the first stored item and the duplicate items are not written to the media. Single instancing is currently supported only on magnetic disk media.

CommVault System Administration Course – R02

Library Management Module - 49

¡ Configuring Media Agents „

Media Agent Options z z

„ „

High Scalability Parallel Transfer Limit

Locating the Index Cache Using Shared Index Cache

Configuring Media Agent Options Parallel Transfer This option allows you to establish the maximum number of concurrent read/write operations to the Media Agent. By default the number of concurrent read/write operations to a Media Agent is 25 and this value can be modified to a maximum of 50, if necessary. If Optimize for Concurrent LAN Backup is enabled from the Control tab, the number of concurrent read/write operations can be set to a maximum of 200. Basically this option controls the maximum number of data streams that can be managed by a Media Agent. Note that if this option is not selected, no maximum limit on the number of concurrent read/write operations to the Media Agent is established, which may result in performance degradation of the Media Agent. High Scalability Using the Optimize for concurrent LAN backups option from the control tab in Media Agent properties is good practice when receiving concurrent data protection operations from a large number of clients. For example, if you have 25 clients concurrently using the Media Agent for 50 or more data transfer streams in a specific operation window, it is recommended that you enable this option. Enabling this option will help to optimize the Operating System resources on the Media Agent. Setting the concurrent backups option allows the Media Agent software to optimize resources on the client during backup to maximize multi-stream data protection operations.

CommVault System Administration Course – R02

50 – Library Management Module Locating the Index Cache The Index Cache is used as a temporary working space for index-based data protection and restore jobs. Indexes located in the cache keep track of where individual objects are written on protected media. Indexes are created, updated, and written to protected media with each data protection job. Indexes are retained in the Index Cache by settings on the Media Agent for subsequent use. A Browse operation will consolidate and read indexes in the Index Cache as required by the scope of the browse. Indexes required, but not found in the Index Cache are automatically restored. The Index Cache should be located on a separate disk and it should be the fastest disk available to the Media Agent.

CommVault System Administration Course – R02

Library Management Module - 51

Using Shared Index Cache „

„

„

Required for Data path Failover and Round Robin CIFS or NFS supported Read & Write permissions required

Index Cache

MA

Using Shared Index Cache A shared index cache may be required if alternate data paths are enabled for failover or load balancing between multiple Media Agents. In both cases, an access path to the current active index may be required by another Media Agent in order to successfully complete a job. In configuring alternate data paths, a new data path can be added if the one of the following conditions are true: • • • •

The data path originates from the same Media Agent (e.g. different library) The new data path is to the same library (i.e. the index can be recovered from storage media by a Media Agent) Index is shared between all data paths (e.g. different Media Agents with different libraries) Use Preferred path is enabled (No failover or load balancing)

In order to share an index cache between two or more Media Agents, one Media Agent in the group must be setup first to share its index cache. If the location of a shared index cache is to be changed, then every Media Agent that required a new access path or user account for the new index cache location must be updated, otherwise their index cache access paths will eventually go offline.

CommVault System Administration Course – R02

52 – Library Management Module Un-sharing an index cache is only possible if there are no storage policy copy failover data paths defined that rely on that index cache being shared. Note that when un-sharing an index cache, a copy of the index is not performed and you will have to rely on index cache restores in order to browse backup data.

CommVault System Administration Course – R02

Library Management Module - 53

¡ Configuring Magnetic Libraries „

Library Level Settings z z z

„ „

Usage pattern Managed Disk Space Allocation Policy

Mount Path Usage Options Mount Paths Settings

Configuring Magnetic Libraries Magnetic Libraries are added through the Library and Drive Configuration control applet. Each Magnetic library has a unique internal ID and an Alias that appears in the CommCell Console. Naming magnetic libraries is a personal choice, but we recommend using a descriptive name that identifies the capacity, type, use and of the library. For example; 600GB RAID 5 Spool, 800GB RAID 3 Archive, etc. Fill & Spill vs. Spill & Fill Writers and mount paths are used in either a “fill & spill” or “spill & fill” allocation manner. Fill & Spill means the first mount path disk space or writers must be filled before the resources of the next mount path is used. For example; let’s say you have a three mount path library with 5 writers allowed for each mount path and 8 writers allowed concurrently for the entire library. If you had 10 concurrent backup streams to the library, the first 5 backup streams would go to the first mount path, 3 more would go to the second mount path and the remaining 2 would wait for an available writer. As the first mount path filled up, the last 2 streams would go to the remaining available writers on the second mount path and the other 3 streams would go to the third mount path. Spill & Fill assigns one writer from each mount path in turn until all mount paths are being used before adding additional writers to each mount path. This distributes the data transfer operations across more resources and could yield better performance.

CommVault System Administration Course – R02

54 – Library Management Module Managed Disk Space Managed disk space is configured in the Magnetic Library Properties and enabled on a per Storage Policy Copy basis. Data Aging operations still need to be run on a regular basis for Storage policies using Managed disk space. Data Aging will flag the eligible data as "Managed" (an internal flag). Managed data is included in restore browse operations along with retained data without the need to include "Aged" data. The periodic Update Free Space operation will check for thresholds being exceeded and, if so, erase the "Managed" data starting with the oldest until either all managed data is erased, or the lower threshold level reached. You can increase or decrease the configuration parameter for Interval (Minutes) between magnetic space updates (default :30 minutes) located in the Service Configuration tab of the Control Panel’s Media Management control. If a job exceeds the high watermark and fills the available disk space before managed data can be erased, the job will go into a waiting state due to "Insufficient media". Once the erase operation is completed and space becomes available, the job will resume. Best Practice - Set your high watermark level to leave at least sufficient disk space for what your hourly throughput rate can fill. This will give some breathing room. Allocation Policy Allocation Policies are used to optimize performance and throughput to each storage device by balancing write demands against available resource. Each mount path has an allocation policy which defines the maximum number of concurrent writers for that mount path. A magnetic writer is the equivalent of a tape device. The default maximum number of writers for a mount path is 5. This can be changed for each mount path via the Allocation Policy tab of the Mount path Properties window. Magnetic libraries also have a Library Allocation Policy. This policy is used to determine the maximum number of writers allowed for the entire library. By default the Library Allocation Policy is set to the Maximum Allowed Writers which is a summation of writers from all mount paths. For example; if you have 4 mount paths with 2 writers each, the Maximum Allowed writers to the library would be 8. You can set the number of writers allowed for a library to a number less than maximum. You might want to do this if the Media Agent or data path have insufficient resources or bandwidth to support the maximum number of writers for all mount paths. There is no similar control/policy for readers. You can have any number of readers for a mount path. However, such as for writers, a practical limit bases on available resources should be used.

CommVault System Administration Course – R02

Library Management Module - 55

Mount Path Usage Options 2 3

1 4

Fill & Spill

5 Media Agent

1 4

2

Spill & Fill

5 Media Agent

3

Mount Path Usage Options If a magnetic library’s “Fill & Spill” option is selected (default) and a new volume folder request is made on that library, then the library’s online, writable mount paths will be searched in the order in which they were created starting with the oldest and the first available writer found will be used to create/write the New Volume Note: Data Multiplexing can be performed on a magnetic library by setting the maximum number of streams on the magnetic storage policy to a value equal to the number of data protection operations that are to be performed simultaneously. This is different than a tape library where the maximum number of streams is set to the number of tape drives being used simultaneously. If no New Volume folder request is made, then the job will attempt to continue to use the active volume folder for that storage policy copy and stream. If the active volume folder is unavailable, then a new volume folder request will be made using the previous logic. If a magnetic library’s “Spill & Fill” option is selected and a new job request is made on that library, then the library’s unreserved online, writable mount paths will be searched in order of (a) Least recently used; (b) Least used; and (c);Oldest created mount path. The first available writer found will be used. Current active folders are not considered and are always marked full. If there are no unreserved, online, writable mount paths then the search will be in the order in which the mount paths were created starting with the oldest and the first available writer found

CommVault System Administration Course – R02

56 – Library Management Module will be used to create/write the New Job. If writers are multiplexed, then all available writers will be used first before another multiplex stream is used.

CommVault System Administration Course – R02

Library Management Module - 57

Mount Paths Settings Unbuffered I/O „ Space Allocation „ Fragmentation „ Writer Allocation Policy „

Mount Paths Settings Unbuffered I/O To increase the speed of operations accessing the mount path, you can enable the Media Agent to bypass the Microsoft Windows file system buffering. Note that this option is only applicable for Windows Media Agents and disks that are directly mounted (not UNC paths). Note that this is a licensed feature and requires a feature license to be available on the CommServe. However, this option is always enabled for shared magnetic libraries with dynamic mount paths. Therefore additional licenses are not required for this feature on this configuration. Space Allocation Space allocation allows you to establish the maximum amount of space that must be used by the mount path. This can be done by specifying the reserve space and selecting either the Use until the mount path reaches the reserved space or Do not consume more than n GB options. In some situations the mount path may consume more space and exceed the specified minimum reserve space. Consider the following example: •

If you have established the minimum reserve space as 512 MB, and run a 2 streamed data protection operation when the disk space is 515 MB, both the streams will use a minimum of 25 MB before spanning to an alternate mount path. In such a situation the free space will fall below the specified 512 MB.

CommVault System Administration Course – R02

58 – Library Management Module This could also happen if 2 concurrent single-streamed data protection operation is initiated to the same mount path, using 2 different storage policies. •

Similarly, the system may consume more than the specified maximum space. Consider the following example: When a data protection job is initiated, the system checks for the available disk space and verifies whether it is less than the specified maximum space. For example if you have specified 1 GB as the maximum space and if you have used .75 GB, the data protection job will be initiated using the mount path. However, if the size of the data is .5 GB, the system will write the entire data. In such a situation the space consumed will be more than the specified maximum space.

Note that a data protection operation will generally write to the mount path until the minimum reserve space is reached, before spanning to an alternate path. Fragmentation This option sets the pre-allocation mode – meaning that this a “reserve ahead” feature. It ensures that an adequate number of contiguous chunk space is available so that all the chunks in a data protection job can be kept together on a write from a Windows Media Agent. When chunks from jobs are pruned there is less fragmentation. Writer Allocation Policy The mount path allocation policy allows you to establish the maximum number of concurrent writers or the maximum number of simultaneous data protection operations on the mount path. If necessary you can also disable the mount path for write operation. This is useful in situations where you wish to retire or phase-out a mount path. This operation is supported by all magnetic libraries.

CommVault System Administration Course – R02

Library Management Module - 59

¡Configuring Removable Media Libraries Library Level Settings „ Drive Settings „

Configuring Removable Media Libraries As removable media (tape cartridges and optical disks) can only be accessed by one tape drive (and consequently one operation) at a time, you must plan carefully to avoid resource contention.

CommVault System Administration Course – R02

60 – Library Management Module

Library Level Settings Active Library Controller „ SCSI Reservation „ Auto Cleaning „ Auto Discovery „

Library Settings Active Library Controller In a SAN environment, the library controller feature can be used to configure the active Media Agent to automatically switch over to a failover Media Agent, in the event of a failure in the active Media Agent. SCSI Reservation When this option is enabled, the Media Agent uses the drive exclusively during data protection and other operations, using SCSI reservation. This option is useful in the SAN environment where multiple computers may try to access the same drive, resulting in data corruption. Refer to the hardware manufacturer's documentation to see if this operation is supported. If this option is enabled and the hardware does not support this type of operation, subsequent data protection jobs may fail. Both SCSS-2 and SCSI-3 reservations are supported. Before enabling this option, ensure the following: •

•

Verify the target device to see whether SCSI-2 or SCSI-3 command set for persistent reservation is supported. Most tape drives are known to support either of these command sets. Inter-connecting hardware, such as storage routers or bridge, which connects standard SCSI devices into FC fabric, should also support this command. If you encounter any differences in behavior with this command, then identify the differences in the

CommVault System Administration Course – R02

Library Management Module - 61

•

interconnecting components from the working host to non-working host. (Refer to the hardware manufacturer's documentation to see if this operation is supported.) When using the SCSI-3 command set, make sure that the device drivers for the target device do not perform an implicit reservation. Some of drivers in Windows as well as on Unix are known to perform SCSI-2 reservation implicitly when the device handle is opened by an application. In this case the SCSI-3 reservation will fail with Reservation conflict, as SCSI-2 reservation is already active in the target. (Check the driver’s behavior using the documentation provided for the driver.) Notably, some IBM LTO family drivers and STK 9x40 family drivers are known to perform SCSI-2 reservation on Windows. On Unix, there are hooks to enable implicit reservation using a known interface through native drivers.

Auto Cleaning You must clean each drive periodically or when necessary to remove the oxides that accumulate on the read-write heads. The drive cleaning operation includes the task of mounting the cleaning media into the drive, cleaning the read-write head and unmounting the cleaning media. When Auto Cleaning is enabled, the cleaning tape is automatically mounted and a cleaning operation is initiated on the drive, whenever the selected options indicate that the drive requires cleaning. The CommVault system automatically mounts the cleaning tape and cleans the drive if the Enable Auto-Cleaning options are enabled for the library. This options is not enabled by default. If you enable this option, verify and ensure that you have a good cleaning media in the Cleaning Media pool. When a drive is successfully cleaned, a message is displayed in the Event Viewer, and the drive cleaning parameters in the Odometers tab of Drive Properties are reset. Keep in mind that if these options are not enabled, the system does not automatically clean the drive when the hardware indicates that the drive requires cleaning. Hence subsequent mount operations in the drive may fail. This option is not available for stand-alone drives and optical libraries. Auto Discovery Before using a new media, the Media Agent must collect certain information about it through a process known as discovery. When a media has been discovered its information is entered into the CommServe database. The media information is permanently retained; media does not have to be rediscovered if it is exported from the library and re-imported. If new media are imported through a library’s mail slot, the import operation triggers a discover operation. This is dependent on whether you have enabled or disabled the Enable AutoDiscover option for the library.

CommVault System Administration Course – R02

62 – Library Management Module If the automatic discovery option is not enabled, the system will prompt you to provide the necessary details for the media. If the automatic discovery option is enabled, the system discovers the media during a subsequent inventory update triggered by a job from the CommCell. If the automatic discovery option is not enabled for the library and if you have some undiscovered media from a previous import, or if you import new media by opening the library door and inserting them, you must initiate a discover operation. Media can be discovered from both the Library and Drive Configuration window and the CommCell Browser.

CommVault System Administration Course – R02

Library Management Module - 63

Drive Settings Drive Allocation „ Active Drive Controller „ Drive Verification „

Drive Settings Drive Allocation simply sets the maximum number of drives that any drive pool can use at any given time in a shared library configuration. Setting allocation ensures that there are adequate drives available for all drive pools during data protection operations. A Drive Pool is a set of drives which are attached or accessible to a Media Agent. Like the Master Drive Pools, drive pools are also logical entities used to facilitate the sharing of a library’s drives between multiple Media Agents. You can modify the following Drive Pool properties: • • •

Drive Pool Name This can be used to uniquely identify a drive pool. Enable/Disable Drive Pool Use this option to activate or deactivate the drive pool. Allocate drives Use these options to assign all or some of the drives in the library to the drive pool.

Active Drive Controller The active drive controller can be viewed by right clicking on the drive in question, selecting properties, and then selecting the Drive controller tab. Alternately (And the best method) you can go to the properties of a Master Drive Pool and view all of the properties of all drives from there.

CommVault System Administration Course – R02

64 – Library Management Module Drive Verification When selected, the drive serial number and access path is verified before reading or writing to the media. It is strongly recommended that this option be enabled at all times to prevent the overwriting of data, when the drive access path is changed due to hardware configuration changes. This option is enabled by default.

CommVault System Administration Course – R02

Library Management Module - 65

Module Summary

Key points to remember

Summary •

Libraries can be configured for Magnetic disk or removable media.

•

These libraries can be direct attached or shared among multiple Media Agents. Support for SAN is easily configured.

•

Single instancing is supported for libraries within the software as well as specifically designed hardware.

•

Media Agents can be multiple platforms and can work together in a single CommCell.

•

Sharing the Index Cache offers the added convenience of Multiple Data Paths and MediaAgent failover, as well as load balancing.

CommVault System Administration Course – R02

66 – Library Management Module

CommVault System Administration Course – R02

Media Management Module - 67

Media Management

www.commvault.com/training

CommVault System Administration Course – R02

68 – Media Management Module

¡ Overview Understanding Media Management „ Using Media Options „ Managing Internal and External Media „

Media Management Module Objectives: • • • • •

List and describe the options available for managing magnetic media. Describe the role and configuration options for Media pools and removable media. List and describe the options for importing and exporting media. Configure and understand the common settings and options for use of a Standalone or PnP Library and associated media. Understand the use and implementation of NDMP or IP-Based libraries.

CommVault System Administration Course – R02

Media Management Module - 69

¡ Understanding Media

Management Types of Media „ How data is stored „ Removable Media States „ Removable Media Pools „ Managing Media Content „ Job States on Media „

Understanding Media Management Media management involves interaction among the CommServe, Media Agents, client programs, storage devices and media.

CommVault System Administration Course – R02

70 – Media Management Module

Types of Media „

Stationary z

„

Magnetic mount paths

Removable Tape z Magnetic Optical z USB Storage Drive z

Types of Media Stationary Magnetic mount paths Magnetic mount paths are Media Agent addressable file systems residing on volumes hosted by individual disks or disk arrays. Data is written to the mount paths using the supporting host operating system. Volumes used as mount paths can be directly mounted on the Media Agent or accessed via network protocol (CIFS/NFS) or 3rd party protocol (Centera, HDS DRU). Note that the maximum size of a volume that can be used as a mount path can be a maximum of 4,096 Terabytes. The primary advantage of magnetic media is random/concurrent access. Multiple data streams can be used to read/write data on magnetic volumes. Data can be randomly erased and the freed capacity repurposed. In most cases, magnetic media will also be the fastest read/write storage. The primary disadvantage of magnetic media is that it is stationary. Offsite protection requirements using magnetic media can only be supported through Wide Area Networks. Removable Tape Tape media is the most common media in current use for protected storage. It’s relatively inexpensive, portable, and for large volume reads or writes, it can be very fast. It meets all the requirements for protected storage media. The primary disadvantage to tape is it’s sequential

CommVault System Administration Course – R02

Media Management Module - 71 access and it’s all-or-nothing limitation for re-use. Additionally, of all protected media options, tape media has the smallest life span. Tape media is also the only data storage media that has physical content with the read/write device and is constantly stressed by winding/rewinding and physical movement. Tape drives are in constant need of cleaning to prevent read/write errors. Additionally, the use of tapes in varied locations demands available drive compatible. Magnetic Optical Magnetic Optical (MO) media has almost all the advantages of tape and magnetic except cost. MO media is rugged, secure, durable, and has a shelf life of 100 years. Besides cost, the other disadvantages to MO is capacity and the access to that capacity. MO platters are dual sided with only one side at a time available for reading/writing. USB Storage Device Removable USB Storage Devices can be used in a manner similar to a standalone tape drive – except none of the disadvantages of tape. The USB drives are durable, portable, and compatible for offsite storage and use. Costs and capacity are comparable to tape media. The primary disadvantage is the software’s current limited use of USB libraries. This will change in the future.

CommVault System Administration Course – R02

72 – Media Management Module

How Data is Stored Data stored in Chunks „ Magnetic vs. Removable „

Media ID OML

File marker

Bytes offset

… chunk

How Data is Stored Data is written to storage in chunks. By default, chunk size for magnetic storage is set to 1GB (configurable by registry key only). For removable media, chunks size is set to 4GB for file system/message/document data and 16GB for database data. CommVault supports a maximum chunk size of 32GB Individual objects within a chunk are identified by offsets from the beginning of the chunk. A restore job will first sort objects by chunk and offset before restore in order to do a single pass read of data.

Magnetic Media Data stored on magnetic mount paths are written in 1GB chunks (files) in uniquely numbered folders (e.g. VOL_123). A new folder will be created: • • • •

For every active writer stream When Start New Media option is selected for a data protection job For subsequent job if the previous job’s Mark Media Full after successful job completion option is selected Every 8 hours to ensure a folder does not have too many files (chunks)

CommVault System Administration Course – R02

Media Management Module - 73

Removable Media States Spare

Undiscovered

Active Full Retired Bad

Appendable

Removable Media States Undiscovered Undiscovered media is media not yet stamped or identified by the software. The discovery operation can be configured to occur automatically upon import or loading of a tape, or can be a manually initiated operation. Undiscovered media will appear in the CommCell Console as Foreign media Spare Spare media resides logically in a scratch pool and if physically in the library it is available for assignment by the system. Spare media located outside the library will not be prompted for import if no spare media exists within the library. Active Active media is that media currently being used by the software to write data. Distinct active media is required for each data stream of each storage policy copy. Hence a 4 stream storage policy copy would have up to 4 active media in use at one time. Appendable Appendable media is that media that has not been written to full capacity and is still eligible for additional writes. Under normal conditions, a storage policy copy requiring active media will check for and use any qualifying appendable media.

CommVault System Administration Course – R02

74 – Media Management Module Full/Bad Full or bad media is media that can be read from for restores, but is not writable. A full state is achieved through writing data to the end of the media (EOT) or by a user/job marking the media full. A bad state is achieved through error or usage levels exceeding set threshold limits – or by a user marking the media bad. Full media will become spare media when all jobs on the media are logically pruned and the media is not overwrite protected. Bad media will become retired media when all jobs on the media are logically pruned and the media is not overwrite protected.

CommVault System Administration Course – R02

Media Management Module - 75

Removable Media Pools „

Physical Media In Library z Exported z

„

Logical Scratch z Cleaning z Assigned z

• Foreign • Retired • Overwrite Protect

Removable Media Pools Physical Each piece of media is a member of a physical location group identified as either “Media in Library” or “Exported Media”. For libraries with internal storage slots, the “Media in Library” location would be listed as a slot or drive. Media in the library can be marked as “Prevent Export” to ensure it remains in the library. Exported media is media external to the library and is identified by an assignable (and editable) location. In the case of a single drive or standalone library, there is no ”Media in Library” location. All standalone library media that is not in the drive itself is considered to be external and thus in an export location. Logical Media is also tracked logically. Removable media available for re-use are kept in Scratch pools. There is a default scratch pool created for each removable media library that is configured. Additional scratch pools can be created for each library to manage use of different spare media by different drive pools and/or storage policy copies. Since having spare media is critical to successful data protection operations, each scratch pool has settings for managing the number of media in the pool and the sequence in which they are used. The Cleaning media pool is for cleaning media. Simple enough. However, you can have different drives within a library and thus different cleaning media. Hence, you can have more than one cleaning media pool. Movement of new cleaning media to this pool can be done

CommVault System Administration Course – R02

76 – Media Management Module manually, or automatically by barcode range. As with spare media, cleaning media is critical to successful data protection operations. Each cleaning pool has media management settings similar to scratch pools. A low watermark for each cleaning and scratch pool can also be established. This parameter represents the minimum number of media that should be available inside the library for that media pool at all times. If the number of available media falls below the low watermark, the system logs a message in the Event Viewer and , if configured, generates an alert. Non-retired media with retained data can be found in the Assigned media pool. This is the normal pool for full, appendable, bad, and active media. Media in the Overwrite Protect Media pool can be placed there manually from the Scratch pool by an administrator or can be sent there automatically from the Assigned media pool, by enabling the media option to Prevent Re-use. The Overwrite Protect Media pool status exempts the data on the media from being re-used in a normal fashion. Media must be manually moved out of the Overwrite Protect Media Pool by the administrator. The Foreign media pool is the logical repository for media that has been used by another application and is not available for use by this CommCell. Media not-recognized or not usable (e.g. media with different format or from another CommCell that optionally is not used) will be placed in this pool. The administrator can also manually move media to this pool. The Retired media pool is for media that has exceeded the vendor recommended threshold values for usage and errors. Media that has exceeded thresholds but still has retained data will not be moved to the retired media pool until all such data has been aged off.

CommVault System Administration Course – R02

Media Management Module - 77

Managing Media Content Viewing Contents „ Deleting Contents „ Writing over Contents „ Erasing Media „

Managing Media Content Viewing Contents You can view the contents of a specific used media. This feature can be used to view a list of data protection operations residing in the media. All the details associated with the data protection operation(s) available in the media are displayed. This includes the following: • • • • • • •

The Job ID associated with the data protection operation Names of the client, agent, instance/backup set and subclient Whether the data protection operation is Full, Incremental, Differential or Synthetic full The archive file type The date and time in which the archive file associated with the data protection operation was created Media side Archive file

Deleting Contents The delete contents option can be used to delete the contents of a media and move it to a specified scratch pool. This option can be used to make media available to complete an important data protection job when there are no spare media available in the library.

CommVault System Administration Course – R02

78 – Media Management Module Writing over Contents When all the data ages on a piece of media that tape/platter is returned to the scratch pool. Once that media is in the scratch pool it is eligible to be over-written. Until that time comes the aged data is still available for recovery. Erasing Media The erase spare media operation ensures that the old data from removable media (tapes and optical platters) are not recoverable once the media is recycled. This is done as follows: •

On tapes, the On-Media-Label (OML) is over-written by a new OML to indicate the erased status.

•

On Optical media, the platter is formatted and a new OML is written.

•

Only spare, retired and recycled media from tape and optical libraries can be erased. Note, however, that the erase media operation cannot be performed on magnetic and stand-alone drives. Once a media is erased by this operation, data cannot be retrieved using Media Explorer.

Note that the Erase Spare Media operation will not erase the following data: •

Data associated with another CommCell, if the option to overwrite media from another CommCell is disabled in the Media tab of the Library Properties dialog box.

•

Data written by other applications, if the option to overwrite media when content verification fails is disabled in the Media tab of the Library Properties dialog box.

The Erase Media operation is a low priority job, and is displayed in the Job Controller window. It can be killed, if necessary.

CommVault System Administration Course – R02

Media Management Module - 79

Job States on Media Retained „ Aged „ Managed „ Failed „ Killed „

Job State on Media Retained Retained jobs have not exceeded retention criteria and are automatically available for a browse or Restore by Job operation. Aged Aged jobs are backup jobs on removable media that have: • •

Exceeded all retention criteria or Been manually pruned

In either of the above cases, a Data Aging operation must be run to mark the job records as pruned in the CommServe database. Since the data is on tape and not actually deleted, aged jobs are available for browse or Restore by Job operations if explicitly included in these operations. This inclusion can be set CommCell wide by default or job specific. Managed Managed jobs are backup jobs in a managed magnetic library that have exceeded all retention criteria and had a Data Aging operation run to mark the job records as managed. Managed jobs are treated the same as retained jobs for browse and restore operations. An administrative process periodically checks the amount of free space available in a library and compares that against the high watermark for managed data. If the high water mark is exceeded, the process erases managed jobs until the low water mark for managed data is reached. Jobs are erased in

CommVault System Administration Course – R02

80 – Media Management Module chronological order starting with the oldest. Use of managed jobs on magnetic libraries maximizes the use of the library capacity for data availability. Failed Failed jobs are jobs that, for some reason, did not successfully write all required data to protected storage. Indexes for failed jobs are not retained in cache or written to the protected media. However, successfully written chunks from failed jobs can be explicitly included in Restore by Job operations. Killed Killed jobs are jobs that are prematurely terminated by the software or a user before successfully writing all required data to protected storage. Indexes for killed jobs are not retained in cache or written to the protected media. However, successfully written chunks from killed jobs can be explicitly included in Restore by Job operations.

CommVault System Administration Course – R02

Media Management Module - 81

¡ Using Media Options „ „ „

Removable Media Usage Media Labeling Threshold Settings

CommVault System Administration Course – R02

82 – Media Management Module

Removable Media Usage Spare Media „ Appendable Media „ Overwrite Media „ Exported Media „ Bad Media „ Mixed Retention „

Removable Media Usage Spare Media Spare media usage is determined by the job type and the usage options selected in the storage policy copy’s associated scratch media pool. Spare media will be used when: • • •

The active job stream fills its current active tape and requires spare media A new active job stream starts with the option to Use New Media The job is a Synthetic Full job. (appendable media may be a source of data, hence not a choice for writing)

Note: The order in which spare media is used is selectable within the Scratch Media pool. The options are to Use new media first (default) or Use recycled media first. In using recycled media, the media with the least amount of usage and/or errors will be used first. Appendable Media Appendable Media will be used if: • • • • •

The host library option to Use Appendable media is enabled. The appendable media is assigned to the same Storage Policy copy and stream. The last write to the appendable media is within the specified time limit. There is available capacity on the appendable media. The user did not use the “Start New Media” option. This will use a tape from the scratch pool vice appendable media.

CommVault System Administration Course – R02

Media Management Module - 83 •

The new media requirement is not in support of a synthetic full backup. This will use a tape from the scratch pool vice appendable media since the appendable media may contain data that is required to create the synthetic full.

Overwrite Media Media will be overwritten if it is: • • •

Newly discovered and the appropriate Overwrite Media option for the library is selected. In a standalone drive and the Overwrite Media in Drive option is selected and the jobs on the media meet the overwrite condition. A previously used media in a scratch media pool and is requested by a data protection job.

Exported Media Exported Media will be recalled if : • • • • • •

The media is required by a Data recovery operation The media is required by an Auxiliary Copy operation The media is required by a Verification operation The media is required by a Content Indexing Operation The media is required by a Synthetic Full The media is required by a Data protection AND the library option to Start New Media When required media is exported is cleared

Bad Media Bad media can be used for data recovery operations only. Bad media will remain in the assigned media pool until such time as all usable data has been aged off. The bad media will then be moved to the retired Media pool awaiting ultimate deletion and removable from the library. Mixed Retention By design, storage policy copies use separate media largely to prevent mixing different retention requirements on the same piece of media. Should this happen, the media is retained for the longest retained job on the media and may not recycle back for use as expected. This condition can be caused if you use different backup and archive retention setting or use extended retention without separating the jobs to different media. CommVault warns you if different backup and archive retention requirements are set. It does not warn you if extended retention is used.

CommVault System Administration Course – R02

84 – Media Management Module

Media Labeling Auto Stamping „ Barcode Patterns „ Updating Barcodes „

Media Labeling Media labels are used to externally identify and track removable media. Exported media is recalled using the external label. Any removable media loaded into a drive will validate the On Media Label to prevent a mis-labeled tape from being used. Auto Stamping Media in Standalone and PnP libraries should be labeled for manual handling. If the option to Auto Stamp Media in Drive is selected and an unidentified (or new) media is loaded into the drive, the system automatically stamps a new label using either the specified Barcode Labeling Scheme or a default label, which is the media creation date and time. If the option is cleared, the unidentified (or new) media must be manually stamped with an appropriate label. Media discovered in blind libraries will automatically be labeled. A default labeling scheme is used unless the user specifies an alternate barcode labeling scheme. Care must be exercised while generating the labeling scheme to ensure that each media label is unique. If the specified labeling scheme is not unique, and if duplicate labels are found, the system automatically appends a media ID to make it unique. Barcode Patterns When you use one or more standard barcode pattern(s) in the libraries in the CommCell, you can define different Barcode Patterns to automate the distribution of media when you have specific purposes for the media in the library. For example:

CommVault System Administration Course – R02

Media Management Module - 85 •

•

•

When you have a library with different drive types, we recommend you have specific barcodes for media associated with each drive type. This will help you to easily manage and administer the media in the library. When you have media that are not used by the Media Agents in this CommCell (referred to as Foreign media in the software) stored in the library and use Vault Tracker to manage the movement of such media. If you already manage media using barcode patterns, you can use this method to automate the process.

Scratch, Cleaning, and Foreign Media Pools support the use of Barcode patterns for automatic distribution of new media. Updating Barcodes Barcodes labels can wear out, fall out and be assigned different barcodes. Media from libraries with different labeling requirements may be required for use in alternate libraries. (For example, a auto stamped media from a standalone library can be assigned a barcode and used by a barcode capable library) Additionally, some libraries recognize a 6 character barcode while others recognize 8 characters. A firmware upgrade may affect all your barcode labels in your existing library. CommVault provides the means to update media barcodes in all of the previously cited conditions. Individual media labels can be edited and scalable tools allow you to add/remove prefix/suffix for all media with the library.

CommVault System Administration Course – R02

86 – Media Management Module

Threshold Settings „

„ „

„

Usage and errors tracked for each removable media Media evaluated before each write Media that exceeds thresholds is “Deprecated” and moved to Retired Media Pool Storage Resources thresholds provided by vendors

Threshold Settings Media that exceeds usage or error thresholds jeopardize data protection and should be removed from use. When the CommVault software evaluates scratch pool media for use, media that has exceeded thresholds will be automatically moved into the retired media pool the software tracks statistics for each tape and optical platter which can be seen in the odometer tab of the media’s property page. The Storage Resources properties list the vendor recommended thresholds for cleaning and expiring media. These thresholds can be changed, but any change should be verified with the media vendor to ensure support and reliability of the media in recording or reading data correctly.

CommVault System Administration Course – R02

Media Management Module - 87

¡ Managing Internal and External Media Exporting Media „ Importing Media „

Managing Internal and External Media CommVault offers several features to automate the process of managing media within the library. You can automate the media discovery process in the library using one of the following methods: When you configure a library, select the default media type and then click Yes in the Discover Media Options dialog box to automatically discover the media in the library. Importing is the process by which you move media that are outside a library into storage slots within the library. Exporting is the process by which you physically remove one or more media from a library. There are two ways of importing/exporting media: • •

You can import/export media through the library’s mail slot (if available). You can open the library door and insert/remove media by hand.

If the library involved is a blind library, then an inventory will be required when media is imported in order to discover and track the media within the library. When media is imported/discovered, the system assigns it in one of the following ways:

CommVault System Administration Course – R02

88 – Media Management Module •

•

•

If it is a media associated with the CommCell, which was previously exported, it will be assigned to the scratch pool or a storage policy copy to which it was assigned before it was exported. If the media is a new media the system assigns it to a scratch pool based on its barcode if barcode ranges are assigned to scratch pools. If not, then the new media will go to the default scratch pool. It can then be manually moved to another scratch pool if needed. If a data protection operation requests an exported media, depending on the options selected for the library in the Media tab of the Library Properties dialog box, the following operation will be performed: •

Automatically mark the exported media as full, and use a new media for the data protection operation. The job will remain in the waiting state, with the Reason for job delay stating that the media is outside the library. However, if a restore or auxiliary copy job requires data from an exported media, the system will prompt you to import the media in order to complete the operation. Information about exported media will be retained in the system; they do not have to be rediscovered if they are re-imported.

CommVault System Administration Course – R02

Media Management Module - 89

Exporting Media Setting Export Locations „ Export by List „ Export using Vault Tracker „

Exporting Media Exporting is the process by which you physically remove one or more media from a library. If a media is reserved by a job for a read or write operation, the media cannot be exported. Media that is left in the mail slot after an export operation will be treated as an exported media. Pop-up messages associated with media outside the library and the media information in the CommCell Console will indicate that the media is in the IEPort associated with the mail slot. You can use Vault Tracker and Media Repository in Vault Tracker to manage media residing outside the library. There are two ways to remove the media from the library: • •

You can remove the media through the library’s mail slot (if available and supported by the library). You can open the library door and remove media from the storage slots by hand.

Removing media through a mail slot offers the following advantage: The inventory update that is triggered by a mail slot export is much less time-consuming than the full inventory operation that is triggered when you close the library door. However, under certain circumstances you may want to open the library door even though a mail slot is available. For example, if you want to remove many media from a library at once, it may be faster to open the door than to use the mail slot.

CommVault System Administration Course – R02

90 – Media Management Module Setting Export Locations When a media is exported an optional entry to specify the storage location for the media is provided. This information, can be viewed or updated in the Media Properties dialog box. This feature helps you to keep track of the exported media. The exported media are displayed with the location in the Exported Media pool in the CommCell Console. Export by List Media can be selected from a list of media and exported. This is useful when you wish to export several media at the same time and know either the media’s barcode, or slot number in which the media resides. This task can be performed from the following levels in the CommCell Console: • • • • •

Library Media in Library Media List dialog box which appears when you select the View Media option. This dialog box can be accessed by right-clicking a Storage Policy copy. Media List dialog box which appears when you select the Change Data Path option. This dialog box can be accessed by right-clicking a Storage Policy copy. Media List dialog box which appears when you select the Media Not Copied option. This dialog box can be accessed by right-clicking a Storage Policy and Storage Policy copy.

Export using Vault Tracker Media can be exported automatically using Vault Tracker. Vault Tracker jobs can be initiated by a Tracking Policy, Data Protection Job or Auxiliary copy job.

CommVault System Administration Course – R02

Media Management Module - 91

Export using Vault Tracker Using Virtual Slots „ Using Containers „ Vault Tracker Jobs „

Export using Vault Tracker Vault Tracker provides the facility to manage media that are removed from a library and stored in offsite locations. In practical terms, Vault Tracker provides the following capabilities in your day-to-day operations: • • • • •

Identifies media that must be sent off-site for storage or brought back from off-site locations. Automatically moves the media in sequence in the library and provides a pick-up list for the operators. Facility to identify and track media during transit. Facility to track record and track containers when the media is stored and moved using containers. Facility to record and track the movement of non-CommVault or Foreign Media.

CommVault System Administration Course – R02

92 – Media Management Module

Using Virtual Mail Slots „

Consolidates exported media to a specified block of internal storage slots for ease of removal Virtual Mail Slots

Exported Media

Library

Library

Using Virtual Mail Slots Exporting multiple media from a library with a single mail slot can be time consuming. Each media moved to the mail slot must be manually acknowledged and removed individually. Opening the library to remove multiple media manually may be quicker, but it requires a visual search of all the slots to find the appropriate media. Virtual Mail Slots alleviate the need to visually search the library for each media by designating a static block of slots within the library to hold the media awaiting export. The Virtual Mail Slot feature is enabled on the Library’s Media properties page. The administrator enters the starting slot number, number of slots, and the direction (Up or Down) to use.

CommVault System Administration Course – R02

Media Management Module - 93

Using Containers Pre-configured or dynamically created „ Assigned via Vault Tracker or Media Properties „ Contents viewed from Vault Tracker „

Using Containers Containers are used to logically group exported media into a single addressable unit. This reduces the number of objects to track and facilitates movement of a set of media in to, or out of, third party offsite storage when a cost is assigned to each unit. Hence, ten media placed in a single container for storage would be a single unit cost in both directions. Some container facts: • • • • •

Containers can be pre-defined by the user or dynamically defined by a Vault Tracker Job. Each container will report the volume of data it contains. Exported Media Properties will report which container the media is in. The contents of each container can be viewed. List Media will identify the required media, container, and location.

CommVault System Administration Course – R02

94 – Media Management Module

Vault Tracker Jobs Reached Reached Destination Destination

Production Library

Container

Transport

Export Location Picked PickedUp Up

Due DueBack Back

Transport

Returned Returned to toSource Source

Disaster Recovery Library

Vault Tracker Jobs Vault Tracker provides the facility to manage media that are removed from a library and stored in offsite locations. In practical terms, Vault Tracker provides the following capabilities in your day-to-day operations: • • • •

Identifies media that must be sent off-site for storage or brought back from off site locations. Automatically moves the media in sequence in the library and provides a pick-up list for the operators. Facility to identify and track media during transit. Facility to record and track the movement of media not used by the Media Agents in this CommCell or Foreign Media.

Vault Tracker jobs can be initiated in three ways: • • •

By a Tracking Policy initiated on demand or scheduled By the Advanced Options of a Data Protection job By the Vault Tracker Options of an Auxiliary Copy job

The first method - Tracking policy - should be used if: •

The source location is other than the library the media was last written to.

CommVault System Administration Course – R02

Media Management Module - 95 • •

The media movement is not directly related to a particular auxiliary copy or data protection job. The media to be moved comes from multiple sources.

Exported media monitored by a Vault Tracker job is reported as being in one of three states: • • •

At Source – Media is in a static location or library At VMS – Media is in the Virtual Media Slots awaiting removal In Transit – Media has been removed from a library or has been marked as Picked up from a location other than a library.

Departure and arrival of media for a library is automatically recorded by the library’s slot inventory mechanism. Hence media removed from a library source is automatically assumed to be in transit (if a transit location is specified) or at the specified export location. Media departing an export location must be manually flagged as removed from that location. It can be marked Picked Up to put it In Transit (if a transit location is specified) or optimistically at its destination (new source). In Transit media, or exported media that has not been marked Picked Up because you want to confirm it’s arrival at it’s intended export location (new source), can be marked as Reached Destination. Individual media in a transit state between two export locations that, for one reason or another, requires the return of the media to the originating source can be marked “Return to Source”. Alternatively, a Vault Tracker job that moved media between to export locations can be “rolled back” to indicate return of all of its media to the source location. Exported media that can be re-used as spare media can be tracked/scheduled with a Due Back tracking policy. When creating a Due Back tracking policy you can include all media that will be recyclable within a specific number of days. For example; you can create a weekly scheduled Due Back tracking policy that includes all media that can be recycled within the week (7 days).

CommVault System Administration Course – R02

96 – Media Management Module

Importing Media Discovering new Media „ Verifying Existing Media „

Importing Media Importing is the process by which you move media that are outside a library into storage slots within the library. There are two ways of importing media: • •

You can import media through the library’s mail slot (if available) You can open the library door and manually insert media into storage slots in the library

Import Media through the Library Mail Slot Importing media through a mail slot offers an advantage - the inventory update that is triggered by a mail slot import is much less time-consuming than the full inventory operation that is triggered when you open and then close the library door. Directly Insert Media by Opening the Library Door Under certain circumstances you may want to open the library door even though a mail slot is available. For example, if you want to add many media to a library at once, it may be faster to open the door than to use the mail slot. Once again, the media will be automatically discovered, if the Enable Auto-Discover option for the library is enabled.

CommVault System Administration Course – R02

Media Management Module - 97 If this option is disabled, the media will be displayed (with a '?' icon) in the Media in Library pool in the CommCell Browser. You must subsequently discover the media, in order to use the media in the library.

CommVault System Administration Course – R02

98 – Media Management Module

Discovering new Media

„ „ „ „

Robotic vs. Manual Blind vs. Barcoded Standalone 3rd party media

Discovering new Media Before using a new media, the Media Agent must collect certain information about it through a process known as discovery. When a media has been discovered its information is entered into the CommServe database. The media information is permanently retained; media does not have to be rediscovered if it is exported from the library and re-imported. If new media is imported through a library’s mail slot, the import operation triggers a discover operation. The system will then acquire the slot map from the library and use this as our media inventory. This is dependent on whether you have enabled or disabled the Enable Auto-Discover option for the library. If the automatic discovery option is not enabled, the system will prompt you to provide the necessary details for the media. In this case when you import media you must initiate a Discover Media from the Library’s task menu. In a blind library the discover media operation is available only at the media level. When you import media in a blind library, ensure that you perform a quick inventory with a full scan, in order to discover the imported media. It is recommended that you initiate a full scan operation (if it is not automatically triggered) whenever you directly import media into a library, especially on libraries without a mail slot.

CommVault System Administration Course – R02

Media Management Module - 99 The discover media operation mounts the media into a drive and if a valid OML is not found, it writes the On-Media-Label (OML) on the media. Once unmounted, the system also keeps track of the media location. A discover inventory job is displayed in the Job Controller window and can be killed if necessary. Also note that both the full and quick inventory processes will automatically discover new media whenever a new media is found. For a Standalone Library there is no discovery process for media. A tape is loaded into the drive and an OML is written only when a job is initiated. The media label can be automatically generated or manually input. Media labels can also be pre-configured by the Stamp Media Operation. If the media being imported was previously used with another application CommVault will not automatically detect that in a barcoded library. The media will be placed in the scratch pool and be available for jobs. Once that media is selected for a job, the system loads the tape and checks for a header. If we do not recognize the format or header information as one of our own the media is placed in the Foreign Media pool for the Administrator to deal with. In order to use this media the tape has to be either formatted or the option to Overwrite Media When Content Verification Fails must be selected from the library properties.

CommVault System Administration Course – R02

100 – Media Management Module

Verifying Existing Media Auto vs. Manual „ Same or Different Libraries „

Verifying Existing Media Robotic, Standalone or Blind libraries The verify media operation is a type of inventory which must be initiated by the user. This operation can be used to verify whether the media information displayed in the CommCell Console matches the OML in the media. You can initiate the verify inventory operation by rightclicking the media that you wish to verify and then choosing the Verify Media option. For blind or standalone libraries the verify inventory operation does not discover new media. A verify inventory operation is displayed as a job in the Job Controller window and can be killed if necessary. The verify inventory operation does not discover new media. Same or Different Libraries Media must have unique labels throughout the entire CommCell to ensure data can be restored from anywhere. If the system discovers duplicate barcodes an event message will alert the administrator who must remedy the situation. When media is loaded into a library other than the one it was written in, the system will generate an event message notifying the user that the media is in a different library. This is just an informational message and not a problem. Though this media cannot be written to, the data contained within can be restored without any additional configuration.

CommVault System Administration Course – R02

Media Management Module - 101

Module Summary

Key points to remember

Summary •

Data is stored in Chunks, 1GB for all data on Magnetic media, 4GB for indexed data to tape and 16GB for Database data to tape.

•

Magnetic Media is faster for writing and recovering data. Tape media has its advantages in being a low cost storage media that is easily transportable for offsite protection and disaster recovery.

•

Parallel streams are written to media simultaneously to separate device streams or media, while multiplexed streams interlace the data into a single device stream or single media.

•

Pro’s and Con’s of mixing retention on a single media. Using basic retention vs. extended retention.

•

Appendable media allows for most efficient use of media.

•

Vault Tracker gives the ability to manage media outside of the library.

CommVault System Administration Course – R02

102 – Media Management Module

CommVault System Administration Course – R02

Storage Policies Module - 103

Storage Policies

www.commvault.com/training

CommVault System Administration Course – R02

104 – Storage Policies Module

¡ Overview Understanding Storage Policies „ Data Retention, Management, and Aging „ Roles of Verification, Encryption and Content Indexing „

Storage Policies Module Objectives: • • • •

List and describe the function and role of Storage Policies in managing protected data. List and describe the common settings and options available for a Storage Policy and Storage Policy copy. Describe how data is retained, managed, and aged within a Storage Policy. List and describe the options for performing Verification, Encryption and Content Indexing on managed data.

CommVault System Administration Course – R02

Storage Policies Module - 105

¡ Understanding Storage Policies Design and Philosophy of Storage Policies „ Types of Storage Policy copies „ Updating Secondary copies „ Managing Data Streams „

Understanding Storage Policies Policies are templates by which administrators can apply a consistent set of rules on a scalable level. Storage Policies are essential for easy and efficient management of large volumes of protected storage.

CommVault System Administration Course – R02

106 – Storage Policies Module

Design and Philosophy of Storage Policies Drive Pools Copies

Media Agent(s) Libraries Day s Cycles

Streams Data Paths

Retention

Storage Policy basic components

Design of Storage Policies A Storage policy is a single addressable entity that encapsulates the physical data paths and storage media along with the logical management and retention of one or more copies of data requiring protected storage. Production data can be easily directed to appropriate protected storage management by simply associating it with a storage policy. There are two types of storage policies: Disaster Recovery and Standard. A Disaster Recovery storage policy is used to separate and manage DR backup data independently from other data protection jobs. A Standard storage policy can manage data from any backup or archive job – or both. A Storage Policy is used to: •

Manage one or more copies of the protected data. Multiple copies are used to provide data redundancy should one copy become unavailable and alternate storage media type options for restore performance and/or location.

•

Manage which copy of the data is used for restore when either more than one copy of the data exists or a specific copy of the data is to be used.

•

Define maximum concurrency level of device streams to use. This can determine resource utilization levels and concurrency capability for restoring data.

CommVault System Administration Course – R02

Storage Policies Module - 107 •

Define Content Indexing engine, options, and source for offline content indexing.

The only permitted relationship between storage polices is through a linkage called Incremental Storage Policy. When a standard storage policy has an incremental storage policy enabled, full data protection jobs are stored and managed by the standard (or parent) storage policy, while all non-full data protection jobs are stored and managed by the associated incremental storage policy. This configuration can effectively use each media type’s concurrency, capacity, and associated costs to store data. For example; you can direct large, less frequent full backups to tape media and smaller more frequent non-full backups to magnetic media. Any standard storage policy can have, or be an incremental storage policy. A single storage policy can be used as an incremental storage policy for multiple standard storage policies. Philosophy of Storage Policies Conceivably, a single storage policy can be used to manage all data for a company. Different data retention and media storage requirements can be accounted for via configurable characteristics of secondary copies. However, in a practical sense, using a single storage policy for complex data management requirements is not recommended. The primary reason for storage policies and the reason for using different storage policies is ease of data management. Data boundaries can be identified along the lines of data type, data source, and management requirement. Additionally, business, legal, and security requirements need to be considered when designing a durable storage policy schema. Taking the time to map out a storage plan will yield both more efficient and easier management for your data.

CommVault System Administration Course – R02

108 – Storage Policies Module

Types of Storage Policy Copies Primary „ Secondary

Primary Copy

„

Synchronous z Selective z

Synchronous Contains All Data

Secondary Copy

Selective Contains Full Data Only

Types of Storage Policy Copies A storage policy can have one or more physical copies of the data assigned to it. Each storage policy copy is a container that provides a distinct set of management tools for protecting it’s associated data. Each storage policy has at least one copy called the Primary copy which is created by default. The Primary copy receives its data directly from a subclient’s data protection job. Additional copies of the data can be created called Secondary copies. Secondary copies can also receive their data from the subclient via an inline copy option; or via an Alternate copy operation from a specified copy source. There are two types of secondary copies - synchronous and selective. A synchronous copy contains all jobs from the source copy for the associated subclients. Data requiring more granular or point-in-time restore capability should use synchronous copies. A selective copy contains only selected full backup jobs from the source copy for the associated subclients. This selection of full backups can be manual, job, or time-pattern based. Selective copies are primarily used for offsite, long term storage for data used in Disaster Recovery. Note that within a storage policy, subclient data can be copied from the Primary copy and between secondary copies. While you can re-associate subclients to different storage policies, you cannot copy/move their data between different storage policies. Storage Policy copies can reside on the same or different libraries and on the same or different media. A common example would be to have the primary copy on magnetic disk media and secondary copy on removable media such as tape.

CommVault System Administration Course – R02

Storage Policies Module - 109

Updating Secondary Copies Subclient

Subclient

Inline „ Auxiliary copy „

Primary MediaAgent

Auxiliary Copy

Secondary

Updating Secondary Copies Secondary copies are updated from the specified source with any data not previously copied as required by the copy type characteristics. The update is conducted as either an Inline Copy or an Auxiliary Copy. Inline Copy An Inline copy requires that the data for the primary and secondary copy pass through the same Media Agent. The Inline Copy option is enabled on the secondary copy’s property page. Anytime a data protection job is performed, the data will be sent to the primary copy and also to this copy set at the same time. If the data protection job to a primary copy fails for any reason, the inline copy will also fail. If only the inline copy fails, the primary copy job will continue and an Auxiliary Copy job can be run to complete the update of missing data to the secondary copy. Auxiliary Copy An Auxiliary copy is an immediate or scheduled operation that copies updated job data from a source copy to one or more target copies. Auxiliary Copy can be performed between any two Media Agents, regardless of their host operating system. A source copy can either be the Primary copy (the default) of a storage policy, or a secondary copy of your choosing – with some limitations as described in this document and CommVault’s Books Online. The destination can be any active secondary copy – or all active secondary copies. If multiple active secondary copies are being updated by the same auxiliary copy

CommVault System Administration Course – R02

110 – Storage Policies Module operation, they are done so in the chronological order in which they were added to the storage policy. An auxiliary copy operation copies only Valid Data – it does not mirror media (i.e. tape for tape). Valid data is defined as data from a data protection job that has not failed or been killed. If a data protection job should not (or can not) be copied, it must be manually marked as Do Not Copy in the source copy Job View. Since a source copy can contain data from failed or killed data protection jobs, an auxiliary copy operation may result in “consolidation” of the copied data on fewer medium. In fact, auxiliary copy can be used as a consolidation method to move only valid data to another copy in order to delete the less efficient source copy. If the source copy has multiple streams, they are, by default, auxiliary copied serially - one stream at a time. If there are multiple streams on the source, you can enable an auxiliary copy job option to copy a number of those streams in parallel. The parallel copy option enables the use of maximum possible or a limited number of data streams as defined by available resources. While performing auxiliary copy operations, priority is given to using LAN-free read data paths. This means that if multiple source and destination Media Agents are involved, the Media Agent with the most LAN-free read paths will be used.

CommVault System Administration Course – R02

Storage Policies Module - 111

Managing Data Streams Job vs. Device Streams „ Parallel vs. Multiplexing „ Combine Streams „

Managing Data Streams During a data protection operation, the source server sends data to the Media Agent for storage on the media. During a data recovery operation, or restore, the Media Agent accesses data on the storage media and sends it to the destination server.

CommVault System Administration Course – R02

112 – Storage Policies Module

Job vs. Device Streams Referred to as Data Pipe „ Two types of Data Streams „

Job Stream z Device Streams z

Job Streams

Client

Device Streams

Media Agent

Tape Library

Job vs. Device Streams The connection set up between the Media Agent, and source or destination server, is called a Data Pipe or what is more commonly referred to as a Data Stream. There are two types of data streams: Job Stream – the connection between a client and a Media Agent. Device Stream – the connection between the Media Agent and the storage device. A job stream can be described as the actual data movement between the Client and Media Agent. Multiple streams provide for multiple channels through which data can flow. When used, multiple streams provide the means to parallelize an operation and thus improve the rate at which data can be written to or retrieved from the storage media. A device stream can be described as the connection between a Media Agent and the library media device. A device can be either a physical tape drive or writer in the case of magnetic library mount paths. LAN vs. SAN Streams The “Optimize for concurrent LAN backups” option on a Media Agent is enabled by default. Enabling this option will help the Media Agent host optimize its resources for concurrent data protection operations from a large number of clients. You might disable this option in a SAN environment when the only Client source is on the same host as the Media Agent.

CommVault System Administration Course – R02

Storage Policies Module - 113

Parallel vs. Multiplexing 8mb/ A sec

8mb/ A sec 15mb/sec

B

7mb/ sec

15mb/sec

A

B

B

7mb/ sec

A, B C

C

C

Dynamic Drive Tape Library

10mb/ sec

Parallel

C

10mb/ sec

Dynamic Drive Tape Library

Multiplexing

Parallel vs. Multiplexing Parallel Given sufficient resources, parallel job streams can improve performance both for data protection and for data recovery jobs. The max number of possible parallel jobs streams per subclient or per Media Agent is configurable. The actual number of parallel jobs streams is dependent upon the concurrency level of the data protection schedule. Note that over saturation of a data path or Media Agent can be more detrimental to performance than under-utilization. Be sure to consider all resources involved when considering using or scheduling parallel jobs. Multiplexing Multiplexing allows multiple job streams to be written to one device stream. This is accomplished by the Media Agent combining the job stream data and transferring it to the device via one device stream. A Multiplexing factor is enabled/set on the Media tab of the Primary storage policy copy. The Multiplexing factor setting has no effect on any auxiliary copy operation other than Inline copy. Note that the maximum multiplexing factor that can be set from the CommCell Console is 10 and the system displays a warning message when the multiplexing factor is set to 5 or above.

Best Practices It is recommended that you keep the following in mind when performing data multiplexing: CommVault System Administration Course – R02

114 – Storage Policies Module

•

Use different storage policies for file system and database type data before performing data multiplexing. Therefore, there wont be differences in the chunk sizes of the different types of data.

•

If possible use the Restore by Jobs option to restore multiplexed data, especially when restoring large amount of data. This will provide the optimum performance during the restore operation as there are fewer tape rewinds to secure the data.

•

It is recommended that you perform data multiplexing for jobs that have similar speeds (i.e. two database jobs), instead of mixing faster jobs (i.e. file systems) with slower jobs (i.e. databases). Mixing faster and slower jobs results in data stored on media that is not uniform. Hence, data recovery operations of slower clients will have added performance penalty.

•

Multiplexing is recommended if you are planning to recover: • Individual items, files and folders. • Entire computers or databases.

•

It is not recommended under the following conditions: • If you are planning to recover scattered folders, since multiplexing will further scatter the data. Also it adds to up to extra tape mounts and rewinding/forwarding on the media. • Clients which undergo very frequent restore requests.

•

The multiplexing factor is determined based on the ratio of how fast the tape drive is compared to the disk. For example, consider the following ratios: • Tape write speed = 75 GB per hour • Disk read speed (backup) = 25 GB per hour • Tape read speed = 100 GB per hour • Disk write speed (restore) = 50 GB per hour • Tape write speed/disk read speed (backup) = 100/25 = 4 Multiplexing Factor • Tape read speed/disk write speed (restore) = 10060 = 2 Multiplexing Factor

•

It is recommended that the lower of the two ratios(2 Multiplexing Factor) should be used as the multiplexing factor if you want no-penalty data recovery operations.

CommVault System Administration Course – R02

Storage Policies Module - 115

Combine Streams Auxiliary Copy A B

A B C

Magnetic Library

A B C

C Auxiliary Copy A B

A B C

Magnetic Library

A, B, C

C

Combine to Streams Whether copying in serial or parallel fashion, an auxiliary copy operation normally copies each source data stream to a separate destination stream. This means if there were four data streams on the source copy, then the auxiliary copy operation would use four data streams on the destination library. Each stream requires its own destination media. In some cases such as copying from a multiple stream magnetic library to a limited device tape library, this may not be the most efficient – or desired method of copying data. As such, the destination storage policy copy has an option to Combined to Streams. Combining streams is done serially and is used to improve media storage efficiency – not throughput. The Combine to streams options is considered a media management tool and is configurable only on secondary copies. Example: 4 subclients with 2 data readers configured for each subclient will have a total of 8 streams. Setting the Combine to Streams option to 2 will result in a 4 to 1 consolidation and the 8 streams will be combined onto 2 tapes.

CommVault System Administration Course – R02

116 – Storage Policies Module

¡ Data Retention, Management, and Aging Determining Data Retention Requirements „ Managing Backup and Archive Retention „ Understanding Cycles and Days „ Data Aging „ Managed Disk Space „

Data Retention, Management, and Aging Most agents follow basic retention rules. Basic retention rules are defined by the user as retention days and retention cycles. These parameters determine how much data is retained and for how long. For a data cycle to be eligible for data aging, both its retention days and retention cycles must be exceeded. Basic Retention Rules for All Backups The amount of time that you want to retain the data that is backed up via this storage policy copy, and the number of full backup cycles that you want to retain the data that is backed up via this storage policy copy. Select from the following options: • • •

Infinite: retains the data for an infinite amount of time n days: allows you to set the length of the retention period n cycles allows you to set the number of retention cycles (not applicable for agents that do not have cycles, such as Data Archiver)

Spool Copy (No Retention): sets the retention rule on a primary copy to 0 days and 0 cycles.

CommVault System Administration Course – R02

Storage Policies Module - 117 Basic Retention Rules for Data/Compliance Archiver Data The amount of time that you want to retain the data that is archived via this storage policy copy. Select from the following options: • •

Infinite Retains the data for an infinite amount of time Retain for n days

Allows you to set the length of the retention period. Extended Retention Rules for Full Backups For Infinite or n Days Keep: All Fulls - All full backups • • Weekly Full - The first or last full backup of every week (each week starts on Friday) Monthly Full - The first or last full backup of every month (each month starts on the • first of the month) • Quarterly Full - The first or last full backup of every quarter (each quarter starts on January 1, April 1, July 1, and October 1.) Half Yearly Full - The first or last full backup of every half year (each half year starts • January 1, and July 1) • Yearly Full - The first or last full backup of each year (each year starts on January 1)

CommVault System Administration Course – R02

118 – Storage Policies Module

Determining Data Retention Requirements Legal Requirements

Risk Assessment

Customer Confidence

Retention Policy

Government Requirements

Company Requirements

Reputation

Industry Requirements

Cost

Determining Data Retention Requirements Understanding data retention and data aging is essential for managing media in a CommCell environment. Data retention is defined by the user for each Storage Policy Copy as a minimum number of Cycles and/or Days. Determining how long to keep your data will vary on many different key components which must be assessed before the creation of your storage policies. Enforcement and implementation of various retention laws have raised the level of attention concerning data retention along with the specter of litigation should you not be in compliance. Some of the major recent additions to retention laws have been: • • •

Sarbanes-Oxley Act of 2002 Gramm-Leach-Bliley Act (GLB Act), also known as the Financial Modernization Act of 1999 Health Insurance Portability and Accountability Act (HIPAA) of 1996

You should be aware of the regulations governing your particular industry. Ignorance of the law will never pass as an alibi. Companies run the risk of paying hefty penalties for noncompliance for reasons including: • •

Lack of awareness De-prioritizing non income-generating projects

CommVault System Administration Course – R02

Storage Policies Module - 119 •

Simple failure to understand the laws

Although experts continue to discuss and translate the ramifications of recent rulings, certain concepts are clear: CEOs and CFOs assume authority and accountability for accuracy of records, and CIOs or IT divisions ensure effective implementation and technical compliance. The responsibility to "get it right" falls largely on technology professionals.

CommVault System Administration Course – R02

120 – Storage Policies Module

Managing Backup and Archive Retention „

Backup Cycles z Days z

„

Archive z

Days

Managing Backup and Archive Retention Backup Each subclient is associated to a storage policy. How long the backup data will be retained on the backup media is determined by the retention rules set in the Storage Policy Copy Properties dialog box. This will affect media usage, and is an important consideration when planning your backups. A longer retention period, for example, could use more media for securing the data over time. If a retention period other than infinite is selected, the data will be pruned according to backup cycles in relation to the retention rules you set in the Storage Policy Copy Properties dialog box. Pruned data can be overwritten on the backup media. The backup data from a subclient will be retained according to the rules set for the storage policy associated with it. The ability to define data in subclients, and then associate them to specific storage policies allows you to prioritize exactly what data is retained and for how long. Backup retention is set to cycles and days. Archive Data Aging rules are unique for the Data Archiver and Compliance Archiver Agents. All data and jobs in a migration archiving or compliance archiving operation, must meet the basic or extended retention rules in days (as specified in a storage policy copy) in order to be aged. The Migration Archiver and Compliance Archiver Agents do not support retention cycles.

CommVault System Administration Course – R02

Storage Policies Module - 121

Understanding Cycles and Days

Example

S M T W T F S

FULL INC INC INC INC INC INC

Cycle 1

FULL INC INC INC INC INC INC

Cycle 2

FULL INC INC INC INC INC INC

Cycle 3

FULL INC INC INC INC INC INC

„

Cycle 4

12 noon 12noon

„

„

Basic Retention set for 2 Cycles/14 Days Full backups scheduled every Sunday Incremental backups scheduled all other days

Understanding Cycles and Days A Retention Cycle is defined as a group of dependent data protection operations starting with a full backup and including all subsequent data protection operations up to, but not including, the next full backup. Dependent operations are incremental, differential, or transactional log backups that support or depend upon a full backup. Retention days specify the minimum number of days to keep all data protections jobs in the cycle – not the individual jobs. The default settings for time and cycle parameters are set to infinite, but can be changed to better suit the retention requirement of the data being secured. Retention time, also referred to as “Days”, is defined as the amount of time that a cycle needs to be available for a single subclient. Retention time is calculated in terms of 24 hour days from the completion time of the last data protection job in a cycle until the start time of the data aging job.

CommVault System Administration Course – R02

122 – Storage Policies Module

Data Aging „ „

Frees storage media for re-use For data that exceeds retention z z

z

Erase data on magnetic storage (default) Use managed disk space to preserve data in magnetic libraries Flags data on removable storage as “Aged Data” until overwritten or erased

Data Aging Unless you have an unlimited storage budget, you’ll want to delete all protected data that is no longer required and free up media for re-use. This is accomplished through the Data Aging operation. Retention and conditional criteria is evaluated and those jobs or cycles that no longer meet this criteria can be Pruned. Pruning will either flag or remove data pointers from the CommServe database and, as appropriate, erase (magnetic) or reclassify (removable) media to spare status”. Note that reclassifying tape media to spare status can only be done when all the data on a specific media is pruned off. While retention criterion is the main factor evaluated by a Data Aging operation, there are many conditional criterion that may exclude a job from being pruned. These include such things as transaction log jobs where the associated database is still retained and data that has not been auxiliary copied to a secondary copy as required. Consult Books Online for conditions that affect each data type. These conditions are not intended to impede re-use of media, but rather ensure complete restoration can be accomplished as expected. Data aging is performed on all active storage policy copies within the CommCell. Storage policy copies can be excluded from data aging by clearing the Enable Data Aging option on the Retention tab of the copy properties page. Different data aging schedules for each Storage Policy copy can be configured in the Aging Options tab of the CommCell’s Data Aging task configuration dialog window.

CommVault System Administration Course – R02

Storage Policies Module - 123 Data on a magnetic library is classified as Managed when it exceeds retention requirements, but storage usage does not exceed the managed disk space threshold limit. Managed data is treated the same as retained data for browse and restore. Disk space is periodically checked for used capacity. When the managed disk space threshold is exceeded, the managed data is erased – oldest first – until the lower threshold limit is reached. Tape media is serial vs. random access and cannot be overwritten until all jobs on the media have been flagged as Pruned in the CommServe database. Until the tape media has either been specifically erased, or is overwritten through re-use, the Aged jobs on the tape can be browsed and restored. Unlike Managed data, the ability to browse or restore Aged data must be specifically enabled or selected. Aged data is accessible in a Restore by Jobs option and can be browsed for individual objects if the Browse/Recovery control option to Show aged data during browse and recovery is selected. To extend the availability of aged jobs on tape, you can mark a specific tape as Prevent Re-use which will move the tape to the Overwrite Protect Media pool

CommVault System Administration Course – R02

124 – Storage Policies Module

Retention Variants „ „ „ „ „

„

Data not copied to secondary storage Transactional log data Failed jobs Manually retained/disabled/pruned Jobs De-configured or re-associated subclients Incremental Storage Policy

Retention Variants There are variants to the Retention/Aging rules. These variants exist to ensure every reasonable expectation for recovering data is met. Some of the more prevalent variants are: Data not copied to secondary storage If an active secondary copy exists within a storage policy, the expectation is that the administrator wants to have a copy of the source. Hence, until a copy is made, the source data cannot be aged. If this is a problem, a solution would be to inactivate the copy. However, doing so will allow the source data to be pruned without a copy being made. Transactional log data Transaction\archive\logical log (log) backups are not considered part of the backup cycle. Therefore, storage policy cycle retention parameters do not apply to them and have the following unique data aging rules: • Logs older than the oldest non-prunable data are prunable regardless of the retention rules of the copy. • For logs that are more recent than the oldest non-prunable data, these logs are not prunable if: o They have not exceeded the retention days criteria. o They reside on a primary copy and have not yet been copied to a secondary copy.

CommVault System Administration Course – R02

Storage Policies Module - 125 o They reside on the copy that has the longest days retention criteria amongst the other synchronous copies within the storage policy. The log data can be valid, partial, or disabled. • If the status is partial or disabled, the valid copy of the log is kept on the copy with the longest days retention criteria among the remaining copies. For each backup copy of the database at least one copy of associated logs will be retained. Those logs will be retained on the copy with the longest retention value setting. Logs are retained beyond that copy's retention settings as long as the associated backup copy of the database exists. Logs on lesser retention value copies are pruned in accordance with the set retention value. Restoring a database with transaction logs requires that you either (a) Do not select a copy precedence for restore, or (b) you select a copy precedence that has the transaction logs. In the case of (a), the restore process will automatically determine which copy holds the required logs and restore those logs. Failed Job data Failed job data on magnetic libraries will be deleted by the Data Aging operation. Failed job data on removable media is immediately eligible for aging, but since it cannot be deleted, it can be restored using the Restore by Job option or restored with the Media Explorer tool. Manually Retained/Disabled/Pruned jobs Individual jobs can be selected for manual retention, disabled for copy, or pruned for data aging. Manually retained jobs are excluded from normal data aging. If a removable media is being prevented from moving to a scratch pool by a manually retained job, it will be indicated as such in the Data Aging and Media Forecast report. Jobs may be disabled for copy by the administrator if an archive copy job fails because the job has corrupt data. Jobs can be pruned manually before their normal data aging event. Pruning a Full backup job will include all dependent incremental or differential jobs. Pruning jobs manually is usually done to free up media. De-configured or re-associated Subclients Subclients who have been de-configured or re-associated to another storage policy can no longer generate full backups to the original storage policy. Still, the data may be required for recover. Galaxy tracks when a storage policy association is changed and knows where the pertinent data is kept. However, the retention criterion of cycles on the original storage policy is ignored and only retention days are used to determine the data aging eligibility. Incremental Storage Policy If data aging is performed on a storage policy that has an incremental storage policy enabled, the data aging operation counts backup cycles across both full and incremental storage policies. The

CommVault System Administration Course – R02

126 – Storage Policies Module data on a full policy is pruned based on the retention of the full policy, and data on the incremental policy is pruned based on the retention rules of the incremental policy. If the incremental storage policy is also being used as a regular storage policy (and has full backups), the full backups will be also pruned according to any basic and extended retention rules that are set. It is recommended that the retention rule for the full storage policy be greater than the incremental storage policy. Data on incremental policy will be pruned earlier if it has shorter retention comparing with full policy. If the incremental storage policy has longer retention than a full storage policy, this may result in dangling incremental jobs.

CommVault System Administration Course – R02

Storage Policies Module - 127

¡ Roles of Verification, Encryption, and Content Indexing

Encryption „ Content Indexing „ Content Indexing and Search „ Data Verification „

Roles of Verification, Encryption and Content Indexing In addition to storing protected data, there are additional actions a Storage policy can be configured to perform to further assist in the protection and recovery of data.

CommVault System Administration Course – R02

128 – Storage Policies Module

Encryption Inline

Copy-based

Subclient B

A C

Subclient A,B

B

A Encrypt C

Magnetic

C

Prim ary Copy Secondary(A,B) Secondary(C)

A,B C Magnetic

Prim ary Copy Secondary(A,B) Secondary (C) Encrypt

Inline vs. Copy-based Encryption Overview The software allows encrypting data both for transmission over non-secure networks and for storage on media. The data can be encrypted online during a data protection job if transmission paths or the primary copy security is required. Keep in mind the encryption may impact the performance of the job. Encryption will always be performed after software compression. Encryption can also be done offline during an auxiliary copy operation to any secondary copy. Encrypting a secondary copy provides protection if storing the copy in an unsecured location as well as protecting the data during transportation. The flexibility of key management schemes makes data encryption useful in a wide variety of configurations. Keys can be retained internally, externally, and on the protected media. During restores, keys can automatically be applied or you can require manual entry. You can select from several algorithms and key lengths. The default encryption method uses the CommServe’s Media password to perform the encryption. A copy of the Media password is kept in the CommServe database and on the protected media. During restore the key is applied automatically to decrypt the data. CommVault’s Media Explorer can also be used to access the data directly from the protected media. CommVault System Administration Course – R02

Storage Policies Module - 129

If you are concerned that media may be misplaced, data can be encrypted before writing it to the media and the keys stored only in the CommServe database. In this way, recovery of the data without the CommServe is impossible - not even with CommVault’s Media Explorer. This No Access mode is also completely transparent. Once enabled, it will work requiring no additional activity on your part. If you need only network security, the encryption keys are randomly chosen for every session. Data is encrypted on the Client and is decrypted on the Media Agent and the keys are discarded at the end. The entire process is completely transparent. All you have to do is to enable encryption, and select the cipher and key. Encryption keys can also be protected with your own pass-phrase (RSA algorithm with 1024-bit keys) before being stored in the database. If the database is accessed by unauthorized users, and the media is stolen, the data will still not be recoverable without the pass-phrase. This highest level of security comes at the price of having to enter the pass-phrase for every recovery operation and not being able to run synthetic full backups. You can also export a file that contains the scrambled pass-phrase of the client computer to a dedicated directory on another computer, the system can recover the client’s data to that (and only that) computer without prompting you for the pass-phrase. Explicitly enabling synthetic full backups in the GUI will create a copy of unlocked encryption keys in the database, which will be accessible only to synthetic full data protection operations. In this case the regular data recovery operations will still prompt you for a pass-phrase, but synthetic full data protection operations will not.

CommVault System Administration Course – R02

130 – Storage Policies Module

Content Indexing Production (online) „ Copy-based (offline) „

Content Indexing Overview Content indexing adds the ability to find files and messages by searching their contents for a keyword or phrase. Before you can search by content, a content index must be created. Content indexing is not installed or activated by default in order to ensure efficient use of your resources. Content indexing can be configured on your Production server (online) or on your protected storage copy (offline). Production (online) The online content indexing module is used to index the live data on Windows computers. The Content Indexing Agent and Data Classification Enabler software must be installed on the computers in the CommCell you wish to content index and search. Once installed the content index function must be enabled and scheduled.

Copy-based (offline) Offline content indexing module is used to index the storage data secured by various data protection operations. For this reason the configuration of the offline content indexing is associated with a storage policy. The Offline Content Indexing is installed along with the Content Indexing Engine and does not require separate installation.

CommVault System Administration Course – R02

Storage Policies Module - 131

Content Indexing and Search Content Indexing Node

OFFLINE

ONLINE Client

Storage Policy Copy

OCI Agent

DCE Metadata

CI Store

Media Agent

Disk Tape Objects Files Messages Documents

Search/Find

Web Search Server

CommServe

Index

Web Search

Queries/Results SQL Database

Content Indexing and Search A Content Indexing Node (CIN) is created with installation of the first Content Indexing Engine (CIE). A CIN may contain multiple CIE’s for scalability and multiple CIN’s may be created to offer a choice of where content indexing will be conducted. Because of the CPU processing and storage requirements, a CIE should be single purposed. Offline content indexing capability is possible with the installation of the first CIE. Content Indexing is selected and configured in the Storage Policy’s dialog box. Any copy within a storage policy can be used as the source for content indexing. Stored objects such as Files, messages, and documents can be indexed. Database objects cannot be indexed. A Content Indexing job for each storage policy must be run to create the content index on the CIN. The CIE used must have local or network access to a Media Agent with data path access to the copy being indexed. Online content indexing capability requires installation of the Online Content Indexing (OCI) Agent and the Data Classification Enabler (DCE) on the Client. The OCI Agent provides for defining subclient content to be indexed and a share name path for accessing search results. Each OCI subclient can be scheduled to index of its respective content. Prior to indexing, subclient content must be scanned and available through the DCE. DCE runs as a background process to collect metadata on objects within each file partition. A small SQL capable database for this metadata is maintained on each

CommVault System Administration Course – R02

132 – Storage Policies Module partition. A client with an OCI agent installed must have network access to the CIS conducting the index. Search Content Indexes can be used by the CommCell Console’s Find and Search features, CommVault’s Outlook Add-in and Archive Mail Browser or via the web using CommVault’s Web Search Server. The Find functionality is used without content indexes to locate protected storage objects by name across multiple storage location indexes. The CommCell Console’s search feature is available at the Backup/Archive Set, Agent, Client Computer, Client Computer Group, and CommCell level. The level you choose to conduct the search defines the initial scope of the content index to be used. Search options allow you to precisely define your search criterion and scope. CommVault’s Outlook Add-in has a search function that uses content indexes for Exchange messages. This search function is also available as a standalone feature called Archive Mail Browser for hosts without Microsoft Outlook. If CommVault’s Web Search Server (WSS) is installed and configured, two more search capabilities are added which are accessible via the web. End User Search can be used to search for both offline and online data owned or read accessible by the logged in user. Compliance Search can be used by legal or corporate IT to conduct a search of all indexed data online and offline regardless of who owns the data. Web Search queries and result set can be saved in a SQL database for later use. Objects returned by a key word search are displayed with either dynamic or static teasers. A dynamic teaser is the text data located near the found keyword. A Static teaser is the first few lines of text from the object itself. Objects identified during an Online search can be opened or saved if a share name path is configured and available to the data. Objects identified during an Offline search must be saved as a result set and then selected for “Prepare for Viewing” to initiate a restore operation. Selected objects are restored to a temporary cache where they can then be selected for view or saved.

CommVault System Administration Course – R02

Storage Policies Module - 133

Data Verification „

„

„

„

Verifies that all selected data is valid for restoring and for being auxiliary copied Data Verification can be scheduled or executed on demand Verification options can be set on Storage Policy copy User can also select individual backups for verification and/or select individual media for verification

Data Verification During a data verification operation, protected data is checked to see that it is available and readable. CommCell metadata is used to determine where the data is stored, what chunks are used and whether the media and chunks can be located and successfully read. Short of restoring the data, this is the best way to verify accessibility of protected data. You can verify all or selected jobs on a specific copy or media. Verification is a schedulable job and can be run serially or in parallel for multiple stream copies. Data Verification is accomplished by selecting a storage policy. All copies in the storage policy can be verified or only a selected copy. Data verification can be run to check if valid data can be successfully auxiliary copied. On the other hand, if an auxiliary copy operation is successfully run, the data can be considered verified.

CommVault System Administration Course – R02

134 – Storage Policies Module

Module Summary Key points to remember

Summary •

Purpose for Storage Policy Copies - Primary (backup), Sync (Specify by client, takes all FID), Selective (specify by client fulls only, good for remote sites with little resources)

•

Retention can be defined as Cycles and Days, Days only, Basic (cycle and days, Infinite, Spool) Extended (weekly, monthly, quarterly, yearly).

•

Content Indexing can be configured at the Primary storage (production data) or Secondary storage (protected data).

•

Data Aging - When does it happen, Why does it happen, What does it do?

•

There are two types of Streams - Job based (from client), Device which is defined at the Storage Policy.

CommVault System Administration Course – R02

Clients Module - 135

Clients

www.commvault.com/training

CommVault System Administration Course – R02

136 – Clients Module

¡ Overview What are Clients? „ Understanding the role of Agents „ Backup and Archive Sets „ Configuring Subclients „

Clients Module Objectives: • • • • •

List and describe the common types of Agents available and their function in protecting/restoring data. Describe the function/role of a Backup/Archive Set in defining and managing Agent data. Describe the role of Data Classification in determining archive and backup set content. Describe a subclient and the purpose of the different property settings used for facilitating data protection and movement. Describe the function and configuration steps for using a subclient policy.

CommVault System Administration Course – R02

Clients Module - 137

¡ What are Clients? „

Host system Physical machines z Clustered Systems and Applications z Virtual machines z

„

Properties

What are Clients? A client is the host system addressable by an IP Address that provides the resources necessary for an Agent to access the data. Clients can be either physical machines, VMWare or Microsoft Virtual Server style virtual machines, or cluster virtual machines in a supported clustered environment. Data accessed through a client can reside on local/SAN disks or available through CIFS/NFS paths. During data protection and restore operations, the Client must have bi-directional access via TCP/IP with the CommServe and supporting Media Agent. Connectivity is required to maintain control over the operation, update indexing, and for tracking the data on these systems. Although CommVault software does have some robust features that can overcome minor communication gaps, extended loss of communications will cause backup or restore jobs to fail. The CommVault software supports various file systems and applications on Windows, Netware, UNIX, Linux, and Macintosh operating systems at various version and service pack levels. Always consult the latest available documentation to see what is supported. Properties In the CommCell Console, Client specific and common shared agent properties are set and selected at the Client level. These properties include Client Priority; Job Results Folder retention, location, and low space alert threshold; Encryption option; and Content Indexing option.

CommVault System Administration Course – R02

138 – Clients Module

¡ Understanding the role of Agents „

Agent Roles File System z Application z Archive z

„

Indexed or Non-indexed Based

Understanding the role of Agents Client Agents are software modules that perform data protection and data recovery operations for specific operating systems or applications. Different file and application data types can not or should not be protected or restored in the same manner. Each data type has unique requirements to ensure consistency and accuracy when collecting or restoring the data. Multiple agents may be required to protect all types of data residing on a computer. A File System Agent provides an indexed-based backup and granular restore capabilities of files and system metadata as may be required to perform full recovery of operating system and user files. An Application Agent provides a database and/or object (document/message/attribute) level backup and restore capability for application managed data. In the case of databases, the application agent ensures referential integrity and consistency during backup and may provide for both restore and application recovery actions as appropriate. Object level protection in an application allows administrators the option of granular object restore instead of restoring the entire database and extracting/using only a small part for recovery. An Archive Agent provides the movement of infrequently accessed data from high cost production storage to lower cost storage with transparent recall capability. A set of rules is defined by the administrator to archive selected objects (files/messages/documents) to alternate storage and optionally leave a “stub object” in their place. The archiving frees up production space and/or slows down the growth of disk space usage. The stub object contains the necessary

CommVault System Administration Course – R02

Clients Module - 139 information to locate the archived object in protected storage. Native or client mechanisms are provided to facilitate the automatic and transparent recall of object when the stub is accessed by a user or application. Indexed or Non-indexed Based An index based agent provides granular recovery of individual objects based on an index maintained by CommVault (e.g. Files, System State objects, E-mail messages, Documents). Non-indexed based agents are ones where the degree of granularity is insignificant and the need to quickly protect or restore the entire data set for consistency purposes is paramount (e.g. Database Instances, Image level agents).

CommVault System Administration Course – R02

140 – Clients Module

¡ Backup and Archive Sets Understanding Backup and Archive sets „ Using On Demand Sets and Protection „ Using Data Classification „

Backup and Archive Sets A backupset is a logical grouping of subclients representing a complete set of data related to a specific agent. For example, a backupset for a file system agent will contain all files on all drives within the Client machine.

CommVault System Administration Course – R02

Clients Module - 141

Understanding Backup and Archive Sets „

„

„

Logical grouping of all subclients Provides for Distinct Management Defines Data access and/or Collection Method

Backup/Archive set

Subclient Content

Understanding Backup and Archive sets A Backup or Archive set is a logical management grouping of all subclient content for the parent agent. Since subclient content can vary as well as their protection schedule and storage, a consistent means must be provided to address (e.g. User login, application name, scan method) and Browse/restore all agent data as a single entity. Some backupset additionally provide options for application wide rules used by all subclients when identifying, protecting, or restoring data. User-defined Backup Sets For agents that support them, you can create additional backup sets, i.e., user-defined backup sets, to back up data using different backup options or schedules. By adding user-defined backup sets to your agent configuration, you can establish: • • •

One or more series of backups for a given client computer One or more subclient groups for a given client computer One or more series of backups for related subclients (i.e., members of a subclient group)

User-defined backup sets provide multiple management options for the same set of data. When a new user defined BackupSet is created, it becomes a complete collection of data on the client for the specific Agent.

CommVault System Administration Course – R02

142 – Clients Module The advantage to using multiple backup sets is when customized backups need to be performed that do not conform to the current subclient design within the default backup set. Additional backup sets can be created and scheduling can be set up to only backup these subclients when needed. Note that duplicate data within separate subclients in separate backup sets can not be backed up simultaneously. Conflicting operations can cause jobs to go into a pending state until running jobs are completed. Caution should be used when creating multiple BackupSets. Improper configuration and scheduling can result in failed jobs or data being backed up multiple times. All Archiver Agents support the ability to create additional archive sets, i.e., user-defined archive sets. This feature is useful for archiving client data using different archive options or schedules, and for workload balancing. If there is an overlap in the data archived between various Archive Sets on the same client, data must be retained on all the storage policies used by various subclients in all Archive Sets on the client in order to recover data.

CommVault System Administration Course – R02

Clients Module - 143

Using On Demand Sets and Protection „

„

„

„

Dynamic content defined at backup time Supports only one subclient Can be executed from command line Not used in Full System Restore

Directive File C:\FileList_A.txt C:\FileList_B.txt

C:\FileList_A.txt D:\DB\data1.ora D:\Logs\012.log D:\Logs\013.log

Using On Demand BackupSets For File System Agents, On Demand backup sets enable the user to define dynamic job-based subclient content. Only one subclient is allowed in an On Demand backup set. The contents of the subclient are defined in a Directive File provided by the user when a backup is executed or scheduled. The directive file is a plain text file that contains one or more full path Content files. The plain text content files define the actual file(s) to be protected. They can contain one or more, full path filenames that you want to back up. Directories are allowed and expanded, but no regular expressions are allowed. Note that the Directive File is not required for the NAS NDMP Agents. They will use a Content File. On Demand backup sets were designed for integration with database application backups where the contents of the backup are event driven. The use of a directive file meets requirements for multiple sources for content. For example; one script could determine which database files to backup and another determine which log files to backup. The results of both scripts generate two content files. Another example might be a tablespace level backup where a script lists only those data files in each tablespace that needs to be protected. Each tablespace generates a separate content file. Any number of On Demand backup sets can be created. For Data Archiver Agents, On Demand Data Protection Operations allow content to be specified as an external input at the time of initiating a data protection operation. Content for On Demand Data Protection operations use a Content File as previously described.

CommVault System Administration Course – R02

144 – Clients Module

The On Demand Instance is a logical entity that allows Oracle data to be protected on demand, with content specified in a RMAN script that is run through the Command Line Interface. The primary difference between an On Demand Instance and a traditional Instance, is that data protection operations for the subclient associated with an On Demand Instance only supports On Demand Data Protection operations.

CommVault System Administration Course – R02

Clients Module - 145

Using Data Classification for Archive and Backup Define data based on attributes „ Minimizes scan time „

Files A*.* After created by 2/1/07

*.txt >1G

Data Classification for Archive and Backup The Data Classification Enabler allows you to look at and define data based on its attributes and independent of where the data is located. In effect, the enabler allows you to do this irrespective of the boundaries imposed by traditional folders, files, and volumes; it looks at all the data available and uses specific characteristics of the data to select data. As such, the enabler allows you to create a dynamic new view of the data. Data Classification involves a real-time monitoring of file systems and maintenance of metadata databases (metafiles) that describe the file systems. Data Classification is integrated with Active Directory, and it allows you to define data for various user types. Data Classification is especially useful when used to define data for user groups on a single share. Data for disparate groups within a company, such as Finance, Sales, and Maintenance, can reside on the same share, and Data Classification can define data for users in these groups using rules that you define specifically for data on this share. Since just one share is being used, an efficient use of disk space is being made. And since Data Classification supports Active Directory, you can authenticate against the Active Directory domain the users whose files you want to archive. Data Classification greatly enhances a significant data management process in the industry: the "Information Life Cycle Management" of data, which is the process of determining the "treatment" of data over its life cycle. Traditionally, decisions regarding such treatment have been made based on the location of the data, typically folders/files. However, the folder/file structure suggests that the associated data belongs to one class or category and is therefore not

CommVault System Administration Course – R02

146 – Clients Module very conducive to categorizing data for classification purposes. By eschewing traditional folder/file boundaries and by searching for common attributes, the Data Classification Enabler provides a much more powerful treatment of data. The Data Classification Enabler also provides the following capabilities: • • •

It can be installed independently of any agent that uses it It minimizes the amount of scan time needed to search for data by querying the meta databases that have been created by the Data Classification services. Data Classification is highly fault-tolerant. For example, if a meta database is deleted while the Data Classification services are running a scan of this database, the services can run another scan of the database. Also, if you disable Data Classification during the initial data scan, the database will restart the initial scan. Finally, if you add a new volume after the Data Classification Enabler is installed, the new volume is automatically detected, and a new database is created.

Support for the Windows File System Agent The Windows File System Agent can use the Data Classification Enabler to scan data before backups. The availability of the Data Classification Enabler for data scanning depends on a number of factors including the data type. If the enabler is not available for data scanning, another scan method is used. To use a Data Classification Enabler scan, you must ensure that the Data Classification Enabler software is installed on the client. Data Classification uses the traditional rules established for the File Archiver for Windows Agent. In addition, it provides several unique rules. These rules are available only when Data Classification is enabled for the supported agent. All the rules for Data Classification are configurable from the DataClassSet subclient properties Rules tab of the File Archiver for Windows Agent. The unique Data Classification rules include the following: • • •

Folders/Files Owned By - Allows you to select and exclude files belonging to specific users and user groups File Paths - Allows you to select and exclude files based on file location and specific file characteristics (e.g., file extensions) SQL Query Strings - Allow you to use SQL query-like commands to define more complex rules based on your requirements

Whenever a DataClassSet subclient is created, a default set of rules is established. These rules are reflected in the various rules tabs. Also, the SQL query string, which has its own tab, is automatically disabled; however, the default set of rules is formulated into the SQL query string. At this point, you can start changing rules either from the various rules tabs or within the SQL query string.

CommVault System Administration Course – R02

Clients Module - 147

¡ Configuring Subclients Defining Content Options „ Using Filters „ Pre/Post Process Commands „ Setting Storage Options „ Using Subclient Policy „

Configuring Subclients If all data had exactly the same protection management requirements there would be no need for subclients. But all data is not created equal. Some data needs to be retained longer while other data needs to protected more frequently. Some data may need to go directly to tape media for immediate offsite storage and others - not. The functionality of subclients allow you separate content and define its management options as required.

CommVault System Administration Course – R02

148 – Clients Module

Defining Content Options „

„

Content is mutually exclusive within a backup set Can use wildcards in defining content z

„

Limited to last level of data path

Archive Content

Defining Content and Content Options Data protection content is defined at the subclient level for File System Agents. The default subclient, created during installation, has the unique characteristic of including all protected data not explicitly covered by other subclients within the backup set. Additional subclients can be created by the administrator to separate and manage a subset of the backup set data. The content of each subclient is mutually exclusive within the backup set. This means the content, as defined by one subclient, will not be protected by another subclient. The only exception to this is the NAS NDMP Agent subclient. Administrators must manually configure NDMP subclients to not have overlapping content. In the case of application subclients, the inclusive feature of the default subclient means it can automatically "discover" all data requiring its protection. In some Agents, this automatic discovery feature can be disabled (for example: SQL Server), filtered (for example: Lotus Notes), or otherwise configured to assign content to other subclients (for example: Exchange 2000/2003 mailboxes and GroupWise databases). For a Data Archiver subclient you can establish the criteria to determine which content will be scanned (i.e., mailboxes, libraries, volumes, folders, and directories) and which data objects will be archived (i.e., messages, items, and files). Depending on the agent type, archive rules can be configured for on variations of disk space, quota, time, and/or size. With Data Archiver for Windows with Data Classification Enabler installed, you can further expand the content rules to include user and group ownership.

CommVault System Administration Course – R02

Clients Module - 149

Using Wildcards to Define Content Regular expressions (or wildcard characters) can be used to define content in user-defined subclients. Wildcards expressions are characters such as * or ?. Regular expressions include patterns such as [a-f] or *. [l-n]df. The terms are interchangeable. The use of regular expressions in defining content is limited to the last level in the data path. For example: F:\Users\[A-L]* is an acceptable use of regular expressions in defining content, whereas *.pst is not an acceptable use. For File System subclients, the use of regular expressions requires the explicit enabling of the subclient option to Treat characters as regular expressions. This is required for some operating systems (for example: MAC OS), because certain characters such as * or ? can be part of a legitimate file path or name. Keep in mind that this option does not appear in the default subclient since use of regular expressions in the default subclient is not supported.

CommVault System Administration Course – R02

150 – Clients Module

Using Filters „

Global Filters z

„

Local Filtering z z

„

Unix, Windows, Netware Exclusion Exception

Subclient policy filters

Inbox User Files System Files

Using Filters Filtering unnecessary data from data protection operations can reduce backup time, storage space, and recovery time. Most, but not all Agents include some filtering capability at the subclient level. File systems' subclient filters can be defined for a path, directory, or file level. The exception is NAS NDMP Agents which only support file names or directory names. Some application Agents such as Exchange Mailbox, Lotus Notes Database, and Lotus Notes Document also allow filters to be defined. There are four basic types of filters that can be used: •

Exclusion filters can be defined at the subclient level and exclude data from being unnecessarily protected.

•

Exception filters can only be defined at the subclient level for supported agents. An Exception filter allows you to define directory or file exceptions to a filter defined in the exclusion section. For example, you can "exclude" the C:\Temp directory with the "exception" of C:\Temp\log directory. In some cases, wildcards are supported in the Exception filter.

•

Subclient Policy filters are made up of Exclusion and Exception filters and applied by a subclient Policy. You can elect to not include subclient Policy filters in a

CommVault System Administration Course – R02

Clients Module - 151 subclient and thereby define your own Exclusion and Exception filter for each subclient. •

Global Filters are made up of Exclusion filters that can be defined for each operating system such as Windows, and Netware, and for the Exchange application. Both Linux and MAC OS fall under the UNIX global filter. Filters defined at global level can be included or excluded in a subclient filter definition.

CommVault System Administration Course – R02

152 – Clients Module

Pre/Post Process Commands Available on Data Protection and Restore jobs „ Supports executable and scripts/batch commands „ Return code evaluated „

z

„

Exception: Netware

No timeout limit

Using Pre/Post Process Commands Data Protection/Archive jobs consist of several sequential phases. Each phase must complete before its successor can begin. The Subclient Properties (Pre/Post Process) tab allows you to use these phases as triggers to start pre/post processes. For example, copying files from a share to your local disk and then deleting the files after the data protection/archive operation has run, or you might want to run a database integrity check just prior to backing up your databases or workspaces. Pre/Post processes can also include tools, services, and messages. For each batch file/shell script that is associated with a subclient's Pre/Post processing phase, the system examines the return code. A return code of zero indicates the phase was run successfully, and the job continues. A non-zero return code is interpreted as a failure condition, the phase is retried, and the job is put into a pending status. Under normal circumstances a specified post process command is only executed upon the successful completion of the respective job phase. However, you may still want to run a post process even if the job phase did not complete successfully, especially if your post process is used to bring a database online or release a snapshot. The system provides you with an option to run post processes when the respective job phase is killed, fails, completes with one or more errors, or is interrupted due to a media reservation issue--in addition to successful job phase runs. The Run Post Process for All Attempts option can be selected on the subclient Properties (Pre/Post Process) tab dialog to enable this functionality.

CommVault System Administration Course – R02

Clients Module - 153

Keep in mind that there are a couple of scenarios where a post process will not be executed for all attempts to run the job phase, even if this option is selected: when a user suspends/stops the job during the respective phase, or if the job is interrupted during that phase due to Operation Window rules. If you encounter either of these conditions and your post process is designed to bring a database online, release a snapshot or execute some other critical process, you may want to manually perform these tasks depending on the situation. NetWare will not wait for pre/post processing commands to complete, unless a command delay is configured for that process. If a command delay is not configured, NetWare will immediately move to the next phase of the backup job after launching the command. Also, command failure will not prevent the next phase of the backup job.

CommVault System Administration Course – R02

154 – Clients Module

Setting Storage Options Data Storage Policy „ Data Transfer Options „ Single Instancing „

Setting Storage Options Data Storage Policy Although this property is editable on the target subclient, once it is changed there, that target subclient will override or ignore any further Storage Policy settings specified or changed in the associated subclient template. Data Transfer Options Several configurable options to efficiently use available resources for transferring data secured by data protection operations are provided in the subclient. This includes the following: • • •

Enable or disable Data Compression either on the client or the Media Agent. Configure the transfer of data in the network using the options for Network Bandwidth Throttling and Network Agents. Although the Data Compression setting is editable on the target subclient, once it is changed there, that target subclient will override or ignore any further Data Compression settings specified or changed in the associated subclient template.

Single Instancing Single Instancing identifies the duplicate items in a data protection operation and maintains references for the duplicate items. When a data protection operation is performed for the first time all the data is stored; if the same data is subsequently identified in another data protection operation, reference counters are incremented and the data itself is physically not stored in the storage media.

CommVault System Administration Course – R02

Clients Module - 155

Single Instance Storage is currently supported for magnetic libraries. Note that there are appliances that support single instancing of data (e.g., Centera). Such hardware can be configured as a magnetic library with the option to write the data in a singleinstance format enabled. Single Instance Storage is supported by both backup and data archival products, specifically for the file system and email attachments, in order to provide optimization in storage when copies of the same data is backed up and stored.

CommVault System Administration Course – R02

156 – Clients Module

Using Subclient Policy Used as a scalability tool „ Defines/Manages set of Subclients for Agent Backupset „ Filters, security, compression and storage policy can be altered independently „

Using Subclient Policy Subclient Policies provide the facility to define various subclient properties such as subclient contents, filters, storage policy, etc. as a common template and attach the subclient policy to appropriate clients based on their file system. Mutual exclusion across different subclients defined in the subclient policy is enforced. A user specifies the file system type (Windows File System, Windows File System with System State, File System, Netware File System) for which a subclient policy is being defined. One of the subclients defined in the subclient policy is designated to be the “default subclient” (Used for Indexing reasons). Cloning of a subclient policy will be supported. Subclient Policies are created at the subclient Policy logical node in the CommCell Console. When a user chooses to associate a subclient policy, the default subclient is changed to point to the subclient that is marked as default within the subclient policy. Other subclients are created on that backup set and pointed to the corresponding subclient Id from the subclient policy. The name of the existing default subclient is also changed to the name of the default subclient within the subclient policy.

CommVault System Administration Course – R02

Clients Module - 157

Module Summary

Key points to remember

Summary •

Clients are systems that contain data to be protected and managed.

•

Clients must communicate with CommServe and MediaAgent to be protected.

•

Backupsets and Archivesets are used to logically group subclient content making data recovery easy.

•

You can define data for various user types independently of where the data is located, by using the Data Classification Enabler.

•

Some common property settings for subclients include defining content, establishing content filters, using pre/post processing commands, and selecting storage options.

•

Subclient policies allow you to define multiple properties in a common template and attach this policy to appropriate clients.

CommVault System Administration Course – R02

158 – Clients Module

CommVault System Administration Course – R02

Job Management Module - 159

Job Management

www.commvault.com/training

CommVault System Administration Course – R02

160 – Job Management Module

¡ Overview Configuring Data Protection Tasks „ Executing Data Protection Tasks „ Task Management Tools „

Job Management Module Objectives: • • • • •

List and describe the common task management tools used for scheduling, controlling and executing jobs. Describe the requirements and steps for performing a successful CommServe Disaster Recovery. Describe the steps for downloading and installing software updates for CommCell Components. Describe the common tools/options for assisting support personnel in identifying and resolving problems. Describe the various job options available when configuring and executing Backup and Archive jobs.

CommVault System Administration Course – R02

Job Management Module - 161

¡ Configuring Data Protection Tasks Types of Data Protection „ Understanding Synthetic Full „ Archive Concept - HSM „

Configuring Data Protection Tasks Data Protection is a comprehensive term used to account for both Backup and Archive jobs. All jobs are managed by the JobManager service run on the CommServe host. Jobs are configured and run in distinct phases where the status of each phase determines whether the next phase will be run.

CommVault System Administration Course – R02

162 – Job Management Module

Types of Data Protection „

Backup z z z z

„ „

Full Incremental Differential Synthetic Full

Archive Automated System Recovery (ASR)

Types of Data Protection Backup Full Full backups are required and are the preferred means of backup in terms of data protection and restore speed. A full backup captures all data as defined by the subclient content. Full backups are the preferred method of data protection given that sufficient resources are available. Incremental Incremental backups are the most efficient form of backup capturing only the data that has changed since the previous backup. While incremental backups save space and time, a full subclient restoration would require access to the most recent full backup plus all incremental backups up to the requested restore time. This may involve more media and time to perform a successful restore. Differential Differential backups combine fast restore with a more efficient use of storage than normal full backups. A differential backup captures all data that has changed since the previous full backup. Each differential backup is then cumulative of all data changes. A full subclient restoration would require the most recent full backup plus the most recent differential backup. This could result in faster restore jobs. Synthetic Full Synthetic Full is listed as a data protection (backup) option for some subclients, but it is not a genuine data protection job since it obtains no data from the source subclient.

CommVault System Administration Course – R02

Job Management Module - 163 When a normal backup operation is executed, an image file is generated. This image file contains a list of all objects within the scope of the subclient’s defined contents – excluding filtered files. These objects are collected from data already in protected storage and written back to protected storage as an equivalent full data protection job. Actual scheduled protection is not achieved until an incremental backup from the client is completed. The synthetic full task includes an option to run an incremental backup just before or after the synthetic full is run. A synthetic full may be used when a normal full backup cannot be reasonably performed due to client or network resource limitation. Archive Archiving has no associated concept of Full, differential, or Incremental collection of a subclient’s content. Defined content is evaluated against a set of rules and those objects which satisfy the rules are archived. Archiving has the two major functions of freeing up production disk space by moving less frequently accessed objects to less expensive storage and managing the accessibility of data in alternate storage over extended periods of time.

Automated System Recovery (ASR) Automated System Recovery (ASR) is a tool that speeds the recovery of Windows XP and Server 2003 client computers by automatically reinstalling the operating system and restoring the hardware configuration settings, system state, program files as well as the Documents and Settings folder. The hardware configuration of the client computer is restored directly from a user-created ASR floppy. The other backed up data is restored from tape media. This procedure does not completely restore the File System, other applications, or application data, which should be restored with the appropriate Agent(s).

CommVault System Administration Course – R02

164 – Job Management Module

Understanding Synthetic Full DA Synthetic Full

Latest Image

Incremental from client before or after Synthetic Full

Full

Inc

Inc

Inc

Tape 1

Inc Tape 2

Tape 3

Understanding Synthetic Full As the name suggests, a synthetic full backup is a synthesized backup. It is created from the most recent full backup (i.e., standard or synthetic) and subsequent incremental and/or differential backups. The resulting synthetic full backup is identical to what would have been created had the last backup for the subclient been a full backup. Unlike full, incremental, and differential backups, a synthetic full backup does not actually transfer data from a client computer to the backup media. In this sense it is not really a backup at all, rather a backup consolidator. Because synthetic full backups consolidate backup data into one archive file, they provide a means of enhancing restore performance, particularly when a given full backup cycle contains many incremental backups. An example of the usefulness of synthetic full backups is the case where a Client is remotely located from a Media Agent, with a WAN connection that is limited in bandwidth. In such a case, you may not want to perform full backups on a regular basis across the WAN, but you do want a archive file of your backup data to enhance restore performance. Synthetic full backups are treated by the system as standard full backups. As a result, they delineate full backup cycles in the same manner as standard backups. This is an important point to remember, since retention periods are defined in part by the number of full backup cycles. Using synthetic full backups in an unguided manner may cause the unintentional expiration of data. Synthetic full backups can either be started manually or scheduled to occur at regular intervals.

CommVault System Administration Course – R02

Job Management Module - 165

Archive Concept - HSM Agent

Tiered Storage „ Immediate reclamation of disk space „

Primary Storage

Secondary Storage

Tape Storage

Archive Concept – HSM Tiered storage refers to access time and/or cost of storing a data object. High speed, durable SCSI drives on production systems are expensive but necessary for a robust and reliable environment. Non-SCSI drives such as SATA or IDE drives are less expensive and less durable, but suitable for less frequently accessed data such as archived or recent backup data. Magnetic Optical and tape media provide removal capability that is essential for remote storage of data as protection against disaster or in environment controlled facilities to extend the life of the media. Tape is usually the economical choice for removable media. Hierarchical Storage Management is the timing and movement of data between the tiered level of storage. As data becomes older, maybe less valuable, and the access time requirement longer you should move the data to more efficient storage media. Archiving There are two key components to Archiving with CommVault. The primary purpose for Archiving is to move data from primary storage to secondary storage to free space and reduce the burden of backing up multiple instances of unused data from the primary host. The first archiving component is Compliance Archiving. This is currently available for Microsoft Exchange Server. Compliance archiving uses Exchange Journaling to store incoming and outgoing email and attachments which are then archived to secondary storage. Compliance Archiving when used with content indexing is a powerful tool for storing and retrieving email

CommVault System Administration Course – R02

166 – Job Management Module data for audit and compliance. Search can be completed by keyword, phrase, or by searching in the To, From, CC, and BCC fields. The second component is Migration Archiving. There are agents currently available for Microsoft Exchange Server Mailbox and Public Folders, Microsoft SharePoint, Novell Netware File System, Unix/Linux File System, Windows File System and File Shares. The purpose of Migration Archiving is to take unused files and remove them from the primary storage, freeing space and decreasing backup times and space. While Compliance Archiving for Exchange merely creates an archived copy of an email, Migration Archiver actually removes the e-mail or file from the server, and leaves a stub in its place. The stub contains the location of the file in storage, and instructions on how to retrieve it. Recalling migrated files becomes a very simple process. The file or e-mail can be recalled seamlessly by the user by simply opening the file or email. This triggers an immediate recall of the file from storage. Files may also be recalled by an authorized user via browse and recovery from the CommCell console. Even though the original file is being replaced by another file (Stub), space is made available because the stubs are only 4-6 kb in size. The available space is reclaimed immediately.

CommVault System Administration Course – R02

Job Management Module - 167

¡ Executing Data Protection Tasks Job Initiation Options „ Advanced Backup/Archive Options „ Creating a Schedule Policy „ Using Command Line „

Executing Data Protection Tasks Individual Data Protection jobs run at the subclient level. If you chose a level higher than a subclient (i.e., backup set, etc.), you are prompted to confirm that you want to back up all the subclients below that level/node.

CommVault System Administration Course – R02

168 – Job Management Module

Job Initiation Options On Demand (Run immediately) „ Schedule „ Schedule Policies „ Command Line „ Automatic (Aux Copy) „

Job Initiation Options On Demand On Demand Data Protection Operations allow content to be specified as an external input at the time of initiating a data protection operation. Whereas traditional backups/archive operations are performed on subclients which have fixed content configured prior to performing the operation, On Demand Data Protection Operations allow you the flexibility of specifying content each time you perform a backup or archive operation. Schedule Scheduled data protection operations provide a convenient means of securing data without user intervention. You can establish data protection schedules for each subclient using the CommCell Console. When scheduling data protection operations, you need to establish a schedule for each subclient. For example, a backup schedule always contains a full backup and may contain one or more other backup type operations. When combined for a given subclient, these backups comprise a full backup cycle. Use the CommCell Browser to schedule or initiate backups at the subclient, instance and/or backupset level depending upon the agent. Selecting the backupset or instance saves you from having to select the individual subclients. If you select a backupset or instance, the same schedule or data protection operation requested is applied to all subclients.

Schedule Policies

CommVault System Administration Course – R02

Job Management Module - 169 If you have a large number of clients/backup sets/subclients or storage policies in your CommCell that require the same backup or auxiliary copy schedule, you can manage their schedules through a Schedule policy. Schedule policies allow you to associate a schedule or groups of schedules to any number of clients/backupsets/subclients or storage policies within your CommCell. There are two types of schedule policies, Data Protection and Auxiliary Copy. During the CommServe install the “System Created” schedule policy covers all supported agent types. Additional schedules can be created to support individual agent or all agent types. A schedule policy allows you to define a maximum of six schedules, and associate them by agent type to any number of client computer groups, client computers, backupsets, or subclients. You also can define the type of schedule such as Data Protection operation or Auxiliary Copy operation, set up an alert, as well as configure Advanced Options as described in the Advanced Backup/Archive options section. In the case of an Auxiliary Copy operation, the Schedule Policy association is to the Storage Policy. Decoupling is used to remove a subclient from its association from the Schedule Policy, but does not delete the scheduled actions. Decoupled scheduled actions are treated as individual actions and can be edited or deleted as such. Command Line The Command Line is useful for incorporating Data Protection and Recovery operations into your own scripts and scheduling programs. Using the Command Line interface you can create an action using a single command with arguments or create scripts that include an answer file. An easier method may be to use the CommCell Browser to generate a script and answer file using the options you select. The commands are integrated with the base component installed with all agents and are therefore available on all computers which have any CommServe, Media Agent, or Agent software installed. In order for the commands to function, the Galaxy Commands Manager service should be up and running on the CommServe. Automatic Automatic auxiliary copy operations are performed every 30 minutes to copy data to the automatic copy. These operations will only occur when new data that needs to be copied is found on the primary copy (source). You can change the interval at which the automatic auxiliary copy operation is performed on the job initiation tab of the auxiliary copy dialog box or by editing the Automatic Copy Schedule.

CommVault System Administration Course – R02

170 – Job Management Module

Advanced Backup/Archive Options Data Options „ Startup Options „ Job Retry „ Media Options „

Using Backup/Archive Options Advanced Backup Options enables the user to address unique requirements regarding data protection operations. Depending on what data is being protected and what type of backup operation is being performed various options will be available for configuration. Data Options By default, Follow Mount Points option is enabled. This option allows the Windows operating systems to be backed up including any drives mounted within that subclient. If a drive is mounted as a folder it will be backed up when this option is enabled. This can sometimes result in data on the drive being backed up twice if the resource also belongs to another subclient. Unix local mount paths are automatically included in the Subclient. NFS and CIFS (UNC format) mount paths can be included in a backup/archive if explicitly added to the subclient content. Startup Options Priorities can be defined at various levels within the CommCell. The priority levels can be set based on the Client, Agent type or job, as well as on the fly through the Job Controller window. When creating a backup operation, use the Priority settings to control the job priority level for the specific operation that is being configured. How CommVault’s priority and pre-emption features function is discussed later in this module . The Startup in Suspended State option will place the job in the Job Controller in a suspended state. This is useful when starting the job is dependent upon other jobs or an external event whose timing cannot be pre-determined or detected electronically. When more than one job is in suspended state, the backup administrator can decide which operations should begin first.

CommVault System Administration Course – R02

Job Management Module - 171

Job Retry Options Use the Total Running Time option to configure the maximum time a job should run before it is killed by the system. This is to prevent hung or delayed jobs from impacting future jobs. If the job is in a running state at the max running time set, it will be allowed to continue to run to completion. If at anytime after the max running time set a job phase fails and the job state changes to pending, the job will be killed. To specify the total running time option to include jobs in a running state as well, select the option Kill Running Jobs When Total Running Time Expires. Using this feature to set a total running time will ensure that current jobs will not conflict with upcoming scheduled jobs. Configure the total running time to kill jobs before the next nights backups are scheduled to start. Media Options Depending on the type of operation being performed (Full/incremental/differential), different options will be available on the Media tab. When performing full backup operations, the option to Create New Index will be available. By default this option is selected. If cleared the previous full index will continue to be used. Over time, the Index files can grow significantly larger requiring more space to backup and slowing down browse operations. If manually creating a new index, consider scheduling a monthly full job that will create a new index, and for each Full task during the month. This will allow you to browse up to the last monthly full, but will keep the Index Cache from growing unmanageably large. When configuring a backup operation the Start New Media and Mark Media Full options can be used to determine which jobs will be placed on a specific media. With careful planning and scheduling a set of data protection operations can be scheduled to run and all of these jobs can be placed on a single media. Use the Start New Media for the first job and Mark Media Full for the last job creating keystone operations. These settings are also useful when performing Auxiliary Copy operations. In 6.1 an option was added to Allow other Schedule to use Media set. This option is enabled by default, and can be disabled only if both Start New Media and Mark Media Full options are selected. Thus, if Start New Media is also selected, all jobs within the same schedule policy event will be allowed to use the same active media started by the first job. If the option to Mark Media Full is enabled, the system will wait 10 minutes after job completion to actually mark the media full. Any job (schedule policy or not) requesting to write to the media within the 10 minute time frame will be allowed to do so. Jobs within the same schedule policy event will not have that limit enforced since the Allow other Schedule to use Media Set option is enabled. Clearing the Allow other Schedule to use Media set option and enabling Start New Media and Mark Media Full will isolate each job on its own media – as you should expect.

CommVault System Administration Course – R02

172 – Job Management Module

Creating a Schedule Policy Template for bulk scheduling jobs „ Scalability tool „ Granular control down to Subclient level „

Creating a Schedule Policy Scheduling individual data protection job patterns for two, three, or maybe a dozen clients is a reasonable task load for an administrator. Doing the same for 300 clients can be challenging without some assistance. The Schedule Policy is a template-based management tool that can be applied across any number of client systems within a CommCell. Schedule policies allow you to associate a Data Protection or Auxiliary Copy job schedule pattern to any number of clients, backupsets, or subclients. You can define which type of data protection operation you want to schedule as well as configure Advanced Backup/Migrate/Archive Options and assign object association in the CommCell Browser. There are two types of Data Protection schedule policies: •

The Agent Specific schedule policy allows you to define a maximum of six schedules, and associate them to any number of client computers, backupsets, and subclients for a particular agent. An Agent specific schedule policy allows use of all applicable data protection job options available for that agent.

•

An All Agent Types schedule policy is a generic data protection schedule policy that has most of the same options as a regular Data Protection policy. It is supported by all agents that support scheduling. Only generic data protection job options applicable to all agents types are available.

Up to 5 schedules can be included in a Schedule Policy so a reasonably complex protection schedule policy can be created.

CommVault System Administration Course – R02

Job Management Module - 173

Using Command Line Extensive control for most tasks available in CommCell Console „ Modes „

Synchronous (wait for completion status) z Asynchronous (verify job submission only) z

Using Command Line QCommands provide access to most CommCell functions through the command line, and can be integrated into your own scripts and scheduling programs. Note that scripts can also be generated through the CommCell Console for some features using the Save as Script option. All commands have consistent output in all situations to facilitate easier script writing. Although most CommCell operations are available through the command line, the recommended method for those not familiar with scripting or command line syntax is to use the CommCell Console as it provides comprehensive support for all options available in the software. Configuration No special configuration is required to use the command line interface. The commands are integrated with the base package, and are therefore available on all computers which have any CommServe, Media Agent, or Agent software installed. In order for the commands to function, the Galaxy Commands Manager service should be up and running on the CommServe. The Galaxy Commands Manager is a service that is installed with the CommServe, and is responsible for handling command line requests and forwarding them to the Event Management Service of the target CommServe. Log in Sessions Using the Qlogin command, you can start a User login session, removing the need to log in for every command. Once the Qlogin command is successful, the login session for all computers

CommVault System Administration Course – R02

174 – Job Management Module remain valid until you explicitly log out using the Qlogout command. This is true for all command sessions, not just the one for which the login took place. Saving a Job as a Script Data Protection, Recovery, Auxiliary Copy, Disaster Recovery Backup, and Data Aging operations and their selected options can be saved as scripted files, which can later be executed from the command line. This method of script generation will create 2 files, the first is the command file that configures the environment and executes the commands, and the second text based file maintains the responses also known as an argument file. These scripts can be saved in a Shell or Batch script format and include error checking. Script Considerations Consider the following before you save a job as a script: •

•

•

When you use the CommCell Console to save a script, only the options that are available for the corresponding QCommand are saved. For example, the Job Retry option is available in the CommCell Console for several operations such as backup and restore. However, this option is not available in the corresponding QCommand. Therefore, if you save a Script with Job Retry options selected, they will be ignored when the script is generated. When you use the CommCell Console to save a script, certain characters (for example, the left bracket [ and number sign #) in front of an object name (for example, #csmacs as the subclient name) may cause the script to not work correctly when it is run from the command line. If your script does not work from the command line, check it for these characters. You may need to rename the object by removing the characters, and then rerun your script. When you use the CommCell Console to save a backup script for a Microsoft Share Point 2003 Document Agent and run it from the command line, the script will only back up the latest version of a document. This will happen even when you choose the "Backup All Versions" option when generating the script.

CommVault System Administration Course – R02

Job Management Module - 175

Advanced Backup/Archive Options Data Options „ Startup Options „ Job Retry „ Media Options „

Using Backup/Archive Options Advanced Backup Options enables the user to address unique requirements regarding data protection operations. Depending on what data is being protected and what type of backup operation is being performed various options will be available for configuration. Data Options By default, Follow Mount Points option is enabled. This option allows the Windows operating systems to be backed up including any drives mounted within that subclient. If a drive is mounted as a folder it will be backed up when this option is enabled. This can sometimes result in data on the drive being backed up twice if the resource also belongs to another subclient. Unix local mount paths are automatically included in the Subclient. NFS and CIFS (UNC format) mount paths can be included in a backup/archive if explicitly added to the subclient content. Startup Options Priorities can be defined at various levels within the CommCell. The priority levels can be set based on the Client, Agent type or job, as well as on the fly through the Job Controller window. When creating a backup operation, use the Priority settings to control the job priority level for the specific operation that is being configured. How CommVault’s priority and pre-emption features function is discussed later in this module. The Startup in Suspended State option will place the job in the Job Controller in a suspended state. This is useful when starting the job is dependent upon other jobs or an external event

CommVault System Administration Course – R02

176 – Job Management Module whose timing cannot be pre-determined or detected electronically. When more than one job is in suspended state, the backup administrator can decide which operations should begin first. Job Retry Options Use the Total Running Time option to configure the maximum time a job should run before it is killed by the system. This is to prevent hung or delayed jobs from impacting future jobs. If the job is in a running state at the max running time set, it will be allowed to continue to run to completion. If at anytime after the max running time set a job phase fails and the job state changes to pending, the job will be killed. To specify the total running time option to include jobs in a running state as well, select the option Kill Running Jobs When Total Running Time Expires. Using this feature to set a total running time will ensure that current jobs will not conflict with upcoming scheduled jobs. Configure the total running time to kill jobs before the next nights backups are scheduled to start. Media Options Depending on the type of operation being performed (Full/incremental/differential), different options will be available on the Media tab. When performing full backup operations, the option to Create New Index will be available. By default this option is selected. If cleared the previous full index will continue to be used. Over time, the Index files can grow significantly larger requiring more space to backup and slowing down browse operations. If manually creating a new index, consider scheduling a monthly full job that will create a new index, and for each Full task during the month. This will allow you to browse up to the last monthly full, but will keep the Index Cache from growing unmanageably large. When configuring a backup operation the Start New Media and Mark Media Full options can be used to determine which jobs will be placed on a specific media. With careful planning and scheduling a set of data protection operations can be scheduled to run and all of these jobs can be placed on a single media. Use the Start New Media for the first job and Mark Media Full for the last job creating keystone operations. These settings are also useful when performing Auxiliary Copy operations. In 6.1 an option was added to Allow other Schedule to use Media set. This option is enabled by default, and can be disabled only if both Start New Media and Mark Media Full options are selected. Thus, if Start New Media is also selected, all jobs within the same schedule policy event will be allowed to use the same active media started by the first job. If the option to Mark Media Full is enabled, the system will wait 10 minutes after job completion to actually mark the media full. Any job (schedule policy or not) requesting to write to the media within the 10 minute time frame will be allowed to do so. Jobs within the same schedule policy event will not have that limit enforced since the Allow other Schedule to use Media Set option is enabled. Clearing the Allow other Schedule to use Media set option and enabling Start New Media and Mark Media Full will isolate each job on its own media – as you should expect.

CommVault System Administration Course – R02

Job Management Module - 177

¡ Task Management Tools Activity Control „ Using Operation Window „ Job Management Control „ Using the Job Controller „

CommVault System Administration Course – R02

178 – Job Management Module

Activity Control CommCell „ Client Level „ Agent „ Subclient „

Activity Controls Use the Activity Control tool to enable or disable all activity, data protection, data recovery, auxiliary copy, or data aging operations within the CommCell (regardless of the client computer from which they originate). If you want to control these operations that originate from a specific client computer or agent, use the Activity Control tab of the property sheet of the specific client computer or agent. The Activity Control feature allows you to enable or disable operations including scheduled operations at the following levels in the CommCell hierarchy: •

CommCell - Allows you to enable/disable all activity, data protection, data recovery, auxiliary copy, data aging and/or content indexing operations for all client computers within the CommCell.

•

Client Computer Group - Allows you to enable/disable all data protection, data recovery and/or online content indexing operations on all client computers that are members of a client computer group.

•

Client Level - Allows you to enable/disable all data protection, data recovery and/or online content indexing operations on a specific client computer.

CommVault System Administration Course – R02

Job Management Module - 179 •

Agent - Allows you to enable/disable the data protection and/or data recovery operations of a specific agent on a specific client computer. Online Content Indexing jobs can be disabled for Online Content Indexing Agents.

•

Subclient - Allows you to enable/disable the data protection of a specific subclient. Online Content Indexing jobs can be disabled for Online Content Indexing subclients.

When disabling operations, the CommCell level has the highest precedence while a subclient has the lowest precedence. For example, if you disable data protection operations at the CommCell level, then all data protection operations throughout the CommCell are disabled regardless of the corresponding settings of the individual client computer groups, client computers, agents, and subclients. If, however, a data protection operation is enabled at the CommCell level, you can still disable data protection operations at the client computer groups, client computer, agent, subclient levels. By default, all operations are enabled at all levels of the CommCell hierarchy.

CommVault System Administration Course – R02

180 – Job Management Module

Using Operation Window Set Globally or at Agent „ Disable jobs during certain time periods „ Place jobs on queue „ Resume jobs after time period „

Operation Windows By default, all operations can run for 24 hours. To prevent certain operation types such as Data Protection or Auxiliary Copy from running during certain time periods within the day, you can define operation rules using Operation Windows so that these operations are suspended during those times. The main purpose of this feature is to help you prevent an unexpected, time consuming operation from disrupting normal operations. Operation Window Rules can be looked at as “Blackout Windows” Operation Window rules are defined at both the CommServe and agent levels. Rules established at the CommServe level apply globally across the entire CommCell. Operation rules established at the agent level apply only to the specified agent. When an operation rule is defined at both the CommServe and agent levels, the job will run outside of the total time frame of both levels. Note that at the agent level: • • •

Not all operations are available to be assigned an operation rule, such as administration and synthetic full operations. You can also elect to ignore the operation rules set at the CommServe level from the Operation Window dialog box. The client time zone is displayed in the Operation Window Rules Details dialog box.

CommVault System Administration Course – R02

Job Management Module - 181 When using Operation Window, consider the following, there are two types of Agents – Indexed base and Non-Indexed base. Simply stated, Index Based agents use the index cache and the Non Index Based Agents will not. An easy example of each might be index based types might include Agents managing granular objects within a job like a file system backup or mail system messages. Non Indexed Agents usually are database applications. A listing of these type can be found within Books Online. Jobs that are started at any time within the Operation Window rules or “Blackout” period will still be added to the Job Manager window in a queued state. Once the “do not run interval” has passed, these queued jobs will resume automatically. Jobs that are started before an operation rule can run to completion if the “Allow running jobs to complete past the operation window” option is enabled from the General tab of the Job Management dialog box. Jobs that are not interruptible (such as Non-Indexed based jobs) will not be terminated if they fall within the time an operation rule is defined.

CommVault System Administration Course – R02

182 – Job Management Module

Job Management Control Queue Options „ Pre-emption Control „ Priorities „ Restarts „ Job Status Updates „

Job Management Control Job Management options are available in the CommCell Console Control Panel. It sets the default actions to take when a job starts and how to automatically manage the job when certain events happen. The areas of job management are: •

Queue Options - If a job conflicts with another job or control setting, what action should the job take? In some cases you may want the job to queue and wait. In other cases you may not want the job to start.

•

Pre-emption Control - If limited resources are being competed for, what action should the Job Controller take when a higher priority task is started? Should lower priority jobs be pre-empted or not? Select or clear the options as necessary. Note that in a Gridstor enabled storage policy, pre-emption is suspended.

•

Priorities - Set whether a client or an application has priority. You can also set the respective application priority. Job Priorities are based on a three-digit integer. The first digit always represents operation priority. If client precedence is chosen, the second digit represents client priority and the third digit represents agent priority. If agent precedence is chosen, the second digit represents agent priority and the third digit represents client priority. The lowest job priority number has the higher priority. Hence, a

CommVault System Administration Course – R02

Job Management Module - 183 job with a priority of 066 will run before a job with a priority of 366 – if a resource conflict is an issue. Job Operation(s) Data Aging, Export Media, Inventory Data Recovery, Disaster Recovery Backups Data Protection Auxiliary Copy Data Verification Erase Media

Assigned Priority 0 (Not applicable) 66 (leading zeros do not appear) 166 266 366 466

Restarts When a job phase fails it will normally go into a pending state awaiting an automatic restart. The frequency interval and number of restarts can be configured. The default is every 20 minutes and 144 times (48 hours). You can also set the option on some job types as to whether they are restart-able or preempt-able. Job Status Updates Job status information is periodically updated in the Job Controller window. The period of the update is based on time, or the amount of data moved – whichever is less. You can reduce the period if you want more frequent status updates. However, this can have a negative performance effect on the job. Managing chunk size, found in the CommCell Console Control Panel under Media Management can also affect job performance.

CommVault System Administration Course – R02

184 – Job Management Module

Using the Job Controller „

Manage Data Protection operations z Data Recovery operations z Administration operations z

Job Controller Window The Job Controller allows you to manage the following types of jobs: • Data protection operations • Data recovery operations • Administration operations You can select a job in the Job Controller and perform a control action on that job individually. If you have multiple jobs in the Job Controller that need actions performed on them, you can simultaneously control groups of these jobs from the Multi-Job Control option displayed when you right-click on a job. You can perform the following actions on jobs: • •

•

Suspend - Temporarily pauses a job. A suspended job is not terminated; it can be restarted at a later time. Only pre-emptable jobs can be suspended. Resume - Resumes a job and returns the status to Waiting, Pending, or Running depending on the available resources or the state of the operation windows or activity control settings. Kill - Terminates the job.

CommVault System Administration Course – R02

Job Management Module - 185 The state of the job reflected in the job controller window are self explanatory (e.g. running, waiting, suspended, completed). Listed below are the less frequently encountered but equally important: • • • •

Dangling Cleanup -The job has been terminated by the job manager, and the job manager is waiting for the completion of associated processors before killing the job. Pending - The Job Manager has suspended the job and will restart it without user intervention. A pending job may be waiting. Queued - The Job Manager has queued the job and will restart the job only if the condition which caused the job to queue has cleared. Waiting - The job is active, waiting for resources (e.g. media or drive) to become available or for internal processes to start. The job many also be started in the waiting state if a queue option was selected from the General tab of the Job Management dialog box.

CommVault System Administration Course – R02

186 – Job Management Module

Module Summary

Key points to remember

Summary •

There are many different options for Data Protection depending on the needs of the organization.

•

Archiving uses hierarchical storage management concepts to move unused data between different tiers of storage.

•

There are many options for initiating jobs within Galaxy.

•

Global options for job queuing, job pre-emption and job priorities can be configured.

•

The Job Controller window of the CommCell Console offers the ability to dynamically suspend, resume, or kill jobs using the Multi-Job feature.

•

The SQL meta data for the CommServe is protected by at least two backup locations with one operation.

•

Data Aging will purge data that has exceeded retention. Data is aged differently depending on what type of media it was written to.

CommVault System Administration Course – R02

Restore Module - 187

Restore

www.commvault.com/training

CommVault System Administration Course – R02

188 – Restore Module

¡ Overview „ „

Finding Data to restore Understanding Restore and Recovery

Restore Module Objectives: • • • •

List and describe the common methods for locating data in protected storage for restore/recall. List and describe the common methods and options for restoring and recalling data. Describe the requirements and steps for performing a full File System/Application Agent restore. Describe how to use and manage stub files for recall of data.

CommVault System Administration Course – R02

Restore Module - 189

¡ Finding data to restore Image/No-Image Browse „ List Media & List Media (Precise) „ Exact Index „ Find/Search „

CommVault System Administration Course – R02

190 – Restore Module

Image/No-Image Browse D:\DIR 1/1/06 1/2/06 1/3/06 1/4/06 No Image A.txt B.txt C.txt D.txt E.txt F.txt G.txt H.txt

FileName

1

2

3

4

A.txt

9 9 9 9

B.txt

9 9

C.txt

9 9 9 9

D.txt

9

E.txt

9 9 9

F.txt

9

G.txt

9 9

H.txt

9

Image/No-Image Browse Image Browse Each indexed backup generates an “image” index of all files/objects scanned at the time of the backup. The image will always contain all files/objects scanned at the time of backup regardless of the backup type (e.g. full, differential, incremental). This image index is used when you: • • •

Browse backup data for a specified time Select to restore the contents of a directory or drive at a specified time Do a full system rebuild of the file system at a specified time

A backup cycle that spans a period of time will have image indexes for each backup job contained with the index. For example; a subclient doing weekly full backups with daily incremental backups will have seven (7) image indexes for the list of files/objects it backed up during the cycle – one for each backup job/day. Selecting a point-in-time browse of the backup data will use the appropriate image index to show you what files/objects existed on the system/application at that time. No-Image Browse What if you’re looking for a deleted file that’s not in the latest backup image? How do you find it amongst the backup data? The basic answer is that you can search for the file by doing pointin-time browses of the backup data. If you are not certain of when it was last backed up, this can be a hit-or-miss exercise. Additionally, if you’re guessing when it was backed up, it might not be the latest version of the file. To ensure you got the latest version would require methodically

CommVault System Administration Course – R02

Restore Module - 191 stepping backwards through each backup job. As you might suspect, this can take time. This is where the Image Browsing option becomes valuable. Disabling the Image Browsing option will create a browse display of all files within the index from that point back to the beginning of the index. Deleted files not in the current backup image but were captured in previous backups, would be displayed in the browse. Essentially, you are browsing all files that were backed up since the index/cycle started to the selected point-in-time. If your deleted file was backed up in this cycle, it should appear in the backup browse display. If not, you can use the Find feature to scan through multiple indexes/range of time to locate the deleted file or all versions of a file. A caution with disabling the Image Browsing option is that if you are restoring directories or drives, more files will be restored and the resultant volume may be more than the capacity of the device.

CommVault System Administration Course – R02

192 – Restore Module

List Media & List Media (Precise) „

List Media z z

„

Media Containing Index Media Containing Jobs within specified time range

List Media (Precise) z

z

Lists media for specified objects selected for restore Lists media for specified subclient

List Media & List Media (Precise) List Media List media option is useful to predict media required for the following operations: • • • •

To restore data associated with a specific backup set, subclient or instance To restore the index required to browse data associated with a specific backup set or subclient To restore a specific file (or specific files and folders) To restore data associated with a specific job

Media prediction is useful in a variety of circumstances, including the following: • • •

To ensure that media required by an operation is available in the library, especially if you are restoring/recovering data across a firewall. In cases where data spans across several media, to identify the exact media necessary to restore/recover a file/folder/sub-section of the data. To identify and restore/recover from a copy that accesses a faster magnetic disk media rather than slower tape/optical media.

To identify media associated with an alternate copy, when the media containing data associated with a specific copy is not readily available due to the following reasons: •

When the media is exported from the library

CommVault System Administration Course – R02

Restore Module - 193 •

When the media is used by another operation

List Media Associated with a Specific Backup Set, Instance or Subclient is referred to as List Media in the CommCell Console and provides the following options: • • •

Search media associated with the latest data protection cycle, starting from the latest full backup. (This is the default option.) Search media associated with data protection operations performed between a specified time range. Search for media associated with a specific storage policy copy, synchronous or selective copies, with the specified copy precedence number.

Keep in mind that when you search media from a secondary copy, the listed media may not reflect the entire instance or backup set data, unless all the storage policies associated with all the subclients have been configured for secondary copies. List Media (Precise) This operation in the CommCell Console is useful to predict precisely which media contains specific files or folders. For example: • When a data protection operation spans across multiple media and you would like to know the exact media in which the files you wish to restore reside. • You have a specific set of files (either a random set or a specific set, such as *.doc or *.txt) that you wish to restore and would like to know the all the media in which the files reside. • You wish to restore a specific version of the file and would like to know the specific media in which the version resides.

CommVault System Administration Course – R02

194 – Restore Module

Exact Index Current Index

Past Index

Object

M

A.txt

X

B.txt

X

C.txt

X

D.txt

X

E.txt

X

F.txt

X

T

W

Th

X X

X

X

X

X

S

Object

M

X

X

A.txt

X

X

X

B.txt

X

C.txt

X

D.txt

X

E.txt

X

F.txt

X

F

X

X

X

Exact Index

S

X X

X

X

X

X

X

X

X

Restore

(Cache & Latest)

T

W X

X

X

X

X

X

X

(Copy on Media)

Exact Index When you browse indexed backup data, the default index set restored/used will be the last index set generated in the backup cycle. This is true even if you are doing a point-in-time browse. This means, that no matter when within a backup cycle you want to browse, the last index set generated in that backup cycle will be used. There could be circumstances where the media containing the last index set in the cycle is not available (e.g. disaster recovery, remote restore) or the index set is lost/corrupt. In this case you can select the Use Exact Index option available in the Advanced Browse options window. The Use Exact Index option is useful in situations when you want to restore data from specific media. For example; when media is off-site and you would like to maximize the chances that the necessary index information comes from the same media.

CommVault System Administration Course – R02

Restore Module - 195

Find/Search „

Find z z

„

Available at backup/archive set level Spans indexes

Search z z z z

Available at all levels Uses content indexing End user or Compliance CommCell or Web-based

Find The Find feature allows you to search the data protection archives for any file or directory name or name pattern. Because Find supports the capability of searching multiple indexes, unlike browse, you can search beyond the last full backup (or new index) as long as the data resides on an index that exists within the retention period. The Find feature is accessible as a right-click option on the browse window, and for supported agents a non-browse find is available from the All Tasks menu. Depending on the agent, the Find option is accessible from either the backup set or archive set level. The Advanced Search tab of the Find feature is accessible only to clients that support Content Indexing and have created a content index for their archived or backed up data. Advanced Search allows you to search by content (keyword or phrase). For Exchange agents, you can also search by To, From, CC, BCC and attachment name. Advanced searches with multiple criteria can be performed. In addition, Exchange agents that support Content Indexing can also narrow search results by specifying a received time range for messages on the Find (Receive Time Range) tab. When using Find, be sure to specify a time range to prevent an unbounded search through all past data. End User Search The main concept behind End-User Searches is the ability for users to search all data objects that were created by them, or that is accessible to them. Because End-User Searches require a

CommVault System Administration Course – R02

196 – Restore Module Windows Security ID (SID) to uniquely identify whether a user created a data object, or has the access rights to the data object, End-User Searches are not currently available for Unix-based agents and Unix-created data objects residing on NFS shares managed by Data Archiver. However, Data Archiver data residing on CIFS shares do support End-User Searches. Note that the search function from the CommCell Console does not provide end user search capability. You must use the Web-based Search Console or Data Archiver’s Outlook Add-in for end user search criteria. Compliance Search The Compliance Search capability allows compliance officers full access to all computers and supported applications for searching, regardless of ownership/access attributes for the piece of data. CommCell Console Search If Content Indexing is installed and available, the Search function is available at all levels of the CommCell from the backup/archive set and up. Web-based Search Console The licensed Web Search Server has features that enables both compliance and end-users to search and restore data based on individual security permissions. The web-based End User and Compliance Search Consoles have an easy-to-use search interface modeled after popular search engines. The web-based Search Console requires that an Active Directory External User Group be associated and configured before is used.

CommVault System Administration Course – R02

Restore Module - 197

¡ Understanding Restore and Recovery „ „ „

Restore/Recall Options Working with stub files Full System/Application Restore

Understanding Restore and Recovery Restore is the act of moving data from protected storage back into a production environment where it can be accessed through normal production tools. Recovery is a subsequent phase applied to application data where the restored data is recognized and functionally used by the application. For example; a set of database files can be restored to production disk space but they are not usable until the application recognizes, validates, and makes them accessible through the application. This is sometimes called a Soft Recovery.

CommVault System Administration Course – R02

198 – Restore Module

Restore/Recall Options „ „ „ „ „ „ „

Restore by Job Mapped Restores Recall Archived Objects Copy Precedence Restore from anywhere Out-of-Place Restores Restore to Non-Client hosts

Restore/Recall Options The simplest form of restore is to copy an object back to its original location. However, not all restores are simple. The process is intricate, by requirements to filter, redirect, alternate source, alternate path, parallel stream and otherwise manipulate the data being restored. Flexibility in restore, recall, or recovery of data is an essential feature in any data protection product.

CommVault System Administration Course – R02

Restore Module - 199

Restore by Job „ „ „ „

Accessible from BackupSet level Multiple jobs can be selected Newer files overwrite older files For CommServe DR use “DR Restore” option

Restore by Job The Restore by Job feature provides the facility to select a specific backup job or set of jobs to be restored. This method of restoring data is faster under some circumstances as it reads continuously from the tape and retrieves the data independently rather than using the indexing subsystem which does individual seek offsets on the media. A key advantage of restore by jobs is restoration of any successfully written data in failed or killed jobs. If multiple job streams were used in parallel for the data protection job, then Restore by Job can use those same multiple streams in parallel for the restore. The degree of parallelism would be a factor of the degree of multiplexing for the jobs involved. The higher the multiplexing, the more “parallel” the restore. Optimally, you’d like each job stream assigned to a device stream on a 1-to-1 basis. Multiple stream parallel restores in these cases would be as expected. On the other hand, if the job streams are multiplexed to the same tape, then a single device/job stream to the client will be used. The “parallel” restore occurs at the client, where multiple threads will be generated to de-mux the single stream and restore the data accordingly. A standard browse/index-based restore for more granular restore may be more appropriate at times and can be faster by not restoring unnecessary objects. However, an index-based restore is

CommVault System Administration Course – R02

200 – Restore Module single stream only, regardless of how many streams may have been used in parallel by the data protection job. Restore by Job cannot be used to restore the Windows System State data.

CommVault System Administration Course – R02

Restore Module - 201

Mapped Restores „

„

„

Restore in-place and out-of-place with single job Supported on Windows, UNIX, and Macintosh File System Agents Available in Browse and Restore options and Restore by Job

Mapped Restores When performing normal restore of the selected data you’re given the option to designate one specific destination. If the restored data needs to go to multiple locations (for example you’re using a restore to restructure your primary storage) you would need to perform multiple restore jobs. Using a map file provides the ability to restore individual files and folders within a single job to different locations. The map files contains a list of files to be restored with their corresponding restore paths. In addition the mapping options also provide the ability to specify whether the unmapped files that were selected for restore should be restored to the selected restore destination. The map file restore options are also supported by Restore by Job feature and restores from the command line. Map file restores are supported by the Macintosh, Unix and Windows Agents. The map file should be a CSV (Comma Separated Value) file. On Windows, the map file can be of any recognizable type, such as .txt, .doc, .xls, .rtf, etc. On Unix, use an editor such as VI to create the map file. The format of the mapping should be as follows: "","“ Points to Remember While Creating the Mapping File Consider the following while creating a map file:

CommVault System Administration Course – R02

202 – Restore Module •

•

• • •

•

•

The specified paths for both the source and destination should be absolute paths within the same client computer. If the specified destination folder is not available, the necessary destination folder will be created during the restore. The source and destination type should be the same. For example, a folder must map to another folder and a file to another file. (The results may be unpredictable if this is not followed.) Computer names or UNC paths should not be used in the source and destination names. Wildcards in source or destination path is not supported. To filter a file, add the source folder and file and then add an empty double quotes as the destination. For example: "C:\dir5","" Use a mapping file only when there is a large number of files to be restored with multiple destination paths. Consider the following examples: You wish to restore 10 files. Within these 10 files, 8 files must be restored to the same location, while 2 files must be restored to separate destinations - in this case select these 10 file in the Browse window, specify the destination path for the 8 files to the common location and only include the 2 files in the map file. As much as possible filter the files from the Browse window, rather than filtering the files using a map file.

CommVault System Administration Course – R02

Restore Module - 203

Recall Archived Objects Temporarily or permanently bring back objects from protected storage „ Persistent Pipeline established to minimize resource impact „

Recall Archived Objects The recall or non-browse recovery feature provides the facility to transparently and either temporarily or permanently bring an object back from an export location for a specific operation. Transparency is provided by a stub object substituting for the actual object which initiates the recall automatically when accessed through normal means. The temporary capability is useful if you need an object for a specific purpose or length of time and do not wish to reset the object’s archive criteria. A permanent recall is useful if the object being recalled will be used multiple times and you wish to reset the object’s archive criteria. When performing a non-browse recovery of archived data from magnetic media or tape, the system maintains a common open pipeline known as a persistent pipeline between the target client and the protected media source for multiple stub recall requests. The pipeline stays persistent for 20 minutes (by default) to reduce the time to open a pipeline for each individual stub recall request. The initial object being restored may take a few seconds more time as the pipeline is established. Subsequent restores are very quick. When performing a non-browse recovery of archived data from a stub, keep in mind that the data can only be recovered in-place to the same path/destination on the same client from which the data was archived. If the message or item you want to recover was archived using the Do not create stub option or the stub was manually deleted after archiving, the object must be restored using the browse recovery method.

CommVault System Administration Course – R02

204 – Restore Module

Stubbed, non-browsed restores can only be recalled in-place to the item’s previous location. However, for the Exchange Mailbox Archiver Agent, the restore destination can be the same mailbox, a different mailbox, or a PST file on another Exchange Server within the same organization and site with a compatible Exchange Mailbox Archiver Agent installed and operational. Important Notes: Prior to performing a PST migration archiving operation or a PST recovery operation with the Exchange 2003/2007 Mailbox Archiver Agent, ensure that the MAPI32.dll file has been copied into the \Base installation directory. After completing a PST archiving or recovery operation, ensure that the MAPI32.dll file is removed from the \Base installation directory on the Client, otherwise non-PST restore/recovery/retrieve jobs for Exchange may go into a pending state.

CommVault System Administration Course – R02

Restore Module - 205

Copy Precedence iDA

The Each If a copy option Storage precedence to specify Policy Copy ais has a restore specified, specific copy onlyprecedence precedence that copy number. will for restore be used If no may forspecific restore be used to precedence whether bypass missing or not is used, or more suspect Galaxy will use each relative backup data datacopy or exists to in force on use precedence other of an alternate copiesorder within data to the path find the restore. Storage for dataPolicy to restore.

1 Client

MA

Magnetic Library

2 Tape Library

Remote Site iDA

3 Tape Library

Client

MA

Copy Precedence Storage Policy Copy precedence is a priority sequence used by a restore job to determine which storage policy copy to use when multiple copies are involved. Copy precedence is initially assigned in the order in which a storage policy copy is created (e.g. 1, 2, 3 …). As such, the primary copy is by default (and normally) the first precedence (#1) copy. However, the primary status of a copy and its copy precedence are not related. While data from the client is always written to the primary copy, there is no requirement that the primary copy also be the first copy used for restore. The precedence sequence of a copy can be changed in the Copy Precedence tab of the Storage Policy Properties dialog window. For a restore request, where the copy precedence is not specified (default), the software assumes any copy will do and will check each copy precedence in sequence until it finds one available to satisfy the restore request. When you manually specify copy precedence in the restore request you turn off the automatic copy selection and force the software to use the copy you selected – regardless of whether it has the most recent data or no relevant data at all. Additionally, when manually specifying copy precedence, you should select the copy precedence before browsing the backup data set. Browsing one copy and restoring from another may yield unexpected results. For example; a restored directory size and content might not be the same as you viewed in the browse.

CommVault System Administration Course – R02

206 – Restore Module Manually specifying copy precedence for restore is useful if: • • • • •

The primary copy is no longer available for a data recovery operation due to a hardware failure. You know that the media containing the data from data protection operations for a particular copy have been removed from the storage library. In this case, you can choose to browse/restore/recover from a copy whose media are inside the library. You want to browse/restore/recover from a copy that accesses faster magnetic disk media rather than slower tape media. You know that the media drives used by a particular copy are busy with another operation and want to browse/restore/recover from a different copy to avoid resource conflicts. You want to perform backups to a primary copy in one location and do restores from a secondary copy in another location. (This can be pre-set for scheduled restores or command line restore (Oracle RMAN) by altering the precedence of the secondary copy on the Storage Policy Properties dialog window.

CommVault System Administration Course – R02

Restore Module - 207

Restore From Anywhere ?

iDA

Export

Client Tape Library MA

?! Import Tape Library

A restore Data The media fromrequest a can client then that isbe written to exported requires data from managed from the the librarywill media library. andcheck imported Bylibrary into “managed” another inventory managed towe see mean first library an inventory within who has the the is same maintained media, then by the library. CommCell. restore the data.

Restore From Anywhere Data can be restored/recovered from any compatible library and drive type in the CommCell. By default the software automatically identifies and restores/recovers data from any configured library in the CommCell, even if the media is not available in the original library in which the data protection operation was performed. Data Recovery operations use a specific data path – Media Agent, Library and Drive Pool - to perform the restore operations. When the default options are selected, the system automatically chooses the most appropriate data path. Media associated with stand-alone libraries can be imported into the regular libraries (and vice versa) to perform Data Recovery operations. When moving media from a standalone to a barcode-enabled library, make sure it has a compatible barcode label and change the Media’s Identifier in the CommCell Console to match the barcode number before importing it. This feature is mostly used in a disaster recovery scenario where a new library is configured and media from previous library(ies) are imported to perform restores. There is no requirement to change a storage policy’s data paths to include the new library unless you intend to continue data protection jobs with the new library.

CommVault System Administration Course – R02

208 – Restore Module

Out-of-Place Restores „ „

Same Client/Different path or name Cross-platform z

Compatible File System ¾ ¾

„

Cross-application z z

„

Windows < - > UNIX/Linux - Yes Netware any other FS - No

Exchange Mailboxes on any version All Databases – Exact version only*

System State z z

Use Browse Options Restore to different client is supported

Out-of-Place Restores By default, data is restored to the same computer from which it was backed up. You can restore to a different parent directory path by clearing the Restore to same paths option. All files restored will have the new path pre-pended to their current path. Hence a file with a fully qualified path name of D:\user\data\document\file.txt restored to a path of F:\Restored would result in a restored file of F:\Restored\user\data\document\file.txt. Multiple files with different path names would have the same new path pre-pended. The Preserve source path option can be used to keep or remove levels of the restored path and file name. This option would best be used in re-organizing directory structures in conjunction with the restore. Files can also be renamed by appending a suffix to the filename itself – not the extension. Maintaining the extension keeps the application association consistent. To change the destination computer, you can select one from a list of established clients within the CommCell, with Operating Systems/Applications that support the out-of-place, crossplatform, or cross-application restore operation from this client. If the client name does not appear, the cross-platform OS or application does not support the out-of-place restore.

CommVault System Administration Course – R02

Restore Module - 209

Restores to Non-client hosts UNC Path Non-client

DA

DA Impersonate User

Restore Backup

MA

Restores to Non-client hosts By default, a restore job can only read/write data from/to the designated client. This is due to the characteristics of the user which the software services run. Data residing on non-client systems can be backed up by a client via a UNC path if an authorized user and password is provided. Conversely, data can be restored via a client system to any non-client systems via UNC path if an authorized user and password is provided. When specifying UNC path content in a subclient, a dialog window will appear enabling you to enter a user name and password. The same user name and password must be valid for all UNC paths specified in the subclient content. For restore, the Advanced Restore options to specify a different restore destination and to impersonate a user must be used. The specified user must have write permission to the share path and directory on the remote client .

CommVault System Administration Course – R02

210 – Restore Module

Working with stub files Stubs point to Archive/Backup file Report.doc

Report.doc

Financial.xls Summary.doc

Financial.xls Summary.doc

Working with stub files When data is archived from a server to another location, a pointer - or what we call a stub - is left in place of the file. This stub enables transparent recalls of the data. Although the data has been moved, it is still available to the user or application as if it was still on the server. After a stub has been recalled, the icon will change back to the normal icon for the respective file type. The only visible difference between a stub and the file is the icon representing the file. The icon is changed to denote the archived status of the data. The icon doesn’t change much, the original application icon will still appear but with a small clock in the left bottom corner. This represents a potential delay in accessing the file while it’s being recalled from remote storage. Windows and Netware File Systems – Will present the data in the fashion depicted above. The file will still appear in the original location, but the icon will change slightly as shown in the illustration. Certain files will not show a stub/icon at all. These files include *.html and *.ttf files. The reason for this is a limitation of the operating system having no icon – archived or not - to apply to these files. UNIX File Systems – There is no special icon denoting a migrated file within UNIX. Currently, there is no visual way to tell if it is a stub. Network Storage - Depending on the software version, the stubs may or may not show a unique icon. For Network Appliance’s ONTAP version 6.5, a migration icon is not supported. For version 7.0 and later, the icon will change and have the same look as Windows migrated files. The user or application accesses the file in the normal way. There is nothing special required to recall an archived file. However, a user should be made aware that the file may take some extra

CommVault System Administration Course – R02

Restore Module - 211 time to open because the system has to recall it from the remote location. Immediately reaccessing the file will not generate multiple recall events – nor will it make the file recall faster. The user may see a pop-up notification informing them that the system is retrieving the file. This pop-up is, by default, turned off in Windows and on in Network Appliance. The administrator has the option to change this setting in the registry key. Stub Retention By default, stub retention length is decided by the retention of the archived messages/items associated with the stub. After the messages/items on the media (all the copies) have expired, the stub will be pruned in the next migration archiving operation. Users can also specify a value in days for stub retention time, from the subclient properties Archiving Rules tab. Keep in mind that the stub could be pruned before this value, if the storage policy retention time is smaller than this value. Stub Rules The following stub rules allow you to manage the creation and pruning of stubs: • • •

Preserve stubs until the media has been pruned Prune stub only after n days Archive files only, do not create a stub

CommVault System Administration Course – R02

212 – Restore Module

Full System/Application Restore Recovers data files, directories and system state „ Performed via Browse at backup set level „ Restores System files and/or Data Files „

Full System Restore The difference between a normal restore and a full system restore is the severity of the problem. Normally, if data is lost or removed, it is recovered from the archives using the normal restore procedures. However, when a normal restore operation cannot correct a software and/or hardware corruption problem, some level of full system restore is required. If the root directory of a UNIX system, or the System State of a Windows system is lost, a Full system restore is required. When you perform a full system restore, the client computer must have a functional and compatible version of the Operating system. With UNIX, the default install partition should not be on the same disk partition that will contain the restored root file system. Full Agent restore always restores file system data in place. If the original path is not present, the file restore will fail. All Partitions/disks included in the restore must be configured, accessible, and of sufficient size to receive the restored data. If you are recreating and repartitioning the system disks and do not know how large each volume was, you can browse the backup data and view the size of each volume on the backup data browse screen. If you are restoring to different hardware, other issues need to be addressed in order to successfully and fully recover a system.

CommVault System Administration Course – R02

Restore Module - 213 Application Restores An application is made up of its managed data set and supporting executable and configuration files. The application must be functional in order to restore its managed data. In some cases this may require using a File System restore to get necessary supporting files in place. Additionally, the application must be in a recovery state in order for a restore and full recovery to work successfully. For example; With Exchange the System Attendant Service must be running in order to restore databases. The System Attendant Service will run without the presence of the databases. However, if the databases do exist they must be enabled for overwrite by a restore. This can be done using Exchange admin tools, or it can be done automatically by selecting the appropriate option during an Exchange database restore. On the other SQL Server service will not run without the master, model, msdb and temp databases. In order to do a full application recovery of SQL Server you must build suitable replacement databases in order to start the SQL Service - and then restore your original databases from protected storage. Microsoft provides utilities to rebuild the necessary databases. The key thing to remember is that restoring an application to full functionality requires cooperation between many parts. Understanding and practicing application recovery is essential to successful recovery.

CommVault System Administration Course – R02

214 – Restore Module

Module Summary

Key points to remember

Summary •

Finding the data to restore is simple, understanding the index cache is helpful.

•

Finding the media is simple and that information can be obtained at various levels within the GUI.

•

CommVault also offers search capabilities for data, but should be used with date restrictions.

•

Data can be restored to many different locations from any library.

•

Full system restores are available for extreme situations where the entire system needs to be recovered.

CommVault System Administration Course – R02

Monitoring Module - 215

Monitoring

www.commvault.com/training

CommVault System Administration Course – R02

216 – Monitoring Module

¡ Overview Monitoring Options „ Alerts „ Reports „

Monitoring Module Objectives: Module Objectives: • • • •

List and describe the common options available to monitor a CommCell operations. List and describe the value/purpose of the more commonly used reports. List and describe the value/purpose of the more commonly used alerts. List and describe options for report generation and alert notification.

CommVault System Administration Course – R02

Monitoring Module - 217

¡ Monitoring Options CommCell Explorer „ CommNet „ Job Controller „ Resource View „ Event Viewer „

CommVault System Administration Course – R02

218 – Monitoring Module

CommCell Explorer Client Views „ Job Views „ Media Views „ Configuration Views „ Database Views „

CommCell Explorer The CommCell Explorer views provide a way to query information on the CommCell components directly from the SQL database. These views are provided in addition to the CommCell Console Report Selection feature. You can use the views provided with CommCell Explorer or customize them to reflect the data in any manner appropriate to your organization. You can query the database with Microsoft SQL Mgmt Console or any SQL database query tool. Query results can be displayed through Explorer or you can use products such as Microsoft Excel or Crystal Reports to format your query output. If you modify a view or create a new view, you must reapply them after each new release. The following views are included with CommCell Explorer • • • • • • • •

CommCellAdminSchedule CommCellAuxCopyInfo CommCellBackupInfo CommCellBkSchedule CommCellClientConfig CommCellClientFSFilters CommCellJobControllerCount CommCellLibraryReservInfo

CommVault System Administration Course – R02

Monitoring Module - 219 • • • • • • • •

CommCellLicense CommCellMediaClient CommCellMediaInDrives CommCellMediaInfo CommCellRestoreInfo CommCellStoragePolicy CommCellSubClientConfig CommCellUpdateInfo

CommVault System Administration Course – R02

220 – Monitoring Module

CommNet

CommNet This management tool has been designed to manage and administer CommCells and QSMCells efficiently and in a timely manner in order to minimize administration costs. Additionally, you can analyze concise summaries and comprehensive reports showing various aspects of secondary and primary storage in order to resolve problems proactively rather than reactively. The software also provides features such as remote administration capabilities, alert mechanisms, user administration and scheduling. Alerts Alert mechanism provides the capability to notify users of critical conditions using E-mail, Pager and SNMP traps. This feature also provides the mechanism to escalate the conditions if it has not been addressed for a specified amount of time or if the condition worsens. The CommNet Server now includes the Administrative Alert, Cell Synchronization Failure Over Time, which enables users to be notified when a specified CommNet entity has encountered cell synchronization failure(s) within a specified time range. Reports The software contains various reports that help you to manage your CommNet domain. Some of the important reports are described below: •

Data Protection Coverage -Data Protection reports can help you to understand the whether or not a particular client has data protection coverage on a given day, week,

CommVault System Administration Course – R02

Monitoring Module - 221 month, or year. The coverage report also displays the status of the coverage on a copy basis to determine if the coverage is from a primary or an auxiliary copy. In addition, you can also view valuable trending analysis such as job success rate, media consumption and primary and secondary storage growth.

•

•

•

•

The software provides flexibility in defining the Data Protection Windows for the CommCells for reporting purposes. You can use these customized reports to fine tune various parameters on a CommCell in order to meet the data protection window. A Window Utilization report is also provided, which can be used to determine the Clients, Agents, and Data Protection operations that are not meeting the specified window within a specified amount of time. Window Utilization - Data Protection Window Utilization Report can help you determine how well the data protection window is being utilized in the CommCells. This report provides a summary of data protection operations relative to the data protection window specified in the CommCell, including the number of jobs completing outside the window. Data Recovery Coverage - Data Recovery reports can help you to understand the details of the recovery operations performed in the CommCells, including information on the restore destination and the job successes associated with the recovery operations. Media Management Performance - Media Management related information, such as Media Agent, library and drive performance are provided in a concise manner in various Media Management reports. The software also generates performance reports for a given Media Agent, library and drive, to help analyze media management aspects such as throughput, data size transferred, number of data protection operations handled, and library and drive usage times. Prediction Capabilities - The software provides the capability to predict the following, by looking at past consumption and usage: • •

•

•

Data growth for each CommCell as well as the sub clients in a CommCell. Media Prediction to forecast media usage (for tape/optical libraries) and capacity usage (for magnetic libraries) in the individual CommCells.

Q-Factor - Q-Factor is a scale to measure how well the data protections operations are performed in a CommCell. This is determined based on the short-term and longterm coverage. In order to customize the calculation based on your specific environment, facility to define the weights for several aspects in the short-term and long-term coverage and an acceptable Q-Factor is also provided. Primary Storage and Media Based Costing - Media based costing mechanism provides the facility to determine the cost associated with hosting an application and its protected data, based on the type of media in which the data resides. This feature includes the facility to define cost categories and the media associated with each cost category, and billable entities, such as departments within a company. A comprehensive Billable Charge Back report can be generated to view the cost, related expense, storage size, and ranking information for all vital primary and

CommVault System Administration Course – R02

222 – Monitoring Module secondary storage entities. The report can be generated for clients or billable entities. The costing model can be defined centrally and automatically distributed to all CommCells and QSMCells within the CommNet domain. Dashboard The Dashboard displays a pictorial view of a CommNet entity’s, the CommNet domain's or a CommCell's, last seven days. It contains the details of successful jobs, data protection coverage and data growth for the CommNet entity specified by the user. This is extremely useful for those needing to quickly obtain the status of their CommNet domain environment and for troubleshooting. The dashboard can display up to six (6) reports at one time. By default, it displays the following reports: Last 7 Days Job Success This bar graph depicts the data protection job success rate for the last seven days, according to the CommCell Browser’s local time. It displays the total number of data protection jobs executed during this time, as well as those that have completed, were killed and failed. If viewing the last seven days of a CommCell rather than the entire CommNet domain, you can click on the graph to see a detailed report of the data protection jobs within this time period. Last 7 Days DP Coverage This bar graph depicts the data protection coverage per subclient for the last seven days, according to the CommCell Browser’s local time. It displays the total number of subclients with data protection coverage as well as those with no activity, or with coverage that failed or was killed. If viewing the last seven days of a CommCell rather than the entire CommNet domain, you can click on the graph to see a detailed report of the data protection jobs per subclient within this time period. Last 7 Days Data Growth - Combined This bar graph depicts the data growth for both incremental and full data protection jobs for the last seven days, according to the CommCell Browser’s local time. It displays the total growth for both incremental and full data protection jobs per day in GB. If viewing the last seven days of a CommCell rather than the entire CommNet domain, you can click on the graph to see a detailed report of the data growth within this time period. Other reports available for display in the Dashboard are: • • • •

Last 7 Days Data Growth - Incremental Last 7 Days Data Growth - Full Last 7 Days DP Activity - Data Size Last 7 Days DP Activity - Job Detail

CommVault System Administration Course – R02

Monitoring Module - 223

Job Controller „ „

„ „

Job status updates every n minutes Displays job information and any errors Filter job Optional display by job z z z

Current media Associated events Log files entries

Job Controller The Job Controller allows you to monitor the following types of jobs: • • •

Data protection operations Data recovery operations Administration operations

You can view detailed information about these jobs as well as job events and the media used for each job. Information about a job is continually updated and available in the Job Controller window. When a job is finished, the job stays in the Job Controller for five minutes. This time can be changed by opening the control panel and selecting the Display control – Job Filter tab. Once a job is finished, more information about that job is obtainable using the Job History. Job Updates By default, the system updates the information in the Job Controller every 5 minutes or 2GBs of data transferred. These values can be changed by opening the Control Panel and selecting the Job Management control –Advanced tab. Job Information You can view the following information about a job in the Job Controller: • •

Job Status Job Details

CommVault System Administration Course – R02

224 – Monitoring Module • • •

Media Events Log Files

Job Status Levels A job in the Job Controller window may have one of the following status levels: • •

• • •

•

• • • •

•

• •

Completed - The job has completed. Completed With One or More Errors -The job has completed with errors. This is relevant only to Exchange 2000 Database, Oracle, and CommServe Express Recovery, and File System backup jobs. Dangling Cleanup - The job has been terminated by the job manager, and the job manager is waiting for the completion of associated processors before killing the job. Failed - The job has failed due to errors or the job has been terminated by the job manager. Interrupt Pending - The job manager is waiting for the completion of associated processors before interrupting the job due to resource contention with jobs that have a higher priority, etc. Kill Pending - The job has been terminated by the user using the kill option, and the job manager is waiting for the completion of associated processors before killing the job. Killed - The job is terminated by the user using the kill option. Pending - The job manager has suspended the job and will restart it without user intervention. A pending job may be waiting. Running - The job is active and has access to the resources it needs. Stop Pending - A job is suspended by a user using the Suspend option, and the Job Manager is waiting for the completion of associated processes before stopping the job. Stopped - A running, waiting, or pending job has been manually stopped by a user using the Suspend option. The job will not complete until it is restarted using the resume option. Waiting - The job is active, waiting for resources (e.g. media or drive) to become available or for internal processes to start. Queued - The job conflicted with other currently running jobs (such as multiple data protection operations for the same subclient), and the Queue jobs if other conflicting jobs are active option was enabled from the General tab of the Job Management dialog box. The Job Manager will automatically resume the job only if the condition that caused the job to queue has cleared.

CommVault System Administration Course – R02

Monitoring Module - 225

Resource View Drives in Library „ Media currently used „

Resource View The resource view displays all the drives in the library and the media that is currently used in the drive at any point in time. The media information includes the barcode\identifier of the media and Job ID associated with the job currently using the media. If the media is not used by a job the resource view displays the Job ID as Cache Mounted. The Resource View includes information on the operation phase to show the specific actions that are occurring on the media in the drive such as writing/verifying/reading OML, loading, unloading, etc. From the drive level, you can release the reservation on resources, if the reservation is not released when a job has failed or killed in the Job Manager. In addition, for regular libraries the resource view also displays the slot information in the library and information on media that is currently in the process of being exported. For Magnetic libraries, the status and free space information for the mount path is displayed. The Resource can be viewed from the drive list in the right-pane of the CommCell Browser when you click Master Drive Pool or Drive Pool in the tree.

CommVault System Administration Course – R02

226 – Monitoring Module

Event Viewer „ „

Used to monitor CommCell activities Filter display by: z z

„

Severity Levels # of Events available

Search events by: z z z z

Time Range Severity Job ID Character pattern

Event Viewer The Event Viewer allows you to monitor activities that are occurring within the CommCell. This information is useful for troubleshooting and informational purposes. For example, you may want to know if your system is experiencing hardware problems, or what jobs have started or completed. Some events will also generate an alert if an alert for that event is configured within the CommCell. The Event Viewer Window Events are displayed in the Event Viewer with the severity levels of Information, Minor, Major, and Critical, along with additional information, such as the subsystem that generated the event. By default, the maximum number of events displayed in the Event Viewer is 200. You can modify this number by opening the control panel and selecting the Display control – Event Filter. You can modify the types of events to be displayed based on the severity level. The default maximum number of events retained in the Event log is 10,000. This number can be changed by opening the Control Panel and selecting System. The Event log is maintained in the metadata database.

CommVault System Administration Course – R02

Monitoring Module - 227 View the Details of an Event If a particular event is not entirely visible in the Event Viewer, the complete description can be viewed in the Event Details dialog box. Event Search Queries To view only selected events from the Event Viewer, a search query can be created to display only those events that correspond to a time range and severity level, or a Job ID. The results of your query are displayed in the All Found Events window. A further search using the Find feature can be run in the window to search for word/phrase patterns. Some search queries can be saved and selected at any time from the Select from this search query field. A search query based on a particular Job ID cannot be saved.

CommVault System Administration Course – R02

228 – Monitoring Module

¡ Alerts Understanding Alerts „ Alert Output Options „ Recommended Alerts „

CommVault System Administration Course – R02

Monitoring Module - 229

Understanding Alerts

Understanding Alerts CommVault’s Alert feature provides real time notification on events occurring within a CommCell. These conditions can range from minor occurrences that do not require intervention to severe occurrences that need immediate intervention. The system detects conditions within two minutes of the occurrence. Alert coverage is hierarchical and can be defined for each alert created. Alerts can be configured for any object that can be seen by the user creating the alert. Best Practices When possible, configure alerts to cover Client computer groups vice individual clients and when using the E-mail output option, use group e-mail addressing vice individual users. Groups are more consistent and won’t require you to modify alerts when individual Clients or users are removed or added.

CommVault System Administration Course – R02

230 – Monitoring Module

Alert Output Options „

Methods z z z z

„ „ „

E-Mail/Pager SNMP Trap Application Log Command Line

Variable data support (Tokens) Customizable content Escalation

Alert Output Options Alerts can be sent to their intended recipients by email/pager; to the CommServe’s application log accessible from the Windows System Event Viewer; by SNMP Trap(s) to a SNMP capable monitoring application; or initiate a command line program, executable script, or batch file. Alert wording can be customized by the user. Relevant, localized information can be added to reports. Tokens are provided when variable data from the alert is required. Tokens can also be passed as arguments for the command line script. Command line arguments support the use of all tokens except attachments (e.g. ) or lists (e.g. ). E-mail alerts can be sent to a CommCell User Group (Enterprise version), or to individual users. In order for a user to receive an alert, the user must be enabled and have a valid SMTP address. If the mail server is down, the system will attempt to send alert e-mail notification for the next four hours. If after four hours, the mail server is still down, these alerts will be removed from the system, and will not be sent to the recipient. Additionally, an individual alert notification can be generated rather than multiple alerts being generated within a single alert notification. In the event that an alert condition persists, an escalation notice can be generated via alternate means.

CommVault System Administration Course – R02

Monitoring Module - 231

Recommended Alerts Insufficient Storage „ Phase or Network Failure „ went Offline „

Recommended Alerts Insufficient Storage Media Management->Library Management The most common reason for data protection job failure is no spare media. You run out of magnetic disk space or have no spare media inside the library. This is particularly prevalent during weekends when large data protection jobs are usually scheduled and no one is about to monitor the status of spare media. The Insufficient Storage alert is dependent upon the Low Watermark threshold value assigned to the magnetic library, Scratch, and Cleaning Media Pools. Ensure you set these values high enough to allow the alert to give you ample warning to replenish the media before jobs start to fail. Phase or Start Job Failure Job Management->Data Protection A Data Protection job consists of two or more phases and requires network connection in order to initiate the job on the Client. If a phase fails, or the Client can’t be reached the job will enter a pending state and retry. If the condition that caused the failure is transitory, a retry may succeed. If not, the entire job will eventually fail if nothing is done to overcome the reason for the network or phase failure. You can be pro-active in setting alerts for repetitive phase and/or network failure to take appropriate action before the entire job fails.

CommVault System Administration Course – R02

232 – Monitoring Module Offline Media Management->Device Status Library, mount paths, or tape devices that are taken offline intentionally, or by themselves due to some problem can have significant impact on your ability to complete jobs in a timely manner. CommVault’s CommCell Readiness Report checks for device status and should be run before each data protection window. However, for more real time notification we recommend you set an alert for offline status of key devices.

CommVault System Administration Course – R02

Monitoring Module - 233

¡ Reports Understanding Reports „ Recommended Reports „

CommVault System Administration Course – R02

234 – Monitoring Module

Understanding Reports CommNet CommServe

Output Form ats HTML CommServe

TEXT XML

Job Metadata Storage

VaultTracker

Output Methods E-mailed

CommCell Explorer

User Group

Saved Folder/File Template

Available Reports A variety of reports can be created, each tailored to a particular aspect of data management. Through filter criteria, you can customize each report to include only the data that is required. Report templates can be customized and are available to you to save, run, schedule, edit and view. CommNet CommVault’s CommNet or QNet is a hierarchical reporting and management application for one or more CommCells. CommNet reports can include data from multiple CommCells and have user selectable graphing capability. CommCell Explorer CommCell Explorer is a set of database views that can be used to create customized reports.

CommVault System Administration Course – R02

Monitoring Module - 235

Recommended Reports CommCell Summary Report „ Job Summary Report „ Data Retention Forecast and Compliance Report „ Media Information Report „ CommCell Readiness Report „

Recommended Reports CommCell Summary Report The CommCell Summary Report provides the properties of the CommServe, Media Agents, clients, agents, subclients, and storage policies within the CommCell based on the selected filter criteria. This report is useful if you require: • • • • • • • • • • •

The number of computers installed in the CommCell. The type of licenses available in the CommCell, installation date for each license type consumed, as well as the expiration date of each license. Administrative schedules. Data interface pairs information. The alerts that are configured in the CommCell. CommCell user and user group information. Service Pack information for each module. Media Agent and library information. Client information such as enabled and/or disabled operations, defined operation rules, installed clients, and subclient contents and filters. Scheduled data protection information for each subclient. Retrieving all Update information from the registry of each machine in the CommCell and updating the CommServe Database with this information now includes NetWare Agents.

CommVault System Administration Course – R02

236 – Monitoring Module • • • • • •

•

A listing of mailboxes per subclient for common mailbox agents. Information that the use of QSnap or VSS was selected through subclient properties. This is indicated by a superscript Q in the subclient column of the report. The remaining index cache space, size of existing index cache, and disk size on each Media Agent. Subclient policy information through an optional checkbox. The number of streams for a subclient. All Updates which have been applied to a machine as well as the components to which they were applied the list of mailboxes per subclient for the Exchange Mailbox and Archiver Agents. The option to retrieve all Update information from the registry of each machine in the CommCell and update the CommServe Database with this information now includes NetWare Agents.

Job Summary Report The Job summary report can be filtered to display the following: • All Jobs - All Data Protection, Data Recovery, and Administrative Jobs are included in the report. •

Data Protection Jobs - Can be filtered to include: Calendar Backup Job Summary, Backup Job Summary, Calendar Migration/Archive Job summary, QR Volume Creation Job Summary or ContinuousDataReplicator Job Summary.

•

Data Recovery Jobs – Can be filtered to include: Restore Job Summary, Recovery/Retrieve Job Summary, QR Recovery Job Summary, ContinousDataReplicator Copyback Job Summary and/or Stub Recall Summary.

•

Administrative Jobs – Can be filtered to include: Disaster Recovery Backup Job Summary, Auxiliary Copy Job Summary, Data Aging Job Summary, Data Verification Job Summary, Download Updates Job Summary and/or Install Updates Job Summary.

This report is useful if you require: • •

To view all data protection, data recovery, and administration jobs during a specified time period. To view the failure reason and associated events for all data protection, data recovery, and administrative jobs.

Data and Aging Forecast Report The Data on Media and Aging Forecast Report provides a history of data protection jobs, associated media, data aging forecast and recyclable media based on the selected filter criteria. Note that regardless of the filter options selected for this report, the report always checks retention in days. This report is useful if you require:

CommVault System Administration Course – R02

Monitoring Module - 237

• • • • •

A list of data protection jobs that are on each media or magnetic volume. A list of data protection jobs that either exceed their retention criteria or need to be retained. A list of media that can be recycled and a list of media that can not be recycled based on the copy retention. The amount of magnetic space that will be freed and kept for each storage policy copy. An estimated date on which a job is due to be aged.

Media Information Report The Media Information Report provides detailed information about the media that is associated with storage libraries, the media repositories and/or locations, and the media that has been written to or reserved by storage policy copies, all within a specified time range. This report is useful if you require: • Media status and the reason for the media status, such as if the media is full, bad, expired, foreign, or undiscovered. • A list of media that has errors. • A list of empty slots in a library. • A list of data that has been aged from media. • Media repository information if media repositories have been configured for the media. • The barcodes and locations of the media. • The storage policies and storage policy copies associated with the media. • An alert of all the media about to be marked expired • Information on which media is exportable. This is indicated by a superscript E in the Barcode column of the report. CommCell Readiness Report The CommCell Readiness Report provides you with vital information about the condition of your CommCell that can serve as a tool to warn you about potential problems that can impact your operations before they occur. This report is useful if you require: • • • •

To check for connectivity between your CommServe, Media Agents, and clients. To check the capacity of your library. To verify that your library controllers, libraries, master drive pools, drive pools, and drives are online. To verify the existence of the index cache location and to check the index cache label's contents against the selected Media Agent's index cache.

CommVault System Administration Course – R02

238 – Monitoring Module

Module Summary

Key points to remember

Summary •

Alerts are configured to notify users of various occurrences within a CommCell.

•

Many different reports are available with an abundance of options to configure. They can be scheduled for email, or saved to a file among other options.

•

The CommCell Explorer views provide a way to query information on the CommCell components directly from the SQL database.

•

The Job Controller window offers the availability to monitor and manage running jobs.

•

The Event Viewer allows you to monitor all activities that are occurring within the CommCell.

CommVault System Administration Course – R02

Appendix - 239

Appendix

www.commvault.com/training

Appendix

CommVault System Administration Course – R02

240 - Appendix

Vault Tracker vs. Vault Tracker Enterprise Supported Features and Reports The following table displays the features and reports available for VaultTracker and VaultTracker Enterprise: Features

VaultTracker

VaultTracker Enterprise

Export Media after Data Protection and Auxiliary Copy Operations

X

Export Media to Virtual Mail Slots

X

Filter Media by Retention Criteria

X

Iron Mountain Support

X

Media Due Back Policy

X

Media Tracking Between Libraries

X

Media Tracking from a Library to an Export Location

X

X

Media Repository

X

Moving Media Using Containers

X

Recall Media

X

Tracking Transit Locations

X

VaultTracker Alerts

X

Reports

VaultTracker

VaultTracker Enterprise

Media Information Report

X

X

Vault Tracking Report

X

X

CommVault System Administration Course – R02

Appendix - 241

CommCell

CommVault System Administration Course – R02

242 - Appendix

Training Environment

CommVault System Administration Course – R02