Clustered ONTAP Admin 8.2 - Student Guide (STRSW-ILT-D8CADM-REV03-SG) PDF

NETAPP UNIVERSITY
Clustered Data ONTAP Administration
Student Guide
Course ID: STRSW-ILT-D8CADM-REV03
Catalog Number: STRSW-ILT-D8CADM-REV03-SG
Content Version: 1.0
ATTENTION
The information contained in this course is intended only for training. This course contains information and activities that,
while beneficial for the purposes of training in a closed, non-production environment, can result in downtime or other
severe consequences in a production environment. This course material is not a technical reference and should not,
under any circumstances, be used in production environments. To obtain reference materials, refer to the NetApp product
documentation that is located at http://now.netapp.com/.
COPYRIGHT
2013 NetApp, Inc. All rights reserved. Printed in the U.S.A. Specifications subject to change without notice.
No part of this document covered by copyright may be reproduced in any form or by any meansgraphic, electronic, or
mechanical, including photocopying, recording, taping, or storage in an electronic retrieval systemwithout prior written
permission of NetApp, Inc.
U.S. GOVERNMENT RIGHTS

Commercial Computer Software. Government users are subject to the NetApp, Inc. standard license agreement and
applicable provisions of the FAR and its supplements.
TRADEMARK INFORMATION
NetApp, the NetApp logo, Go further, faster, AdminNODE, Akorri, ApplianceWatch, ASUP, AutoSupport, BalancePoint,
BalancePoint Predictor, Bycast, Campaign Express, ChronoSpan, ComplianceClock, ControlNODE, Cryptainer, Data
ONTAP, DataFabric, DataFort, Decru, Decru DataFort, DenseStak, Engenio, E-Stack, FAServer, FastStak, FilerView,
FlexCache, FlexClone, FlexPod, FlexScale, FlexShare, FlexVol, FPolicy, GatewayNODE, gFiler, Imagine Virtually
Anything, Infinivol, Lifetime Key Management, LockVault, Manage ONTAP, MetroCluster, MultiStore, NearStore, NetApp
Select, NetCache, NetCache, NOW (NetApp on the Web), OnCommand, ONTAPI, PerformanceStak, RAID DP,
SANscreen, SANshare, SANtricity, SecureAdmin, SecureShare, Securitis, Service Builder, Simplicity, Simulate ONTAP,
SnapCopy, SnapDirector, SnapDrive, SnapLock, SnapManager, SnapMirror, SnapMover, SnapProtect, SnapRestore,
Snapshot, SnapValidator, SnapVault, StorageGRID, StorageNODE, StoreVault, SyncMirror, Tech OnTap, VelocityStak,
vFiler, VFM, Virtual File Manager, WAFL, and XBB are trademarks or registered trademarks of NetApp, Inc. in the United
States and/or other countries.
All other brands or products are either trademarks or registered trademarks of their respective holders and should be
treated as such.
2 Clustered Data ONTAP Administration: Welcome
2013 NetApp, Inc. This material is intended only for training. Reproduction is not authorized.
TABLE OF CONTENTS
WELCOME ........................................................................................................................................................ 1
MODULE 1: OVERVIEW ................................................................................................................................ 1-1
MODULE 2: INSTALLATION AND CONFIGURATION ................................................................................ 2-1
MODULE 3: CLUSTER ADMINISTRATION BASICS ................................................................................... 3-1
MODULE 4: ARCHITECTURE ....................................................................................................................... 4-1
MODULE 5: PHYSICAL DATA STORAGE ................................................................................................... 5-1
MODULE 6: LOGICAL DATA STORAGE ..................................................................................................... 6-1
MODULE 7: PHYSICAL NETWORKING ....................................................................................................... 7-1
MODULE 8: LOGICAL NETWORKING ......................................................................................................... 8-1
MODULE 9: NAS PROTOCOLS .................................................................................................................... 9-1
MODULE 10: SAN PROTOCOLS ................................................................................................................ 10-1
MODULE 11: STORAGE EFFICIENCY ....................................................................................................... 11-1
MODULE 12: DATA PROTECTION: SNAPSHOT AND SNAPMIRROR COPIES ..................................... 12-1
MODULE 13: DATA PROTECTION: BACKUPS AND DISASTER RECOVERY ....................................... 13-1
MODULE 14: CLUSTER MANAGEMENT ................................................................................................... 14-1
MODULE 15: RECOMMENDED PRACTICES ............................................................................................ 15-1
APPENDIX: TECHNICAL REPORTS AND KNOWLEDGE BASE ARTICLES ........................................... A-1
Clustered Data
ONTAP
Administration
Course ID:
STRSW-ILT-D8CADM-REV03
NetApp Confidential
CLUSTERED DATA ONTAP ADMINISTRATION
Classroom Logistics
Schedule Safety
Start time Alarm signal
Stop time Evacuation procedure
Break times Electrical safety
guidelines
Facilities
Food and drinks
Restrooms
Phones
NetApp Confidential 2
CLASSROOM LOGISTICS
Course Objectives
1 of 2
By the end of this course, you should be able to:
Explain the primary benefits of a Data ONTAP cluster
Create a cluster
Implement role-based administration
Manage the physical and logical resources within a
cluster
Manage features to guarantee nondisruptive
operations
Discuss storage and RAID concepts
Create aggregates
List the steps that are required to enable storage
failover (SFO)
COURSE OBJECTIVES: 1 OF 2
Course Objectives
2 of 2
Create a flash pool
Build a namespace using multiple volumes
Configure FlexCache
Create an infinite volume
Identify supported cluster interconnect switches
Set up and configure SAN and NAS protocols
Configure the storage-efficiency features
Administer mirroring technology and data protection
Explain the notification capabilities of a cluster
Scale a cluster horizontally
Configure the storage QoS feature
COURSE OBJECTIVES: 2 OF 2
Course Agenda: Day 1
Morning
Module 1: Overview
Afternoon
Module 2: Installation and Configuration
Module 3: Cluster Administration Basics
COURSE AGENDA: DAY 1
Morning
Module 4: Architecture
Module 5: Physical Data Storage
Afternoon
Module 6: Logical Data Storage
Module 7: Physical Networking
Morning
Module 8: Logical Networking
Module 9: NAS Protocols
Afternoon
Module 10: SAN Protocols
Morning
Module 11: Storage Efficiency
Module 12: Data Protection: Snapshot and
SnapMirror Copies
Afternoon
Module 13: Data Protection: Backups and
Disaster Recovery
Morning
Module 14: Cluster Management
Afternoon
Module 14: Cluster Management (Continued)
Module 15: Recommended Practices
NetApp University Information Sources
NetApp Support
http://support.netapp.com
NetApp University
http://www.netapp.com/us/services-
support/university/
NetApp University Support

http://netappusupport.custhelp.com
NETAPP UNIVERSITY INFORMATION SOURCES
Module 1
Overview
MODULE 1: OVERVIEW
1-1 Clustered Data ONTAP Administration: Overview
Module Objectives
After this module, you should be able to:

Explain the primary benefits of Data ONTAP
clustering
Identify the scope of Data ONTAP concepts
such as node virtual storage servers
(Vservers), administrative Vservers, and data
Vservers
MODULE OBJECTIVES
Clustered Data ONTAP
A Reliable, Optimized Computing

Foundation
Provides unlimited connectivity
Dynamically controls itself
Integrates business workflows
Simplified End-User Experience

Access control
Audit policies
File-level and storage systems
Data management
Information lifecycle management
CLUSTERED DATA ONTAP

The Data ONTAP operating system is the foundation of NetApp Unified Storage Architecture. Data ONTAP
8.2 adds a number of critical new features to that foundation. Before reviewing the new features, recall the
significance of a unified storage architecture. NetApp Unified Storage Architecture includes unique and
innovative features that distinguish it from the competition. This single architecture spans the entire hardware
lineup, from the smallest entry-level model to the largest enterprise model. The architecture provides Fibre
Channel SAN, iSCSI, and NAS protocol support, scalable performance and capacity, and a single application
interface for the entire product lineup. Unmatched reliability, scalability, and flexibility make the Data
ONTAP operating system the best choice for all data storage environments.
Clustered Data ONTAP Highlights
Protocols Scalability Storage Efficiency

CIFS Performance scaling Capacity scaling Deduplication
FC
Compression
FCoE NFS
Thin provisioning
iSCSI pNFS Operational Cloning
scaling
Cost and Management and Integrated Data Protection
Performance Ecosystem Integration
Snapshot copies
Flash Cache Unified management
Asynchronous mirroring
Solid-state drives Secure multi-tenancy
Disk-to-disk and disk-to-
Flash Pool Multivendor
tape backup
FlexCache virtualization
SAS and SATA
CLUSTERED DATA ONTAP HIGHLIGHTS
Primary Reasons to Use Clustered
Data ONTAP
Scalability: performance and capacity
Flexibility: data management and movement
Transparency: namespaces, storage failover,
NAS LIF failover and migration, resource use
and balancing, nondisruptive operation
PRIMARY REASONS TO USE CLUSTERED DATA ONTAP

Clustered Data ONTAP can scale to meet the needs of customers. Adding additional disk shelves to a single
system has always been a method of scaling capacity. The scalability of a cluster greatly multiplied by the
ability to add nodes to a cluster without any disruption or downtime for the cluster. You can also scale
computing power by adding nodes to a cluster.
Clusters also provide incredible flexibility in how nodes and storage are managed.
Finally, the ability to move volumes among aggregates and nodes means that an administrator isnt locked
into a particular data scheme. Changing a scheme is transparent and simple.
Scalability
A single system image for 1 to 24 nodes

Throughput that scales linearly to multiple gigabytes per second
Capacity that scales to petabytes
The ability to support continuous operation
Online (transparent) load balancing and scaling
A fully integrated, single-vendor solution
Note: Not all platform combinations can scale up to 24 nodes.
SCALABILITY
Clustered Data ONTAP solutions can scale from 1 to 24 nodes, and are mostly managed as one large system.
More importantly, to client systems, a cluster looks like a single file system. The performance of the cluster
scales linearly to multiple gigabytes per second of throughput, and capacity scales to petabytes.
Clusters are built for continuous operation; no single failure on a port, disk, card, or motherboard will cause
data to become inaccessible in a system. Clustered scaling and load balancing are both transparent.
Clusters provide a robust feature set, including data protection features such as Snapshot copies, intracluster
asynchronous mirroring, SnapVault backups, and NDMP backups.
Clusters are a fully integrated solution. This example shows a 20-node cluster that includes 10 FAS systems
with 6 disk shelves each, and 10 FAS systems with 5 disk shelves each. Each rack contains a high-availability
(HA) pair with storage failover (SFO) capabilities.
Scalability: Performance (NAS)
R A Six-Node Data ONTAP Cluster
A B C F R A R B
B B R
C G H E F D
D E G H
Linearly scale read performance

with load-sharing mirror
relationships.
Linearly scale aggregate
read/write performance in a single
namespace.
SCALABILITY: PERFORMANCE (NAS)

In this example, volume R is the root volume of a virtual storage server and its corresponding namespace.
Volumes A, B, C, and F are mounted to R through junctions. This clustered Data ONTAP solution provides
performance scaling in two ways:
Volume B and root Volume R each have two read-only mirror relationships, which enables read requests
to be spread across multiple volumes and nodes (the B mirrors) rather than taxing one volume (and node)
with all read requests. High-performance computing (HPC) applications often require scaling of read
throughput well beyond the write throughput.
Volumes A, C, D, E, F, G, and H provide scaled performance when applications are accessing all six
nodes. Because the volumes are distributed across six nodes, processing is spread across those nodes, but
all of the nodes are in a single namespace. Volumes D and E are mounted to C through junctions.
Likewise, volumes G and H are mounted to F.
Scalability: Capacity
The ability to rapidly and
seamlessly deploy new
storage or applications or
Projects
both
No required downtime A B C
Movement that is transparent

A1 A2 A3 B1 B2 C1 C2 C3
to clients and does not alter
the namespace
A B C2 B2
A1 A2 C C3
A3
C1 B1
Offload (move) volumes to newly added storage.

Grow the volumes.
SCALABILITY: CAPACITY
In the example on this slide, more capacity is needed for project B. Follow these steps to scale the capacity:
1. Add two nodes to make a 10-node cluster with additional disks.
2. Transparently move some volumes to the new storage.
3. Expand volume B in place.
This movement and expansion is transparent to client machines.
Flexibility: The Virtual Storage Tier
Data-Driven Real-Time Self-Managing Flash Cache
Storage-level RAID-protected
The Virtual cache
Storage Tier
PCI-e modules
Capacities of up to 2 TB
Flash Pool
A RAID-protected aggregate
A solid-state drive (SSD) tier
that is used as cache
A hard disk tier that is used as
storage
Hard Disk Storage
FLEXIBILITY: THE VIRTUAL STORAGE TIER

The NetApp Virtual Storage Tier provides fully automated use and optimization of flash
technologycontroller-based and based on Peripheral Component Interconnect Express (PCIe) and solid-
state drives (SSDs).
NetApp flash PCIe modules improve performance for workloads that are random-read-intensive, which
reduces latency by a factor of 10 or more compared to hard disks. Flash Cache modules are available in
capacities of up to 2 TB and provide controller-based caching.
NetApp Flash Pool enables the caching of random read and write operations through the automated use of
SSDs, which enables the use of capacity-optimized hard disk technology across the majority of
application workloads. Flash Pool enables the creation of a Data ONTAP RAID-protected aggregate that
consists of a combination of hard disks and SSDs.
Transparency: Load Optimization
Optimized performance
Maximized disk use
Transparency to
Projects applications
A B C
A1 A2 A3 B1 B2 C1 C2 C3
A
B1 B A1 C2 C3
A3
C1 C B2 A2
Project A gets dedicated resources.
TRANSPARENCY: LOAD OPTIMIZATION

In this example, project A needs more computer power than other projects need. With a cluster, you can:
Transparently move volumes that are not project-A volumes to free up processing power on other nodes
Give project A dedicated resources
Move volumes as needed when project A stops being critical or if another project becomes critical
Volume movement is transparent to client machines.
Nondisruptive Operation
DataMotion for Volumes

NAS logical interface (LIF) migration
Storage failover (SFO)
NONDISRUPTIVE OPERATION
Nondisruptive operation is a key feature of Data ONTAP clustering. Three critical components of
nondisruptive operation include DataMotion for Volumes (volume move), logical interface (LIF) migration,
and SFO.
SFO is covered in Module 6: Logical Data Storage
NAS LIF Migration is covered in Module 8: Logical Networking
Volume move is covered in Module 14: Cluster Management
Virtual Storage Servers
Virtual Storage Servers (Vservers)
Represent groupings of physical and logical resources
Are conceptually similar to vFilers
Node Vservers
Represent each physical node
Are associated with cluster LIFs, node management LIFs, and
intercluster LIFS
Administrative Vserver
Represents the physical cluster
Is associated with the cluster management LIF
Data Vservers
Are a virtual representation of a physical data server
Are associated with data LIFs
VIRTUAL STORAGE SERVERS

There are three types of Vservers. Data Vservers are used to read and write data to and from the cluster. Node
Vservers simply represent node-scoped resources, and administrative Vservers represent entire clusters.
Unless the documentation refers specifically to an admin Vserver or node Vserver, the general term
Vservers refers to data Vservers.
Cluster Resources
Data
Network
cmg
mg1 lif7 lif8 lif15 mg1 lif16
lif4 lif5 lif6 lif12 lif13 lif14
lif1 lif2
HA
lif3 lif9 lif10 lif11
Interconnect
cl1 cl2
cl1 cl2
Cluster
aggr1 aggr2 Interconnect aggr3 aggr4
n1aggr0 n2aggr0
n1vol0 n2vol0
Data Vservers:
vserverA
vserverB
vserverC
CLUSTER RESOURCES
The example on this slide shows many of the key resources in a cluster: three types of Vservers (node, data,
and administrative), plus nodes, aggregates, volumes, and data LIFs.
Physical and Logical Elements
Physical Logical
Nodes Clusters
Disks Volumes
Aggregates Snapshot copies
Network ports Mirror relationships
FC ports Vservers
Tape devices LIFs
PHYSICAL AND LOGICAL ELEMENTS

Physical elements of a systemsuch as disks, nodes, and ports on those nodescan be touched and seen.
Logical elements of a system cannot be touched, but they do exist and use disk space.
Volumes, Snapshot copies, and mirror relationships are areas of storage that are divided from aggregates.
Clusters are groupings of physical nodes.
Vservers are virtual representations of resources or groups of resources.
A LIF is an IP address that is associated with a single network port.
vserver show (Summary View)
cluster1::> vserver show
Admin Root Name Name
Vserver Type State Volume Aggregate Service Mapping
----------- ------- --------- ---------- ---------- ------- -------
cluster1 admin - - - - -
cluster1-01 node - - - - -
vs1 data running vs1 aggr1a file file
4 entries were displayed.
VSERVER SHOW (SUMMARY VIEW)

Notice the types of Vservers. When a cluster is created, the administration Vserver is automatically created.
When a node is joined to the cluster, a node Vserver is automatically created to represent it. Administrators
create data Vservers to build global namespaces.
Administrative Vserver
Cluster
Management
LIF cmg
Administrative Vserver (Physical Cluster)
ADMINISTRATIVE VSERVER
Node Vservers
Data
Network Node
Network Ports Management
mg1 mg1 LIFs
Cluster LIFs
lif1 lif2
lif1 lif2
Cluster
Network
Ports
n1aggr0 n2aggr0
n1vol0 n2vol0
Aggregates
NODE VSERVERS
Data Vservers
lif7 lif8 lif15 lif16

Data LIFs lif4 lif5 lif6 lif12 lif13 lif14
Volumes
Data Vservers:
vserverA
vserverB
vserverC
DATA VSERVERS
Data Vserver Details
Formerly referred to as cluster Vservers

Are not necessarily associated with any single
node
Contain most resources within their scope:
Namespace
Volumes
Data LIFs (for client access)
Protocol servers: NFS, CIFS, FC, FCoE, and
iSCSI
DATA VSERVER DETAILS

Data Vservers are not necessarily associated with any node or group of nodes within the cluster, but there can
be circumstances where an administrator chooses to limit a Vservers volumes and data LIFS to specific
nodes.
Putting It All Together
Data
Network
cmg
mg1 lif7 lif8 lif15 mg1 lif16
lif4 lif5 lif6 HA lif12 lif13 lif14
Interconnect
lif1 lif2
lif1 lif2
Node Vserver Node Vserver
Cluster
n1aggr0 n2aggr0
n1vol0 n2vol0
Data Vservers:
vserverA
vserverB
vserverC
PUTTING IT ALL TOGETHER
Module Summary
Now that you have completed this module, you

should be able to:
Explain the primary benefits of Data ONTAP
clustering
Identify the scope of Data ONTAP concepts
such as node virtual storage servers
(Vservers), administrative Vservers, and data
Vservers
MODULE SUMMARY
Exercise
Module 1: Overview
Time Estimate: 10 Minutes
EXERCISE
Please refer to your exercise guide.
Module 2
Installation and Configuration
MODULE 2: INSTALLATION AND CONFIGURATION
2-1 Clustered Data ONTAP Administration: Installation and Configuration
Module Objectives

Configure nodes and install the operating
system
Create a cluster
to the Expand a cluster two nodes
Set the dates, times, and time zones of the
nodes in the cluster
MODULE OBJECTIVES
Basic Steps for Setting Up a Cluster
1. Connect controllers, disks, and cables.

2. Set up and configure nodes.
3. Install software onto nodes (software is pre-
installed on most systems).
4. Initialize disks.
5. Create a cluster.
6. Join additional nodes to the cluster.
7. Create aggregates and volumes.
8. Configure data Vservers.
BASIC STEPS FOR SETTING UP A CLUSTER
Hardware Setup
Connect:
Controllers to disk shelves
High-availability (HA) interconnect
Controllers to the networks
Any tape devices
Controllers and disk shelves to power
HARDWARE SETUP
Connect controllers to disk shelves. Verify that shelf IDs are set properly.
If required for your controller type, connect nonvolatile RAM (NVRAM) high-availability (HA) cable
between partners. The connections can be 10-GbE or InfiniBand, depending on your storage controllers.
Connect controllers to the networks.
If present, connect any tape devices. This task can be performed later.
Connect controllers and disk shelves to power.
Communication Connections
Console connection (using ANSI-9600-8N1)

Remote management device connection,
dependent on model
Service Processor (SP)
Remote LAN Module (RLM)
Management network connections
Cluster network connections
Data network connections
COMMUNICATION CONNECTIONS
Each controller should have a console connection, which is required to get to the firmware and to get to the
Boot menu (for the setup, installation, and initialization options, for example). A remote management device
connection, although not required, is helpful in the event that you cannot get to the UI or console. Remote
management enables remote booting, the forcing of core dumps, and other actions.
Each node must have two connections to the dedicated cluster network. Each node should have at least one
data connection, although these data connections are necessary only for client access. Because the nodes are
clustered together, its possible to have a node that participates in the cluster with its storage and other
resources but doesnt field client requests. Typically, however, each node has data connections.
The cluster connections must be on a network that is dedicated to cluster traffic. The data and management
connections must be on a network that is distinct from the cluster network.
Disk Cabling
node1 node2 node3 node4
NVRAM* Interconnect
FC or SAS (simplified)
For more detailed information on disk cabling, see module 5 in this

course, and also the Data ONTAP Cabling web-based course.
DISK CABLING
A large amount of cabling must be done with a Data ONTAP cluster. Each node has NVRAM
interconnections to its HA partner. Each node has FC or SAS connections to its disk shelves and to those of
its HA partner.
In a multipath high-availability (MPHA) cabling strategy, each storage controller has multiple ways to
connect to a disk. An I/O module failure does not require a controller failover. This method is the most
resilient and preferred method of shelf cabling.
Ethernet cabling for alternate control path (ACP) requires one connection to each controller, connected in a
series through all shelves. First you connect stack to stack. Then you connect between I/O modules from top
to bottom in each stack.
Network Cabling
1 of 2
Cluster Interconnect
Management
Data
Management Data Network

Network
NOTE: NetApp recommends switch redundancy for data and

management networks.
NETWORK CABLING: 1 OF 2
For customers with strict security requirements, management ports can be connected to a network that is
separate from the data network. In that case, management ports must have a role of management, and network
failover cannot occur between data and management interfaces.
Network Cabling
2 of 2
Management
Data
Data and
Management
Network
NOTE: NetApp recommends switch redundancy for data and

NETWORK CABLING: 2 OF 2
When you cable the network connections, consider the following:
Each node is connected to at least two distinct networks: one for management (the UI) and data access
(clients) and one for intracluster communication. NetApp supports two 10-GbE cluster connections to
each node to create redundancy and improve cluster traffic flow.
The cluster can be created without data network connections but not without cluster network connections.
Having more than one data network connection to each node creates redundancy and improves client
traffic flow.
Powering On a Node and Cluster
1. Power on network switches.

2. Power on disk shelves.
3. Power on tape devices (if present).
4. Power on storage controllers.
POWERING ON A NODE AND CLUSTER
Firmware
Use LOADER firmware.

Press any key to enter the firmware.
Two boot device images exist: flash0a and
flash0b.
Use printenv to show the firmware
environment variables.
Use setenv to set the firmware environment
variables, for example, setenv AUTOBOOT
true
FIRMWARE
1. Use LOADER firmware.
2. From the console, early in the booting process, press any key to enter the firmware.
3. Use version to show the firmware version.
4. Two boot device images exist (depending on platform): flash0a and flash0b.
CompactFlash
USB flash
5. Use printenv to show the firmware environment variables.
6. Use setenv to set the firmware environment variables; for example, setenv AUTOBOOT true.
To copy flash0a to flash0b, run flash flash0a flash0b. To flash (put) a new image onto the
primary flash, you must first configure the management interface. The auto option of ifconfig can be
used if the management network has a Dynamic Host Configuration Protocol (DHCP) or BOOTP server. If it
doesnt, you must run ifconfig <interface> addr=<ip> mask=<netmask> gw=<gateway>.
After the network is configured, ensure that you can ping the IP address of the TFTP server that contains the
new flash image. To then flash the new image, run
flash tftp://<tftp_server>/<path_to_image> flash0a.
The Setup Procedure
From the Boot menu:

1. If necessary, run option 7 (Install new
software first).*
2. Run option 4 (Clean configuration and
initialize all disks).*
3. Run the cluster setup wizard.
* Steps 1 and 2 are usually done by NetApp before

hardware arrives at a customer site.
THE SETUP PROCEDURE
The Boot Menu
Please choose one of the following:
1. Normal Boot.
2. Boot without /etc/rc (no effect in Clustered ONTAP).
3. Change password.
4. Clean configuration and initialize all disks.
5. Maintenance mode boot.
6. Update flash from backup config.
7. Install new software first.
8. Reboot node.
Selection (1-8)?
THE BOOT MENU
Installing the Data ONTAP
Operating System on a Node
You need:
Access to an FTP, TFTP, or HTTP server
The software image file on that server
From the boot menu, complete the following:
1. Select option 7.
2. When prompted, enter a URL to a Data ONTAP tgz image.
3. When complete, allow the system to boot.
INSTALLING THE DATA ONTAP OPERATING SYSTEM ON A NODE

After you boot the system, if the node stops at the firmware prompt by itself (which happens if the firmware
environment variable AUTOBOOT is set to false), type boot_primary to enable the node to continue to the
boot menu. If AUTOBOOT is set to true, the node goes straight to the boot menu.
If you use TFTP, beware of older TFTP servers that have limited capabilities and might cause installation
failures.
Initializing a Node
From the Boot menu, select option 4:
Initialization clears the three disks that the system
uses for the first aggregate that it creates.
NOTE: This action requires time, depending on disk size.
Initialization creates one aggregate (for this node) and
a vol0 root volume on the aggregate.
Initialization must be run on both nodes of each HA
pair.
INITIALIZING A NODE
Because all disks are initialized parallel to each other, the time that is required to initialize the disks is based
on the size of the largest disk that is attached to the node, not on the sum capacity of the disks. After the disks
are initialized, the nodes first aggregate and its vol0 volume are automatically created.
The Cluster Setup Wizard
1 of 3
From the Boot menu of an initialized
controller:
1. Boot normally.
2. Log in as admin with no password.
3. Follow the prompts.
You can also run cluster setup from the
CLI.
THE CLUSTER SETUP WIZARD: 1 OF 3

An initialized storage controller automatically boots to the cluster setup wizard. You can type exit to stop
the wizard and stay at the command prompt. The node is not yet part of a cluster. You can restart the wizard
from the CLI by typing cluster setup.
2 of 3
The first node creates the cluster.
You need the:
Cluster name
Cluster network ports and MTU size (usually best to use
default MTU)
Cluster base license key
Cluster management interface port, IP address, network
mask, and default gateway
Node management interface port, IP address, network
mask, and default gateway
DNS domain name
IP addresses of the DNS server

To complete the cluster setup wizard, you need the following information for the first node in the cluster:
The cluster name. The wizard automatically names the node clustername-0x, where x is the order in
which the node joins the cluster. The node that creates the cluster is 01, the next node to join the cluster is
02, and so on.
Cluster network ports. On a private, nonroutable cluster network, the wizard automatically assigns cluster
IP addresses.
The cluster base license key. Additional license keys can be added here.
The cluster management interface port, IP address, network mask, default gateway, and home port
The node management interface port, IP address, network mask, default gateway, and home port
DNS domain name
IP addresses of the DNS server
3 of 3
Subsequent nodes join the cluster.
You need the:
Cluster network ports and MTU size
Node management interface port, IP address,
network mask, and default gateway
For more detailed information on cluster setup and configuration,

enroll in the Clustered Data ONTAP Installation Workshop.

To complete the cluster setup wizard, you need the following information for subsequent nodes:
Cluster network ports. On a private, nonroutable cluster network, the wizard automatically assigns cluster
IP addresses.
The node management interface port, IP address, network mask, default gateway, and home port
The Normal Boot Sequence
1. The firmware loads the kernel from the boot device.
2. The kernel mounts the / root image from rootfs.img on
the boot device.
3. Init is loaded, and startup scripts run.
4. NVRAM kernel modules are loaded.
5. The /var partition on NVRAM is created and mounted
(restored from boot device if a backup copy exists).
6. The management gateway daemon (mgwd) is started.
7. The data module, the network module, and other
components are loaded.
8. The vol0 root volume is mounted from the local data
module.
9. The CLI is ready for use.
THE NORMAL BOOT SEQUENCE
System Setup Tool
The System Setup tool and simple instructions are included with every FAS2200 shipment.
SYSTEM SETUP TOOL

System Setup is a new utility tool that lets partners and end-user customers set up their NetApp FAS2220 or
FAS2240 system quickly and easily. System Setup takes the user through a series of simple steps to set up
and provision their FAS2200 series system. The System Setup tool is designed to improve the initial overall
customer experience.
System Setup Benefits
Set up your FAS2200 three times faster.
You dont need to be a storage expert.
Defaults take the guesswork out of the setup
process.
You get NetApp best practices for optimal
performance.
Deduplication, flexible volumes, auto grow,
and storage provisioning
SYSTEM SETUP BENEFITS

System Setup is big news for NetApp partners and customers alike. With System Setup, installation and
configuration of your FAS2200 series is completed three times faster than previously. You dont need to be a
storage expert either. Numerous IT generalists used System Setup and were able to set up and configure a
FAS2200 series in minutes.
For NetApp partners, this means more options. They can choose to continue to sell their own professional
services, knowing that System Setup will make installation much easier. Alternatively, NetApp partners can
choose not to sell installation services and allow their customers to do their own installation and setup.
System Setup is designed for a positive out-of-the-box experience. Default settings mean that the guesswork
is taken out of the setup process. These default settings also mean that the customer benefits by having
NetApp best practices for optimal performance of their new FAS2200 system. System Setup helps customers
set up efficiency technologies such as deduplication, flexible volumes, auto grow, and storage provisioning.
System Setup Installation Requirements
System Setup:
Runs on the following systems:
Windows XP
Windows 7
Windows Server 2008 R2 x64
Requires .NET Framework 3.5 SP1
Can configure FAS2200 systems running:
Clustered Data ONTAP 8.2
Data ONTAP 8.1 7-Mode
Data ONTAP 8.1.1 7-Mode
Data ONTAP 8.1.2 7-Mode
SYSTEM SETUP INSTALLATION REQUIREMENTS

Note that a few installation requirements apply to System Setup.
The utility is only supported on Windows platforms running Windows XP, Windows 7, or Windows Server
2008 R2 x64. System Setup also requires the .NET Framework, 3.5 SP1.
The systems to be configured should be running Data ONTAP 8.1, 8.1.1, or 8.1.2 in 7-Mode. System Setup
will configure the 10-GbE mezzanine card in FAS2220 systems with Data ONTAP 8.1.2.
More Information About System Setup
support.netapp.com
Download System Setup
Access documentation
fieldportal.netapp.com
Slides
Sales FAQs
MORE INFORMATION ABOUT SYSTEM SETUP

Download the System Setup utility tool and access documentation on the NetApp Support site at
support.netapp.com
NetApp employees and partners can also access sales tools, including slides and an FAQ, on the NetApp
Field Portal at fieldportal.netapp.com.
Network Time Protocol
Manually set the date, time, and time zone with
system date modify.
Kerberos is time-sensitive and typically requires the
Network Time Protocol (NTP).
NTP is disabled by default.
NTP enablement and disablement are cluster-wide.
The commands for verifying and monitoring NTP are:
system services ntp config show
system services ntp server show
NETWORK TIME PROTOCOL
7-Mode Transition Tool
Transition
Data ONTAP 7-Mode Clustered Data ONTAP
7-MODE TRANSITION TOOL

You can use the new Data ONTAP 7-Mode Transition Tool to migrate NAS workloads from Data ONTAP
operating in 7-Mode to clustered Data ONTAP 8.2. The transition tool is a CLI wizard and GUI that
simplifies and expedites complex transition tasks. The tool collects volume and IP address information from
the Data ONTAP 7-Mode environment and verifies the configuration. The tool then applies the configuration
to the Vserver in the cluster. The tool uses SnapMirror software to seed the cluster with FlexVol volumes, and
regular replication to keep the cluster current until the workload is cut over to the cluster. After all data
configuration information is staged on the target cluster, the tool initiates and manages a cut-over event to
transfer the Data ONTAP 7-Mode workload to the cluster.
Note that the transition tool supports source controllers that run Data ONTAP 7.3.3, Data ONTAP 7.3.7, Data
ONTAP 8.0.3 7-Mode, and Data ONTAP 8.1 7-Mode. Use third-party tools to move LUNs from a Data
ONTAP 7-Mode controller to the cluster.
To learn more about transitioning from Data ONTAP operating in 7-Mode to a cluster, see the NetApp
Transition Fundamentals web-based course.
Module Summary

should be able to:
Configure nodes and install the operating
system
Create a cluster
to the Expand a cluster two nodes
Set the dates, times, and time zones of the
nodes in the cluster
MODULE SUMMARY
Exercise
Module 2:
Installation and Configuration
Time Estimate: 30 minutes
EXERCISE
Module 3
Cluster Administration Basics
MODULE 3: CLUSTER ADMINISTRATION BASICS
3-1 Clustered Data ONTAP Administration: Cluster Administration Basics
Module Objectives
Describe and utilize the various tools to manage a
cluster
Determine which commands are available for a
command directory
Determine whether parameters are required or
optional for a command
Switch among privilege levels
Describe the Vserver administrative roles
Explore policies and job schedules
Discuss the enhanced node-locked licensing model
MODULE OBJECTIVES
Lesson 1
LESSON 1
Overview
You can manage resources within a cluster by
using the CLI or the GUI.
The CLI accesses the hierarchical command
structure.
You can access an entire cluster from a cluster
management or node management logical
interface (LIF).
A cluster management LIF can fail over to a
surviving node if its host node fails.
The three administrative privilege levels are
admin, advanced, and diagnostic.
OVERVIEW
The CLI and the GUI provide access to the same information, and you can use both to manage the same
resources within a cluster.
The hierarchical command structure consists of command directories and commands. A command directory
might contain commands, more command directories, or both. In this way, command directories resemble file
system directories and file structures.
Command directories provide groupings of similar commands. For example, all commands for storage-related
actions fall somewhere within the storage command directory. Within that directory are directories for disk
commands and aggregate commands. The command directories provide the context that enables you to
use similar commands for different objects. For example, you use create commands to create all objects
and resources and delete commands to remove objects and resources, but the commands are unique
because of the context (command directory) in which the commands are used. Therefore, storage
aggregate create is different from network interface create.
The cluster login is accessible from a cluster management logical interface (LIF). You can also log in to each
node by using the node management LIF for the node.
Shells
The cluster shell

Former name: ngsh (the ng shell)
Scope: the entire cluster
The node shell
A subset of the Data ONTAP 7G and Data
ONTAP 7-Mode commands
Scope: a single node at any one time
The system shell
A means of access to the BSD shell
Scope: a single node at any one time
SHELLS
The Cluster Shell
Secure Shell (SSH) is the default method for

nonconsole logins.
Remote Shell (RSH) is supported with Data
ONTAP 8.2 and later.
The root user is not permitted.
The admin user is predefined, with a
password that is chosen during cluster setup.
You can access the cluster shell through the
cluster management LIF or the node
management LIFs.
THE CLUSTER SHELL

The best way to manage a cluster is to use Secure Shell (SSH) to log in to the cluster shell with the cluster
management LIF IP address. If a node cannot communicate with the rest of the cluster, you can use the node
management LIF for that node. If you cannot use the node management LIF, you can use a remote
management device.
Management LIFs
The cluster management LIF:
Is a persistent LIF to use for SSH access
Is unique within the cluster
Is assigned to a data port
Can fail over and migrate among nodes
The node management LIF:
Is unique for a node
Is assigned to a data or node-mgmt port
Can only fail over or migrate to a port on the same
node
Can access the entire cluster
MANAGEMENT LIFS
Clustered Data ONTAP has one management virtual interface on each node that is called a node
management LIF. Node management LIFs do not fail over to other nodes.
Clustered Data ONTAP also includes a management LIF, the cluster management LIF, that has failover and
migration capabilities. Therefore, regardless of the state of each individual node (for example, if a node is
rebooting after an upgrade or is halted for hardware maintenance), a LIF address can always be used to
manage the cluster, and the current node location of that LIF is transparent.
Management LIFs
The Output of net int show
cluster1::> net int show
(network interface show)
Logical Status Network Current Current Is
Vserver Interface Admin/Oper Address/Mask Node Port Home
----------- ---------- ---------- ------------------ ------------- ------- ----
cluster1
cluster_mgmt up/up 192.168.239.20/24 cluster1-01 e0c true
cluster1-01
clus1 up/up 169.254.165.103/16 cluster1-01 e0a true
clus2 up/up 169.254.185.207/16 cluster1-01 e0b true
mgmt up/up 192.168.239.21/24 cluster1-01 e0c true
cluster1-02
clus2 up/up 169.254.126.156/16 cluster1-02 e0b true
mgmt up/up 192.168.239.22/24 cluster1-02 e0c true
vs1
vs1_lif1 up/up 192.168.239.74/24 cluster1-01 e0d true
vs1_lif2 up/up 192.168.239.75/24 cluster1-01 e0d false
The Node The Cluster This LIF has failed

Management LIF Management LIF over or migrated.
MANAGEMENT LIFS: THE OUTPUT OF NET INT SHOW

The two mgmt1 LIFs that are shown here are the node management LIFs. Each is associated with a node
virtual storage server (Vserver).
The cluster management LIF, cluster_mgmt in this example, is not associated with any one node Vserver
but with the administration Vserver, cluster1, which represents the entire physical cluster.
The Node Shell
You access the node shell from within the

cluster shell.
The scope of access is limited to one node at
a time, but you can access any node in the
cluster.
You can enter the node shell for an interactive
session or execute individual commands
directly from the cluster shell.
The command for accessing the node shell is
cluster1::> system node run.
THE NODE SHELL
system node run
A single command directly from the cluster shell:
cluster1::> system node run node cluster1-02
hostname
cluster1-02
An interactive session:
cluster1::> system node run node cluster1-02
Type 'exit' or 'Ctrl-D' to return to the CLI
cluster1-02> hostname
cluster1-02
SYSTEM NODE RUN

In these examples, the hostname command is invoked from the CLI of one node and executed on the other
node. In the first example, the command is invoked from the cluster shell. In the second example, the
administrator enters the node shell of the other node and then runs the command interactively.
The System Shell
The diag user can access the system shell from
within the cluster shell.
From any node, the diag user can access the
system shell on any other node.
To access the system shell, do the following:
1. Unlock the diag user and set the password:
cluster1::> security login unlock username
diag
cluster1::> sec log pass user diag
2. From the cluster shell, use the advanced command:
cluster1::*>system node systemshell
3. Can only be accessed by the "diag" user.
THE SYSTEM SHELL

You can use the system shell to access the BSD environment that the Data ONTAP operating system runs in.
You should access the system shell only under the supervision of NetApp technical support.
You can access the system shell only as the diag user and only from within the cluster shell. Root access to
the system shell is not available from Data ONTAP clusters.
Use the security login unlock -username diag cluster shell command to unlock the
diag user. Then use the security login password -username diag command to set the diag
user password.
OnCommand System Manager
Clustered Data ONTAP works with

OnCommand System Manager.
Start OnCommand System Manager:
1. Enter the IP address (or host name) of the
cluster management LIF.
2. Enter a user name and password.
The admin user is predefined.
ONCOMMAND SYSTEM MANAGER

NetApp System Manager 2.0R1 and later supports clustered Data ONTAP.
OnCommand System Manager Login Page
1of 3
ONCOMMAND SYSTEM MANAGER LOGIN PAGE: 1 OF 3

1. From the Windows Start menu, start OnCommand System Manager.
2 of 3

2. Click Add.
3. Enter the IP address for the cluster management LIF.
4. Click the More button.
5. Select Credentials.
6. Enter the user name admin and the password.
7. Click Add.
3 of 3

You should see your cluster appear on the Discover page.
OnCommand System Manager 3.0
ONCOMMAND SYSTEM MANAGER 3.0

Clustered Data ONTAP management tools have been updated for clustered Data ONTAP 8.2. System
Manager has been updated to version 3.0 to support new features such as SnapVault, CIFS Local users and
Groups, SMB 3.0 and the new licensing model. Clustered Data ONTAP 8.2 no longer supports Element
Manager (including ClusterView). Use System Manager (or the Data ONTAP CLI) to manage the cluster.
OnCommand Unified Manager
ONCOMMAND UNIFIED MANAGER

OnCommand Unified Manager provides automated monitoring, provisioning, and protection for NetApp
storage systems. You can script commands by using the CLI of the Unified Manager server software that runs
on a server separate from the cluster.
You can use Unified Manager to monitor tasks on nodes in a Data ONTAP cluster. Unified Manager cluster
monitoring, alerts, and reporting help you adopt clustered Data ONTAP scale-out technology. Unified
Manager helps you review objects such as clusters, Vservers, and large aggregates. In addition, the Unified
Manager Performance Advisor capability has been enhanced for clustered Data ONTAP. Performance
Advisor can track input/output operations per second, latency, and throughput for physical and logical cluster
objects.
Lesson 2
LESSON 2
Cluster Shell Features
1 of 2
Has a history buffer
Enables you to easily reissue commands
Enables you to retrieve commands and then
easily modify and reissue the commands
Provides context-sensitive help when you
press the question mark (?) key
Enables you to reduce the required amount of
typing and get context-sensitive assistance
when you press the Tab key
CLUSTER SHELL FEATURES: 1 OF 2

The cluster shell has features that are similar to the popular tcsh shell for UNIX machines, such as the ability
to pull previous commands out of a command history buffer and then edit and reissue those commands.
Editing of commands is similar to editing through tcsh or Emacs, with key combinations, such as Ctrl-A and
Ctrl-E, that move the cursor to the beginning and end of a command, respectively. The up and down arrows
enable you to cycle through the command history.
Simple online help is also available. You can press the question mark (?) key almost anywhere to get
contextual help. In many places, you can press the Tab key to complete a command or parameter, which
reduces the required amount of typing.
Cluster Shell Features
2 of 2
The cluster shell uses named parameters.
You can abbreviate a command directory,
command, or parameter to its shortest
unambiguous sequence of characters.
The search path enables you to run
commands out of context.
You can run queries with patterns and
wildcards.
CLUSTER SHELL FEATURES: 2 OF 2

The cluster shell uses named parameters for every command.
Every command directory, command, and parameter can be abbreviated to the shortest string of characters
that is unambiguous within the context. For example, from the top level, the storage aggregate show
command can be abbreviated to sto a s and the network interface show command to n i s.
Commands can be run out of context. If you are at the top level of the command hierarchy and enter disk
show, the shell runs the storage disk show command, because the shell can resolve the disk
command as being unique within the whole command hierarchy. Likewise, if you type disk and press Enter,
you are put into the storage disk command directory. This method works even if youre in an unrelated
command directory, such as the network interface directory.
The cluster shell supports queries and UNIX-style patterns and wildcards to enable you to match multiple
values of specific parameters. For example, assume that you use a naming convention for volumes such that
every volume that is owned by the Accounting department has the prefix acct_. You can show only those
volumes by using the volume show vserver * volume acct_* command (which shows all
volumes that begin with acct_ on all Vservers). To further limit your query to volumes that have more than
500 GB of data, you can run this command:
cluster1::> volume show vserver * -volume acct_* -used >500gb.
Pressing the ? Key at the Top Level
1 of 2
cluster1::> ?
up Go up one directory
cluster> Manage clusters
dashboard> Display dashboards
event> Manage system events
exit Quit the CLI session
history Show the history of commands for this CLI session
job> Manage jobs and job schedules
lun> List LUN (logical unit of block storage) commands
man Display the on-line manual pages
network> Manage physical and virtual network connections
qos> QoS settings
redo Execute a previous command
rows Show/Set the rows for this CLI session
run Run interactive or non-interactive commands in
the node shell
security> The security directory
set Display/Set CLI session settings
sis Manage volume efficiency
snapmirror> Manage SnapMirror
statistics> Display operational statistics
PRESSING THE ? KEY AT THE TOP LEVEL: 1 OF 2

Press the? key at the top level of the command hierarchy to show the command directories and commands
that are available at that top level.
Pressing the ? Key at the Top Level
2 of 2
storage> Manage physical storage, including disks,
aggregates, and failover
system> The system directory
top Go to the top-level directory
volume> Manage virtual storage, including volumes,
snapshots, and mirrors
vserver> Manage Vservers
PRESSING THE ? KEY AT THE TOP LEVEL: 2 OF 2
Press ? for Commands and Directories
1 of 2
cluster1::> cluster
cluster1::cluster> ?
contact-info Manage contact information for the
cluster.
create Create a cluster
ha Manage high-availability configuration
identity Manage the cluster's attributes,
including name and serial number
join Join an existing cluster using the
specified member's IP address
modify Modify cluster node membership attributes
peer Manage cluster peer relationships
setup Setup wizard
show Display cluster node members
statistics Display cluster statistics
PRESS ? FOR COMMANDS AND DIRECTORIES: 1 OF 2

You can also press the ? key to show the available commands and command directories at any other level of
the hierarchy.
Press ? for Commands and Directories
2 of 2
cluster1::cluster> statistics
cluster1::cluster statistics> ?
show Display cluster-wide statistics
PRESS ? FOR COMMANDS AND DIRECTORIES: 2 OF 2
Press ? and the Tab Key for Parameters
1 of 2
cluster1::> storage aggregate
cluster1::storage aggregate> modify ?

[ -aggregate] <aggregate name> Aggregate
[ -raidtype|-t {raid_dp|raid4} ] RAID Type
[ -maxraidsize|-s <integer> ] Max RAID Size
[ -hybrid-enabled {true|false} ] Hybrid Enabled
[ -snapshot-autodelete {on|off} ] Automatic Snapshot Deletion
[ -ha-policy {sfo|cfo} ] HA Policy
[ -free-space-realloc {on|off} ] Free Space Reallocation
[ -percent-snapshot-space <percent> ] Space Reserved for Snapshot
Copies
PRESS ? AND THE TAB KEY FOR PARAMETERS: 1 OF 2

You can press the ? key to show required and optional parameters and valid keyword values for parameters
that accept keywords.
Press ? and the Tab Key for Parameters
2 of 2
cluster1::storage aggregate> modify -aggr aggr1a -state ?
offline
online
restricted
cluster1::storage aggregate> modify -aggr aggr1a -state rest<TAB>
cluster1::storage aggregate> modify -aggr aggr1a -state restricted
PRESS ? AND THE TAB KEY FOR PARAMETERS: 2 OF 2

Press the Tab key to show directories, commands, and parameters that are available or to automatically
complete a command (or a portion of a command).
Changing Privilege Levels in the CLI
1 of 2
cluster1::> storage disk ?
assign Assign ownership of a disk to a system
fail Fail the file system disk
modify Modify disk attributes
option> Manage disk options
remove Remove a spare disk
removeowner Remove disk ownership
replace Initiate or stop replacing a file-system disk
set-led Turn on a disk's red LED for a number of minutes
show Display a list of disk drives and array LUNs
updatefirmware Update disk firmware
zerospares Zero non-zeroed spare disks
CHANGING PRIVILEGE LEVELS IN THE CLI: 1 OF 2

This slide shows the set adv command (an abbreviation of set -privilege advanced) in the
cluster shell. Note the options that are available for the security directory before you change the privilege
level (using admin privileges) and after you change the privilege level (to advanced privileges). With
advanced privileges, the certificate-related commands are available.
If the command prompt contains an asterisk (for example, cluster1::*>), you are not currently using the
admin privilege level.
Changing Privilege Levels in the CLI
2 of 2
cluster1::> set advanced
Warning: These advanced commands are potentially dangerous; use them

only when directed to do so by NetApp personnel.
Do you want to continue? {y|n}: y
cluster1::*> storage disk ?

assign Assign ownership of a disk to a system
fail Fail the file system disk
modify Modify disk attributes
option> Manage disk options
reassign *Change the default owner of all disks from one
node to another
remove Remove a spare disk
removeowner Remove disk ownership
replace Initiate or stop replacing a file-system disk
set-led Turn on a disk's red LED for a number of
minutes
show Display a list of disk drives and array LUNs
unfail *Unfail a broken disk
updatefirmware Update disk firmware
zerospares Zero non-zeroed spare disks
CHANGING PRIVILEGE LEVELS IN THE CLI: 2 OF 2

This slide shows the set adv command (an abbreviation of set -privilege advanced) in the
cluster shell. Note the options that are available for the security directory before you change the privilege
level (using admin privileges) and after you change the privilege level to advanced privileges. With advanced
privileges, the certificate-related commands are available.
If the command prompt contains an asterisk, you are currently using the advanced privilege level.
OnCommand System Manager
Dashboard Page
ONCOMMAND SYSTEM MANAGER DASHBOARD PAGE

This is the initial page that appears when you log in to a cluster in OnCommand System Manager. Its a
dashboard view of the system health of the entire cluster. The left pane contains the command directories and
commands.
Note the three tabs in the left pane:
Cluster: Cluster-wide central admin tasks (administrative Vserver)
Vservers: Vserver or virtual array management (data Vservers)
Nodes: Node and hardware focused tasks (node Vservers)
Storage
STORAGE
Notice the expanded Storage directory in the left pane.
Storage Aggregate
STORAGE AGGREGATE
Notice the Aggregates pane on the right.
Editing an Aggregate
EDITING AN AGGREGATE
If you right-click an aggregate and select Edit, the Edit Aggregate dialog box appears.
The Edit Aggregate Dialog Box
THE EDIT AGGREGATE DIALOG BOX

In the Edit Aggregate dialog box, you can change the RAID type and rename the aggregate.
Lesson 3
LESSON 3
Data Vserver-Scoped Roles
vsadmin
vsadmin-protocol
vsadmin-readonly
vsadmin-volume
cluster1::> security login role show vserver vs1
DATA VSERVER-SCOPED ROLES

Clustered Data ONTAP includes administrative access-control roles that can be used to subdivide
administration duties for Vserver administration tasks.
Custered Data ONTAP 8.1 and later supports the vsadmin role. The vsadmin role grants the data Vserver
administrator full administrative privileges for the Vserver. Additional roles include the vsadmin-protocol
role, the vsadmin-readonly role, and the vsadmin-volume role. Each of these roles provides a unique
Vserver administration privilege.
vsadmin
This role is the superuser role for a Vserver. A Vserver
administrator with this role has the following capabilities:
Manages its own user account, local password, and public
key
Manages volumes, quotas, qtrees, Snapshot copies,
FlexCache devices, and files
Manages LUNs
Configures protocols
Configures services
Monitors jobs
Monitors network connections and network interfaces
Monitors the health of a Vserver
DATA VSERVER-SCOPED ROLES: VSADMIN
vsadmin-protocol
A Vserver administrator with this role has the
following capabilities:
Configures services
Manages LUNs
Monitors network interfaces
DATA VSERVER-SCOPED ROLES: VSADMIN-PROTOCOL
vsadmin-readonly
Views volumes and LUNs
Views services and protocols
DATA VSERVER-SCOPED ROLES: VSADMIN-READONLY
vsadmin-volume
Manages volumes, quotas, qtrees, Snapshot
copies, FlexCache devices, and files
Manages LUNs
Configures services
DATA VSERVER-SCOPED ROLES: VSADMIN-VOLUME
Cluster-Scoped Roles
admin
readonly
none
cluster1::> security login role show vserver cluster1
CLUSTER-SCOPED ROLES
admin
Grants all possible capabilities
Is a cluster superuser
CLUSTER-SCOPED ROLES: ADMIN
readonly and None
A Cluster administrator with the role of
readonly can grant read-only capabilities.
A Cluster administrator with the role of None
cannot grant capabilities.
CLUSTER-SCOPED ROLES: READONLY AND NONE
Policy-Based Storage Services
Policies are a collection of rules created and

managed by the cluster or Vserver admin
Pre-defined or created to manage data
access
Policy examples:
Firewall and security
Export, quota, file and data
Snapshot copy and SnapMirror
QoS
POLICY-BASED STORAGE SERVICES

Examples of services that are policy based:
firewall
system health
snapmirror
volume efficiency
volume flexcache
volume quota
volume snapshot
vserver cifs group
vserver data
vserver export
vserver fpolicy
vserver security file-directory
qos policy-group
failover
Policy Example
policyA fwall_policy1
Rule1 criteria1 192.168.1.0/24 ssh
Rule2 criteria2 192.168.1.0/24 http
policyB fwall_policy2
Rule3 criteria3 Rule3 criteria3
Rule1 criteria1
property 192.168.21.0/24 ssh
property
Rule2 criteria2
property 192.168.22.0/24 ssh
property
propertyRule3 criteria3 property
192.169.23.0/24 ssh
property allow
property
property
Assign a policy to service or resource

A rule criteria within the policy will match the service or resource
The matching rules properties will apply to the service or resource
The example is a firewall to allow or deny access to a protocol for
specific IP address ranges
POLICY EXAMPLE
Job Schedules
Job schedules can be used:
Globally (by all Vservers)
For functions that can be automated
SnapShot, SnapMirror, and SnapVault, for example
Note the following job schedule syntax:
@:00,:05,:10...:55 means every five minutes on the
five-minute marks
@2 means daily at 2:00 a.m.
@0:10 means daily at 12:10 a.m.
@:05 means hourly at five minutes after the hour
JOB SCHEDULES
The job schedule show Command
cluster1::> job schedule show
Name Type Description
----------- --------- ------------------------------------------------
5min cron @:00,:05,:10,:15,:20,:25,:30,:35,:40,:45,:50,:55
8hour cron @2:15,10:15,18:15
daily cron @0:10
hourly cron @:05
weekly cron Sun@0:15
THE JOB SCHEDULE SHOW COMMAND
Revised Licensing Model
Proof-of-sale is recorded as a
license entitlement record
License keys are now also linked
to the controller serial number
License keys are locked to nodes
License keys have been
lengthened to 28 characters
Nondisruptive upgrades from
Data ONTAP 8.1 to 8.2 do not
require new license keys
REVISED LICENSING MODEL

Because of the change in license key format in Data ONTAP 8.2, if you upgrade a controller from Data
ONTAP 8.1 to Data ONTAP 8.2 and that system fails or is repurposed, license keys for Data ONTAP 8.2
must be issued.
Next, if a controller that runs Data ONTAP 8.2 fails and must be replaced, or if you upgrade the controller,
the existing license keys will not work with the replacement controller. New license keys for the replacement
controller are issued during the hardware order for upgrades or during the transfer process for a head swap.
Finally, if a controller that was originally licensed to run Data ONTAP 8.2 is reverted to Data ONTAP 8.1, it
will require new Data ONTAP 8.1 license keys.
Implications of License Key Format Changes
When you upgrade from Data

ONTAP 8.1 to Data ONTAP 8.2, you
need new license keys that can be
used if the system fails or is
repurposed.
Replacement systems for failed
controllers that run Data ONTAP 8.2
need new license keys.
Reverting from Data ONTAP 8.2
requires new Data ONTAP 8.1
license keys.
IMPLICATIONS OF LICENSE KEY FORMAT CHANGES
License Commands
cluster1::> license ?
(system license)
add Add one or more licenses
clean-up Remove unnecessary licenses
delete Delete a license
show Display licenses
status> Display license status
LICENSE COMMANDS
System Manager License Page
SYSTEM MANAGER LICENSE PAGE

The License Page in System Manager is located within the Cluster tab. The upper pane shows licensed
features. The lower pane shows licensed nodes and expiration date. It also shows whether a license is a valid
legacy license from a previous version of clustered Data ONTAP.
Module Summary
Now that you have completed this module, you should
be able to:
Describe and utilize the various tools to manage a
cluster
Determine which commands are available for a
command directory
Determine whether parameters are required or
optional for a command
Switch among privilege levels
Describe the Vserver administrative roles
Explore policies and job schedules
Discuss the enhanced node-locked licensing model
MODULE SUMMARY
Exercise
Module 3: Cluster Administration
Basics
EXERCISE
Module 4
Architecture
MODULE 4: ARCHITECTURE
4-1 Clustered Data ONTAP Administration: Architecture
Module Objectives

Show the end-to-end path of a file write
request through a cluster
Answer questions about replicated database
(RDB) concepts
Identify the differences between a vol0 root
volume and a data virtual storage server
(Vserver) root volume
MODULE OBJECTIVES
Lesson 1
LESSON 1
Components
Three major software components on every

node:
The network module
The data module
The SCSI module
Other key software components on every
node:
The cluster session manager (CSM)
The replicated database units (RDB)
COMPONENTS
The modules refer to separate software state machines that are accessed only by well defined APIs. Every
node contains a network module, a SCSI module, and a data module. Any network or SCSI module in the
cluster can talk to any data module in the cluster.
The network module and the SCSI module translate client requests into Spin Network Protocol (SpinNP)
requests and vice versa. The data module, which contains the WAFL (Write Anywhere File Layout) file
system, manages SpinNP requests. The cluster session manager (CSM) is the SpinNP layer between the
network, SCSI, and data modules. The SpinNP protocol is another form of RPC interface. It is used as the
primary intranode traffic mechanism for file operations among network, SCSI, and data modules.
The members of each replicated database (RDB) unit on every node in the cluster are in constant
communication with each other to remain synchronized. The RDB communication is like the heartbeat of
each node. If the heartbeat cannot be detected by the other members of the unit, the unit corrects itself in a
manner that is discussed later in this course. The four RDB units on each node are the blocks configuration
and Operations Manager (BCOM), the volume location database (VLDB), VifMgr, and management.
Single Node Components (Illustrated)
Node
Network and
Client Access (Data) SCSI
modules
Management M-Host
CSM Cluster Traffic
Data
module
RDB Units:
Mgwd
VLDB
Data Vserver
VifMgr
Root Volume
BCOM
Vol0
Root
Vol1
Vol2
SINGLE NODE COMPONENTS (ILLUSTRATED)
The Network Module
Manages networking, NFS, and CIFS

Speaks:
TCP/IP and UDP/IP
NFS and CIFS
SpinNP
THE NETWORK MODULE
The SCSI Module
Manages networking, FC, Fibre Channel over

Ethernet (FCoE), and iSCSI
Speaks:
FC
SCSI
SpinNP
TCP/IP
THE SCSI MODULE
The Data Module
Manages the WAFL (Write Anywhere File

Layout) file system, RAID, and storage
Speaks:
SpinNP
FC and SAS to disk and tape devices
THE DATA MODULE
The CSM
Provides a communication mechanism

between any network or SCSI module and
any data module
Provides a reliable transport for SpinNP traffic
Is used regardless of whether the network or
SCSI module and the data module are on the
same node or on different nodes
THE CSM
The Path of a Local Write Request
Node1 Node2
Network and Network and
Requests SCSI SCSI
Responses
modules modules
NAS and SAN
Clients
CSM CSM
Data Data
module module
Vol0 Vol0
Root Root Vol3
Vol1 Vol4
Vol 1
Vol2
THE PATH OF A LOCAL WRITE REQUEST

A NAS or SAN client sends a write request to a data logical interface (LIF). The network module (NAS) or
SCSI module (SAN) that is currently associated with that LIF translates the NFS or CIFS (NAS), FC, FCoE,
or iSCSI (SAN) request to a SpinNP request. The SpinNP request goes through the CSM to the local data
module. The data module sends the data to nonvolatile RAM (NVRAM) and to the disks. The response works
its way back to the client.
The Path of a Remote Write Request
Node1 Node2
Network and Network and
Requests SCSI SCSI
Responses
modules modules
NAS and SAN
Clients
CSM CSM
Data Data
module module
Vol0 Vol0
Root Root Vol3
Vol1 Vol4
Vol 1
Vol2
THE PATH OF A REMOTE WRITE REQUEST

A NAS or SAN client sends a write request to a data LIF. The network module or SCSI module that is
currently associated with that LIF translates the NFS or CIFS, FC, FCoE, or iSCSI request to a SpinNP
request. The SpinNP request goes through the CSM to the remote data module by means of the remote CSM.
The data module sends the data to NVRAM and to the disks. The response works its way back to the client.
Clustered Data ONTAP Modules
NAS
Network
The network module:
WAFL
Module
SAN
RAID Is called the N-blade
Storage
Module N
V Provides NAS protocols
R
The SCSI module:
A
Network M
WAFL
Module
SAN
RAID Is called the SCSI-blade
Storage
Module
Provides SAN protocols
Network
The data module:
WAFL
Module
SAN
RAID Is called the D-blade
Storage
N
Module
V Provides storage access
R
A
to shelves (WAFL file
Network
Module M
WAFL system, RAID
RAID
SAN
Module Storage subsystems, and storage
shelves subsystems)
CLUSTERED DATA ONTAP MODULES
Data ONTAP Architecture
CSM
Cluster Traffic
Network and SCSI Data module

modules
Network Protocols WAFL RAID Storage
Clients
To HA partner
NVRAM
Physical
Memory
Management
DATA ONTAP ARCHITECTURE
The Vol0 Volume
Contains data for managing the node and

cluster:
Is used for RDB databases and log files
Doesnt contain user or client data
Cannot be accessed by NAS or SAN clients
Exists on every nodeone vol0 per node
Must not be confused with the root volume of
a data Vserver
Cannot be mirrored, moved, or backed up
Can be re-created after a disaster
THE VOL0 VOLUME

The vol0 volume of a node is analogous to the root volume of a Data ONTAP 7G operating system. The vol0
volume contains the data that is needed for the node to function.
The vol0 volume does not contain any user data, nor is it part of the namespace of a Vserver. The vol0
volume resides permanently on the initial aggregate that is created when each node is initialized.
The vol0 volume is not protected by mirror relationships or tape backups, which is valid. Although vol0 is an
important volume (a node cannot boot without its vol0 volume), the data that is contained on vol0 is largely
re-creatable. If the data is lost, the log files are indeed gone. But because the RDB data is replicated on every
node in the cluster, that data can be automatically re-created on this node.
Data Vservers
1 of 2
Formerly known as cluster Vservers
Are virtual entities within a cluster
Can coexist with other cluster data Vservers
in the same cluster
Are independent of nodes
Are independent of aggregates
Contain all the volumes of their namespaces
DATA VSERVERS: 1 OF 2
Think of a cluster as a group of hardware elements (nodes, disk shelves, and more). A data Vserver is a
logical piece of that cluster, but a Vserver is not a subset or partitioning of the nodes. A Vserver is more
flexible and dynamic. Every Vserver can use all the hardware in the cluster, and all at the same time.
Example: A storage provider has one cluster and two customers: ABC Company and XYZ Company. A
Vserver can be created for each company. The attributes that are related to specific Vservers (volumes, LIFs,
mirror relationships, and others) can be managed separately, while the same hardware resources can be used
for both. One company can have its own NFS server, while the other can have its own NFS, CIFS, and iSCSI
servers.
Data Vservers
2 of 2
Represent unique namespaces
Can and should have multiple data logical
interfaces (LIFs), each of which is associated
with one Vserver
Can and do have multiple volumes, each of
which is associated with one Vserver
DATA VSERVERS: 2 OF 2
A one-to-many relationship exists between a Vserver and its volumes. The same is true for a Vserver and its
data LIFs. Data Vservers can have many volumes and many data LIFs, but those volumes and LIFs are
associated only with this one data Vserver.
Building a Namespace with Volumes and
Junctions
R A Data ONTAP Cluster
A B C F
D E G H
A
B R
C G H E F D
R is the root of a data Vserver.

A, B, C, and F are mounted to R through junctions.
D and E are mounted to C through junctions.
G and H are mounted to F through junctions.
BUILDING A NAMESPACE WITH VOLUMES AND JUNCTIONS

These nine volumes are mounted together through junctions. All volumes must have a junction path
(mountpoint) to be accessible within the Vservers namespace.
Volume R is the root volume of a Vserver. Volumes A, B, C, and F are mounted to R through junctions.
Volumes D and E are mounted to C through junctions. Likewise, volumes G and H are mounted to F.
Every Vserver has its own root volume, and all nonroot volumes are created within a Vserver. All nonroot
volumes are mounted into the namespace, relative to the Vserver root.
In this example if volume C goes offline, clients who are mounted to R or C will not be able to access D or E.
Clients who are mounted directly to D or E will have uninterrupted access to D or E.
Vservers, Namespaces, and Volumes
PopCo PetCo RonCo QuekCo
Namespace Namespace Namespace Namespace

Vserver Vserver Vserver Vserver
Root Root Root Root
Volume Volume
Volume
Volume
Volume
Volume
VSERVERS, NAMESPACES, AND VOLUMES

NOTE: This slide is a representation of logical concepts and is not meant to show any physical relationships.
For example, all of the objects that are shown as part of a Vserver are not necessarily on the same physical
node of the cluster. In fact, that situation is unlikely.
This slide shows four distinct Vservers and namespaces. Although the hardware is not shown, these four
Vservers might reside in a single cluster. These namespaces are not separate entities of the Vservers but are
shown merely to indicate that each Vserver has a namespace. The volumes, however, are separate entities.
Each volume is associated with one Vserver. Each Vserver has one root volume, and some Vservers have
additional volumes. Although a Vserver might have only one volume (the Vservers root volume), in real life,
it is more likely that a Vserver consists of multiple volumes, possibly thousands. Typically, a new volume is
created for every distinct area of storage. For example, every department and employee might have volume
separate volume in a Vserver.
Namespaces
A namespace is the file system of a data

Vserver.
A namespace consists of many volumes.
A namespace is independent of the
namespaces of other data Vservers.
The root of the namespace is the cluster
data Vserver root volume.
A client mount or mapping can be to the data
Vserver root volume or to a point further into
the tree.
NAMESPACES
A namespace is a file system. A namespace is the external, client-facing representation of a Vserver. A
namespace consists of volumes that are joined together through junctions. Each Vserver has one namespace,
and the volumes in one Vserver cannot be seen by clients that are accessing the namespace of another
Vserver.
The Data Vserver Root Volume
Exists on each data Vserverone per

data Vserver
Is the root of the data Vserver namespace
Is a normal flexible volume
Contains junctions
Can be moved, copied, and backed up
Can have Snapshot copies
Is usually mirrored
THE DATA VSERVER ROOT VOLUME

Each Vserver has one namespace and, therefore, one root volume. This volume is separate from the vol0
volume of each node.
Lesson 2
LESSON 2
The RDB
The RDB is the key to maintaining high-
performance consistency in a distributed
environment.
The RDB maintains data that supports the cluster,
not the user data in the namespace.
Operations are transactional (atomic): entire
transactions are either committed or rolled back.
Four RDB units exist: the volume location
database (VLDB), management, VifMgr, and
blocks configuration and operations manager
(BCOM).
THE RDB
The RDB units do not contain user data. The RDB units contain data that helps to manage the cluster. These
databases are replicated; that is, each node has its own copy of the database, and that database is always
synchronized with the databases on the other nodes in the cluster. RDB database reads are performed locally
on each node, but an RDB write is performed to one master RDB database, and then those changes are
replicated to the other databases throughout the cluster. When reads of an RDB database are performed, those
reads can be fulfilled locally without the need to send requests over the cluster interconnects.
The RDB is transactional in that the RDB guarantees that when data is written to a database, either it all gets
written successfully or it all gets rolled back. No partial or inconsistent database writes are committed.
Four RDB units (the VLDB, management, VifMgr, and BCOM) exist in every cluster, which means that four
RDB unit databases exist on every node in the cluster.
Management Gateway
Is also known as the M-host

Enables management of the cluster from any
node
Provides the CLI
Runs as mgwd (the management gateway
daemon) on every node
Stores its data in the management RDB unit
MANAGEMENT GATEWAY
The management RDB unit contains information that is needed by the management gateway daemon (mgwd)
process on each node. The kind of management data that is stored in the RDB is written infrequently and read
frequently. The management process on a given node can query the other nodes at run time to retrieve a great
deal of information, but some information is stored locally on each node, in the management RDB database.
Volume Location Database
The VLDB is:

One of the RDB units
An index of which aggregate owns a volume
An index of which node hosts an aggregate
VLDB content is cached in memory on each
node for instant access by each network and
SCSI module to speed up the lookup process
during data access by clients.
VOLUME LOCATION DATABASE

Although each RDB unit consists of a process and a database on each node in the cluster, an RDB unit is
considered a single entity. One of the RDB units is the VLDB.
The VLDB tracks where the volumes and aggregates are.
Because the VLDB is potentially referenced (read) frequently for client requests, the VLDB content is cached
in memory on each node so that the network and SCSI modules can avoid RDB lookups during client
requests.
VIF Manager
Runs as vifmgr
Stores and monitors LIF configuration
Stores and administers LIF failover policies
VIF MANAGER
The VifMgr is responsible for creating and monitoring NFS, CIFS, and iSCSI LIFs. It also handles automatic
NAS LIF failover and manual migration of NAS LIFs to other network ports and nodes.
Blocks Configuration and Operations
Management
Runs as bcomd
Stores LUN map definitions
Stores initiator groups (igroups)
BLOCKS CONFIGURATION AND OPERATIONS MANAGEMENT

The BCOM RDB unit hosts the SAN ring that contains the replicated configuration information data for block
data access, including LUN maps and initiator groups( igroups).
The RDB: Details
1 of 2
Each RDB unit has its own replication ring.
For each of the units, one node is the master
and the other nodes are secondaries.
The master node for each unit might be
different than the master nodes for the other
units.
Writes for an RDB unit go to its master and
are then propagated to the secondaries
through the cluster interconnect.
THE RDB: DETAILS: 1 OF 2

Each RDB unit has its own ring. An RDB ring is the total of all RDB units of each type across the cluster. For
example, in an eight-node cluster, the eight vldb units make up the vldb ring. Each of the four RDB rings
elects a master. The master is considered the "official" copy of the database in case of discrepancies.
If n is the number of nodes in the cluster, each unit or ring consists of n databases and n processes. At any
given time, one of those databases is designated as the master, and the others are designated as secondary
databases. Each RDB units ring is independent of the other RDB units. For example, if node X has the
master database for the VLDB unit, node Y might have the master for the VifMgr unit, and node Z might
have the master for the management unit and the BCOM unit.
The master of a given unit can change. For example, when the node that is the master for the management
unit is booted, a new management master must be elected by the remaining members of the management unit.
Note that a secondary can become a master and a master can become a secondary. Nothing is special about
the database itself; the database that is designated as the master is the role of the process that manages the
database (master versus secondary).
When data must be written to a unit, the data is written to the database on the master, and then the master
immediately replicates the changes to the secondary databases on the other nodes. If a change cannot be
replicated to a specific secondary, the entire change is rolled back everywhere, which is what no partial
writes means. Either all databases of an RDB unit get the change, or none gets the change.
The RDB: Details
2 of 2
An RDB unit is considered to be healthy only
when it is in quorum (when a master can be
elected).
In quorum means that a simple majority of
nodes are communicating with each other.
When the quorum is lost or regained, the master
might change.
If a master has communication issues, a new
master is elected by the members of the unit.
One node has a tie-breaking ability (epsilon) for
all RDB units.
THE RDB: DETAILS: 2 OF 2

RDB Terminology and Definitions
A master can be elected only when a quorum of member nodes is available (and healthy) for a particular RDB
unit. Each member votes for the node that it thinks should be the master for this RDB unit. One node in the
cluster has a special tie-breaking ability called epsilon. Unlike the master, which might be different for each
RDB unit, epsilon is a single node that applies to all RDB units.
Quorum means that a simple majority of nodes are healthy enough to elect a master for the unit. The epsilon
power is used only in the case of a voting tie. If a simple majority does not exist, the epsilon node (process)
chooses the master for a given RDB unit.
When cluster communication is interruptedfor example, because of a booting or a cluster interconnect
hiccup that lasts for a few secondsa unit goes out of quorum. When the cluster communication is restored,
the unit comes back into quorum automatically.
RDB Databases
node1
node2 mgwd VLDB VifMgr BCOM node3
mgwd VLDB VifMgr BCOM mgwd VLDB VifMgr BCOM

node4
mgwd VLDB VifMgr BCOM
RDB DATABASES
This slide shows a four-node cluster. The four databases that are shown for each node are the four RDB units
(management, VLDB, VifMgr, and BCOM). Each unit consists of four distributed databases. Each node has
one local database for each RDB unit.
The databases that are shown on this slide with dark borders are the masters. Note that the master of any
particular RDB unit is independent of the master of the other RDB units.
The node that is shown on this slide with a dark border has epsilon (the tie-breaking ability).
On each node, all the RDB databases are stored in the vol0 volume.
Quorum
1 of 2
A quorum is a simple majority of connected, healthy, and
eligible nodes.
Two RDB quorum concepts exist: a cluster-wide quorum
and an individual RDB unit that is in or out of quorum.
RDB units never go out of quorum as a whole; only local
units (processes) do.
When an RDB unit goes out of quorum, reads from the
RDB unit can still occur, but changes to the RDB unit
cannot.
Example: If the VLDB goes out of quorum, during the brief
time that the database is out, no volumes can be created,
deleted, or moved; however, access to the volumes from
clients is not affected.
QUORUM: 1 OF 2
A master can be elected only when a majority of local RDB units are connected and healthy for a particular
RDB unit on an eligible node. A master is elected when each local unit agrees on the first reachable healthy
node in the RDB site list. A healthy node is one that is connected, can communicate with the other nodes,
has CPU cycles, and has reasonable I/O.
The master of a given unit can change. For example, when the node that is the master for the management
unit is booted, a new management master must be elected by the remaining members of the management unit.
A local unit goes out of quorum when cluster communication is interrupted for a few seconds, for example,
because of a booting or a cluster interconnect hiccup that lasts for a few seconds. Because the RDB units
always work to monitor and maintain a good state, the local unit comes back in quorum automatically. When
a local unit goes out of quorum and then comes back into quorum, the RDB unit is synchronized again. Note
that the VLDB process on a node might go out of quorum although the VifMgr process on that same node has
no problem.
When a unit goes out of quorum, reads from that unit can be performed, but writes to that unit cannot. That
restriction is enforced so that no changes to that unit happen during the time that a master is not agreed upon.
In addition to the example above, if the VifMgr goes out of quorum, access to LIFs is not affected, but no LIF
failover can occur.
Quorum
2 of 2
The members of each RDB unit vote to
determine which node will be their master;
each unit elects its own master.
Each master might change when a local unit
goes out of and into quorum.
Before you take a node down for an extended
period of time, you should mark it as ineligible
(so the node doesnt factor into quorum):
cluster1::> system node modify node
<node> -eligibility false
QUORUM: 2 OF 2
Marking a node as ineligible (by using the cluster modify command) means that the node no longer
affects RDB quorum or voting. If you mark the epsilon node as ineligible, epsilon is automatically given to
another node.
The Epsilon Node
When half of the nodes in a cluster are

isolated from the other half, no simple majority
exists.
NOTE: This situation is rare.
One node has a weighted vote (epsilon).
The epsilon node is epsilon for the entire
cluster, not only for individual RDB units (such
as the masters).
THE EPSILON NODE

One node in the cluster has a special voting weight called epsilon. Unlike the masters of each RDB unit,
which might be different for each unit, the epsilon node is the same for all RDB units. This epsilon vote is
used only in the case of an even partitioning of a cluster, where, for example, four nodes of an eight-node
cluster cannot talk to the other four nodes. This situation is rare, but in this situation, a simple majority does
not exist, and the epsilon node sways the vote for the masters of the RDB units.
Which Cluster Is In Quorum?
4+
3
2+
2
WHICH CLUSTER IS IN QUORUM?
Two-Node Clusters
Two-node clusters are a special case:

No majority exists in the event of a cluster
interconnect partition or during a failover
situation.
The RDB manages this case under the
covers but must be told that this cluster
contains only two nodes.
cluster1::> cluster ha modify configured true
See TR3450 for more information
TWO-NODE CLUSTERS
From Ron Kownacki, author of the RDB:
Basically, quorum majority doesnt work well when down to two nodes and theres a failure, so RDB is
essentially locking the fact that quorum is no longer being used and enabling a single replica to be artificially
writable during that outage.
The reason we require a quorum (a majority) is so that all committed data is durable: if you successfully
write to a majority, you know that any future majority will contain at least one instance that has seen the
change, so the update is durable. If we didnt always require a majority, we could silently lose committed
data. So in two nodes, the node with epsilon is a majority and the other is a minorityso you would only
have one-directional failover (need the majority). So epsilon gives you a way to get majorities where you
normally wouldnt have them, but it only gives unidirectional failover because its static.
In two-node (high-availability mode), we try to get bidirectional failover. To do this, we remove the
configuration epsilon and make both nodes equaland form majorities artificially in the failover cases. So
quorum is two nodes available out of the total of two nodes in the cluster (no epsilon involved), but if theres
a failover, you artificially designate the survivor as the majority (and lock that fact). However, that means you
cant fail over the other way until both nodes are available, they sync up, and drop the lockotherwise you
would be discarding data.
Putting It All Together
Node
Network and
Client Access (Data) SCSI
modules
Management M-Host
CSM Cluster Traffic
Data
module
RDB Units:
Mgwd
VLDB
VifMgr
Data Vserver BCOM
Vol0
Root Volume Root
Vol1
Vol2
PUTTING IT ALL TOGETHER
Module Summary

should be able to:
Show the end-to-end path of a file write
request through a cluster
Answer questions about replicated database
(RDB) concepts
Identify the differences between a vol0 root
volume and a data virtual storage server
(Vserver) root volume
MODULE SUMMARY
Exercise
Module 4: Architecture
Time Estimate: 15 Minutes
EXERCISE
Module 5
Physical Data Storage
MODULE 5: PHYSICAL DATA STORAGE
5-1 Clustered Data ONTAP Administration: Physical Data Storage
Module Objectives
Draw the connections from a high-availability (HA) pair of
controllers to the disk shelves
Create aggregates
List the steps that are required to enable storage failover
(SFO)
Explain and enable two-node HA mode for two-node
clusters
Create a flash pool
MODULE OBJECTIVES
Lesson 1
LESSON 1
The FAS3270 System
Hardware Diagram
HA c0b HA c0a FC 0c Management e0M Data Cluster X1107A Dual-
e1b e1a Port 10-GbE Card
Data e0a e2b
X1107A
LINK/ACT
LINK/ACT
1 Chelsio
Communications
c0a 0c e0a
X1107A
LINK/ACT
LINK/ACT
0a 0b 2 Chelsio
Communications
LNK LNK
c0b 0d e0b
X1139A
10GbEE CNA
3 5
SAN LAN
SAN LAN
PORT 1
PORT 2
4 6
SAS 0a SAS 0b Data e3a and e3b FC 0d Data e0b ACP* Console Cluster
e2a
X1139A Dual-
Port UTA Card
*alternate control path
THE FAS3270 SYSTEM: HARDWARE DIAGRAM

The example on this slide shows a FAS3270 system controller. Use this example as a reference, but
remember that as new cards are supported, some of this information might change.
The FAS62x0 System
Hardware Diagram
3
0 e0a e0b
e0c e0d e0e e0f 0a 0b 0c 0d
LINK LINK LINK LINK

LINK LINK LINK LINK
7 13
8 14
9 15
10 16
Management: e0M 1Gb Data: e0a and e0b 10-Gb Cluster: e0c, e0e 10-Gb Data: e0d, e0f FC: 0a, 0b, 0c, and 0d Console
THE FAS62X0 SYSTEM: HARDWARE DIAGRAM

This example shows a FAS62x0 series controller. Use this example as a reference, but remember that as new
cards are supported, some of this information might change.
A Typical Disk Shelf with SAS Connection
ACP ACP SAS SAS ACP ACP SAS SAS
Circle Square Circle Square Circle Square Circle Square
IOM6 IOM6
LNK LNK LNK LNK
A
IOM6 B IOM6
DC AC DC AC
1
x2 x2
2 2 2
IOM6 IOM6
A B
A TYPICAL DISK SHELF WITH SAS CONNECTION

This slide shows the back of a typical disk shelf that has a SAS connection to the host storage controller. Each
of the IOM6 modules uses the same orientation. The SAS and alternate control path (ACP) connections use a
circle and square symbol to make cabling connections within a series easier. Each SAS port with a circle
symbol connects to the next shelfs SAS port with a square symbol. ACP connections work exactly the same
way.
An HA Pair
SAS Storage Configuration
X1107A
X1107A
LINK/ACT
LINK/ACT
LINK/ACT
LINK/ACT
1 Chelsio Chelsio
Communications 1 Communications
c0a 0c e0a c0a 0c e0a
X1107A
X1107A
LINK/ACT
LINK/ACT
LINK/ACT
LINK/ACT
0a 0b 2 Chelsio
Communications 0a 0b 2 Chelsio
Communications
LNK LNK LNK LNK
c0b 0d e0b c0b 0d e0b
Controller 1 Controller 2
D
A
D
A
B
X2065A
X1139A
X2065A
10GbEE CNA
3 5 3 5
SAN LAN
SAN LAN
PORT 1
PORT 2
4 6 4 6
IOM6 IOM6 IOM6 IOM6

LNK LNK LNK LNK
A LNK LNK
A LNK LNK
Shelf 1 Shelf 1
IOM6 B IOM6 IOM6 B IOM6
DC AC DC AC DC AC DC AC
1 1
x2 x2 x2 x2
2 2 2 2 2 2
IOM6 IOM6 IOM6 IOM6

LNK LNK LNK LNK
A LNK LNK
A LNK LNK
Shelf 2 Shelf 2
IOM6 B IOM6 IOM6 B IOM6
DC AC DC AC DC AC DC AC
1 1
x2 x2 x2 x2
2 2 2 2 2 2
Stack 1 Stack 2
Starting with shelf ID 10 Starting with shelf ID 20
SAS
ACP
VTIC
AN HA PAIR: SAS STORAGE CONFIGURATION

The diagram on this slide combines SAS and ACP shelf cabling and the high-availability (HA) cabling for
controller 1 and controller 2 of an HA pair. For a complete course in cluster cabling, see the Data ONTAP 8
Cabling course (STRHW-WBT-DOTCABL).
HA Interconnect Links
InfiniBand links connect the two nodes of each
HA pair.
InfiniBand for FAS6000 and V6000 series
Dedicated 10-Gb links for FAS3200 and V3200 series
The HA links are used to mirror nonvolatile RAM
(NVRAM).
The HA links provide a channel for certain types of
communication traffic between the nodes in a pair:
Failover
Disk firmware
Heartbeats
Version information
HA INTERCONNECT LINKS
InfiniBand links connect the two nodes of each HA pair for all models except the FAS and V-Series 32x0
series model controllers. FAS and V-Series 32x0 model controllers use a dedicated 10-GbE link, internal or
external, depending on the model and enclosure. Visit the NetApp Support site to see the appropriate
hardware configuration guide for your model storage controller.
The types of traffic that flow over the HA interconnect links are:
Failover: The directives are related to performing storage failover (SFO) between the two nodes,
regardless of whether the failover is:
Negotiated (planned and in response to an administrator request)
Not negotiated (unplanned and in response to an improper system shutdown or booting)
Disk firmware: Nodes in an HA pair coordinate the update of disk firmware. While one node is updating
the firmware, the other node must not perform any I/O to that disk.
Heartbeats: Regular messages demonstrate availability.
Version information: The two nodes in an HA pair must be kept at the same major and minor revision
levels for all software components.
Disks
Every disk, volume, and aggregate in an HA pair is
assigned a home and is owned by that node.
Designated disks in the HA pair serve as mailbox disks:
A mailbox disk provides persistent storage for information
about the SFO state, including some specific replicated
database (RDB) data when in two-node HA mode.
Each node of an HA pair designates two disks in the first
RAID group in the root aggregate as mailbox disks .
Attempts are made to write SFO state information to all
mailbox disks for configuration and status changes.
Quorum techniques are used to guarantee that at least three
of the four mailbox disks must be available for SFO.
DISKS
Each node of an HA pair designates two disks in the first RAID group in the root aggregate as the mailbox
disks. The first mailbox disk is always the first data disk in RAID group RG0. The second mailbox disk is
always the first parity disk in RG0. The mroot disks are generally the mailbox disks.
Each disk, and therefore each aggregate and volume that is built upon the disk, can be owned by one of the
two nodes in the HA pair at any given time. This form of software ownership is made persistent by writing
the information onto the disk itself. The ability to write disk ownership information is protected by the use of
persistent reservations. Persistent reservations can be removed from disks by power-cycling the shelves or by
selecting maintenance mode while in boot mode and then issuing manual commands. If the node that owns
the disks is running in normal mode, the node reasserts its persistent reservations every 30 seconds. Changes
in disk ownership are managed automatically by normal SFO operations, although commands exist to
manipulate disk ownership manually if necessary.
Each node in an HA pair can perform reads from any disk to which the node is connected, even if the disk
isnt that disks owner; however, only the node that is marked as that disks current owner is allowed to write
to that disk.
Disk Names
The system assigns the disk ID automatically based on
the node name, slot and port number, and either the
loop ID (FC-AL) or the shelf ID and bay number (SAS).
cluster1::> disk show -instance
Disk: cluster1-01:0c.18
Container Type: aggregate
Owner/Home: cluster1-01 / cluster1-01
...
FC-AL Disk ID = <node>:<slot><port>.<loopID>
DISK NAMES
Disks are numbered in all storage systems. Disk numbering enables you to:
Interpret messages displayed on your screen, such as command output or error messages
Quickly locate a disk that is associated with a displayed message
Disks are numbered based on a combination of their node name, slot number, and port number, and either the
loop ID for FC-AL-attached shelves or the shelf ID and bay number for SAS-attached shelves.
Disk Names: Slot and Port
The slot and port designate where an adapter is located
on the host storage controller.
3
6 0c
0 e0a e0b
e0c e0d e0e e0f 0a 0b 0c 0d
LINK LINK LINK LINK

LINK LINK LINK LINK
7 13
8 14
9 15
10 16
FAS62x0
DISK NAMES: SLOT AND PORT
Disk Names: Shelf ID and Bay (SAS)
The shelf ID and bay designate the specific shelf and
bay number where the disk is located.
Bay 0 3.0TB
Bay1 3.0TB
Bay 2 3.0TB
Bay 3 3.0TB
DS4486
Bay 4 3.0TB
Bay 5 3.0TB
Bay 6 3.0TB
Bay 7 3.0TB
3.0TB 3.0TB 3.0TB 3.0TB
3.0TB 3.0TB 3.0TB 3.0TB
3.0TB 3.0TB 3.0TB 3.0TB
Bay 20 3.0TB
Bay 21 3.0TB
Bay 22 3.0TB
Bay 23 3.0TB
DS4486
Shelf ID
DISK NAMES: SHELF ID AND BAY (SAS)
Disk Ownership
Software disk ownership is made persistent by writing
the ownership information onto the disk.
The ability to write disk ownership information is
protected by the use of persistent reservations.
Changes in disk ownership are managed
automatically by normal SFO operations, although
commands exist to manipulate disk ownership
manually if necessary.
It is possible for disks to be unowned.
DISK OWNERSHIP
A disks data contents are not destroyed when the disk is marked as unowned; only the disks ownership
information is erased. Unowned disks that reside on an FC-AL, where the owned disks exist, have ownership
information applied automatically to guarantee that all disks on the same loop have the same owner.
Lesson 2
LESSON 2
Write Requests
The Data ONTAP operating system receives write
requests through multiple protocols:
CIFS
NFS
Fibre Channel (FC)
iSCSI
HTTP
Write requests are buffered into:
System memory
Nonvolatile RAM (NVRAM)
WRITE REQUESTS
Write Request Data Flow: Write Buffer
Network
Network Protocols NVLOG N
Stack NVLOG V
Memory Buffer /
NVLOG R
SAN Cache
Service
NVLOG A
RS-232
NVLOG M
NVRAM Full
SAN Host NFS WAFL
HBA Service
CIFS
UNIX NIC Service
Client
RAID
Windows
Client
Storage
WRITE REQUEST DATA FLOW: WRITE BUFFER

Write requests are received from clients. Each write request is stored in a buffer in memory. A copy of each
request is made in the NVLOG. The WAFL file system acknowledges receipt as requests are received.
Consistency Point
A CP is a completely self-consistent image of a
file system.
Creating a CP is equivalent to capturing the
structure of a file system at a moment in time.
When a CP is created, designated data is written
to a disk, and a new root inode is chosen.
A CP can be created for many reasons, including:
Half of the NVRAM card is full.
Ten seconds have elapsed.
A Snapshot copy has been created.
The system has been halted.
CONSISTENCY POINT
A consistency point (CP) is a completely self-consistent image of the entire file system. A CP is not created
until data has been written to disk and a new root inode has been chosen.
Although CPs are created for many reasons, a few of the major reasons are:
Half of the nonvolatile RAM (NVRAM) card is full.
Ten seconds have elapsed.
A Snapshot copy has been created.
The system has been halted.
CPs in the Data ONTAP Operating System
For a CP, the Data ONTAP operating system

flushes writes to disk
It always writes to new data blocks.
The volume is always consistent on the disk.
When the Data ONTAP operating system
flushes memory to disk:
It updates the file system atomically, meaning
that the entire write must be completed or the
entire write is rolled back.
Flushed memory includes all metadata.
It checks and then clears the NVRAM.
CPS IN THE DATA ONTAP OPERATING SYSTEM

At least once every 10 seconds, the WAFL file system generates a CP (an internal Snapshot copy) so that
disks contain a completely self-consistent version of the file system. When the storage system boots, the
WAFL file system always uses the most recent CP on the disks, so you dont have to spend time checking the
file system, even after power loss or hardware failure. The storage system boots in a minute or two, with most
of the boot time devoted to spinning up disks and checking system memory.
The storage system uses battery-backed NVRAM to avoid losing data write requests that might have occurred
after the most recent CP. During a normal system shutdown, the storage system turns off protocol services,
flushes all cached operations to disk, and turns off the NVRAM. When the storage system restarts after
power loss or a hardware failure, it replays into system RAM any protocol requests stored in NVRAM that
are not on the disk. CPs are regularly triggered by the timer, a Snapshot copy, or internal synchronization.
Other events also trigger CPs, though not at set intervals.
Atomic Operations
An atomic operation is actually a set of operations that can be combined so that they appear to the rest of the
system as a single operation, with only two possible outcomes: success or failure.
For an operation to be atomic, the following conditions must be met:
Until the entire set of operations is complete, no other process can be aware of the changes being made.
If any single operation fails, then the entire set of operations fails, and the system state is restored to its
state prior to the start of any operations.
Source: http://en.wikipedia.org/wiki/Atomic_operation
Write Request Data Flow: WAFL to RAID
Network
Stack NVLOG V
Memory Buffer /
NVLOG R
SAN Cache
Service
NVLOG A
RS-232
NVLOG M
NVRAM Full
SAN Host NFS WAFL
HBA Service
CIFS
UNIX NIC Service
Client
RAID
Windows
Client
Storage
WRITE REQUEST DATA FLOW: WAFL TO RAID

The WAFL file system provides short response times to write requests by saving a copy of each write request
in system memory and battery-backed NVRAM and immediately sending acknowledgments. This process is
different from the process used by traditional servers, in which requests must be written to the disk before
they are acknowledged. The WAFL file system delays the writing of data to the disk, which provides more
time to collect multiple write requests and determine how to optimize storage of data across multiple disks in
a RAID group. Because NVRAM is battery-backed, you dont have to worry about losing data.
In the WAFL file system:
Data has no fixed location, except in the superblock.
All data, including metadata, is stored in files.
Layouts can always be optimized.
By combining batch writes, the WAFL file system:
Allows the Data ONTAP operating system to convert multiple small file writes into one sequential disk
write
Distributes data across all disks in a large array, which prevents overloaded disks and hotspots (uneven
utilization of disks in an array)
CPs from the WAFL File System to RAID
The RAID layer calculates the parity of the data:
To protect it from one or more disk failures
To protect stripes of data
The RAID layer calculates checksums, which are
stored using the block or zone method.
If a data disk fails, the missing information can be
calculated from parity.
The storage system can be configured in one of two
ways:
RAID 4: The system can recover from one disk failure in
the RAID group.
RAID-DP: The system can recover from up to two disk
failures in the RAID group.
CPS FROM THE WAFL FILE SYSTEM TO RAID

The WAFL file system then transfers data to the RAID subsystem. The RAID subsystem calculates parity and
passes the data and parity to the data module, where the data is committed to the disks. RAID uses parity to
reconstruct broken disks. Parity scrubs, which proactively identify and solve problems, are performed at the
RAID level using checksum data.
Write Request Data Flow:
RAID to Storage
Network
Stack NVLOG V
Memory Buffer /
NVLOG R
SAN Cache
Service
NVLOG A
RS-232
NVLOG M
NVRAM Full
SAN Host NFS WAFL
HBA Service
CIFS
UNIX NIC Service
Client
RAID
4 KB
Windows
Client Checksum
computed Storage
WRITE REQUEST DATA FLOW: RAID TO STORAGE

Storage drivers move data between system memory and storage adapters, and ultimately to disks. The disk
driver component reassembles writes into larger I/O operations and also monitors which disks have failed.
The SCSI driver applies the appropriate SCSI commands to synchronize with the reads and writes that it
receives.
CPs from RAID to Storage
1. The data module commits data and parity to

the physical disks.
2. The root inode is updated to point to the new
file inodes on the disk.
3. NVRAM is flushed and made available, after
which the CP is complete.
CPS FROM RAID TO STORAGE
Write Request Data Flow: Storage Writes
Network
Stack NVLOG V
Memory Buffer /
NVLOG R
SAN Cache
Service
NVLOG A
RS-232
NVLOG M
NVRAM Full
SAN Host NFS WAFL
HBA Service
CIFS
UNIX NIC Service
Client
RAID
Windows
Client
Storage
WRITE REQUEST DATA FLOW: STORAGE WRITES

The data module transfers data to physical disks. After data is written to the disks, a new root inode is
updated, a CP is created, and NVRAM is flushed.
NVRAM
The Data ONTAP operating system writes

from system memory:
NVRAM is never read during normal write
operations.
NVRAM is backed up with a battery.
If a system failure occurs before the
completion of a CP, the data is read from
NVRAM and added back to the system
memory buffer when the system is brought
back online (or by the HA partner machine).
NVRAM
NVRAM is best viewed as a log. This log stores a subset of incoming file actions.
When a request comes in, two things happen:
The request is logged to NVRAM. NVRAM is not read during normal processing. It is simply a log of
requests for action (including any data necessary, such as the contents of a write request).
The request is acted upon. The storage system's main memory is used for processing requests. Buffers are
read from the network and from the disk and processed according to the directions that came in as CIFS
or NFS requests. NVRAM holds the instructions that are necessary if the same actions need to be
repeated.
If the storage system does not crash, the NVRAM is eventually flushed without ever being read back. If the
storage system crashes, the data from NVRAM is processed as if the storage system were receiving those
same CIFS or NFS requests again. The same response is made by the storage system for each request in
NVRAM, just as if it had come in through the network.
Read Requests
Every time a read request is received, the

WAFL file system does one of two things:
Reads the data from the system memory (also
known as the cache)
Reads the data from the disks
The cache is populated by:
Data recently read from disk
Data recently written to disk
READ REQUESTS
The Data ONTAP operating system includes several built-in, read-ahead algorithms. These algorithms are
based on patterns of usage. The algorithms help ensure that the read-ahead cache is used efficiently.
The response to a read request is composed of four steps:
1. The network layer receives an incoming read request. (Read requests are not logged to NVRAM.)
2. The WAFL file system looks for the requested data in the read cache:
If it locates the data, it returns the data immediately to the requesting client.
If it does not locate the data, it initiates a read request from the disk.
3. Requested blocks and intelligently chosen read-ahead data are sent to cache.
4. The requested data is sent to the requesting client.
NOTE: In the read process, cache is system memory.
Read Request Data Flow: Read from Disk
Network
Network Protocols N
Stack V
Memory Buffer /
R
SAN Cache
Service
A
Console
M
SAN Host NFS WAFL

HBA Service
CIFS
UNIX NIC Service
Client
RAID
Windows
Client
Storage
READ REQUEST DATA FLOW: READ FROM DISK

Read requests that can be satisfied from the read cache are retrieved from the disk. The read cache is then
updated with new disk information for subsequent read requests.
Read Request Data Flow: Cache
Network
Network Protocols N
Stack V
Memory Buffer /
R
SAN Cache
Service
A
RS-232
M
SAN Host NFS WAFL

HBA Service
CIFS
UNIX NIC Service
Client
RAID
Windows
Client
Storage
READ REQUEST DATA FLOW: CACHE

When a read request is received from a client, the WAFL file system determines whether to read data from
the disk or respond to the request using the cache buffers. The cache can include data that was recently
written to or read from the disk.
Lesson 3
LESSON 3
RAID Groups
RAID groups are a collection of data disks

and parity disks
RAID groups provide protection through parity
Data ONTAP organizes disks into RAID
groups
Data ONTAP supports:
RAID 4
RAID-DP
RAID disk types: data, hot spare, parity, and
double-parity
RAID GROUPS
A RAID group includes several disks that are linked together in a storage system. Although there are different
implementations of RAID, Data ONTAP supports only RAID 4 and RAID-DP. To understand how to manage
disks and volumes, it is important to first understand the concept of RAID.
Data ONTAP classifies disks as one of four types for RAID: data, hot spare, parity, or double-parity. The
RAID disk type is determined by how RAID is using a disk.
Data disk: A data disk is part of a RAID group and stores data on behalf of the client.
Hot spare disk: A hot spare disk does not hold usable data but is available to be added to a RAID group in an
aggregate. Any functioning disk that is not assigned to an aggregate, but is assigned to a system, functions as
a hot spare disk.
Parity disk: A parity disk stores data reconstruction within a RAID group.
Double-parity disk: A double-parity disk stores double-parity information within RAID groups if NetApp
RAID software, double-parity (RAID-DP) is enabled.
RAID 4 Technology
RAID 4 protects against data loss that results from a
single-disk failure in a RAID group.
A RAID 4 group requires a minimum of three disks:
Two data disks
One parity disk
Data Data Data Data Data Data Data Parity
RAID 4 TECHNOLOGY
RAID 4 protects against data loss due to a single-disk failure within a RAID group.
Each RAID 4 group contains the following:
Two or more data disks
One parity disk (assigned to the largest disk in the RAID group)
Using RAID 4, if one disk block goes bad, the parity disk in that disk's RAID group is used to recalculate the
data in the failed block, and then the block is mapped to a new location on the disk. If an entire disk fails, the
parity disk prevents any data from being lost. When the failed disk is replaced, the parity disk is used to
automatically recalculate its contents. This is sometimes referred to as row parity.
RAID-DP Technology
RAID-DP protects against data loss that results from
double-disk failures in a RAID group.
A RAID-DP group requires a minimum of five disks for
clustered Data ONTAP 8.2 and later:
Three data disks
One parity disk
One double-parity disk
Data Data Data Data Data Data Parity Double-

Parity
RAID-DP TECHNOLOGY
RAID-DP technology protects against data loss due to a double-disk failure within a RAID group.
Each RAID-DP group contains the following:
Three data disks
One parity disk
One double-parity disk
RAID-DP employs the traditional RAID 4 horizontal row parity. However, in RAID-DP, a diagonal parity
stripe is calculated and committed to the disks when the row parity is written.
RAID Group Size
RAID-DP
NetApp Minimum Maximum Default
Platform Group Size Group Size Group Size
All storage systems
5 16 14
(with SATA disks)
All storage systems
5 28 16
(with FC or SAS disks)
RAID 4
NetApp Minimum Maximum Default
Platform Group Size Group Size Group Size
All storage systems
3 7 7
(with SATA)
All storage systems
3 14 8
(with FC or SAS)
RAID GROUP SIZE

RAID groups can include anywhere from 3 to 28 disks, depending on the platform and RAID type. For best
performance and reliability, NetApp recommends using the default RAID group size.
Aggregates
Aggregates:
Are the same as with the Data ONTAP 7G operating
system
Have storage containers that consist of disks
Can use RAID 4 or RAID-DP technology
Contain volumes
Can be taken over by their nodes HA partner
Can be grown by adding disks
32-bit and 64-bit aggregates are supported.
Nondisruptive, in-place aggregate expansions are
available from 32-bit aggregates to 64-bit aggregates.
AGGREGATES
In the Data ONTAP 8.1 operating system and later releases , nondisruptive, in-place aggregate expansions are
available from 32-bit aggregates to 64-bit aggregates. During the conversion, the volumes on the aggregate
remain online and continue to serve data.
For clustered Data ONTAP, storage administrators can initiate expansion through the cluster shell by enabling
the diagnostic mode and then running the storage aggregate 64bit-upgrade start command.
The expansion runs in the background but can affect overall cluster performance.
After an aggregate is converted to 64-bit, you can grow the aggregate beyond 16 TB by adding disks through
the storage aggregate add-disks command.
The storage aggregate show Command
cluster1::> storage aggregate show
Aggregate Size Available Used% State #Vols Nodes RAID Status
--------- -------- --------- ----- ------- ----- ---------- -----------
cluster1-01_aggr0
56.76GB 2.59GB 95% online 1 cluster1-01 raid_dp
cluster1-01_aggr2
113.5GB 113.2GB 0% online 1 cluster1-01 raid4
cluster1-01_aggr3
cluster1-02_aggr0
cluster1-02_aggr1
cluster1-02_aggr2
113.5GB 113.5GB 0% online 0 cluster1-02 raid4
THE STORAGE AGGREGATE SHOW COMMAND

Here you can see that each node contains three aggregates.
In-Place 32-Bit-to-64-Bit Aggregate Expansion
Features:
You can expand 32-bit aggregates to 64-bit
aggregates.
You can expand while an aggregate is online and
serving data.
Considerations:
64-bit aggregates consume more space than 32-bit
aggregates do.
The process works in the background but affects
performance.
IN-PLACE 32-BIT-TO-64-BIT AGGREGATE EXPANSION
Lesson 4
LESSON 4
SFO
1 of 2
Two nodes are connected as an HA pair.
Each node is a fully functioning node in the larger
cluster.
Clusters can consist of heterogeneous hardware,
but both nodes of an HA pair must be the same
controller model.
SFO can be enabled from either node in the pair.
SFO takeover can be initiated from any node in
the cluster.
A manual storage takeover forces a booting of
the node that is taken over.
SFO: 1 OF 2
Enabling SFO is done within pairs, regardless of how many nodes are in the cluster. For SFO, the HA pairs
must be of the same model; for example, two FAS32x0 systems or two FAS62x0 systems. The cluster itself
can contain a mixture of models, but each HA pair must be homogeneous. The version of the Data ONTAP
operating system must be the same on both nodes of the HA pair, except for the short period of time during
which the pair is upgraded. During that time, one of the nodes is booted with a later version than its partners
version, with the partner to follow shortly. The nonvolatile RAM (NVRAM) cards must be installed in the
nodes. Two interconnect cables are required to connect the NVRAM cards (except for FAS and V-Series 32x0
models with single-enclosure HA).
Remember that this cluster is not simply the pairing of machines for failover; this cluster is the Data ONTAP
cluster.
SFO
2 of 2
Automatic giveback is enabled by default for
2 Node clusters.
Both nodes of an HA pair must be booted before
SFO can be enabled for the pair.
SFO: 2 OF 2
According to the High-Availability Configuration Guide for Clustered ONTAP 8.2:
If your cluster consists of a single HA pair:
Cluster high availability (HA) is activated automatically when you enable storage failover on clusters that
consist of two nodes, and you should be aware that automatic giveback is enabled by default. On clusters that
consist of more than two nodes, automatic giveback is disabled by default, and cluster HA is disabled
automatically.
HA Pairs
A high availability (HA) pair contains two nodes
whose controllers are directly connected through
an HA interconnect.
A node can take over its partner's storage to
provide continued data service if the partner goes
down.
HA pairs are components of the cluster, but only
the nodes in the HA pair can take over each
other's storage.
Single-node clusters are supported in Data
ONTAP 8.2, but non-HA nodes are not supported
in clusters that have two or more nodes.
HA PAIRS
HA pair controllers are connected to each other through an HA interconnect. This allows one node to serve
data that resides on the disks of its failed partner node. Each node continually monitors its partner, mirroring
the data for each others nonvolatile memory (NVRAM or NVMEM). The interconnect is internal and
requires no external cabling if both controllers are in the same chassis.
HA pairs are components of the cluster, and both nodes in the HA pair are connected to other nodes in the
cluster through the data and cluster networks. But only the nodes in the HA pair can take over each other's
storage. Non-HA nodes are not supported in a cluster that contains two or more nodes. Although single-node
clusters are supported, joining two single-node clusters to create one cluster is not supported, unless you wipe
clean one of the single-node clusters and join it to the other to create a two-node cluster that consists of an HA
pair.
HA Policy: CFO and SFO
Aggregates are automatically assigned an HA policy.
Root aggregates (aggr0) are always assigned CFO
(controller failover) policy.
Aggr0 is given back at the start of the giveback process to
allow the taken-over system to boot.
Data aggregates are assigned SFO (storage failover)
policy.
Data aggregates are given back one at a time during the
giveback process, after the taken-over system boots.
The HA policy of an aggregate cannot be changed
from SFO to CFO in normal operation.
Hardware-assisted takeover can be used to speed up
the takeover process.
Do not store data volumes on aggr0
HA POLICY: CFO AND SFO

Takeover is the process in which a node takes over the storage of its partner.
Giveback is the process in which that storage is returned to the partner.
Both processes can be initiated manually or configured for automatic initiation. A takeover is manually
initiated with the storage failover takeover command; an automatic takeover occurs after a node
failure condition.
Hardware-assisted takeover speeds up the takeover process by using a node's remote management device (SP
or RLM) to detect failures and quickly initiate the takeover rather than waiting for Data ONTAP to recognize
that the partner's heartbeat has stopped. Without hardware-assisted takeover, if a failure occurs, the partner
waits until it notices that the node is no longer giving a heartbeat, confirms the loss of heartbeat, and then
initiates the takeover.
Do not store data volumes on vol0. Due to aggr0 being a CFO aggregate, volumes on aggr0 are not available
to clients during failover.
Ownership of aggr0 During Failover
cluster1::> aggr show -aggregate aggr0
Aggregate: aggr0
Checksum Style: block
Number Of Disks: 3
Nodes: cluster1-02
Disks: cluster1-02:1b.16,
cluster1-02:1b.17,
cluster1-02:1b.18
Free Space Reallocation: off
HA Policy: cfo
Space Reserved for Snapshot Copies: -
Hybrid Enabled: false
Available Size: 5.57GB
Checksum Enabled: true
Checksum Status: active
Has Mroot Volume: false
Has Partner Node Mroot Volume: true
Home ID: 1579305252
Home Name: cluster1-01
OWNERSHIP OF AGGR0 DURING FAILOVER

Notice that the Nodes field reflects the HA partner, but the Home Name field retains the name of the failed
node.
Unplanned Event
Node 1 and 2 own their Node 1
root and data aggr0
aggregates.
aggr1
Node 1 fails.
aggr2
Node 2 takes over root
and data aggregates.
H
A
Node 2 aggr0
aggr3
UNPLANNED EVENT
Clustered ONTAP 8.2 performs takeovers a little differently than past versions. Prior to 8.2, an unplanned
event (e.g node failure) and a planned event (manual takeover initiated by an administrator) followed the
same process. In clustered ONTAP 8.2, planned events use a different process.
When a node fails, an unplanned event or automatic takeover is initiated (8.2 and prior). Ownership of data
aggregates is changed to the HA partner. After the ownership is changed, the partner can read and write to the
volumes on the partners data aggregates. Ownership of aggr0 disks remain with the failed node, but the
partner takes over control of the aggregate which can be mounted from the partner for diagnostic purposes.
Giveback
Automatic or manual Node 1
giveback is initiated aggr0
with storage
failover aggr1
giveback
aggr2
command.
Aggr0 is given back to
H
node 1 to boot the A
node.
Node 2 aggr0
Data aggregate
giveback occurs one aggr3
aggregate at a time.
GIVEBACK
Giveback is initiated by the storage failover giveback command or by automatic giveback if the
system is configured for it. The node must have access to its root volume on aggr0 to fully boot. The CFO HA
policy ensures that aggr0 is given back immediately to the allow the node to boot.
After the node has fully booted, the partner node returns ownership of the data aggregates one at a time until
giveback is complete. You can monitor the progress of the giveback with the storage failover
show-giveback command. I/O resumes for each aggregate when giveback is complete for that aggregate,
thereby reducing the overall outage window of each aggregate.
Aggregate Relocation
Aggregate relocation (ARL) moves the
ownership of storage aggregates within the
HA pair.
This occurs automatically during manually
initiated takeover and giveback operations to
reduce downtime during maintenance.
ARL cannot move ownership of the root
aggregate.
To avoid ARL, use the -bypass-
optimization parameter with the storage
failover takeover command.
AGGREGATE RELOCATION
Aggregate relocation operations take advantage of the HA configuration to move the ownership of storage
aggregates within the HA pair. Aggregate relocation occurs automatically during manually initiated takeover
and giveback operations to reduce downtime during maintenance. Aggregate relocation can be initiated
manually for load balancing. Aggregate relocation cannot move ownership of the root aggregate.
During a manually initiated takeover, before the target controller is taken over, ownership of each aggregate
that belongs to the target controller is moved to the partner controller one aggregate at a time. When giveback
is initiated, the ownership is automatically moved back to the original node. To suppress aggregate relocation
during the takeover, use the -bypass-optimization parameter with the storage failover takeover command.
The aggregate relocation requires additional steps if the aggregate is currently used by an infinite volume with
SnapDiff enabled.
Planned Event in 8.2 with ARL
Node 1 and 2 own Node 1
their root and data aggr0
aggregates
aggr1
Manual takeover is STOP
initiated using the aggr2
storage
failover
takeover H
A
command
Node 2 aggr0
Data aggregates
change ownership aggr3
to node 2 one at a
time
Root aggregate is failed over once all

user data containing aggregates are
failed to the partner node
PLANNED EVENT IN 8.2 WITH ARL

When a node takes over its partner, it continues to serve and update data in the partner's aggregates and
volumes. To do this, it takes ownership of the partner's data aggregates, and the partner's LIFs migrate
according to network interface failover rules. Except for specific SMB 3.0 connections, existing SMB (CIFS)
sessions are disconnected when the takeover occurs.
The following steps occur when a node takes over its partner:
1. If the negotiated takeover is user-initiated, aggregate relocation is performed to move data aggregates one
at a time from the target node to the node that is doing the takeover. The current owner of each aggregate
(except for the root aggregate) is changed from the target node to the node that is doing the takeover.
There is a brief outage for each aggregate as ownership is changed. This outage is less than that accrued
during a takeover that does not use aggregate relocation. You can monitor the progress by using the
storage failover show-takeover command. To avoid the aggregate relocation during this
takeover instance, use the -bypass-optimization parameter with the storage failover takeover
command. To bypass aggregate relocation during all future planned takeovers, set the
-bypass-takeover-optimization parameter of the storage failover command to true.
NOTE: Aggregates are relocated serially during planned takeover operations to reduce client outage. If
aggregate relocation is bypassed, it will result in longer client outage during planned takeover events.
2. If the takeover is user-initiated, the target node gracefully shuts down, followed by takeover of the target
node's root aggregate and any aggregates that were not relocated in Step 1.
3. Data LIFs migrate from the target node to the node that is doing the takeover, or any other node in the
cluster based on LIF failover rules, before the storage takeover begins. To avoid LIF migration, use the
-skip-lif-migration parameter with the storage failover takeover command.
4. Existing SMB (CIFS) sessions are disconnected when takeover occurs. NOTE: Due to the nature of the
SMB protocol, all SMB sessions, except for SMB 3.0 sessions connected to shares with the
Continuous Availability property set, will be disruptive. SMB 1.0 and SMB 2.x sessions cannot
reconnect after a takeover event. Therefore, takeover is disruptive, and some data loss could occur.
5. SMB 3.0 sessions established to shares with the Continuous Availability property set can
reconnect to the disconnected shares after a takeover event. If your site uses SMB 3.0 connections to
Microsoft Hyper-V, and the Continuous Availability property is set on the associated shares,
takeover will be nondisruptive for those sessions.
If the node that is performing the takeover panics within 60 seconds of initiating takeover, the following
events occur:
The node that panicked reboots.
After it reboots, the node performs self-recovery operations and is no longer in takeover mode.
Failover is disabled.
If the node still owns some of the partner's aggregates, after enabling storage failover, return these
aggregates to the partner by using the storage failover giveback command.
Giveback in 8.2 with ARL
Manual giveback Node 1
aggr0
is initiated with
the storage aggr1
STOP
failover
aggr2
giveback
command.
H
Aggr0 is given A
back to node 1 to Node 2 aggr0

boot the node. aggr3
Node 2 data
aggregate
giveback occurs
one aggregate at
a time.
GIVEBACK IN 8.2 WITH ARL

The local node returns ownership of the data aggregates and volumes to the partner node after any issues on
the partner node are resolved or maintenance is complete. In addition, the local node returns ownership when
the partner node has booted up and giveback is initiated either manually or automatically.
The following process takes place in a normal giveback. In this discussion, node 2 has taken over node 1. Any
issues on node 1 have been resolved, and it is ready to resume serving data.
1. Any issues on node B have been resolved, and it is displaying the message Waiting for giveback.
2. The giveback is initiated by the storage failover giveback command, or by automatic giveback if the
system is configured for it. This initiates the process of returning ownership of the node 1's aggregates
and volumes, from node 2 back to node 1.
3. Node 2 returns control of the root aggregate first.
4. Node 1 proceeds to complete the process of booting up to its normal operating state.
5. As soon as Node 1 is at the point in the boot process where it can accept the nonroot aggregates, node 2
returns ownership of the other aggregates one at a time until giveback is complete. You can monitor the
progress of the giveback with the storage failover show-giveback command.
I/O resumes for each aggregate when giveback is complete for that aggregate, thereby reducing the overall
outage window of each aggregate.
HA Best Practices
Do not use the root aggregate for storing data.
Follow recommended limits for volumes,
Snapshot copies, and LUNs to reduce the
takeover or giveback time.
Use LIFs with defined failover policies to provide
redundancy and improve availability of network
communication.
Avoid using the -only-cfo-aggregates
parameter with the storage failover
giveback command.
Use the Config Advisor tool to help ensure that
failovers are successful, and test failover
routinely.
HA BEST PRACTICES
See the Clustered Data ONTAP Logical Storage Management Guide for current information on storage
limits.
Find Config Advisor here: http://support.netapp.com/NOW/download/tools/config_advisor/
Storage Failover Event Summary
HA Event Event Description
Unplanned Event All aggregates failover to partner node in
parallel
Planned Event (cDOT All aggregates failover to partner node in
8.1) parallel
Planned Event (cDOT Each aggregate is failed over serially, the root
8.2) aggregate is failed over once all user data
containing aggregates are failed over to the
partner node
Giveback Root aggregate is given back first; once a
node is assimilated back into the cluster each
data containing aggregate is given back
serially to the partner node
STORAGE FAILOVER EVENT SUMMARY
The storage failover show Command
cluster1::> storage failover show
Takeover InterConn
Node Partner Enabled Possible Up State
-------------- -------------- ------- -------- --------- --------------
cluster1-01 cluster1-02 true true true connected
cluster1-02 cluster1-01 true true true connected
THE STORAGE FAILOVER SHOW COMMAND

This example shows a two-node cluster, which is also an HA pair. Note that SFO is enabled on both nodes.
Two-Node HA
Is an additional configuration step for two-node
clusters only
Must be configured to enable the cluster to
operate properly when one of the two nodes is
down
Is needed because of the way that the RDB units
maintain quorum. (RDB units operate differently
when only two nodes exist compared to when
more than two nodes exist.)
Must be enabled for SFO to work properly in a
two-node cluster
TWO-NODE HA
For clusters of only two nodes, the replicated database (RDB) units rely on the disks to maintain quorum
within the cluster in the case that a node is booted or goes down. This process is enabled by configuring the
two-node HA mechanism. Because of the reliance on the disks, SFO enablement and automatic giveback are
also required by two-node HA and are configured automatically when two-node HA is enabled. For clusters
that are larger than two nodes, quorum can be maintained without using the disks. Do not enable two-node
HA for clusters that are larger than two nodes. When expanding a cluster beyond two nodes, the HA state
must be changed manually. Nodes cannot be added while HA is enabled.
For More Information
Clustered Data ONTAP 8.2 High-Availability Configuration Guide
TR-3450: High-Availability Overview and Best Practices
FOR MORE INFORMATION
Lesson 5
LESSON 5
NetApp Virtual Storage Tier
Flash Cache Flash Pool
What is it? What is it?
A controller-based PCIe card Storage-level, RAID-protected
A plug and play device cache (specific to aggregates)
What does it do? What does it do?

Per controller cache Caches random reads and
Caches random reads overwrites
Provides cached data
persistence through failovers
Where does it fit? Where does it fit?
With random-read workloads, With random-overwrite-heavy
for example, File Services workloads, for example, OLTP
With volumes that span multiple With consistent performance
aggregates (required)
NETAPP VIRTUAL STORAGE TIER
Flash Cache
Is a 256-GB, 512-GB, or 1-TB PCIe module
Is a plug-and-play device (no required configuration)
Is for Data ONTAP 8.0.2 clusters or later
Supports all protocols
Acts as an extention to the WAFL buffer cache, and saves evicted
buffers
Shared by all volumes on a node
http://www.netapp.com/us/products/storage-systems/flash-cache/.
FLASH CACHE
Flash Cache intelligent caching intelligent caching is a solution that combines software and hardware within
NetApp storage controllers to increase system performance without increasing the disk count. The Flash
Cache plug-and-play PCIe module requires no configuration to use the default settings, which are
recommended for most workloads. The original Flash Cache module is available in 256-GB, 51-GB, or 1-TB
capacities and accelerates performance on all supported Data ONTAP client protocols. The Flash Cache
controller-based solution is available to all volumes that are hosted on the controller.
Flash Cache 2
512-GB, 1-TB, and 2-TB densities
FLASH CACHE 2
Flash Cache 2 is the second generation of Flash Cache performance accelerators. The new architecture of
Flash Cache 2 accelerators enables them to provide even higher throughput. Flash Cache 2 accelerators
provide 512-GB, 1-TB, and 2-TB densities.
Flash Cache Benefits
Benefits:
Increase I/O throughput by up to 75%
Use up to 75% fewer disks without compromising performance
Increase email users by up to 67% without adding disks
Key Points:
Use for random read-intensive workloads (databases, email, file
services)
Reduce latency by a factor of 10 or greater compared to hard disks
Increase I/O throughput and eliminate performance bottlenecks
Lower costsuse SATA disks with Flash Cache for important workloads
Save power, cooling, and rack space by using fewer, larger disks
FLASH CACHE BENEFITS

Flash Cache speeds data access through intelligent caching of recently read user data or NetApp metadata. No
setup or ongoing administration is needed, and operations can be tuned. Flash Cache works with all NetApp
storage protocols and software, enabling you to increase I/O throughput by up to 75%, use up to 75% fewer
disks without compromising performance, and increase email users by up to 67% without adding disks. Key
points for Flash Cache usage include:
Use for random read-intensive workloads (databases, email, file services)
Reduce latency by a factor of 10 or greater compared to hard disks
Increase I/O throughput and eliminate performance bottlenecks
Lower costsuse SATA disks with Flash Cache for important workloads
Save power, cooling and rack space by using fewer, larger disks
Flash Pool
Flash Pool is an aggregate-level
read and write cache.
Like Flash Cache, Flash Pool
uses 4-KB block granularity and
Performance Capacity real-time caching.
+ Flash Pool is not a replacement
HDD for Flash Cache
Flash Pool Cache remains populated and
available during SFO events.
Random overwrite data is cached.
FLASH POOL
WAFL (Write Anywhere File Layout) aggregates are built with disks of the same type: SATA, hard disks,
and FC and SAS hard disks or solid-state drives (SSDs). Flash pools allow the mixing of SSDs and hard disks
within the same aggregate. The SSD tier aggregate is used as a cache and doesnt contribute to usable space.
When an aggregate is converted to hybrid, the usable space in the aggregate does not change. The disks that a
hybrid aggregate consists of are treated like any disks in a NetApp storage array, and any class of disk can be
added on demand, subject to best practices around data, such as parity ratios and RAID types.
Flash pools provide:
Improved cost performance with fewer spindles, less rack space, and lower power and cooling
requirements
Highly available storage with a simple administrative model
Improved cost-to-performance and cost-to-capacity ratios compared to those of an SSD and SATA
combination with pure FC SAS
Predictable and better degraded mode operation across controller failures and with takeover and giveback
Automatic, dynamic, policy-based placement of data on appropriate tiers of storage (hard disks or SSDs)
at WAFL-block granularity for either data or system metadata
Flash Pool Components
A flash pool is an aggregate with:
One or more hard disk RAID groups
An SSD RAID group
Only one type of hard disk can be
used per flash pool:
High capacity (SATA)
Performance (SAS)
SSDs cache random data
Previously written data (overwrites)
Read data expired from main memory
Existing aggregates can be non-
disruptively converted to flash pools.
FLASH POOL COMPONENTS

Flash pools consist of one or more hard-disk RAID groups and one or more SSD RAID groups. Each RAID
group within the aggregate must be homogeneous (all SATA disks, all SAS disks, or all SSDs). Additionally,
with the Data ONTAP 8.1.1 operating system, the RAID group type of the aggregate must be the same across
both the hard-disk and SSD RAID groups (either RAID 4 or RAID-DP). You can nondisruptively convert
existing aggregates to flash pools but, after conversion, SSDs cannot be removed.
Blocks in the SSD Tier
Flash pool metadata

Read-cached blocks:
Are a cached copy of the blocks from the hard
disk tier
Still exist on the hard disk tier
Write-cached blocks:
Are written directly to the SSD tier
Are not yet written to the hard disk tier
BLOCKS IN THE SSD TIER

The following blocks are stored in the SSD tier of the Flash Pool:
Flash pool metadata: All metadata that is associated with the flash pool is stored in the SSD tier of the
aggregate.
Read-cached blocks: These blocks are cached copies of a block in the hard disk tier of the aggregate and are
stored in the SSD tier. Almost all data from the active file system in a read/write volume is eligible to be read-
cached into the SSD tier.
Write-cached blocks: These blocks are associated with a FlexVol volume that is written directly to the SSD
tier of the aggregate. Only one copy of the block exists. The copy is in the SSD tier. A hard-disk block is
reserved for write-cached blocks for an eventual move into the hard-disk tier after access to the block ceases.
Enabling Flash Pools
1. Turn on the hybrid-enabled option.
2. Add a new RAID group with SSDs.

modify aggr aggr3 hybrid-enabled
true

-add-disks aggr aggr3 disktype
SSD diskcount 12
ENABLING FLASH POOLS

Enable flash pools on aggregates by completing the following steps:
1. Turn on the hybrid_enabled option on the aggregate.
2. Add a new RAID group with SSDs.
This action converts the aggregate to a flash pool.
Flash Pools: Additional Considerations
Available space
Flash Cache
HA takeover and giveback
Volume move
Volume SnapMirror relationships
Aggregate Snapshot copies
Data compression
V-Series
RAID4 for SSD tier
FLASH POOLS: ADDITIONAL CONSIDERATIONS

The capacity of the SSD tier is not reflected in the total aggregate size. For example, if the original aggregate
has a 10-TB capacity and you add an SSD raid group with a 1-TB capacity, the amount of capacity in the
aggregate that can be provisioned is still 10 TB.
Flash pools can coexist in the same cluster or on the same storage controller as Flash Cache, but blocks from
flash pools are not included in Flash Cache caching. Flash Cache continues to serve all non-flash-pool
aggregates on the controller.
Not only are flash pools compatible with takeover and giveback, but they also provide performance
acceleration during such events.
Regardless of whether the source aggregate is a flash pool, a volume can be moved to a flash pool aggregate.
The volume is not immediately cached after the move, and performance might degrade slightly until the cache
is repopulated.
SnapMirror destination volumes can reside in flash pools, but the SnapMirror destination is not cached. When
the volume is promoted to read/write, then the data from the new active file system is cached.
Because SSD blocks can become trapped in Snapshot copies, the aggregate Snapshot feature should either be
disabled or configured with automatic deletion so that SSD blocks continue to be recycled.
Flash pools support read caching of uncompressed blocks in a compression-enabled volume, but they do not
yet support caching of blocks that are compressed in the hard-disk tier. Compressed blocks are never write-
cached.
Flash pools are supported on V-Series systems with NetApp storage but not with third-party storage.
When using Data ONTAP 8.2 or a subsequent release, the RAID policies for the SSD RAID group and HDD
RAID groups in a Flash Pool aggregate are independent. That means an SSD RAID group could be RAID 4
protected, while the HDD RAID groups in the same Flash Pool aggregate use RAID-DP protection.
Nevertheless, the added protection of RAID-DP makes it a best practice to use RAID-DP for SSD RAID
groups as well. An uncorrectable error in an SSD RAID group that is configured with RAID 4 and has
experienced the failure of one SSD will result in the entire Flash Pool aggregate being taken offline. And it
could also cause a loss of data that is cached in write cache. Therefore, NetApp recommends using RAID-DP
protection for SSD RAID groups and HDD RAID groups.
Module Summary
Now that you have completed this module, you should be
able to:
Draw the connections from an HA pair of controllers to the
disk shelves
Create aggregates
List the steps that are required to enable SFO
Explain and enable two-node HA mode for two-node
clusters
Create a flash pool
MODULE SUMMARY
Exercise
Module 5: Physical Data Storage
EXERCISE
Module 6
Logical Data Storage
MODULE 6: LOGICAL DATA STORAGE
6-1 Clustered Data ONTAP Administration: Logical Data Storage
Module Objectives

Create a volume
Build a namespace by using multiple volumes
Configure FlexCache storage
MODULE OBJECTIVES
Lesson 1
LESSON 1
Virtual Servers
The three types of virtual storage servers

(Vservers) are:
Node: represents a node
Admin: represents a physical cluster
Data: is the context in which volumes are
created
Creating a data Vserver causes a root volume
for that Vserver to be created.
The root volume is the root of the data
Vservers namespace.
VIRTUAL SERVERS
A data virtual storage server (Vserver) connects volumes, logical interfaces (LIFs), and other elements for a
namespace. No volumes can be created until a data Vserver exists with which to associate the volumes.
The vserver show Command
Summary View
cluster1::> vserver show
Admin Root Name Name
Vserver Type State Volume Aggregate Service Mapping
----------- ------- --------- ---------- ---------- ------- -------
cluster1 admin - - - - -
vs7 data running vs7 aggr1b file file
THE VSERVER SHOW COMMAND: SUMMARY VIEW

Note the types of Vservers that are shown here. For each node in a cluster, Data ONTAP automatically
creates a node Vserver to represent the node. The administrative Vserver is automatically created when the
cluster is created. The data Vservers are created by an administrator to build global namespaces.
Volumes
Flexible volumes in clustered Data ONTAP are the
same as in Data ONTAP 7G or 7-Mode.
Any single volume can exist within only a single data
Vserver.
Volumes are joined together through junctions to
create the namespace of a Vserver.
Volumes are the unit of data management: Volumes
can be moved, copied, mirrored, backed up, or copied
by using Snapshot copies.
Data ONTAP 7-Mode volumes cannot be used in
clustered Data ONTAP systems, and vice versa.
VOLUMES
Clustered Data ONTAP flexible volumes are functionally equivalent to flexible volumes in the Data ONTAP
7-Mode and the Data ONTAP 7G operating system. However, clustered Data ONTAP systems use flexible
volumes differently than Data ONTAP 7-Mode and Data ONTAP 7G systems do. Because Data ONTAP
clusters are inherently flexible (particularly because of the volume move capability), volumes are deployed as
freely as UNIX directories and Windows folders are deployed to separate logical groups of data.
Volumes can be created and deleted, mounted and unmounted, moved around, and backed up as needed. To
take advantage of this flexibility, cluster deployments typically use many more volumes than traditional Data
ONTAP 7G deployments use. In a high-availability ( HA) pair, aggregate and volume limits apply to each
node individually, so the overall limit for the pair is effectively doubled.
Volumes
Sizes and Limits
Volume Volume
Aggregat Aggregate
Memory Limit Memory Limit
Platform e Size Platform Size Limit
(GB) (per (GB) (per
Limit (TB) (TB)
node) node)
FAS6280/V6280 96 1000 400 FAS3170 16 1000 240
FAS6240/V6240 48 1000 400 FAS3160 8 500 180
FAS6210/V6210 24 1000 324 FAS3140 4 500 150
FAS3270/V3270 20 1000 240 FAS6080 32 1000 324
FAS3250/V3250 20 1000 240 FAS6040 16 1000 240
FAS3240/V3240 8 500 180 FAS2220 6 500 120
FAS3220/V3220 12 500 180 FAS2240 6 500 120
FAS3210/V3210 4 500 150
VOLUMES: SIZES AND LIMITS

In clustered Data ONTAP 8.2, storage capacity per node and per aggregate has been increased for many
NetApp storage platforms.
Enterprise-class storage systems such as the 6200 series can now support 1,000 volumes per node when
storage has not failed over. Capacity is 2,000 volumes per node when storage fails over to a high-availability
partner. Each cluster can have a maximum of 12,000 volumes, so beyond 12 nodes, the number of volumes
per node must decrease below 1,000.
The maximum aggregate size on enterprise-class platforms, such as the 6280 series, has been increased to 400
TB. For aggregate size limits for other systems, see the table.
For the most up-to-date limits, see the Clustered Data ONTAP Logical Storage Management Guide.
The volume show Command
Summary View
cluster1::> volume show
Vserver Volume Aggregate State Type Size Available Used%
--------- ------------ ------------ ---------- ---- ---------- ---------- -----
cluster1-01
vol0 aggr0 online RW 851.5MB 514.8MB 39%
cluster1-02
vol0 aggr0_cluster1_02_0
online RW 851.5MB 587.1MB 31%
vs7 vs7 aggr1b online RW 20MB 18.88MB 5%
vs7 vs7_vol1 aggr1b online RW 400MB 379.8MB 5%
THE VOLUME SHOW COMMAND: SUMMARY VIEW

This example shows some volumes. You can see that the Type values are all RW, which shows that these
volumes are read/write volumes, as opposed to load-sharing mirror copies or data-protection mirror copies.
The difference between the Size and Available values is the amount of the volume that is used, but the
difference also reflects administrative space that is used by the Write Anywhere File Layout (WAFL ) file
system and the space that is reserved for Snapshot copies.
Junctions
Junctions connect volumes to form the

namespace of a data Vserver.
Junctions appear as directories in the
junctions parent volumes.
When a client traverses a junction, the client
has entered a different volume.
Snapshot copy access does not traverse
junctions.
A volume can be mounted to the namespace
of its Vserver in one place.
JUNCTIONS
Junctions are conceptually similar to UNIX mountpoints. In UNIX, a disk can be divided into partitions, and
then those partitions can be mounted at multiple places relative to the root of the local file system, including
in a hierarchical manner. Likewise, the flexible volumes in a Data ONTAP cluster can be mounted at junction
points within other volumes to form a single namespace that is distributed throughout the cluster. Although
junctions appear as directories, junctions have the basic functionality of symbolic links.
A volume is not visible in the namespace of its Vserver until the volume is mounted within the namespace.
Instance View 1 of 3
cluster1::> volume show -vserver vs7 -volume vs7_vol1
Vserver Name: vs7

Volume Name: vs7_vol1
Aggregate Name: aggr1b
Volume Size: 400MB
Volume Data Set ID: 1026
Volume Master Data Set ID: 2147484674
Volume State: online
Volume Type: RW
Volume Style: flex
Is Cluster-Mode Volume: true
Export Policy: default
User ID: root
Group ID: daemon
Security Style: unix
UNIX Permissions: ---rwxrwxrwx
THE VOLUME SHOW COMMAND: INSTANCE VIEW 1 OF 3

This is a detailed volume show command. Type this to show a summary view of all volumes. Show a
specific Vserver and volume to see the instance (detailed) view of the volume rather than the summary list of
volumes.
Junction Path: /vol1
Junction Path Source: RW_volume
Junction Active: true
Parent Volume: vs7root
Comment:
Available Size: 18.88GB
Total User-Visible Size: 19GB
Used Size: 120MB
Used Percentage: 5%
Autosize Enabled (for flexvols only): false
Maximum Autosize (for flexvols only): 23.91GB
Autosize Increment (for flexvols only): 1020KB
Total Files (for user-visible data): 566
Files Used (for user-visible data): 96
Space Guarantee Style: volume
Space Guarantee in Effect: true
Space Reserved for Snapshots: 5%
Snapshot Reserve Used: 63%
Snapshot Policy: default
Creation Time: Tue Oct 11 14:34:35 2011
Clone Volume: false
NVFAIL Option: off
Is File System Size Fixed: false
Extent Option: off
Reserved Space for Overwrites: 0B
Fractional Reserve: 100%
Snapshot Cloning Dependency: off
Primary Space Management Strategy: volume_grow
Read Reallocation Option: off
Block Type: 64-bit
Mounting a Volume
You should specify a junction path when you

create a volume. This action immediately puts
the volume into the namespace.
If you create a volume without mounting the
volume to the namespace of the Vserver, the
volume is inaccessible to clients until the
volume is mounted.
You use the volume mount and volume
unmount commands to change the junction
path of a volume in its namespace.
MOUNTING A VOLUME
When volumes are created by using the volume create command, a junction path is usually specified.
The junction path is optional; a volume can be created and not mounted into the namespace. To put a volume
without a junction path into use, you must use the volume mount command to assign a junction path to the
volume.
When you unmount a volume, you take the volume out of the namespace. An unmounted volume is
inaccessible to NFS and CIFS clients but is still online and can be mirrored, backed up, moved, and so on.
You can then mount the volume again to the same location or a different location in the namespace and in
relation to other volumes. For example, you can unmount a volume from one parent volume and then mount
the volume to another parent volume.
Volumes, Junctions, and Namespaces
1 of 3
Volume: root:
Junction path (relative to the root): /
NFS mount command:
mount <hostname>:/ /mnt/vserver1
NFS path: /mnt/vserver1
Volume: smith:
Junction path: /user/smith
User: a directory in the root volume in this example, not a
junction
NFS path: /mnt/vserver1/user/smith
NOTE: The NFS mount command is different from a cluster

shell volume mount command.
VOLUMES, JUNCTIONS, AND NAMESPACES: 1 OF 3

You must be able to differentiate how the volume hierarchy looks to an administrator (internally) from how
the namespace looks to an NFS or CIFS client (externally).
An administrator can choose the name of the root volume of a Vserver (and therefore, the root of the
namespace). However, the junction path of the root volume is always /. Note that the junction path for (the
mountpoint of) a volume is not tied to the name of the volume. In this example, the prefix smith_ is used
for the volume smith_mp3 to associate it with the volume smith, but this approach is only a convention to
make the relationship between the smith volume and its mp3 volume more obvious to a cluster administrator.
Ultimately the volume smith_mp3 is mounted at a directory (media) that was created in the volume named
smith.
2 of 3
cluster1::>
cluster1::>
cluster1::>
cluster1::> mount
mount
mount
mount vserver
vserver
vserver
vserver vs1
vs1 vol
vs1 volvs1 vol
vol smith
smith_mp3
smith_jpg acct
client% mkdir /user/smith/media
/user
junction-path
junction-path
junction-path
junction-path /user/smith/media/music
/user/smith/media/photos
/user/smith
/acct
root
user
acct /acct smith /user/smith

media
smith_mp3 smith_jpg
/user/smith/media/music /user/smith/media/photos

This slide shows a representation of the volumes of this namespace. The volume names are shown inside the
circles, and the junction paths are listed outside of them. Note that no volume is called user. The user
entity is a directory within the root volume, and the junction for the smith volume is located in that directory.
The acct volume is mounted directly at the /acct junction path in the root volume.
3 of 3
Volume: smith_mp3:
Junction path: /user/smith/music
NFS path:
/mnt/vserver1/user/smith/media/music
CIFS path (with a share that is called root_share):
\\<data_ip>\root_share\user\smith\media\
music
User and Group Quotas
Limit resource use

vol1
Track resource use
Notify users of high
resource use
qtree3
qtree2
qtree1
USER AND GROUP QUOTAS

Quotas provide a way for you to restrict and track the disk space and number of files that are used by users,
groups, and qtrees. You apply quotas to specific volumes and qtrees. Clustered Data ONTAP 8.0 introduced
the ability to create qtrees on clustered Data ONTAP volumes but lacked full quota support. Clustered Data
ONTAP enables you to apply user and group quota rules to qtrees. You can use quotas to:
Limit the amount of disk space or the number of files that can be used by a user or group
Limit the amount of disk space or the number of files that can be contained by a qtree
Track the amount of disk space or the number of files that are used by a user, group, or qtree without
imposing a hard limit
Warn users when their disk use or file use reaches a predefined threshold
Quota Policies
vserver1
quota policy - unassigned
quota policy - assigned quota policy
quota rule--unassigned
vol1
quota policy
quota rule unassigned
vol1
quota rule vol1 quota
quota rule-unassigned
policy vol1
vol2
quota rule
quota rule vol2
quota rule vol2 quota rule vol1
quota rule vol2
vol3
quota
quota rule vol3
quota rule vol3 quotarule
quota rule vol2
vol3
quota
quota
quota quota
quota
quota
quota
quota quota
quota
quota
quota quota
A set of quota rules for all of the volumes of a Vserver

No sharing among Vservers
Up to five quota policies per Vserver:
The Vserver enables backup copies of quota policies.
One quota policy can be active on a Vserver at a time.
QUOTA POLICIES
Quotas are defined by quota rules. Quota rules are collected in the quota policy of a Vserver and are specific
to a volume. A quota rule has no effect on the volume until the quota rule is activated.
A quota policy is a collection of quota rules for all of the volumes of a Vserver. Quota policies are not shared
among Vservers. A Vserver can have up to five quota policies, which enables you to have backup copies of
quota policies. One quota policy is assigned to a Vserver at any given time.
A quota is the actual restriction that the Data ONTAP operating system enforces, the actual tracking that the
system performs, or the actual threshold that triggers the system to send a warning message. A quota rule
always results in at least one quota and might result in many additional derived quotas.
Activation is the process of triggering the Data ONTAP operating system to create enforced quotas from the
current set of quota rules in the assigned quota policy. Activation occurs on a volume-by-volume basis. The
first time that quotas are activated on a volume is called initialization. Subsequent activation of quotas on the
same volume is called either re-initialization or resizing, depending on the scope of the changes.
Lesson 2
LESSON 2
FlexCache Volumes
A A
F1 F2 F1 F2
a b c d e f d' e f
g h i j k l
m n o p q r m n q r
s t u v w x F3 s t u y F3
y z 1 2 3 4 y z
Origin volume FlexCache volume
FLEXCACHE VOLUMES
A FlexCache volume is a sparsely-populated volume on a cluster node, that is backed by a FlexVol volume. It
is usually created on a different node within the cluster. A FlexCache volume provides access to data in the
origin volume without requiring that all the data be in the sparse volume. You can use only FlexVol
volumes to create FlexCache volumes. However, many of the regular FlexVol volume features are not
supported on FlexCache volumes, such as Snapshot copy creation, deduplication, compression, FlexClone
volume creation, volume move, and volume copy. You can use FlexCache volumes to speed up access to
data, or to offload traffic from heavily accessed volumes. FlexCache volumes help improve performance,
especially when clients need to access the same data repeatedly, because the data can be served directly
without having to access the source. Therefore, you can use FlexCache volumes to handle system workloads
that are read-intensive. Cache consistency techniques help in ensuring that the data that is served by the
FlexCache volumes remains consistent with the data in the origin volumes.
Benefits of FlexCache
Accelerate Simplify Data

Performance Management Reduce TCO
______________ ______________ ______________
Scale Single vendor Eliminate
application No rip and overhead of full
performance replace replication
Decrease Simple storage Reduce
latency at management hardware costs
remote sites Adjust
automatically to
changing
workloads
Clustered Data ONTAP 8.2 Logical Storage Management Guide
BENEFITS OF FLEXCACHE
Use FlexCache to accelerate performance:
Scale application performance easily
Decrease latency at remote sites
Simplify data management
Single vendor:
No rip and replace
Common and simple storage management
Use FlexCache to reduce TCO:
Eliminate overhead of full replication
Reduce hardware costs, power, and cooling
Adjust automatically to changing workloads
For considerations and limitations when using FlexCache, consult the Clustered Data ONTAP 8.2 Logical
Storage Management Guide.
Supported Protocols
NFS
NFSv3
NFSv4
CIFS
SMB 1.0
SMB 2.x
SMB 3.0
SUPPORTED PROTOCOLS
FlexCache volumes support client access using the following protocols: NFSv3, NFSv4.0, and CIFS (SMB
1.0, 2.x, and 3.0).
Reasons to Deploy FlexCache
A. Decrease latency
B. Increase IOPs
C. Balance resources
A A A A
REASONS TO DEPLOY FLEXCACHE

A data volume, when created, is stored on a specific node of the cluster. That volume can move within the
cluster but, at any point in time, only one node contains the source data.
FlexCache volumes decrease latency by enabling multiple nodes of a cluster to respond to read requests
efficiently, without having to send data over the cluster interconnect.
Nodes can occasionally encounter spikes of high performance during certain tasks or activities to a
specific data set. By caching copies of data throughout the cluster, FlexCache volumes take advantage of
cycles available on all nodes within the cluster, thereby increasing capacity to maximize I/O operations.
If there is intensive access to the data on the origin volume, then the host node can get overloaded, and
develop a performance bottleneck. FlexCache volumes balance resources by spreading the workload
across the cluster, smoothing out the performance despite heavy read or metadata access.
Types of Volumes
A Origin volume
FlexCache volume
A
A A A A
TYPES OF VOLUMES
Two types of volume relevant to FlexCache are the origin volume and the FlexCache volume. The origin
volume is a FlexVol volume that is the primary copy of the volume. A FlexCache volume maps to a single
origin volume, so files can partially exist on the FlexCache volume, based on use patterns, but are seen by the
client as an entire file.
FlexCache Configuration
Commands to manage FlexCache volumes:
::> volume flexcache create

::> volume flexcache show
::> volume create
::> volume flexcache cache-policy create
::> volume flexcache cache-policy show
::> volume modify
FLEXCACHE CONFIGURATION
You use the volume flexcache commands to create, delete, and display information about FlexCache
volumes on all nodes in the cluster, or to create, modify, and delete cache policies. You can use the volume
family of commands to perform many of the same operations on individual volumes.
Create a FlexCache volume on all the nodes spanned by a Vserver in a cluster:
volume flexcache create
Display information about all FlexCache volumes in the cluster: flexcache show
Create a FlexCache volume on a single node: volume create
Create a cache policy: volume flexcache cache-policy create
Display the cache policies for all Vservers: volume flexcache cache-policy show
Apply a cache policy to a single volume: volume modify
Cache Policies
Define properties of FlexCache volumes:
Staleness of data
Timeout for unused cache delegation
Enable the FlexCache volume to serve read
requests that are local to the origin volume
::> vol flexcache cache-policy create
vserver vs1 policy vs1uselocal
-prefer-local-cache true
::> vol modify vserver vs1 volume vol1

-flexcache-cache-policy vs1uselocal
CACHE POLICIES
A cache policy is a set of parameters that help you define properties of FlexCache volumes, such as the extent
of staleness of data in FlexCache volumes, the time after which an unused delegation is returned to the origin,
and the parameter that enables the FlexCache volume to serve read requests from a node that also has the
origin volume. Cache policies are defined for the Vserver that contains the volumes. You can use the default
cache policy or configure your own cache policies and apply them to FlexCache volumes in a Vserver.
Every Vserver has a default cache policy. The default cache policy is a special cache policy that is created and
deleted along with the Vserver. FlexCache volumes use the default cache policy when no other cache policies
are present. The default cache policy can be modified but not deleted.
Lesson 3
LESSON 3
Infinite Volumes
Definition
Applications Applications OnCommand System Manager or Cluster Shell
NFSv3, NFSv4.1, pNFS, CIFS
Clustered Data ONTAP 8.2 Operating System
Single Namespace, Single Storage Pool
Up to 2 Billion Files (16-TB Maximum File Size)
NAS Container (More than 20 PB of Raw Capacity)
Two to Ten Nodes in

a Cluster
.......
Data constituent at the silver storage service level

Namespace constituent Data constituent at the gold storage service level
INFINITE VOLUMES: DEFINITION

Infinite volumes are boundless, easily administered, scalable containers that exceed the current Data ONTAP
limits for FlexVol capacity. Infinite volumes are ideal for customers who need a single storage pool in a
single namespace across an entire cluster. Infinite volumes consist of many FlexVol volumes but are
presented to the storage administrator as a single manageable volume. The constituents of an infinite
volume are standard FlexVol volumes. Individual constituents are not managed by the storage administrator.
Although the constituents are not visible in the default displays, the administrator can choose options to view
the constituents.
Infinite Volumes
Constituent View
NAS client access to a single external mountpoint
No client access
directly to the data
constituents
Infinite Volume
Namespace Data Data Data

Constituent Constituent-1 Constituent-2 Constituent-3
Aggregate-1 Aggregate-2 Aggregate-3 Aggregate-4
Node-1 Node-2
INFINITE VOLUMES: CONSTITUENT VIEW

An infinite volume is composed of one namespace constituent and several data constituents. The namespace
constituent contains the namespace of the infinite volume. The namespace constituent maps path names to
data that is stored on data constituents. A data constituent contains the data of one or more regular files and is
one of the containers that contribute to the storage capacity of an infinite volume. Access from a client is
made to a redirector file within the namespace constituent. The I/O requests are routed to one of the many
data constituents that are referenced by the redirector file. This redirection happens transparently from the
clients perspective. By introducing this redirection technology, an infinite volume can present a single
junction namespace that can grow to over 20 PB.
Redirector and Data Files
Data Constituents
/NS/
/NS/bak/ /NS/tmp/
Data File Data File
/NS/bak/img /NS/tmp/file
Redirector
Redirector
Namespace Constituent
Because of storage failover (SFO), access to data continues if only one

node fails.
Directory operations are unaffected.
File operations to other data constituents are unaffected.
The infinite volume continues to be available until all constituents fail.
REDIRECTOR AND DATA FILES

Infinite volumes have two types of constituents:
Data:
Has multiple data constituents (usually at least one per storage node in a cluster)
Contains the data of each file
Namespace:
The namespace is built on a single FlexVol volume.
The FlexVol volume contains a pointer (per file) to a file in the data constituent.
As new files are written, infinite volumes balance capacity across each data constituent and distribute files
across the cluster. Individual files are not striped; they are fully contained within a data constituent.
Resiliency
The Loss of a Data Constituent
Data Constituents
/NS/
/NS/bak/ /NS/tmp/ data file
Data File
/NS/bak/img /NS/tmp/file
Redirector
Redirector
Because of SFO, access to data continues if just one node fails.

Directory operations are unaffected.
File operations to other data constituents are unaffected.
The infinite volume continues to be available until all constituents fail.
RESILIENCY: THE LOSS OF A DATA CONSTITUENT

When a data constituent comes offline:
Directory operations such as mkdir are unaffected.
File operations (lookup, read, write, getattr, and setattr) to the remaining data
constituents are unaffected.
Operations that are distributed over multiple constituents (create and readdirplus) might require
more time and yield partial results.
Resiliency
The Loss of a Namespace Constituent
Data Constituents
Data File Data File
If the host of the namespace constituent and its SFO partner are both down:
Namespace and directory operations do not work.
File operations on recently accessed files continue to work.
If only the host fails and the SFO partner takes over, access to the namespace,
and the infinite volume, functions normally.
RESILIENCY: THE LOSS OF A NAMESPACE CONSTITUENT

When a namespace constituent comes offline:
Namespace and directory operations such as mkdir do not work.
File operations on recently accessed files (read, write, getattr, and setattr) continue to work
because of the caching of file-handle mapping in memory.
Infinite Volumes
Feature Summary
Manageability NFSv3, NFSv4.1, CIFS
A single namespace Configuration
A simplified setup through A single container of more than 20
OnCommand System Manager 2.1 PB (raw)
Management through cluster shell, Support for up to 2 billion files
similar to that of a FlexVol volume
Up to 10 nodes
Constituent management with
diagnostic privilege Reliability: SFO
Automatic capacity balancing at Flash Cache
file-creation time Efficiency
Data protection Deduplication
Snapshot copies Compression
SnapMirror (intercluster) software
NFS-mounted tape backups
See TR4078 for Infinite Volume

recommended practices
INFINITE VOLUMES: FEATURE SUMMARY
Infinite Volumes
New 8.2 Features
Sharing of a cluster
Multiple Vservers
Sharing of aggregates with FlexVol volumes
Unified security style
Data protection
Namespace mirror constituents
Fan-out and bidirectional mirror relationships
Multiple hardware platforms (not supported for the
FAS2000 series)
INFINITE VOLUMES: NEW 8.2 FEATURES

Infinite volumes can now coexist with FlexVol volumes on aggregates, and Vservers that are enabled for
infinite volumes can coexist with Vservers that serve FlexVol volumes.
Infinite Volumes always use unified security style. Unified security style allows all users to view and set file
permissions regardless of whether they are CIFS or NFS clients. Further, it allows the file permissions to
include both Windows and UNIX users and groups, allowing more simplified security management.
Another feature of unified security is support for unified ACLs, which consolidates file permissions for both
UNIX and Windows users and groups in a single access control list. Unified ACLs facilitate access checks
using both Windows and UNIX credentials. They provide multiprotocol access to files without relying on
user mapping between UNIX and Windows users.
Clustered Data ONTAP 8.2 enables you to perform disaster recovery, from the namespace mirror constituent,
of a namespace constituent that has been permanently lost. Infinite volumes can have mirror relationships
with infinite volumes in other clusters in addition to fan-out and bidirectional mirror relationships.
Infinite volumes are also now supported on all current NetApp platforms, except the FAS2000 series.
Infinite Volumes
Unsupported Features
SMB 2.x and SMB 3 FAS2000 series platforms
NFS 4.1 delegations Sharing of root volume via SMB
Qtrees and quotas Active capacity rebalancing
CIFS home directories Change notification
Fpolicy Scalable SAN
File movement across data constituents SMB 1.0 or NFSv4.1 on active filesystems of read-
only volumes
FlexCache SMB tools such as sectrace and fsecurity
SnapVault Single file snapshot restore
Single-node clusters
INFINITE VOLUMES: UNSUPPORTED FEATURES

Infinite volumes in clustered Data ONTAP 8.2 do not support some features, including single-node clusters,
qtrees and quotas, FlexCache software, and SnapVault software. Review the list for additional unsupported
features.
Infinite Volumes
Limitations
Clusters of up to 10 nodes in 5 high-availability (HA) pairs
One infinite volume per Vserver
One infinite volume constituent per aggregate
Total raw capacity of approximately 21 PB in a 10-node cluster:
Each aggregate has 2 RAID-DP groups; each group has 18 data and
2 parity disks; each disk is 3-TB SATA.
One constituent exists per aggregate or 175 total constituents.
The maximum useable capacity is approximately 13.15 PB.
Support for up to 2 billion data files:
The namespace constituent can have up to 2 billion redirector files.
Each data constituent can have up to 100 million data files.
A maximum file size of 16 TB
INFINITE VOLUMES: LIMITATIONS

Infinite volumes have some limitations, depending on the size of the cluster. Each aggregate can contain only
one namespace or data constituent, but up to 20 constituents can reside on each node. A data Vserver can
contain only one infinite volume.
Infinite Volumes
Example
cluster1::> aggr create -aggregate aggr1 -diskcount 70
cluster1::> aggr create -aggregate aggr2 -diskcount 70
cluster1::> vserver create -vserver vs0 -rootvolume vs0_root -is-repository true ...
Cluster1::> set advanced
cluster1::*> volume create -vserver vs0 -volume repo_vol -junction-path /NS -size 768GB
cluster1::*> volume show -volume repo_vol

--------- ------------ ------------ ---------- ---- ---------- ---------- -----
vs0 repo_vol - online RW 768.0GB 729.3GB 5%
cluster1::*> volume show is-constituent true

Vserver Volume Aggregate State Role Type Size Available Used%
-------- ------------ --------- ------- --------- ---- -------- --------- -----
vs0 repo_vol_default_data0001
aggr2 online data RW 109.7GB 104.2GB 5%
repo_vol_default_data0002
repo_vol_ns aggr1 online namespace RW 110.0GB 104.4GB 5%
INFINITE VOLUMES: EXAMPLE

Creating infinite volumes and the aggregates that host infinite volumes is similar to creating FlexVol
volumes. You create aggregates throughout the cluster to host constituent volumes. You create a Vserver that
can serve infinite volumes by using the -is-repository switch. Then you create the infinite volume to fit
the capacity of the constituent aggregates. The volume show command shows you the infinite volume. The
volume show command with -is-constituent true displays the list of constituent volumes.
Module Summary

should be able to:
Create a volume
Build a namespace by using multiple volumes
Configure FlexCache storage
MODULE SUMMARY
Exercise
Module 6: Logical Data Storage
EXERCISE
Module 7
Physical Networking
MODULE 7: PHYSICAL NETWORKING
7-1 Clustered Data ONTAP Administration: Physical Networking
Module Objectives

Draw the connections of the network cables
from the three networks to a controller
Explain port roles
Create an interface group
Configure virtual LAN (VLAN) tagged ports
Identify supported cluster interconnect
switches
Discuss switchless two-node clusters and
single-node clusters
MODULE OBJECTIVES
Network Ports
Physical network ports exist on a controller, with
corresponding network port definitions in the Data
ONTAP operating system:
Node-management ports (by default, one for each node)
Cluster ports (by default, two for each node)
Data ports (by default, two for each node)
Intercluster ports (by default, none)
The defaults might not be the optimal configuration for
your particular installation.
FC SAN environments use host bus adapter (HBA) ports
as data ports.
NETWORK PORTS
Clustered Data ONTAP distinguishes between physical network ports and logical interfaces (LIFs). Each port
has a role that is associated with the port by default, although that situation can be changed through the UI.
The role of each network port should align with the network to which the port is connected.
Node-management ports are for administrators to connect to the node or cluster; for example, through Secure
Shell (SSH) or a web browser.
Cluster ports are strictly for intracluster traffic.
Data ports are for NAS and SAN client access and for the cluster management LIF.
Intercluster ports are used to communicate with another cluster.
The Network Ports of a Node
FC 0c Management e0M Data Cluster X1107A Dual-

e1b and e1a Port 10-GbE Card
Data e0a e2b
X1107A
LINK/ACT
LINK/ACT
1 Chelsio
Communications
c0a 0c e0a
X1107A
LINK/ACT
LINK/ACT
0a 0b 2 Chelsio
Communications
LNK LNK
c0b 0d e0b
X1139A
10GbEE CNA
3 5
SAN LAN
SAN LAN
PORT 1
PORT 2
4 6
Data e3a and e3b FC 0d Data e0b Console Cluster

e2a
X1139A Dual-
Port UTA Card
THE NETWORK PORTS OF A NODE

Using the FAS3270 system as an example, the e1a and e2a ports are defined as having the role of a cluster.
The e0a, e0b, e1b, e2b, e3a, and e3b ports are defined for data. The e1a and e1b ports are on a network
interface card (NIC) in slot 1 at the top right of the controller. The e2 and e3 ports are also found on NICs.
The e0M port is, by default, defined with the role of management. In an FC SAN environment, the FC ports
can also assume the role of serving data.
Intercluster (or cross-cluster) mirror relationships are created over ports that have the role of intercluster or
data. Intercluster port roles are similar to data port roles, except that intercluster port roles are reserved for
intercluster traffic. Intercluster ports can fail over to data ports.
The Roles of Network Ports
Cluster ports are used for communication to

the private cluster network.
Data ports are used to communicate with
client hosts.
Node-management ports are used by
administrators to configure the node.
Intercluster ports are used for communication
to a peer cluster.
THE ROLES OF NETWORK PORTS

Default port roles vary by platform but can be reassigned by using the network port modify command.
Two cluster ports are required per node.
The network port show Command
cluster1::> net port show
(network port show)
Auto-Negot Duplex Speed (Mbps)
Node Port Role Link MTU Admin/Oper Admin/Oper Admin/Oper
------ ------ ------------ ---- ----- ----------- ---------- ------------
cluster1-01
e0a cluster up 9000 true/true full/full auto/1000
e0b cluster up 9000 true/true full/full auto/1000
e0c data up 1500 true/true full/full auto/1000
e0d data up 1500 true/true full/full auto/1000
e1a node-mgmt up 1500 true/true full/full auto/1000
e1b data down 1500 true/true full/half auto/10
cluster1-02
e0a cluster up 9000 true/true full/full auto/1000
e0b cluster up 9000 true/true full/full auto/1000
e0c data up 1500 true/true full/full auto/1000
e0d data up 1500 true/true full/full auto/1000
e1a node-mgmt up 1500 true/true full/full auto/1000
e1b data down 1500 true/true full/half auto/10
THE NETWORK PORT SHOW COMMAND

The network port show command shows the summary view of the ports of this two-node cluster. All
the ports are grouped by node. You can see the roles that are assigned to the ports and the ports statuses and
maximum transmission unit (MTU) sizes. Note that the e1b data ports are on the nodes but are not connected.
The network fcp adapter show
Command
cluster1::> network fcp adapter show
Connection Host
Node Adapter Established Port Address
------------ ------- ----------- ------------
cluster1-01 0c ptp 4b0038
cluster1-01 3a ptp 4b0036
cluster1-01 3b loop 0
cluster1-02 0c ptp 4b0061
cluster1-02 4a ptp 4b005f
THE NETWORK FCP ADAPTER SHOW COMMAND

The network fcp adapter show command shows the summary view of the FC ports of this two-node
cluster. All the ports are grouped by node. You can see the port addresses that are assigned to the ports and
the ports connection types that are established. In an FC SAN environment, these ports can assume a data
port role.
Modifying Network Port Attributes
cluster1::> net port modify ?
(network port modify)
[-node] <nodename> Node
[-port] {<netport>|<ifgrp>} Port
[[-role] {cluster|data|node-mgmt|intercluster|cluster-mgmt}]
Role
[ -mtu <integer> ] MTU
[ -autonegotiate-admin {true|false} ]
Auto-Negotiation Administrative
[ -duplex-admin {auto|half|full} ] Duplex Mode Administrative
[ -speed-admin {auto|10|100|1000|10000} ]
Speed Administrative
[ -flowcontrol-admin {none|receive|send|full} ]
Flow Control Administrative
cluster1::> network port modify -node cluster1-01 -port e0b

-flowcontrol-admin none
MODIFYING NETWORK PORT ATTRIBUTES

You can modify the MTU, autonegotiation, duplex, flow control, and speed settings of a physical network or
interface group. You can modify only the MTU settings and not other port settings of a virtual LAN (
VLAN). The port to be modified must not be hosting any LIFs.
You should not modify the following characteristics of a network port:
The administrative settings of either the 10-GbE or the 1-GbE network interfaces
The administrative settings of the underlying physical ports in an interface group
The MTU size of the management port, e0M
The MTU size of a VLAN cannot exceed the value of the MTU size of its base port.
The values that you can set for duplex mode and port speed are referred to as administrative settings.
Depending on network limitations, the administrative settings can differ from the operational settings (that is,
the duplex mode and speed that the port actually uses).
NOTE: You should set the flow control of all ports to none. By default, the flow control is set to full.
The Interface Group
Is the same as a virtual interface (vif) in the

Data ONTAP 7G operating system
Is the same as a port trunk in the Data
ONTAP GX operating system
Combines two or more physical ports and
presents them as a single logical port
Provides redundancy and more bandwidth
THE INTERFACE GROUP

An interface group (ifgrp) combines multiple physical ports into a single logical port.
An ifgrp is also known as an Ethernet trunk, 802.3ad link aggregation, or port bonding. This term is the Data
ONTAP 8.1.1 name for what was called a virtual interface (vif) in Data ONTAP 7G architecture and a port
trunk in Data ONTAP GX architecture. The network module groups multiple physical ports and makes them
available as a single port. The grouping is configured as either an active port and a standby port (active-
passive or single-mode) or with all of the ports in the ifgrp sharing the load (active-active or multimode).
When using multimode, you can activate the Link Aggregation Control Protocol (LACP) to negotiate port
groupings with the switch. The switch must be configured to support link aggregation.
VLANs
VLANS
A port can be subdivided into multiple VLANs. Each VLAN has a unique tag that is communicated in the
header of every packet. The switch must be configured to support VLANs and the tags that are in use. In
clustered Data ONTAP, a VLAN's ID is configured into the name. So VLAN "e0a-25" is a VLAN with tag 25
configured on physical port e0a. VLANs that share a base port can belong to the same or different IP spaces,
and it follows that the base port could be in a different IP space than its VLANs.
VLANs and Interface Groups
vlan vlan ifgrp
port port port
vlan vlan
ifgrp
port port
VLANS AND INTERFACE GROUPS

Ports are the physical network adapters that are cabled to a switch or a virtualization that groups or subdivides
the physical ports. Ports can be arranged in a hierarchy, with VLANs subdividing ifgrps. In clustered Data
ONTAP, ifgrps cannot be created from other ifgrps, and VLANs cannot be created from other VLANs.
Additionally, ifgrps cannot be created from VLANs.
NOTE: You cannot create VLANs or interface groups on cluster ports.
Cluster Network Standardization
Approach
This configuration is standard for cluster interconnect
switches in clustered Data ONTAP configurations.
New clusters require the standard switch configurations for
the cluster and management network.
Benefits
This solution is engineered by NetApp.
Using this solution guarantees that best practices for
networking design are followed:
Dual-cluster interconnect switches for redundancy
Sufficient Inter-Switch Link ( ISL) bandwidth
Standard hardware, software, and configurations
Faster problem resolution (using known configurations)
CLUSTER NETWORK STANDARDIZATION

To provide a complete storage solution, the Data ONTAP 8 operating system introduced a cluster
interconnect and management switch-configuration standard to which all clustered Data ONTAP deployments
are required to adhere. Clustered Data ONTAP 8.2 continues this policy to provide the best solution in terms
of reliability and serviceability.
The NetApp Cluster Interconnect and
Optional Cluster Management Switch
Whats New
A lower-cost solution for eight-node or smaller cluster sizes
Support that starts in the Data ONTAP 8.1.1 operating system
Sixteen ports of 10-GbE cluster or GbE management
Cluster Network: CN1610 Management Network: CN1601

Low-latency, nonblocking Sixteen ports GbE RJ-45
sixteen 10-GbE enhanced small Dual fans; a single power
form-factor pluggable ( SFP+) supply
Ethernet ports
Dual fans and power supplies
THE NETAPP CLUSTER INTERCONNECT AND OPTIONAL CLUSTER

MANAGEMENT SWITCH
Cluster Interconnect Requirements
Cluster interconnect
NetApp CN1610
Cisco Nexus 5596 (New with Data ONTAP 8.1.2)
Wire-rate 10-GbE connectivity between storage controllers
A 1 x 10-GbE connection from each node to each switch
(2 ports per node total)
Interswitch bandwidth: four ports per switch with CN1610; eight
ports per switch on Cisco Nexus 5010 and 5020
Cluster management switch for:
Management connections for storage controllers and shelves
NetApp CN1601
Cisco Catalyst 2960
CLUSTER INTERCONNECT REQUIREMENTS

The FAS2040 system connects into a cluster by using onboard 1-GbE ports. The first 8 ports of the Cisco
Nexus 5010 and the first 16 ports of the Cisco Nexus 5020 can be 1-GbE or 10-GbE, depending on the small
form-factor pluggable (SFP) that is used. NetApp has released a new 1-GbE SFP to enable the FAS2040
system to participate in clusters. All other controllers remain at 10-GbE. The best practice is to not mix 1-G
and 10-G nodes.
A cluster that uses Nexus 5010 switches for the cluster network can have a maximum of eight FAS2040
system nodes in the cluster.
A cluster that uses Nexus 5020 switches for the cluster network can have a maximum of 16 FAS2040 system
nodes in the cluster.
The NetApp CN1610 and Cisco Nexus 5596 will replace the 5010 and 5020 respectively.
Cluster Configuration Overview
2 to 8 Nodes 2 to 24 Nodes
Interconnect
Cluster
2 NetApp CN1610 2 Cisco Nexus 5596
16 x 10-Gbps Ethernet 48 x 10-GbE ports:
ports enhanced (SFP+): Eight ports are used for
Four ports are used for ISLs.
Inter-Switch Links (ISLs). 2 rack units each
1 rack unit each
Network
Management
2 NetApp CN1601 2 Cisco Catalyst 2960

16 ports of 1-Gbps 24 ports of 10/100
Ethernet RJ-45 Ethernet RJ-45
1 rack unit each 1 rack unit each
CLUSTER CONFIGURATION OVERVIEW

Cisco Nexus 5010 switches are no longer available, but are still supported for clusters up to 18 nodes as of
Data ONTAP 8.2.
Cisco Nexus 5010 switches are no longer available, but are still supported for clusters up to 24 nodes as of
Data ONTAP 8.2.
Configuration Overview
Ability to Be Supported
Maximum
Function Switch Configured in a Network Interface
Nodes
NetApp Cabinet Cards (NICs)
Cluster X1117A-R6 X1107A-
NetApp CN1610 8 Yes R6 X1008A-R6
interconnect
X1117A-R6
Cluster
Cisco Nexus 5596 24 Yes X1107A-R6
interconnect X1008A-R6
Management NetApp CN1601 16 Yes On-board ports only
Cisco Catalyst On-board ports only

Management 24 Yes
2960-24TT
For switch setup and configuration information:

Clustered Data ONTAP Switch Setup Guide for Cisco Switches
CN1601 and CN1610 Switch Setup and Configuration Guide
CONFIGURATION OVERVIEW
Network Cabling
Cluster Interconect
Cisco Nexus 5010
SLOT2
STAT
PS1
PS2
CONSOLE
L1 L2 MGMTO MGMT1 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 10 19 20
Cisco Nexus 5010
SLOT2
STAT
PS1
PS2
CONSOLE
L1 L2 MGMTO MGMT1 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 10 19 20
node1 node2 node3 node4

X1107A
X1107A
X1107A
LINK/ACT
LINK/ACT
LINK/ACT
LINK/ACT
LINK/ACT
LINK/ACT
1 1 1
X1107A
Chelsio Chelsio Chelsio
LINK/ACT
LINK/ACT
Communications Communications Communications 1 Chelsio
Communications
c0a 0c e0a c0a 0c e0a c0a 0c e0a c0a 0c e0a
X1107A
X1107A
X1107A
LINK/ACT
LINK/ACT
LINK/ACT
LINK/ACT
LINK/ACT
LINK/ACT
2 2 2
X1107A
0a 0b Chelsio 0a 0b Chelsio 0a 0b Chelsio
LINK/ACT
LINK/ACT
Communications Communications Communications 0a 0b 2 Chelsio
Communications
LNK LNK LNK LNK LNK LNK
LNK LNK
c0b 0d e0b c0b 0d e0b c0b 0d e0b c0b 0d e0b
3 5 3 5 3 5 3 5
4 6 4 6 4 6 4 6
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24
Catalyst 2960-S Series PoE+ 10G
CONSOLE M
G Cisco Nexus 5010
SLOT2
M
T
SYST 1X 11X 13X 23X B
A
RPS
STAT
S
PS1
E
MSTR T
STAT
DPLX
SPED
STCK
PoE
PS2
2X 12X 14X 24X
MODE CONSOLE
1 SFP+ 2 L1 L2 MGMTO MGMT1 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 10 19 20
Management Data Network

Network
NOTE: NetApp recommends switch redundancy for both data and

NETWORK CABLING
This slide shows a four-node cluster. Typically, two distinct networks exist for a cluster. The cluster traffic
must always be on its own network, but the management and data traffic can coexist on a network.
Two cluster connections to each node are required for redundancy and improved cluster traffic flow.
For proper configuration of the NetApp CN1601 and CN1610 switches, refer to the CN1601 and CN1610
Switch Setup and Configuration Guide.
Single-Node Clusters
Not supported on Single-node clusters:
Cluster ports
Cluster logical interfaces (LIFs)
High availability
Nondisruptive operations
Nondisruptive upgrades
Storage failover
Single-node clusters differ from one-node clusters
SINGLE-NODE CLUSTERS
Before Data ONTAP 8.2, each cluster required two cluster ports and two cluster LIFs. The Single-Node
Cluster feature eliminates the requirement for cluster LIFs in one-node configurations. The cluster ports are
free to be configured as additional data ports.
You can create a single-node cluster with the cluster setup wizard. Creating a single-node cluster from the
cluster setup wizard results in a node without cluster LIFs. The ports that would otherwise be created as
cluster ports are instead created as data ports. The node is configured as non-high availability ( non-HA). A
single-node cluster is the only supported cluster configuration without an HA partner.
Note that with a single-node cluster, some operations are disruptive. For example, because there is no HA
partner, there is no storage failover. When the single node reboots on a panic or during an upgrade, there is a
temporary interruption of service.
A single-node cluster is different from a one-node cluster. A single-node cluster is a one-node cluster that
does not have cluster LIFs configured, and therefore has no connection to the cluster interconnect used in
multinode clusters. A one-node cluster is attached to the cluster interconnect, with the expectation of growing
the cluster beyond one node. A one-node cluster (with cluster LIFs) is not a supported configuration.
Switchless Two-Node Clusters
Cluster
Interconnect
clus1 clus2 clus1 clus2

e0a e0b e0a e0b
HA Interconnect
cluster1::> set diag
cluster1::*> network options switchless-cluster modify enabled true
SWITCHLESS TWO-NODE CLUSTERS

Two-node switchless clusters enable you to directly attach cluster ports to each other. A two-node switchless
cluster can greatly reduce the cost of a two-node configuration.
To make an existing two-node cluster switchless, directly cable the cluster ports and run the network
options switchless-cluster diagnostic command from the cluster shell. To execute the procedure
nondisruptively, use the following steps:
1. Disable autorevert on all four cluster LIFs.
2. Migrate cluster LIF 1 on each node to cluster port e0b.
3. Disconnect ports e0a on both nodes from the switch, and cable the ports to each other.
4. Enable the switchless cluster feature.
5. Revert cluster LIF 1 on both nodes back to port e0a.
6. Migrate cluster LIF 2 on each node to cluster port e0a.
7. Disconnect ports e0b on both nodes from the switch, and cable the ports to each other.
8. Revert cluster LIF 2 on both nodes back to port e0b.
9. Re-enable autorevert on all four cluster LIFs.
Switchless Two-Node Clusters: Cluster
Setup Wizard
Step 1 of 5: Create a Cluster

You can type "back", "exit", or "help" at any question.
Enter the cluster name: clus1

Enter the cluster base license key: XXXXXXXXXXXXXXXXXXXXXXXXXXXX
System Defaults: Private cluster network ports [e0a,e0b].

Cluster port MTU values will be set to 9000.
Cluster interface IP addresses will be automatically generated.
The cluster will be connected using switches.
Do you want to use this default? {yes, no} [yes]: no

Do you want to setup cluster interfaces for this node? {yes, no} [yes]:
Will the cluster network use switches? {yes, no} [yes]: no
SWITCHLESS TWO-NODE CLUSTERS: CLUSTER SETUP WIZARD

To configure a new two-node switchless cluster, directly cable the cluster ports and run the Cluster Setup
Wizard. In the wizard, reply no when asked if you want to use switches.
IPv6 Support
Supported networks: Supported protocols:

Data network NFS, CIFS, and
Management iSCSI
network SSH, Telnet, RSH,
and SNMP
Networks that are not NDMP, DNS, and
supported: NIS
Intercluster traffic
IPV6 SUPPORT
The IPv6 standard replaces IPv4. IPv6 has a 128-bit address space, which relieves the exhaustion of IPv4
addresses. IPv6 also has other features that make it a rich and complex protocol to deploy and manage.
Clustered Data ONTAP 8.2 extends IPv6 support to cluster data network protocols, including the NFS, CIFS,
and iSCSI, and cluster management network protocols, including SSH, Telnet, RSH, and SNMP. Clustered
Data ONTAP 8.2 also supports NDMP, DNS, and NIS protocols. Clustered Data ONTAP 8.2 does not
support IPv6 on the cluster interconnect or for intercluster mirroring traffic.
Module Summary
should be able to:
Draw the connections of the network cables from
the three networks to a controller
Explain port roles
Create an interface group
Configure VLAN tagged ports
Discuss switchless two-node clusters and single-
node clusters
MODULE SUMMARY
Exercise
Module 7: Physical Networking
EXERCISE
Module 8
Logical Networking
MODULE 8: LOGICAL NETWORKING
8-1 Clustered Data ONTAP Administration: Logical Networking
Module Objectives

Create NAS data logical interfaces (LIFs)
Create a LIF failover group
Migrate and revert a NAS data LIF
Configure DNS load balancing
MODULE OBJECTIVES
LIF Characteristics
An IP address or World Wide Port Name (WWPN) is
associated with a LIF.
One node-management LIF exists per node. It can fail over
to other data or node-management ports on the same
node.
One cluster-management LIF exists per cluster. It can fail
over or migrate throughout the cluster.
Two cluster LIFs exist per node. They can fail over or
migrate only within their node.
Multiple data LIFs are allowed per data port.
The are client-facing (NFS, CIFS, iSCSI, and Fibre Channel
access).
NAS data LIFs can migrate or fail over throughout the cluster.
LIF CHARACTERISTICS
Each logical interface (LIF) has an associated role and must be assigned to the correct type of network port.
Data LIFs can have a many-to-one relationship with network ports: Many data IP addresses can be assigned
to a single network port. If the port becomes overburdened, NAS data LIFs can be transparently migrated to
different ports or different nodes. Clients know the data LIF IP address but do not know which node or port is
hosting the LIF. If a NAS data LIF is migrated, the client might unknowingly be contacting a different node.
The NFS mountpoint or CIFS share is unchanged.
A node can have a maximum of 128 LIFs, regardless of the type of LIF.
The network interface show Command
1 of 2
cluster1::> net int show
(network interface show)
----------- ---------- ---------- ------------------ ------------- ------- ----
cluster1
cluster_mgmt up/up 192.168.239.20/24 cluster1-01 e0M true
cluster1-01
mgmt up/up 192.168.239.21/24 cluster1-01 e0a true
cluster1-02
mgmt up/up 192.168.239.22/24 cluster1-02 e0a true
vs7
vs7_lif1 up/up 192.168.239.74/24 cluster1-01 e3a true
vs7_lif2 up/up 192.168.239.75/24 cluster1-01 e3b false
THE NETWORK INTERFACE SHOW COMMAND: 1 OF 2

LIF names must be unique within their scope. For data LIFs, the scope is a data virtual storage server
(Vserver). The scope of a cluster LIF or management LIF is limited to its node. Thus, the same name (for
example, mgmt1) can be used for all the nodes.
The network interface show Command
2 of 2
----------- ---------- ---------- ------------------ ------------- ------- ----
vs7
vs7_lif1 up/up 192.168.239.74/24 cluster1-01 e3a true
vs7_lif2 up/up 192.168.239.75/24 cluster1-01 e3b false
vs7_fclif1 up/up 20:0f:00:a0:98:13:d5:d4
cluster1-01 0c true
vs7_fclif2 up/up 20:10:00:a0:98:13:d5:d4
cluster1-01 0d true
vs7_fclif3 up/up 20:14:00:a0:98:13:d5:d4
cluster1-02 0c true
vs7_fclif4 up/up 20:12:00:a0:98:13:d5:d4
cluster1-02 0d true
THE NETWORK INTERFACE SHOW COMMAND: 2 OF 2

Instead of IP addresses, FC LIFs use worldwide port names (WWPNs).
Network Ports and Data LIFs
192.168.1.55 (vs1_d2)
192.168.1.51 (vs1_d1)
192.168.1.56 (vs2_d2)
192.168.1.52 (vs2_d1)
192.168.1.57 (vs2_d3)
21:00:00:2b:34:26:a6:54 (vs1_d4)
192.168.1.53 (vs1_d3)
192.168.1.54 (vs3_d1)
e0c e0d e0c 0f
node1 node2
NETWORK PORTS AND DATA LIFS

In the environment that is shown on this slide, each of the nodes contains two data network ports. Network
port node2 has three data LIFs that are assigned to one port and two on the other port. This slide shows the
many-to-one relationship between LIFs and network ports. The data LIF name is in parentheses after each
network address.
In a NAS environment, the name is not the actual host name that is associated with the IP address. The name
is an internal name that can be used as the host name for the IP address in the DNS. In a NAS environment,
all these IP addresses can share one host name, such that a DNS round robin is used and picks an IP address
every time that the host name is used; for example, for an NFS mount command.
This slide shows how an environment can randomly distribute client connections across a cluster while the
cluster looks to every user and every client as if there is only one storage host.
LIF Roles and Compatible Ports
LIF Roles Compatible Port Roles

Cluster Cluster
Data Data
Node-management Node-management or
Data
Cluster-management Node-management or
Data
Intercluster Intercluster or
Data
LIF ROLES AND COMPATIBLE PORTS
LIF Static Routes
Are defined paths between LIFs and specific
destination IP addresses through gateways
Can improve the efficiency of network traffic that
travels through complicated networks
Have preferences that are associated with them:
When multiple routes are available, the metric
specifies the preference order of the route to use.
Are defined within routing groups
Are created or chosen automatically when a LIF
is created
LIF STATIC ROUTES

You can control how LIFs in a Vserver use your network for outbound traffic by configuring routing groups
and static routes. A set of common routes are grouped in a routing group that simplifies the administration of
routes.
A routing group is a routing table in which each LIF is associated with one routing group and uses only
the routes of that group. Multiple LIFs can share a routing group.
NOTE: If, for purposes of backward compatibility, you want one route per LIF, you can create a separate
routing group for each LIF.
A static route is a defined route between a LIF and a specific destination IP address. The route can use a
gateway IP address.
Routing groups are created automatically as new LIFs are created, unless a routing group covers that port and
role or network combination.
The node-management LIFs on each node have static routes that are automatically set up for them through the
same default gateway.
If more than one static route is defined for a LIF, each static route has a metric value. The administrator
uses the metric values to configure routes so that one is preferred over another. The lower the metric value,
the more preferred the route is. The metric value for a node management LIF is 10. When routes are created
for data LIFs, if no metric is defined, the default value is 20.
The network routing-groups show
Command 1 of 2
cluster1::> network routing-groups show
Routing
Vserver Group Subnet Role Metric
--------- --------- --------------- ------------ -------
cluster1
c192.168.81.0/24
192.168.81.0/24 cluster-mgmt 20
cluster1-01
c169.254.0.0/16
169.254.0.0/16 cluster 30
i192.168.81.0/24
192.168.81.0/24 intercluster 40
n192.168.81.0/24
192.168.81.0/24 node-mgmt 10
THE NETWORK ROUTING-GROUPS SHOW COMMAND: 1 OF 2
The network routing-groups show
Command 2 of 2
Routing
Vserver Group Subnet Role Metric
--------- --------- --------------- ------------ -------
cluster1-02
c169.254.0.0/16
169.254.0.0/16 cluster 30
i192.168.81.0/24
192.168.81.0/24 intercluster 40
n192.168.81.0/24
192.168.81.0/24 node-mgmt 10
vs1
d192.168.81.0/24
192.168.81.0/24 data 20
vs2
d192.168.81.0/24
192.168.81.0/24 data 20
THE NETWORK ROUTING-GROUPS SHOW COMMAND: 2 OF 2
The network routing-groups
route show Command
cluster1::> network routing-groups route show
Routing
Vserver Group Destination Gateway Metric
--------- --------- --------------- --------------- ------
cluster1
c192.168.81.0/24
0.0.0.0/0 192.168.81.1 20
cluster1-01
n192.168.81.0/24
0.0.0.0/0 192.168.81.1 10
cluster1-02
n192.168.81.0/24
0.0.0.0/0 192.168.81.1 10
vs1
d192.168.81.0/24
0.0.0.0/0 192.168.81.1 20
vs2
...
THE NETWORK ROUTING-GROUPS ROUTE SHOW COMMAND

As with the network interface show output command, node-management LIFs have a server that
is the node itself. The data LIFs are associated with a data Vserver that the data LIFs are grouped under.
NAS Data LIF Failover and Migration
LIF failover means automatic failover that

occurs because of an outage or reboot.
LIF migrate means manual migration: The
node is not rebooted during manual migration.
LIF revert means manually or automatically
sending a LIF back to its home (node and
port).
LIF failover is controlled by the failover group.
NAS DATA LIF FAILOVER AND MIGRATION

Why migrate a NAS data LIF? It might be needed for troubleshooting a faulty port or to offload a node whose
data network ports are being saturated with other traffic. The LIF fails over if its current node is rebooted.
Unlike storage failover (SFO), LIF failover or migration does not cause a reboot of the node from which the
LIF is migrating. Also unlike SFO, LIFs can migrate to any node in the cluster, not just within the high-
availability (HA) pair. After a LIF is migrated, the LIF can remain on the new node for as long as the
administrator wants.
NAS Data LIF Failover and Migration
Limits
Node-management LIFs cannot fail over or
migrate to a port on a different node.
Cluster-management LIFs and NAS data LIFs
can fail over and migrate across ports and
nodes.
Cluster LIFs can fail over and migrate only
across ports on the same node.
Data LIFs are bound to a Vserver and do not
fail over or migrate between Vservers.
SAN data LIFs never fail over or migrate.
NAS DATA LIF FAILOVER AND MIGRATION LIMITS

Data LIFs arent permanently tied to their home ports. However, the port to which a LIF is migrating is tied to
a node. This example shows the line between the physical and the logical. Also, a port has a node Vserver
scope; a data LIF has a data Vserver scope.
LIF Failover Groups
data1
e0c e0d e0c e0d e0c e0d e0c e0d
LIF FAILOVER GROUPS

LIF failover means the automatic migration of a NAS data LIF in response to a link failure on the LIFs
current network port. When such a port failure is detected, the LIF is migrated to a working port. A failover
group contains a set of network ports (physical, VLANs, and interface groups) on one or more nodes. A LIF
can subscribe to a failover group. The network ports that are present in the failover group define the failover
targets for the LIF. You can manage failover groups by adding ports to them, removing ports from them,
renaming them, and displaying information about them.
Types of Failover Groups
System-defined
User-defined
Cluster-wide
TYPES OF FAILOVER GROUPS

Failover groups for LIFs can be system-defined or user-defined. Additionally, one failover group that is called
cluster-wide is maintained automatically.
Failover groups are of the following types:
System-defined failover groups: These failover groups automatically manage LIF failover targets on a
per-LIF basis and contain data ports from a maximum of two nodes. The data ports include all the data
ports on the home node and all the data ports on another node in the cluster for redundancy.
User-defined failover groups: These customized failover groups can be created when the system-defined
failover groups do not meet your requirements. For example, you can create a failover group that consists
of 10-GbE ports and enables LIFs to fail over only to the high-bandwidth ports.
The cluster-wide failover group: This failover group consists of all the data ports in the cluster and
defines the default failover group for the cluster management LIF.
LIF Roles and Failover Groups
LIF Roles Failover Group Failover Target Role Failover Target Nodes
Cluster System-defined (default) Cluster Home node
System-defined (default) Node-management Home node

Node-
management Node-management or
User-defined Home node
data
Cluster-wide (default) Any node
Cluster- Data or
System-defined
management Node-management Home node or any node
User-defined
System-defined (default)
Data Data Home node or any node
User-defined
System-defined (default) Intercluster

Intercluster Home node
User-defined Intercluster or data
LIF ROLES AND FAILOVER GROUPS
Failover Policies
nextavail
priority
disabled
NOTE: Use -failover-policy priority and a user-defined

failover group to control the order of failover within a failover group.
FAILOVER POLICIES
nextavail (default): Enables a LIF to fail over to the next available port, preferring a port on the current node.
In some instances, a LIF configured with the nextavail failover policy selects a failover port on a remote node,
even though a failover port is available on the local node. No outages will be seen in the cluster, because the
LIFs continue to be hosted on valid failover ports.
priority: Given the list of failover targets, if the home port goes down then select the next port from the list in
order, always starting with the first port in the list.
disabled: Disables (prevents) a LIF from failing over.
Creating and Deleting Failover Groups
Creating or adding a port to a failover group:
cluster1::> net int failover-groups create failover-group
customfailover1 node cluster1-02 port e0d
Renaming a failover group:

cluster1::> net int failover-groups rename failover-group
customfailover1 new-name prodfailover1
Removing a port from a failover group:

cluster1::> net int failover-groups delete failover-group
customfailover1 node cluster1-02 port e0d
Deleting a failover group:

cluster1::> net int failover-groups delete failover-group
customfailover1 *
CREATING AND DELETING FAILOVER GROUPS
Enabling and Disabling Failover of a LIF
Enable a LIF to use a user-defined failover

group:
cluster1::> net int modify vserver vs2 lif data1
-failover-policy nextavail failover-group customfailover1
Enable a LIF to use a system-defined failover

group:
-failover-policy nextavail failover-group system-defined
Disable a LIF from failing over:

failover-policy disabled
ENABLING AND DISABLING FAILOVER OF A LIF

The values of the following parameters of the network interface modify command together
determine the failover behavior of LIFs:
-failover-policy: Enables you to specify the order in which the network ports are chosen during a
LIF failover or enables you to prevent a LIF from failing over. This parameter can have one of the
following values:
nextavail (default): Enables a LIF to fail over to the next available port, preferring a port on the
current node. In some instances, a LIF configured with the nextavail failover policy selects a
failover port on a remote node, even though a failover port is available on the local node. No outages
will be seen in the cluster as the LIFs continue to be hosted on valid failover ports.
priority: Enables a LIF to fail over to the first available port specified in the user-defined failover
group (failover targets can be shown with the network interface show -failover
command).
disabled: Disables a LIF from failing over.
-failover-group: Specifies the failover behavior configured for the LIF. The value can be set to:
system-defined: specifies that the LIF uses the implicit system-defined failover behavior for the
LIF's role
[empty]: specifies that the LIF is not configured to use a failover group
[user-defined failover group]: specifies that the LIF is configured to fail over to any
available port present in the failover group
The network interface show
Command
cluster1::> net int show -vserver vs2 -lif vs2_lif1
Vserver Name: vs2
Logical Interface Name: vs2_lif1
Role: data
Data Protocol: nfs, cifs
Home Node: cluster1-02
Home Port: e0d
Current Node: cluster1-02
Current Port: e0d
Operational Status: up
Extended Status: -
Is Home: true
Network Address: 192.168.81.32
Netmask: 255.255.255.0
IPv4 Link Local: -
Bits in the Netmask: 24
Routing Group Name: d192.168.81.0/24
Administrative Status: up
Failover Policy: nextavail
Firewall Policy: data
Auto Revert: false
Fully Qualified DNS Zone Name: none
DNS Query Listen Enable: false
Failover Group Name: customfailover1
THE NETWORK INTERFACE SHOW COMMAND
The network interface failover-
group show Command
cluster1::> net int failover-groups show
(network interface failover-groups show)
Failover
Group Node Port
------------------- ----------------- ----------
clusterwide
cluster1-02 e0c
cluster1-02 e0d
cluster1-02 e0e
cluster1-01 a0a
cluster1-01 e0c
customfailover1
cluster1-02 e0c
cluster1-01 e0c
THE NETWORK INTERFACE FAILOVER-GROUP SHOW COMMAND
NAS Load Balancing
DNS load balancing

NFSv3, NFSv4, NFSv4.1
SMB 1.0, SMB 2.0
Automatic LIF rebalancing
NFSv3 only
NAS LOAD BALANCING

With DNS load balancing enabled, a storage administrator can choose to enable the built-in load balancer to
balance the client LIF network access on the basis of the load of the cluster. Supported NAS protocols include
NFSv3, NFSv4, NFSv4.1, CIFS, and SMB 2.0.
In automatic LIF rebalancing, LIFs are automatically migrated to a less-utilized port, based on the configured
failover rules. Automatic LIF rebalancing allows even distribution of the current load. NFSv3 is the only
supported protocol.
DNS Load-Balancing Characteristics
Uses internal DNS zones that contain multiple data
IP addresses (data LIFs):
The actual data LIF that is used for an NFS mount is
chosen at NFS mount time.
NAS data LIFs can be automatically migrated among
nodes to maintain a balanced load.
Is based on LIF weights:
Weight can be manually or automatically set (based on the
current load in the cluster).
Provides balanced cluster-wide data LIFs
DNS LOAD-BALANCING CHARACTERISTICS

The DNS server resolves names to LIFs based on the weight of a LIF. A Vserver can be associated with a
DNS load-balancing zone, and LIFs can be created or modified to be associated with a specific DNS zone. A
fully-qualified domain name can be added to a LIF to create a DNS load-balancing zone by specifying a
dns-zone parameter in the network interface create command.
Two methods can be used to specify the weight of a LIF: The storage administrator can specify a LIF weight,
or the LIF weight can be generated based on the load of the cluster. Ultimately, this feature helps to balance
the overall use of the cluster. This feature does not increase the performance of any one individual node;
rather, this feature guarantees that each node is used more evenly. The result is better performance use from
the entire cluster.
DNS load balancing also improves the simplicity of maintaining the cluster. Instead of manually determining
which LIFs are used when mounting a specific global namespace, the administrator can let the system
dynamically decide which LIF is the most appropriate. And after a LIF is chosen, that LIF can be migrated to
a different node automatically to guarantee that the network load remains balanced throughout the cluster.
DNS Load-Balancing Commands
Assigning a weight to a LIF by using the network
interface modify command:
cluster1::> net int modify vserver vs2 lif data1 lb-weight 7
Creating a DNS load-balancing zone by using the

network interface create command:
cluster1::> net int create vserver vs2 lif data1 -role data -home-
node cluster1-01 -home-port e0c -address 192.0.2.129 -netmask
255.255.255.128 -dns-zone storage.company.com
Adding a LIF to a load-balancing zone by using the

network interface modify command:
cluster1::> net int modify vserver vs2 lif data2 dns-zone
storage.company.com
DNS LOAD-BALANCING COMMANDS

See KB article 1013801 for step-by-step configuration information.
Automatic LIF Rebalancing
LIFs are automatically migrated to a less-utilized port.
Migration allows even distribution of the current load.
LIFs are migrated based on the weights.
Automatic LIF rebalancing is available only under the
advanced privilege level of operation.
AUTOMATIC LIF REBALANCING

In automatic LIF rebalancing, LIFs are migrated based on the weights assigned to the LIFs.
When new NICs are added to the cluster, these network ports are automatically included when load is
calculated dynamically the next time. You must ensure that the new network ports are a part of the failover
group to which the LIFs belong.
Automatic LIF rebalancing is available only under advanced privilege level of operation.
Automatic LIF Rebalancing Commands
Enabling automatic LIF rebalancing by using

the network interface modify
command:
cluster1::*> net int modify vserver vs2 lif data1 failover-
policy priority failover-group failover-group_2 allow-lb-
migrate true
Enabling automatic LIF rebalancing on

multiple LIFs (data1, data2, data3) by using
the network interface modify command:
cluster1::*> net int modify vserver vs2 lif data1..data3
failover-policy priority failover-group failover-group_2
allow-lb-migrate true
AUTOMATIC LIF REBALANCING COMMANDS

Because automatic LIF rebalancing is disabled for CIFS, automatic LIF rebalancing should not be enabled on
the DNS load-balancing zone that is configured for CIFS connections.
Verifying the LIF Rebalancing Setting: The
network interface show Command
cluster1::*> network interface show lif data1 instance
Vserver Name: vs2

Logical Interface Name: data1
Role: data
...
Auto Revert: -
Sticky Flag: -
Fully Qualified DNS Zone Name: none
DNS Query Listen Enable: false
Load Balancing Migrate Allowed: true
Load Balanced Weight: load
Failover Group Name: failover-group_2
FCP WWPN: -
Address family: ipv4
Comment: -
VERIFYING THE LIF REBALANCING SETTING:

THE NETWORK INTERFACE SHOW COMMAND
Module Summary

should be able to:
Create NAS data LIFs
Create a LIF failover group
Migrate and revert a NAS data LIF
Configure DNS load balancing
MODULE SUMMARY
Exercise
Module 8: Logical Networking
EXERCISE
Module 10
SAN Protocols
MODULE 10: SAN PROTOCOLS
10-1 Clustered Data ONTAP Administration: SAN Protocols
Module Objectives

Explain the differences between the
supported SAN protocols
Identify the components that implement
scalable SAN on a cluster
Configure iSCSI on a cluster and create
a LUN
Configure a Windows iSCSI initiator
Create a portset and an igroup
MODULE OBJECTIVES
Lesson 1
LESSON 1
Unified Storage
NFS iSCSI
Corporate CIFS
LAN FCoE
FC
NAS SAN
UNIFIED STORAGE
A SAN is a block-based storage system that uses FC, Fibre Channel over Ethernet (FCoE), and iSCSI
protocols to make data available over the network. Starting with the Data ONTAP 8.1 operating system,
clustered Data ONTAP systems began supporting SANs on clusters of up to four nodes. In the Data ONTAP
8.2 operating system, SAN is supported in clusters of up to eight nodes.
SAN Protocol Support
Either FC or IP can be used to implement a SAN:
FC:
Uses FC protocol to communicate FC SAN is
covered in
Physical Data FC Frame SCSI-3 SAN Scaling
and
Uses FCoE to communicate Architecting.
Physical Ethernet FCoE FC Frame SCSI-3
IP uses iSCSI to communicate.

Physical Ethernet IP TCP iSCSI SCSI-3
SAN PROTOCOL SUPPORT
Scalable SAN Enhancements
Data ONTAP 8.1.1 and 8.1.2 Data ONTAP 8.2
Windows Red Hat VMware ESX HPUX Solaris Windows Red Hat VMware ESX HPUX Solaris AIX
FC, iSCSI, or FCoE FC, iSCSI, or FCoE
LUNs LUNs LUNs LUNs LUNs LUNs LUNs
SCALABLE SAN ENHANCEMENTS

Clustered Data ONTAP 8.1.1 supported scalable SAN on clusters of up to six nodes and supported the FC,
iSCSI, and FCoE protocols. In Data ONTAP 8.2, scalable SAN support has been expanded to up to eight
nodes, increasing capacity for storage, CPU cycles, and network bandwidth for clustered SAN solutions, with
no need to increase management and administrative resources.
Clustered Data ONTAP 8.2 continues to support Windows, Red Hat Linux, VMware ESX, HPUX, and
Solaris hosts, and also added support for AIX as a SAN host. To function with scalable SAN, all SAN client
stacks must support Asymmetric Logical Unit Access (ALUA).
Consult the NetApp Supportability Matrix for details about supported versions of SAN hosts.
Ports
Application
Initiator File System
TCP/IP Driver iSCSI Driver SCSI Driver FC Driver
Ethernet Port FC Port
SAN Services
Target TCP/IP Driver iSCSI Driver WAFL File System FC Driver
IP LUN FC
SAN SAN
PORTS
Data is communicated over ports. In an Ethernet SAN, the data is communicated over Ethernet ports. In an
FC SAN, the data is communicated over FC ports. For FCoE, the initiator has a converged network adapter
(CNA), and the target has a unified target adapter (UTA).
Nodes and Portals in iSCSI
Application
SCSI Driver
Local Network Connection iqn.1999-04.com.a:system

Portals Node Name
Target Portal Group (TPG) iqn.1992-08.com.netapp:sn.x:vs.1
Data Vserver
SAN Services
Target
WAFL File System
IP
LUN
SAN
NODES AND PORTALS IN ISCSI

In IP SAN, the node name describes a data Vserver, and the portal describes a logical interface ( LIF). Each
iSCSI node must have a node name. Two node-name formats are possible.
IQN-Type Designator
The format of this node name is conventionally:
iqn.yyyy-mm.backward_naming_authority: unique_device_name
This format is the most popular node-name format and the default that is used by a NetApp storage system.
The following are the components of the logical name:
Type designator, IQN, followed by a period (.)
The date when the naming authority acquired the domain name, followed by a period
The name of the naming authority, optionally followed by a colon (:)
A unique device name
Eui-Type Designator
The format of this node name is:
eui.nnnnnnnnnnnnnnnn
The following are the components of the logical name:
The type designator itself, eui, followed by a period (.)
Sixteen hexadecimal digits
Example: eui.123456789ABCDEF0
Connectivity Between the Initiator and
the Target
Application
SCSI Driver
Switch
SAN Services
Target WAFL File System
IP LUN
SAN
CONNECTIVITY BETWEEN THE INITIATOR AND THE TARGET
Direct and Indirect Paths
1 of 3
MPIO
ALUA
Direct
LUN
DIRECT AND INDIRECT PATHS: 1 OF 3

You must configure SAN clients to use:
Multipath I/O (MPIO) to access LUNs
ALUA to determine the state of a given data path to the LUNs
The active-optimized path to a LUN means the path for which the LIF and LUN are hosted by the same
node. The active-nonoptimized path represents the path for which the LIF and LUN are hosted on separate
nodes.
Unlike NAS LIFs, SAN LIFs do not migrate between interfaces or nodes. Therefore, the client host uses
ALUA to determine the most efficient path (or paths) to communicate to the LUN. The active-optimized
paths become the primary paths for data transfer between the host and the LUN.
When a volume that is hosting a LUN is moved to an aggregate on a different node, the virtual storage server
(Vserver) updates the path status, and the client polls the Vserver for the change. In this way, the new active-
optimized and active-nonoptimized paths are chosen, and the client selects the best possible paths.
When a node goes down and storage fails over to the partner node, the nodes paths also come offline. If an
appropriately zoned SAN LIF is available on the partner node, the path to the takeover node becomes the
active-optimized path until the aggregate is returned to its home node. If the paths to a node become
unavailable so that only nonoptimized paths remain, but the storage doesnt fail over, the client chooses a
nonoptimized path, and the data traverses the cluster network until an optimized path is restored.
2 of 3
MPIO
ALUA
Indirect
LUN
3 of 3
MPIO
ALUA
Indirect Indirect Indirect

LUN
Path Priority Selection
ALUA (also called Target Port Group Support or

TPGS) has these features:
One target port group per state per controller
Possible states:
Active-optimized (direct)
Active-nonoptimized (indirect)
Standby (not implemented in the Data ONTAP
operating system)
Unavailable
PATH PRIORITY SELECTION

ALUA is the Asymmetric Logical Unit Access command set. ALUA is also known as Target Port Group
Support (TPGS) in the SCSI Primary Commands (SPC) standard.
The basic design that is described is for one target port group (TPG) per state per controller. Each of the
controller heads in a high-availability (HA) pair has its own group for each of the possible states:
Active-optimized
Active-nonoptimized
Standby
Unavailable (For example, a partner port can become unavailable when the interconnect is down.)
With this model, when ports change to a different state, the ports are moved to the new group that represents
that new state.
Lesson 2
LESSON 2
Clustered Data ONTAP Support
Clustered Data ONTAP 8.1 and later versions support iSCSI.
To configure iSCSI by using NetApp System Manager or
the CLI:
1. Add the iSCSI licenses for the cluster.
2. Create or designate an aggregate for the root volume of a Vserver.
3. Create or designate a Vserver for iSCSI.
4. Enable iSCSI traffic for the Vserver.
5. Create iSCSI logical interfaces (LIFs).
6. Create an initiator group (igroup)
7. Create and bind port sets.
8. Create or designate an aggregate and volume for a LUN.
9. Create a LUN.
10.Map the LUN to the appropriate igroup.
CLUSTERED DATA ONTAP SUPPORT
Licensing iSCSI
The cluster must have the iSCSI license installed.
Install the license by using:
NetApp System Manager
The CLI
LICENSING ISCSI
Identifying an Aggregate for Vserver Use
If needed, create an aggregate:
cluster1::> storage aggregate create -aggregate
aggr_iscsi_2 -node cluster1-02 -diskcount 7
Verify the aggregate:
cluster1::> aggr show
Aggregate Size Available Used% State #Vols Nodes RAID Status
--------- -------- --------- ----- ------- ------ ------ ----- ------
aggr0 900MB 43.54MB 95% online 1 cluster1-01 raid_dp, normal
aggr0_scaling_02_0
900MB 43.55MB 95% online 1 cluster1-02 raid_dp, normal
aggr_iscsi_1 4.39GB 4.25GB 3% online 2 cluster1-01 raid_dp, normal
aggr_iscsi_2 4.39GB 4.39GB 0% online 0 cluster1-02 raid_dp, normal
IDENTIFYING AN AGGREGATE FOR VSERVER USE
Creating an iSCSI-Enabled Vserver
Create a Vserver:
cluster1::> vserver create -vserver vsISCSI2 -rootvolume
vsISCSI2_root -aggregate aggr_iscsi_2 -ns-switch file
-nm-switch file -rootvolume-security-style ntfs
Enable iSCSI protocol:

cluster1::> vserver iscsi create -vserver vsISCSI2
-target-alias vsISCSI2 -status up
Verify the iSCSI-enabled Vserver:

cluster1::> vserver iscsi show
Target Target Status
Vserver Name Alias Admin
-------- ---------------- --------------------- ------
vsISCSI2 iqn.1992-08.com.netapp:sn.d7d67711cf2:vs.4
vsISCSI2 up
CREATING AN ISCSI-ENABLED VSERVER

All volumes within the Data ONTAP operating system have a default security style. In this example, NTFS
was used; however, for a Vserver that is used to expose only LUNs, it does not matter which security style
you use.
Creating iSCSI LIFs
Create an iSCSI LIF:
cluster1::> network interface create -vserver vsISCSI2 -lif i2LIF1 -
role data -data-protocol iscsi -home-node cluster1-01 -home-port e0c -
address 192.168.239.40 -netmask 255.255.255.0 -status-admin up
Verify the iSCSI LIFs:

cluster1::> net int show -vserver vsISCSI2
-------- --------- --------- --------------- ----------- ------ ----
vsISCSI2
i2LIF1 up/up 192.168.239.40/24 cluster1-01 e0c true
i2LIF2 up/up 192.168.239.41/24 cluster1-01 e0d true
i2LIF3 up/up 192.168.239.42/24 cluster1-02 e0c true
i2LIF4 up/up 192.168.239.43/24 cluster1-02 e0d true
NOTE: Three additional LIFs were created as shown,

but the creation commands are not shown.
CREATING ISCSI LIFS
iSCSI LIFs Considerations
Failover groups do not apply to SAN LIFs.

SAN LIFs are single-protocol only, but ports
can be assigned a NAS and a SAN LIF.
Recommendation: Use at least one LIF per
node but more if you have separate subnets
because of multiple iSCSI initiators per host.
ISCSI LIFS CONSIDERATIONS
Creating Port Sets
Create a port set:
cluster1::> lun portset create -vserver vsISCSI2
-portset portset_iscsi2 -protocol iscsi -port-name i2LIF1
i2LIF2 i2LIF3 i2LIF4
Verify a port set:

cluster1::> lun portset show
Vserver Portset Protocol Port Names Igroups
--------- ------------ -------- ----------------------- -------
vsISCSI2 portset_iscsi2
iscsi i2LIF1, i2LIF2,
i2LIF3, i2LIF4
-
Not yet associated with an igroup
CREATING PORT SETS
Lesson 3
LESSON 3
Windows Native Multipath I/O
Windows Server can be configured to support multipath
I/O (MPIO).
Right-click
Features, and
then select Add
Feature. Multipath I/O
added
WINDOWS NATIVE MULTIPATH I/O
Device-Specific Modules
A device-specific module (DSM) is a driver that plugs
into an MPIO framework.
Windows MPIO supports:
Windows DSM 3.5 or later
NetApp Host Utilities Kit for Windows
Use the Interoperability Matrix Tool to verify the
recommended version.
DEVICE-SPECIFIC MODULES
Host Utilities
NetApp Host Utilities provides Perl scripts to

diagnose and troubleshoot Windows.
For example, windows_info provides
diagnostic information.
Use the Interoperability Matrix Tool to verify
compatible host utilities.
Download and install compatible host utilities.
HOST UTILITIES
The iSCSI Software Initiator: Discovery
Click here to
configure.
Add the Vservers

LIF address.
Windows Server 2008 R2

Example
THE ISCSI SOFTWARE INITIATOR: DISCOVERY

To configure the Microsoft iSCSI Software Initiator:
Install the Microsoft iSCSI Software Initiator.
On the Discovery tab, specify the Vservers LIF IP address as a target portal.
The iSCSI Software Initiator: Connection
Click here to
The Vserver is enable
discovered. multipath.
Click here to
connect.
Click here to
accept the
connection
method.
THE ISCSI SOFTWARE INITIATOR: CONNECTION

The Vserver node name is displayed in the target table.
Select the node name, and then click Connect.
The iSCSI Software Initiator:
Favorite Targets
THE ISCSI SOFTWARE INITIATOR: FAVORITE TARGETS

In the Connect To Target dialog box, if you select Add this connection to the list of Favorite Targets, the
connection appears on the Favorite Targets tab.
Creating an igroup
Create an igroup:
cluster1::> lun igroup create -vserver vsISCSI2 -igroup
ig_myWin2 -protocol iscsi -ostype windows -initiator iqn.1991-
05.com.microsoft:win-frtp2qb78mr portset portset_iscsi2
Verify an igroup:
cluster1::> igroup show
Vserver Igroup Protocol OS Type Initiators
--------- -------- -------- -------- -------------------------
vsISCSI2 ig_myWin2 iscsi windows iqn.1991-
05.com.microsoft:win-frtp2qb78mr
CREATING AN IGROUP
Verifying Connectivity
1 of 2
Verify the target portal groups:
cluster1::> vserver iscsi tpgroup show -vserver vsISCSI2
TPGroup TPGroup Logical
Vserver Name Tag Interface
--------- ---------------- ------- ----------
vsISCSI2 i2LIF1 1032 i2LIF1
Verify igroup connection:

cluster1::> igroup show instance ig_myWin2
Vserver Name: vsISCSI2
Igroup Name: ig_myWin2
Protocol: iscsi

ALUA: true
Initiators: iqn.1991-05.com.microsoft:win-
frtp2qb78mr (logged in)
VERIFYING CONNECTIVITY: 1 OF 2
Verifying Connectivity
2 of 2
Verify sessions:
cluster1::> vserver iscsi session show -vserver vsISCSI2
Tpgroup Initiator Initiator

Vserver Name TSIH Name ISID Alias
--------- ------- ---- ------------------------ --------- -------------
vsISCSI2 i2LIF1 5 iqn.1991-05.com.microsoft:win-frtp2qb78mr
40:00:01:37:00:00
Verify connections:
cluster1::> vserver iscsi connection show -vserver vsISCSI2
Tpgroup Conn Local Remote TCP Recv
Vserver Name TSIH ID Address Address Size
------------ --------- ----- ----- --------------- ----------- --------
vsISCSI2 i2LIF1 5 1 192.168.239.40 192.168.239.145 13140
VERIFYING CONNECTIVITY: 2 OF 2
Creating a Volume
Create a volume:
cluster1::> vol create -vserver vsISCSI2 -volume vol1
-aggregate aggr_iscsi_2 -size 150MB -state online -type RW
-policy default -security-style ntfs
Verify a volume:
cluster1::> vol show
--------- ---------- --------- -------- ---- ------- --------- -----

vsISCSI2 vol1 aggr_iscsi_2 online RW 150MB 142.4MB 5%

CREATING A VOLUME
Creating a LUN
Create a fully provisioned LUN:
cluster1::> lun create -vserver vsISCSI2 -volume vol1
-lun lun_vsISCSI2_1 -size 50MB
-ostype windows_2008 -space-reserve enable
Verify a LUN:
cluster1::> lun show -vserver vsISCSI2
Vserver Volume Qtree LUN State Mapped Type Size
--------- ------ ----- ------------ ------ -------- -------- -------
vsISCSI2 vol1 "" lun_vsISCSI2_1 online unmapped windows_2008 54.91MB
For information about creating thin-provisioned LUNs,

see the SAN Scaling and Architecting course.
CREATING A LUN
Mapping a LUN
Map a LUN to an igroup:
cluster1::> lun map -vserver vsISCSI2 -volume vol1
-lun lun_vsISCSI2_1 -igroup ig_myWin2
Verify mapping:
cluster1::> lun show -instance /vol/vol1/lun_vsISCSI2_1
Vserver Name: vsISCSI2
LUN Path: /vol/vol1/lun_vsISCSI2_1
OS Type: windows_2008
Space Reservation: enabled
Serial Number: BGMc1]-hUDrf
Comment:
Space Reservations Honored: true
Space Allocation: disabled
State: online
LUN UUID: 9d426342-cf8d-11e0-90b1-123478563412
Mapped: mapped
Block Size: 512B
cluster1::> lun show

or use:
MAPPING A LUN
Lesson 4
LESSON 4
Scanning for a New LUN
Select Disk
Management.
Right-click, and then select Rescan Disks.
SCANNING FOR A NEW LUN

After creating a LUN with the lun setup command or with NetApp System Manager, use Windows Disk
Management on the host to prepare the LUN for use. The new LUN should be visible as a local disk. If the
new LUN is not visible, in the toolbar, click the Action button, and then click Rescan Disks.
Initializing a New LUN
The LUN
appears.
The LUN
is offline.
Right-click,
and then select
Initialize.
INITIALIZING A NEW LUN
Provisioning a New LUN
The wizard
launches.
Right-click, and
then select New
Simple Volume.
PROVISIONING A NEW LUN

To open the New Simple Volume Wizard, right-click the bar that represents the unallocated disk space, and
then select New Simple Volume. Or, from the Action menu in the Computer Management window, select
All Tasks > New Simple Volume.
The Volume Size and Mount Options
Specify the volume size.
Specify the method

to mount.
THE VOLUME SIZE AND MOUNT OPTIONS

Choose the partition size and drive letter. Accept the default drive assignment, or use the list to select a
different drive.
The Format and Summary Pages
Specify the format.
Verify the configuration, and

then click Finish.
The LUN is now ready to use.
THE FORMAT AND SUMMARY PAGES
Additional SAN Resources
The SAN Implementation instructor-led course:
Implementation details for when you use Windows, vSphere,
and Linux as initiators
Information about SnapDrive for Windows and SnapDrive for
UNIX
The SAN Scaling and Architecting instructor-led course:
Details about FC and FCoE implementation
Steps for troubleshooting:
LIF failure
Storage failover
Volume move
Take both courses and prepare for the NCIE-SAN
certification exams.
ADDITIONAL SAN RESOURCES
Module Summary
be able to:
Explain the differences between the supported SAN
protocols
Identify the components that implement scalable SAN
on a cluster in a clustered Data ONTAP environment
Configure iSCSI on a cluster and create
a LUN
Configure a Windows iSCSI initiator
Create a portset and an igroup
MODULE SUMMARY
Exercise
Module 10: SAN Protocols
Estimated Time: 45 minutes
EXERCISE
Module 11
Storage Efficiency
MODULE 11: STORAGE EFFICIENCY
11-1 Clustered Data ONTAP Administration: Storage Efficiency
Module Objectives

Discuss the storage-efficiency features that
are built into clustered Data ONTAP
Identify the commands that are needed to
manage storage efficiency
MODULE OBJECTIVES
Storage Efficiency Features
Thin Provisioning FlexClone Software
Deduplication Compression
STORAGE EFFICIENCY FEATURES

Clustered Data ONTAP supports several storage-efficiency features, including thin provisioning,
deduplication, data compression, and cloning. Each of these powerful features makes clustered Data ONTAP
even more attractive to new NetApp customers, current Data ONTAP 7-Mode customers, and current
clustered Data ONTAP customers.
Thin Provisioning
Typical: 40% Use NetApp: More than 70% Use
Buy 50% Less Storage
waste
Save 50% in Power, Cooling,
App 3 8 spindles and Space
waste Shared
App 2 6 spindles capacity
App 3
12 spindles
waste
App 1 6 spindles App 2
App 1
Standard Volume Manager NetApp Thin Provisioning
THIN PROVISIONING
If you compare the NetApp storage use approach to the competitions approach, you find one feature that
stands out. Flexible dynamic provisioning with FlexVol technology provides high storage use rates and
enables customers to increase capacity without the need to physically reposition or repurpose storage devices.
NetApp thin provisioning enables users to oversubscribe data volumes, which results in high use models. You
can think of this approach as just-in-time storage.
To manage thin provisioning on a cluster, use the volume command.
Deduplication
cluster1::> volume efficiency on -vserver vs1

-volume vol1
DEDUPLICATION
Deduplication improves physical storage-space efficiency by eliminating redundant data blocks within a
FlexVol volume. Deduplication works at the block level on an active file system and uses the Write
Anywhere File Layout (WAFL ) block-sharing mechanism. Each block of data has a digital signature that is
compared with all of the other blocks within the data volume. If an exact match is identified, the duplicate
block is discarded, and a data pointer is modified so that the storage system references the copy of the data
object that is stored on disk. The deduplication feature works well with datasets that have large quantities of
duplicated data or white space. You can configure deduplication operations to run automatically or according
to a schedule. You can run deduplication on new data or existing data on any FlexVol volume.
The deduplication feature enables you to reduce storage costs by reducing the actual amount of data that is
stored over time. For example, if you create a 100-GB full backup one night and 5 GB of data changes the
next day, the second nightly backup needs to store only the 5 GB of changed data. This approach amounts to a
95% spatial reduction on the second backup. In operational environments, deduplication of full backups can
save more than 90% of the required space, and deduplication of incremental backups saves an average of 30%
of the space. In nonbackup scenarios, such as the creation of virtual machine images, you can save 40% of the
space. To estimate your own savings, visit the NetApp deduplication calculator at http://www.secalc.com.
The Benefits of Deduplication
Can reduce space consumption by 20 times or
greater for backups (TR3966)
Is integrated with the Data ONTAP operating system:
General-purpose volume deduplication
Identification and removal of redundant data blocks
Is application-agnostic:
Primary storage
Backup data
Archival data
Runs as a background process and is transparent
to clients
THE BENEFITS OF DEDUPLICATION
Data Compression
Data Compression
cluster1::> volume efficiency on -vserver vs1 -volume vol1

-compression true
DATA COMPRESSION
Data compression enables you to reduce the physical capacity that is required to store data on a cluster by
compressing data blocks within a FlexVol volume. Data compression is available only on FlexVol volumes
that are created on 64-bit aggregates. Data compression optimizes the storage space and bandwidth that is
required to replicate data during volume operations, such as moving volumes and performing SnapMirror
transfers. You can compress standard data files, virtual disks, and LUNs, but not file system internal files, NT
streams, or metadata.
To manage compression on a cluster, use the volume efficiency command.
Characteristics of Data Compression
Inline compression
Parallelism is increased.
Path length is decreased.
Latency is increased.
Postprocess compression
Uncompressed data is compressed during idle time.
Only previously uncompressed blocks are compressed.
Compression is done before deduplication.
Data ONTAP 8.2 and later can detect incompressible data
before wasting cycles.
See TR3966 for more information.
CHARACTERISTICS OF DATA COMPRESSION

There are two types of data compression: inline compression and postprocess compression.
With inline compression, all writes to a volume are compressed immediately before being written to the
volume. Inline compression increases parallelism because all compression and decompression algorithms are
multiprocessor-capable and because writes are compressed outside of the consistency point. Because
operations do not have to be suspended and resumed, inline compression also reduces path length. However,
because processing is required for compression and decompression, there is a latency impact on performance.
Postprocess compression runs as a background task. Uncompressed data that is written after deduplication is
compressed and rewritten to the volume when the controller is not busy. If inline and postprocess
compression are enabled for the same volume, postprocess compression compresses only the blocks on the
volume that were not compressed previously. If compression and deduplication are enabled, compression is
always executed before deduplication.
Cloning
Aggregate
Aggregate
Aggregate
vol1
vol1
vol1 vol1clone
vol1clone
vol1 clone
Data Blocks
Data Blocks
cluster1::> volume clone create -vserver vs1 -flexclone vol1clone -parent-volume vol1
cluster1::> volume clone split start -vserver vs1 -flexclone vol1clone
cluster1::> volume clone split show -vserver vs1 -flexclone vol1clone
CLONING
A FlexClone volume is a point-in-time, space-efficient, writable copy of the parent volume. The FlexClone
volume is a fully functional stand-alone volume. Changes that are made to the parent volume after the
FlexClone volume is created are not reflected in the FlexClone volume, and changes to the FlexClone volume
are not reflected in the parent volume.
FlexClone volumes are created in the same virtual server (Vserver) and aggregate as the parent volume, and
FlexClone volumes share common blocks with the parent volume. While a FlexClone copy of a volume
exists, the parent volume cannot be deleted or moved to another aggregate. You can sever the connection
between the parent and the FlexClone volume by executing a split operation.
A FlexClone split causes the FlexClone volume to use its own disk space, but the FlexClone split enables you
to delete the parent volume and to move the parent or the FlexClone volume to another aggregate.
To manage cloning on a cluster, use the volume clone command.
Module Summary

should be able to:
Discuss the storage-efficiency features that
are built into clustered Data ONTAP
Identify the commands that are needed to
manage storage efficiency
MODULE SUMMARY
Exercise
Module 11: Storage Efficiency
EXERCISE
Module 12
Data Protection: Snapshot and
SnapMirror Copies
MODULE 12: DATA PROTECTION: SNAPSHOT AND SNAPMIRROR COPIES
12-1 Clustered Data ONTAP Administration: Data Protection: Snapshot and SnapMirror Copies
Module Objectives
Create a Snapshot copy of a volume and create
Snapshot policies
Create load-sharing (LS) and data-protection (DP)
mirror copies
Manually and automatically replicate mirror copies
Promote an LS mirror copy to replace its read/write
volume
Restore a Snapshot copy to be a read/write volume
Configure Vserver and cluster peering for data
protection
MODULE OBJECTIVES
Data-Protection Methods
Snapshot copies
Mirror copies for data protection and load sharing
SnapVault backup copies
Tape backups through third-party software
Restores:
volume snapshot restore
LS mirrors: snapmirror promote
DP mirrors: snapmirror resync
Vault backups: snapmirror restore
NDMP restore
DATA-PROTECTION METHODS
A customers data-protection plan is likely to use all of the methods of protecting data that are shown here.
Disaster Recovery
Clustered Data ONTAP uses NDMP with

third-party software for disaster recovery.
A clustered Data ONTAP system can be an
ndmpcopy source or destination.
Data-protection intracluster and intercluster
mirrors are built-in, disk-to-disk backups.
DISASTER RECOVERY
No native tape backup or restore commands are currently available in clustered Data ONTAP. All tape
backups and restores are performed through third-party NDMP applications.
Lesson 1
LESSON 1
Snapshot Technology
A Snapshot copy is a read-only image of the active
file system at a point in time.
The benefits of Snapshot technology are:
Nearly instantaneous application data backups
Fast recovery of data that is lost due to:
Accidental data deletion
Accidental data corruption
Snapshot technology is the foundation for these
NetApp products:
SnapRestore SnapManager
SnapDrive SnapMirror
FlexClone SnapVault
SNAPSHOT TECHNOLOGY
Snapshot technology is a key element in the implementation of the WAFL (Write Anywhere File Layout) file
system:
A Snapshot copy is a read-only, space-efficient, point-in-time image of data in a volume or aggregate.
A Snapshot copy is only a picture of the file system, and it does not contain any data file content.
Snapshot copies are used for backup and error recovery.
The Data ONTAP operating system automatically creates and deletes Snapshot copies of data in volumes to
support commands that are related to Snapshot technology.
Volume Snapshot Functionality
Snapshot copies can be created:
Manually
Automatically based on a schedule defined by Snapshot
policies
A user can restore files and directories through a client:
UNIX: .snapshot directory (visibility set at the volume)
Windows: ~snapshot directory (visibility set at the share)
A cluster administrator can restore an entire volume with
SnapRestore:
Restores an entire volume (or an individual file)
Command: volume snapshot restore
Requires the SnapRestore license
VOLUME SNAPSHOT FUNCTIONALITY

Snapshot functionality is controlled by management, which provides the UI for manual Snapshot copies and
the Job Manager policies and schedules for automated Snapshot operations. Each volume can have a Snapshot
policy associated with it. A policy can include multiple schedules, so that Snapshot copies are created hourly,
daily, weekly, and so on. A policy also indicates how many of each type of Snapshot copy to retain before
deleting older copies. For example, you can keep four hourly Snapshot copies, and when the fifth one is
created, the oldest one is removed, such that a continuously updated group of the previous four hourly
Snapshot copies is retained.
Clients can see and use the .snapshot directories, so users can restore their own data without administrator
intervention. When an entire volume needs to be restored from a Snapshot copy, an administrator uses the
volume snapshot promote command, which is basically the same as restoring the volume using
SnapRestore technology. The entire Snapshot copy is promoted, replacing the entire volume. Individual files
can only be restored through a client.
Data ONTAP Snapshot Copy
PROD SNAP 1 SNAP 2
Production: Active File System
A A A
Prod
B B B
C C C
WRITE WRITE D D D
A B C D E F F E EE E E
FF F F
F
S1 S2
SNAP #1 SNAP #2
DATA ONTAP SNAPSHOT COPY

Before a Snapshot copy is created, there must be a file system tree that points to data blocks, which contain
content. When the Snapshot copy is created, the file structure metadata is saved. The Snapshot copy points to
the same data blocks as the file structure metadata that existed when the Snapshot copy was created.
Creating a Snapshot copy has no significant impact on disk space. Because the file structure takes up little
space, and no data blocks must be copied to disk, a new Snapshot copy consumes almost no additional disk
space. In this case, the phrase consumes no space really means no appreciable space. The so-called top-
level root inode, which contains metadata that is necessary to define the Snapshot copy, is 4 KB.
Snapshot copies begin to use space when data is deleted or modified. The WAFL file system writes the new
data to a new block (F) on the disk and changes the root structure for the active file system to point to the
new block.
Meanwhile, the Snapshot copy still references the original block F. Any time that a Snapshot copy references
a data block, that block remains unavailable for other uses, which means that Snapshot copies start to
consume disk space only when the file system data changes after a Snapshot copy is created.
Restore from a Snapshot
PROD SNAP
PROD1 SNAP 2
Production: Active File System
A A A
Prod
B B B
C C C
D D D
A B C D E F F E E
E E E
F
F F F
F
Prod
S1 S2
Production:
SNAP #1 SNAP #2
Active
File System
RESTORE FROM A SNAPSHOT
CLI: Snapshot Copy Creation
To manually create Snapshot copies:
cluster1::> volume snapshot create -vserver vs0
-volume vol3 -snapshot vol3_snapshot
To rename Snapshot copies:

cluster1::> volume snapshot rename -vserver vs0
-volume vol3 -snapshot vol3_snap -new-name
vol3_snap_copy
CLI: SNAPSHOT COPY CREATION

EXAMPLE RESULT
Creates a Snapshot copy called test in the
snap create engineering test
engineering volume.
Lists all available Snapshot copies in the
snap list engineering
engineering volume.
Deletes the Snapshot copy test in the
snap delete engineering test
engineering volume.
snap delete a vol2 Deletes all Snapshot copies in vol2.
snap rename engineering Renames the Snapshot copy from nightly.0 to
nightly.0 firstnight.0 firstnight.0 in the engineering volume.
Changes the Snapshot copy reserve to 25% on
snap reserve vol2 25
vol2.
Sets the automatic schedule on vol2 to save these
snap sched vol2 0 2 6 @ 8,
weekly Snapshot copies: 0 weekly, 2 nightly, and
12, 16, 20
6 hourly at 8 a.m., 12 p.m., 4 p.m., and 8 p.m.
Snapshot Disk Consumption
Snapshot Reserve Aggregate Space
Default is 5% Active File

System 95%
Deleting a snapshot
snapshot delete Snap Reserve 5%
volume snapshot autodelete WAFL Reserve 10%
volume snapshot compute-reclaimable

Calculates the volume space that can be reclaimed
if one or more specified Snapshot copies are deleted
Available in advanced mode
Can not use the * wildcard
SNAPSHOT DISK CONSUMPTION
The volume snapshot show Command
netappu::> volume snap show -vserver vs7 -volume vs7_vol1
---Blocks---
Vserver Volume Snapshot Size Total% Used%
-------- ------- ---------------------------------- ------------ ------ -----
vs7 vs7_vol1
weekly.2011-09-22_0015 88KB 0% 37%
5min.2011-09-23_1120 76KB 0% 34%
5min.2011-09-23_1125 72KB 0% 33%
5min.2011-09-23_1130 92KB 0% 38%
weekly.2011-09-29_0015 56KB 0% 27%
daily.2011-10-02_0010 56KB 0% 27%
daily.2011-10-03_0010 52KB 0% 26%
hourly.2011-10-03_0605 52KB 0% 26%
hourly.2011-10-03_0705 52KB 0% 26%
hourly.2011-10-03_0805 52KB 0% 26%
hourly.2011-10-03_0905 52KB 0% 26%
hourly.2011-10-03_1005 52KB 0% 26%
hourly.2011-10-03_1105 52KB 0% 26%
THE VOLUME SNAPSHOT SHOW COMMAND

The Snapshot copies shown here are scheduled Snapshot copies. The copies include two daily Snapshot
copies, six hourly Snapshot copies, and one weekly Snapshot copy.
Snapshot Policies
Created at the cluster level
Assigned at the volume level
Can be created with the CLI or OnCommand System manager
netappu::> volume snapshot policy show
Number Of Is
Name Schedules Enabled Comment
----------------- ---------- ------- ----------------------------------------
-
default 3 true Default policy with hourly, daily &
weekly schedules.
Schedule: hourly Count: 6 Prefix: hourly
daily 2 daily
weekly 2 weekly
none 0 false Policy for no automatic snapshots.

Schedule: - Count: - Prefix: -
SNAPSHOT POLICIES
Two Snapshot policies are automatically created: default and none. If a volume uses none as its Snapshot
policy, no Snapshot copies of it will be created. If a volume uses the default policy, after two weeks, there
will be a total of ten Snapshot copies of it (six hourly copies, two daily copies, and two weekly copies).
Snapshot Directory View
from a Windows Client
Snapshot copies are

visible to Windows
clients that have the
folder options
configured to display
hidden files.
SNAPSHOT DIRECTORY VIEW FROM A WINDOWS CLIENT

Snapshot directories are hidden on Windows clients. To view them, you must first configure the folder
options to display hidden files, then navigate to the root of the CIFS share and find the directory folder.
The subdirectory for Snapshot copies appears to CIFS clients as ~snapshot. Files that are displayed here are
those files that are manually created Snapshot copies and those that are created automatically for specified
intervals.
Restoring a File
To restore a file from the ~snapshot directory, rename or move the original file, then copy the file from the
~snapshot directory to the original directory.
Lesson 2
LESSON 2
The SnapMirror Engine
Is used for the volume move, volume
copy, and snapmirror commands
Uses SpinNP as the transport protocol
between the source and destination volumes
(intracluster only)
Uses a Snapshot copy of the source,
determines the incremental differences, and
transfers only the differences
Executes SnapVault backups
THE SNAPMIRROR ENGINE

Internally, the incremental data transfer algorithm is used for moving, mirroring, and copying volumes. Data
is transferred over the network by using the proprietary SpinNP protocol. On the source side, Snapshot copies
of the volumes are used as the source of the data transfers so that the source volume itself can still be read
from and written to while the data transfer from the Snapshot copy occurs.
SnapMirror Considerations
No qtree SnapMirror replication, but

replication by using volumes
Asynchronous mirroring only
DP and LS mirror copies
Support for Vserver to Vserver replication
Support for intercluster SnapMirror replication
SNAPMIRROR CONSIDERATIONS
LS and DP Mirror Copies
1 of 2
Mirror copies are read-only copies of a volume.
Mirror copies are volumes that have SnapMirror
relationships with source volumes.
Mirror copies are updated from source volumes
manually, or automatically based on a scheduled.
LS mirror relationships stay within the Vserver of the
source volume.
DP mirror relationships can be within a Vserver,
between Vservers within the cluster, and between
Vservers of two different clusters.
Mirrors cannot be cascaded.
LS AND DP MIRROR COPIES: 1 OF 2

Mirror copies are read-only volumes. Each mirror copy has an association read/write volume and is labeled as
an LS or data-protection mirror copy. LS and data-protection mirror copies are similar conceptually, but the
type dictates how the mirror copy is used and maintained.
Mirror copies are copies of read/write volumes. Mirror copies are synchronized with the read/write source
volumes only to the extent that an administrator keeps them synchronized through manual replication or
scheduled (automated) replication. Generally, data-protection mirror copies do not need to be as up-to-date as
LS mirror copies do because of their different purposes.
Each mirror copy can have a replication schedule that is associated with it, which determines when (cron) or
how often (interval) replications are performed on the mirror copy. All LS mirror copies of a volume are
treated as a unified group and use the same schedule. The schedule is enforced by the UI, so if you choose a
different schedule for one LS mirror copy, the other LS mirror copies of that volume are automatically
changed to match. Each data-protection mirror copy is independentit does not have to use the same
schedule as other data-protection mirror copies.
LS mirror relationships stay within the virtual storage server ( Vserver) of the source volume. Data-protection
mirrors can be within a Vserver, between Vservers within the cluster, and between Vservers of two different
clusters.
LS and DP Mirror Copies
2 of 2
A volume must be created before the volume can be
used as a mirror destination.
A SnapMirror volume must be created as type DP, a
RW volume cannot be changed to a DP mirror.
Creating a mirror relationship does not cause an initial
update to be performed.
An LS mirror copy can be promoted to become the
source volume using the snapmirror promote
command.
A DP mirror copy can be converted to a writable
volume using the snapmirror break command.
A mirror copy can be restored to its source.
LS AND DP MIRROR COPIES: 2 OF 2

All replication is performed directly from the read/write volume to the appropriate mirrors. This method is
different from the cascading that occurs within the Data ONTAP 7G operating system.
Creating a mirror copy, associating the mirror copy with a source volume, and replicating to it are separate
steps.
An LS or mirror copy can be promoted to replace its read/write volume. This is similar to using SnapRestore
technology to perform a restore.
The snapmirror promote Command
For LS Mirrors Only
The snapmirror promote command:
Performs a failover to a destination volume
Changes the destination volume to the new source
volume
Read-only volume becomes read-write
New source volume assumes the identity and SnapMirror
relationships of the original source volume
Destroys the original source volume
The destination volume must be an LS volume.
Client accesses are redirected from the original
source volume to the promoted destination volume.
THE SNAPMIRROR PROMOTE COMMAND: FOR LS MIRRORS ONLY
Mirror Creation Steps
1. Create a (mirror) volume: volume create
2. Create a mirror relationship: snapmirror create
3. Perform baseline replication:
Data protection: snapmirror initialize
Load sharing: snapmirror initialize-ls-set
4. Perform incremental replication:
Data protection: snapmirror update
Load sharing: snapmirror update-ls-set
NOTE: The update commands work for the baseline

synchronization, too, so the initialize step can be skipped.
MIRROR CREATION STEPS
The snapmirror show Command
cluster1::> snapmirror show
Source Destination Mirror Relationship Total
Path Type Path State Status Progress Healthy
------------- ---- ------------ ------------- -------------- ---------- -------
cluster1://vs2/vs2root
DP cluster1://vs2/vs2root_dp1
Snapmirrored Idle - true
cluster1://vs2/vs2root_dp2
LS cluster1://vs2/vs2root_ls2
cluster1://vs2/vol227
XDP cluster2://vs7/xdp_vol227
THE SNAPMIRROR SHOW COMMAND

Notice that the volume that is called vs2root has three mirror copies: two data-protection mirror copies and
one LS mirror copy.
The snapmirror show instance
Command
cluster1::> snapmirror show -source-volume vs2root -type ls -instance
Source Path: cluster1://vs2/vs2root
Destination Path: cluster1://vs2/vs2root_ls2
Relationship Type: LS
Managing Vserver: vs2
SnapMirror Schedule: 5min
Tries Limit: 8
Throttle (KB/sec): unlimited
Mirror State: Snapmirrored
Relationship Status: Idle
Transfer Snapshot: -
Snapshot Progress: -
Total Progress: -
Snapshot Checkpoint: -
Newest Snapshot: snapmirror.79deda29-e8a6-11e0-b411-123478563412_4_2147484676.2011-10-05_023500
Newest Snapshot Timestamp: 10/05 02:35:00
Exported Snapshot: snapmirror.79deda29-e8a6-11e0-b411-123478563412_4_2147484676.2011-10-05_023500
Exported Snapshot Timestamp: 10/05 02:35:00
Healthy: true
THE SNAPMIRROR SHOW INSTANCE COMMAND

The instance view of the vs2root_ls2 mirror copy shows when the mirror copy was last replicated and other
information.
LS Mirror Copies
LS mirror copies are primarily used for load sharing
(balancing) when client read access is used.
Read access requests for a volume are distributed to the
volumes LS mirror copies, unless the special . admin path is
used.
LS mirror copies are automatically available in the
namespace.
LS mirror copies are implicitly accessed by clients (for read
access).
Junctions are accessible in LS mirror copies.
LS mirror copies are always replicated as a group.
A source volume can have a maximum of one LS mirror copy
per node.
LS MIRROR COPIES
The purpose of LS mirror copies is to offload volumes (and a single data module) of read activity. Therefore,
all mirror copies must be synchronized at the same data-version level. When a volume is replicated to its LS
mirror copies, all LS mirror copies of the volume are synchronized directly from the volume (without
cascading).
The way that NFS is mounted on a client, or which CIFS share is mapped to the client, changes which data is
accessedeither the read/write volume or one of its LS mirror copies. NFS is usually mounted at the root of a
Vserver by using a command such as mount <host>://myvserver. This command causes the LS
selection algorithm to be invoked. If, however, the NFS mount command is executed by using the . admin
path, such as mount <host>:/.admin /myvserver, this mount from the client always accesses the
read/write volumes when traversing the namespace, even if there are LS mirror copies for volumes.
For CIFS, the difference is not in how a share is accessed but in which share is accessed. If you create a share
for the . admin path and use that share, the client always has read/write access. If you create a share without
using . admin, the LS selection algorithm is used.
Unless the special .admin path is used, clients are transparently directed to an LS mirror copy for read
operations rather than to the read/write volume.
LS Mirror Selection
The Data ONTAP operating system:
If an LS mirror copy is on the same node as the network module
that fields the request, the network module uses that LS mirror
copy.
If no LS mirror copy is on the same node as the network module
that fields the request, the network module uses an up-to-date
LS mirror copy on another node.
NFS and CIFS:
NFS: A new LS mirror can be selected even if a file remains
open.
CIFS: A new LS mirror is not selected while a file remains open.
LS MIRROR SELECTION
When the / path is used (that is, the . admin path is not used) and a read or write request comes through that
path into the network module of a node, the network module first determines if there are any LS mirror copies
of the volume that it needs to access. If there arent any LS mirror copies of that volume, the read request is
routed to the read/write volume. If there are LS mirror copies of the volume, preference is given to an LS
mirror copy on the same node as the network module that fielded the request. If there isnt an LS mirror copy
on that node, an up-to-date LS mirror copy from another node is chosen.
If a write request goes to an LS mirror copy, it returns an error to the client, which indicates that the file
system is read-only. To write to a volume that has LS mirror copies, you must use the . admin path.
For NFS clients, an LS mirror copy is used for a set period of time (minutes), after which a new LS mirror
copy is chosen. After a file is opened, different LS mirror copies can be used across different NFS operations.
The NFS protocol can manage the switch from one LS mirror copy to another.
For CIFS clients, the same LS mirror copy continues to be used for as long as a file is open. After the file is
closed, and the period of time expires, a new LS mirror copy is selected before the next time that a file is
opened. CIFS clients use this process because the CIFS protocol cannot manage the switch from one LS
mirror copy to another.
Typical LS Mirror Issues
1 of 2
Client machines cannot see volumes that have
been created.
The volume must be mounted (given a
junction path) to the namespace.
Replicate the parent volume.
NOTE: You should schedule synchronization of

Vserver root volumes to run hourly.
TYPICAL LS MIRROR ISSUES: 1 OF 2
Typical LS Mirror Issues
2 of 2
Client requests always go to the source volume rather than
to the LS mirror copy. This issue occurs when the client is
mounted by using the .admin path or share.
Because the client is read-only, client write requests fail.
This issue occurs when the client is not mounted by using the
.admin path or share.
For read/write NFS access to a volume that has LS mirror
copies, clients must be mounted by using the .admin path.
For read/write CIFS access to a volume that has LS mirror
copies, a specific volume .admin CIFS share must be created,
and the clients must connect to that share.
TYPICAL LS MIRROR ISSUES: 2 OF 2
FlexCache Volumes and LS Mirror Volumes
1 of 2
A Origin volume
FlexCache volume
A
B LS mirror volume
A A A A
B B B B B
FLEXCACHE VOLUMES AND LS MIRROR VOLUMES: 1 OF 2

FlexCache volumes and LS mirror volumes can serve hosts from a local node in the cluster, instead of using
the cluster interconnect to access the node that stores the primary source of data. However, there are essential
differences between the two volume types and how they are used in a cluster.
FlexCache Volumes and LS Mirror Volumes
2 of 2
LS mirror volume FlexCache volume
Complete copy of source Only cached blocks

data Not a disaster recovery
Potential disaster option
recovery option Read and write-through
Read-only Can simultaneously
Created one at a time create on all nodes
FLEXCACHE VOLUMES AND LS MIRROR VOLUMES: 2 OF 2

LS mirror volumes:
Serve client requests by using a complete copy of the source data
Can be promoted to a source volume and then used as a disaster-recovery solution
Are read-only volumes, with the exception of admin privileges for write access or bypass of the LS mirror
Are created by users one volume at a time
FlexCache volumes:
Serve client requests by using a cached copy of the source data that contains only data blocks that are
accessed by clients
Cannot be used for disaster recovery because they do not contain a complete copy of the source data
Are read and write-through cache volumes
Can be created by users one volume at a time, or can be simultaneously created on all the nodes that are
spanned by the Vserver that contains the origin volume
DP Mirror Copies
1 of 2
DP mirror copies are not implicitly accessed by
clients.
DP mirror copies can be mounted (through a junction)
into the namespace by the administrator.
In DP mirror copies, junctions are not accessible.
Each DP mirror copy replication is independent of the
LS mirror copies and of other DP mirror copies of the
same source volume.
DP MIRROR COPIES: 1 OF 2
Data-protection mirror copies are not meant for client access, although they can be mounted into the
namespace by an administrator. Junctions cannot be followed in a data-protection mirror copy, so access is
given to only the data that is contained in that data-protection mirror copy, not to any other volumes that are
mounted to the source read/write volume.
Data-protection mirror copies are primarily meant for disk-based online backups. Data-protection mirror
copies are simpler, faster, more reliable, and easier to restore than tape backups are, although data-protection
mirror copies are not portable for storing offsite. A typical use of data-protection mirror copies is to put them
on aggregates of SATA disks that use RAID-DP technology and then mirror data to them daily during the
least active time in the cluster. One data-protection mirror copy per volume is generally sufficient.
DP Mirror Copies
2 of 2
Consider using inexpensive, high-capacity (and slower)
SATA disks for DP mirror copies.
DP mirror copies can be restored or resynchronized:
To restore a mirror copy is to re-create a broken SnapMirror
relationship such that destination changes overwrite the
source data.
To resynchronize a mirror copy is to re-create a broken
SnapMirror relationship such that source changes overwrite
the destination data.
You can restore and resynchronize to a new volume.
DP MIRROR COPIES: 2 OF 2
A feature that is available only for data-protection mirror copies is the ability to perform a SnapMirror restore.
This action can restore a broken mirror relationship between a source and destination and perform an
incremental overwrite of the source volume with the current contents of the mirror destination. If the restore is
performed between a source and destination that didnt formerly have a SnapMirror relationship, a baseline
copy of the destination contents are performed to the source volume.
Resynchronizing a source and destination is similar to restoring a source and destination, except that the
source content overwrites the destination content.
Intercluster Logical Interfaces and Ports
Intercluster LIFs Ports Cluster LIFs
New in Data
ONTAP 8.1
Share data ports ifgrp1
with data LIFs or
e0a e0b e0c e0d
use dedicated
intercluster ports
Node scoped!
failover only to
other intercluster
capable ports on Data LIFs
Intercluster LIFs
same node (IP Addresses)
INTERCLUSTER LOGICAL INTERFACES AND PORTS
Intercluster SnapMirror Replication
Replication between clusters for DR
Data transfers on intercluster network
RW Source volume
Intercluster LIF Intercluster

connection WAN network
DP Destination volume
INTERCLUSTER SNAPMIRROR REPLICATION

Intercluster SnapMirror replication, as opposed to traditional intracluster mirroring, gives you the flexibility
to create an asynchronous SnapMirror volume on a cluster other than the source volumes cluster for data
protection. The replication is carried out across the WAN by using intercluster logical interfaces (LIFs). You
can use intercluster SnapMirror replication to store online copies of your data off-site for disaster recovery.
To use intercluster SnapMirror replication, you must license the feature on both participating clusters.
You need a full mesh intercluster network to support node failover and volume moves of the source or
destination volumes. For the network to be full mesh, every intercluster LIF on every node in the cluster must
be able to connect to every intercluster LIF on every node in the peer cluster.
Intercluster Networking for SnapMirror
Requires the clusters to be peered together

Requires full mesh connectivity of intercluster
LIFs
Every intercluster LIF on every

node in the cluster
WAN
should be able
to connect to every intercluster LIF
on every node in the other cluster.
INTERCLUSTER NETWORKING FOR SNAPMIRROR
Cluster and Vserver Peering
Supported relationships include:
Intercluster (Cluster and Vserver peers required)
Intracluster (Vserver peer required)
Source Destination Destination Destination
Vserver Peering is Required
Cluster and Vserver Peering is Required

CLUSTER AND VSERVER PEERING

Supported configurations include intracluster and intercluster mirror and vault relationships. In intracluster
relationships, source and destination volumes can be on the same Vserver or different Vservers. In intercluster
relationships, source and destination volumes can be in different clusters.
Vserver peering is an infrastructure that enables you to manage the relationship between two Vservers and
provides authorization for applications to function between two Vservers. Cluster peering provides similar
intercluster management and authorization.
Establishing cluster peering is a one-time operation that must be performed by the cluster administrators. A
peer relationship can be created in two ways. In one method, a peer relationship is created by a cluster
administrator who has security credentials (a cluster admin login and password) for the other cluster. The
other method allows two administrators who do not want to exchange cluster admin passwords to peer their
clusters. In this method, each administrator enters the cluster peer create command specifying
intercluster IP addresses of the other cluster.
Vserver peer requirements include the following:
A cluster peering relationship must exist before any Vserver peer relationships involving two clusters can
be created. This is not required if the Vservers reside in the same cluster.
If Vservers are on different clusters, then execute vserver peer create and vserver peer accept. If
Vservers are on the same cluster, then execute ONLY vserver peer create.
Vserver names involved in Vserver peering relationships must be unique.
The languages of the two Vservers must be the same.
Use the vserver peer create command.
Starting in clustered Data ONTAP 8.2, more granularity in SnapMirror security is provided. Replication
permission must be defined by peering Storage Virtual Machines together. Before creating any SnapMirror
relationships between a pair of Storage Virtual Machines, you must have a Storage Virtual Machine peer
relationship between the pair of Storage Virtual Machines. These Storage Virtual Machines can be local
(intracluster) or remote (intercluster). Storage Virtual Machine peering is a permission-based mechanism and
is a one-time operation that must be performed by the cluster administrators.
SnapMirror Snapshot Copies
1 of 2
bcluster1::> vol snap show -vserver vs2 -volume vs2root
(volume snapshot show)
---Blocks---
-------- ------- ---------------------------------- ------------ ------ -----
vs2 vs2root
weekly.2011-10-02_0015 84KB 0% 1%
daily.2011-10-04_0010 80KB 0% 1%
snapmirror.79deda29-e8a6-11e0-b411-
123478563412_4_2147484684.2011-10-04_052359
0% 1%
92KB
hourly.2011-10-04_2105 72KB 0% 1%
hourly.2011-10-04_2205 72KB 0% 1%
hourly.2011-10-04_2305 72KB 0% 1%
hourly.2011-10-05_0005 72KB 0% 1%
daily.2011-10-05_0010 72KB 0% 1%
hourly.2011-10-05_0105 72KB 0% 1%
SNAPMIRROR SNAPSHOT COPIES: 1 OF 2

The Snapshot copies that are shown on this slide are (mostly) scheduled Snapshot copies; three of them are
SnapMirror Snapshot copies. A separate reference Snapshot copy of a read/write volume (vs2root in this
example) is needed for every mirror copy that exists and that has a data version other than the read/write
volume. Because the replication process uses the SnapMirror Snapshot copy for a given mirror copy to
determine what has changed since the previous replication was performed, incremental mirroring is achieved
in this way. Because LS mirrors are synchronized, there should be only one SnapMirror Snapshot copy for all
of the LS mirrors. In this example, the other two SnapMirror Snapshot copies are for data-protection mirror
copies. If all LS and data-protection mirror copies are synchronized, there is one SnapMirror Snapshot copy,
and if there are no mirror copies, there are no SnapMirror Snapshot copies.
SnapMirror Snapshot Copies
2 of 2
---Blocks---
-------- ------- ---------------------------------- ------------ ------ -----
123478563412_4_2147484683.2011-10-05_020500
0% 1%
60KB
hourly.2011-10-05_0205 72KB 0% 1%
123478563412_4_2147484676.2011-10-05_023500
0% 1%
72KB
SNAPMIRROR SNAPSHOT COPIES: 2 OF 2
Module Summary
be able to:
Create a Snapshot copy of a volume and create
Snapshot policies
Create load-sharing (LS) and data-protection (DP)
mirror copies
Manually and automatically replicate mirror copies
Promote an LS mirror copy to replace its read/write
volume
Restore a Snapshot copy to be a read/write volume
Configure Vserver and cluster peering for data
protection
MODULE SUMMARY
Exercise
Module 12: Data Protection:
Snapshot and SnapMirror Copies
EXERCISE
Module 13
Data Protection: Backups and
Disaster Recovery
MODULE 13: DATA PROTECTION: BACKUPS AND DISASTER RECOVERY
13-1 Clustered Data ONTAP Administration: Data Protection: Backups and Disaster Recovery
Module Objectives

Configure and replicate to SnapVault backups
Answer questions about NDMP backup
Discuss disaster recovery in a clustered
environment
MODULE OBJECTIVES
Lesson 1
LESSON 1
SnapVault Software for Clusters
SNAPVAULT SOFTWARE FOR CLUSTERS

SnapVault software leverages block-level incremental replication for a reliable, low-overhead backup
solution. It provides efficient data protection by copying only the data blocks that have changed since the last
backup, instead of entire files. As a result, you can back up more often while reducing your storage footprint
because no redundant data is moved or stored. With direct backups between NetApp systems, disk-to-disk
vault backups minimize the need for external infrastructure and appliances. By default, vault transfers retain
storage efficiency on disk and over the network, further reducing network traffic. You can also configure
additional deduplication, compression, or both on the destination volume. However, if additional compression
is configured on the destination volume, storage efficiencies from source to destination are not retained over
the network.
The key advantages of vault backups for clusters include reduction of backup times from hours or days to
minutes, 100% success rates for backup reliability, reduction of disk capacity requirements by 90% or more,
simplified management across enterprise applications, and minimized network traffic.
For more information about backing up FlexVol volumes to a backup vault, see the Clustered Data ONTAP
Data Protection Guide.
SnapVault Benefits
Reduction of backup times from hours to days

or minutes
100% success rates for backup reliability
Reduction of disk capacity requirements by
90% or more
Simplified management across enterprise
applications
Minimized network traffic
SNAPVAULT BENEFITS
Use SnapMirror Commands
SnapVault was added to the SnapMirror

architecture and UI.
SnapVault functions are accomplished with
SnapMirror commands.
SnapVault is specified with transfer type
XDP.
cluster1::> snapmirror create
-source-path vs7:vs7_vol1
-destination-path vs8:vs8_vol1
-type XDP
-schedule 5min -policy vs7-vs8-vspolicy
USE SNAPMIRROR COMMANDS

Because SnapVault was added to the new SnapMirror architecture and UI, there are no SnapVault commands.
SnapVault functions are accomplished with SnapMirror commands. SnapVault is specified with the transfer
type XDP. Architecture, UI, and various behaviors were changed to accommodate scalability and server
virtualization.
SnapMirror Types
DP: asynchronous data protection mirror

relationships
LS: load-sharing mirror relationships
XDP: backup vault relationships
TDP: transition relationships
RST: transient restore operations
relationships
SNAPMIRROR TYPES
In clustered Data ONTAP, SnapMirror technology is organized to include several types of replication
relationships.
DP is for asynchronous data protection mirror relationships.
LS is for load-sharing mirror relationships.
XDP is for backup vault relationships.
TDP is for transition relationships from Data ONTAP running in 7-Mode to clustered Data ONTAP.
RST is a transient relationship for restore operations.
SnapMirror Commands for SnapVault
snapmirror create
snapmirror initialize
snapmirror modify
snapmirror policy -type XDP
snapmirror show
snapmirror update
snapmirror restore
SNAPMIRROR COMMANDS FOR SNAPVAULT

SnapMirror commands, with the type XDP [pronunciation tip: dash type X.D.P.] option, are used to
configure SnapVault. The basic SnapMirror commands include snapmirror create, snapmirror
initialize, snapmirror modify, snapmirror policy, snapmirror show,
snapmirror update, and snapmirror restore.
Snapshot Policy on the Source Volume
cluster1::> snapshot policy show
Vserver: cluster1
Number of Is
Policy Name Schedules Enabled Comment
------------------------ --------- ------- ---------------------------------
default 3 true Default policy with hourly, daily
& weekly schedules.
Schedule Count Prefix SnapMirror Label
---------------------- ----- ---------------------- ----------------
hourly 6 hourly -
daily 2 daily daily
weekly 2 weekly weekly
default-1weekly 3 true Default policy with 6 hourly, 2

daily & 1 weekly schedule.
Schedule Count Prefix SnapMirror Label
---------------------- ----- ---------------------- ----------------
hourly 6 hourly -
daily 2 daily -
weekly 1 weekly -
Cron Schedules
SNAPSHOT POLICY ON THE SOURCE VOLUME

On the primary volume, you must configure a Snapshot policy that specifies what copies to create and when
to create them by using cron schedules, and assign labels to specify which copies should be transferred by
SnapVault. Notice that in the output of the snapshot policy show command, the last column shows
the SnapMirror label. This parameter is used by the SnapVault secondary volume to locate the matching
Snapshot copy name to transfer to the secondary volume.
On the secondary, create a SnapMirror policy that specifies the labels of Snapshot copies to be transferred and
how copies should be retained on the secondary. A flow chart describing these steps and detailed setup
instructions are available in the SnapVault Express Guide and the Data Protection Guide.
Steps for Creating a SnapVault
Relationship on the Destination Cluster
1. Create a destination data aggregate.
2. Create a SnapVault Vserver.
3. Create a destination volume.
4. Create a SnapMirror policy (type -XDP).
5. Create the SnapVault relationship.
6. Initialize the SnapVault relationship.
STEPS FOR CREATING A SNAPVAULT RELATIONSHIP ON THE DESTINATION

CLUSTER
After setting up the primary volume, follow these steps on the destination node to set up the SnapVault
relationship:
1. Create a data aggregate in the destination cluster;
2. Create a SnapVault destination Vserver;
3. Create a destination volume;
4. Create a SnapMirror policy specifying type XDP for SnapVault relationships;
5. Create the SnapVault relationship with the Snapmirror create command, again, specifying type
XDP for SnapVault relationships; and then
6. Initialize the SnapVault relationship with the snapmirror initialize, type XDP, command.
Fan-In Deployments
Multiple clusters/Vservers backing up to single cluster or
Vserver
Affected by cluster peer limit (7:1 cluster fan-in limit in 8.2)
FAN-IN DEPLOYMENTS
Clustered Data ONTAP supports system level Fan-in. Since replication is now done at the volume level, you
cannon have multiple source volumes backing up to the same destination volume. Similar to the way you
could have multiple source Qtrees backing up to one volume with 7-Mode SnapVault, you can have volumes
from different Vservers and different clusters backing up to volumes on the same vserver. To configure Fan-
in, you must set up cluster peers. Note that in 8.2 the number of cluster peers is limited to 8, so volumes from
a maximum of 8 clusters can back up to a single cluster. Because the current limit in Data ONTAP is 8 cluster
peers, this means that volumes from a maximum of 7 different clusters can back up to a single destination
cluster.
Fan-Out Deployments
SnapMirror and SnapVault of single primary volume
1:4 fan-out possible (Can be any combination of
SnapMirror and SnapVault)
FAN-OUT DEPLOYMENTS
Up to four SnapVault destination volumes can be replicated from the same source volume. The limit of four
destination volumes is shared between SnapMirror and SnapVault, therefore the 1:4 ratio applies to the total
number of SnapMirror and SnapVault relationships of any combination.
Cascades
SnapMirror SnapVault
SnapVault SnapMirror
Only one SnapVault replication supported in a cascade
Cascade configuration transfers the SnapMirror base
Snapshot copy to SnapVault destination
SnapMirror to SnapVault SnapVault to SnapMirror
CASCADES
Supported cascade relationships include SnapMirror to SnapVault and SnapVault to SnapMirror.
Cascade relationships can contain only one instance of a SnapVault relationship; however, you can include as
many mirror copies as you require.
The cascade function is designed to guarantee that all volumes in a cascade chain have a common Snapshot
copy. The common Snapshot copy makes it possible for any pair of end points in a cascade to establish a
direct relationship.
The snapmirror restore Command
Using the snapmirror restore command you:

Specify the destination and the source paths.
Restore the latest Snapshot copy.
cluster1::> snapmirror restore

-destination-path vs7:vs7_vol1
-source-path vs8:vs8_vol1
THE SNAPMIRROR RESTORE COMMAND

Using the snapmirror restore command you specify the destination and then the source. You can
restore the latest Snapshot copy either from the source volume or the SnapVault secondary volume.
Comparing SnapVault to SnapMirror
SnapMirror Functions SnapVault Functions
SnapMirror provides data protection SnapVault accumulates copies of the

for disaster recovery. source volume for archival purposes.
SnapMirror can asynchronously copy If the source FlexVol volume is lost,

the source FlexVol volume to a clients can restore data from the
disaster recovery replica. secondary to primary storage.
If the source FlexVol volume is lost The read-only SnapVault copy can
or destroyed, clients can connect to be rendered writable only by creating
the mirror image of the source data. a FlexClone volume copy.
COMPARING SNAPVAULT TO SNAPMIRROR

SnapMirror and SnapVault are data-protection replications. SnapMirror is used for disaster recovery and
maintains only one read-only replica of the source volume. If a disaster occurs at your data center, you break
the SnapMirror relationship to render the copy writable and quickly connect your client servers to the now
writable SnapMirror replica. SnapVault is used for creating read-only archival copies of your source volume.
If a disaster occurs at your data center, you can select one of the read-only backup versions and restore the
data from secondary to primary storage. The read-only SnapVault copy can be rendered writable only by
creating a FlexClone volume copy of the SnapVault copy and splitting the cloned volume away from the
parent Snapshot copy.
SMTape to Seed Baselines
SnapMirror
Resync SMTape
SMTape
SMTAPE TO SEED BASELINES

In clustered Data ONTAP, you can attach a tape device to the source node and use SMTape in a process that
is called tape seeding. By using SMTape commands and a tape device, you can establish mirror and vault
relationships for large source volumes without sending the initial baseline transfer from the source node to the
destination node over the network.
For more information on using vault backups, enroll in the web-based course Technical Overview of
SnapVault on Clustered Data ONTAP.
Lesson 2
LESSON 2
Backup and Restoration with NDMP
You can perform local NDMP, remote NDMP, and
three-way NDMP backups.
NDMPv4
Direct Access Recovery (DAR)
A clustered Data ONTAP system does not
provide native NDMP backup and restoration,
only NDMP through third-party software.
Backups do not traverse junctions; you must list
every volume to be backed up.
You should not back up directly through NFS
or CIFS.
BACKUP AND RESTORATION WITH NDMP

Backups can be performed across a cluster by using three-way NDMP, provided that the third-party backup
application is given access to the cluster network.
A backup of a source volume does not include all of the volumes that are mounted to the source volume.
NDMP backups do not traverse junctions. Therefore, every volume that is to be backed up must be listed
explicitly. However, if the backup vendor software supports automatic discovery of file systems or the use of
wildcards, not every volume must be specified.
Although backing up through an NFS or CIFS client is possible, doing so uses all of the cluster resources that
are meant to serve data and fill the network module caches with data that most clients arent actually using.
You should send the data through a dedicated FC connection to the tape device or devices by using NDMP,
which doesnt use the resources of the network module, data network, or cluster network.
Vserver-aware NDMP
CAB* Enabled Remote NDMP
DMA Server
Control
Control Connection
Data or Intercluster LIF Connection
Data or Intercluster LIF

Data
Connection
Target
Volume
*CAB: Cluster-Aware Backup
VSERVER-AWARE NDMP
Clustered Data ONTAP now enables NDMP to function at the Vserver level. Resources, including FlexVol
volumes, can be backed up, restored, and scoped. Vserver-aware backups are critical for implementing multi-
tenancy.
For NDMP to be aware of a Vserver, the NDMP data management application software must be enabled with
cluster-aware backup (CAB) extensions, and the NDMP service must be enabled on the Vserver. After the
feature is enabled, you can back up and restore all volumes that are hosted across all nodes in the Vserver. An
NDMP control connection can be established on different LIF types. An NDMP control connection can be
established on any data or intercluster LIF that is owned by a Vserver that is enabled for NDMP and owns the
target volume. If a volume and tape device share the same affinity, and if the data-management application
supports the cluster-aware backup extensions, then the backup application can perform a local backup or
restore operation and, therefore, you do not need to perform a three-way backup or restore operation.
Vserver-aware NDMP user authentication is integrated with the role-based access control mechanism. For
more information about Vserver-aware NDMP and cluster-aware backup extensions, see the Clustered Data
ONTAP Data Protection Tape Backup and Recovery Guide.
Configuring for NDMP
1. Enable and configure NDMP on the node or nodes:
cluster1::> system services ndmp modify
2. Identify tape and library attachments:
cluster1::> system node hardware tape drive
show
cluster1::>system node hardware tape library
show
3. Configure the data management application (such as
Symantec NetBackup) for NDMP.
NOTE: The smallest backup level is the volume.
CONFIGURING FOR NDMP

Enabling or disabling NDMP on a node is simple. You must enable NDMP on each node in the cluster that
will be used for backups (probably all of them). The NDMP data management application uses the NDMP
user name and password that you choose during this configuration.
You must know the tape drive and library paths to configure the data management application.
Clustered Data ONTAP and NDMP
Clustered Data ONTAP supports the Symantec
NetBackup and IBM Tivoli Storage Manager (TSM)
data management applications, and more are being
added.
Clustered Data ONTAP supports local NDMP, remote
NDMP, and three-way NDMP backup.
A data management application with DAR can restore
selected files without sequentially reading entire
tapes.
CLUSTERED DATA ONTAP AND NDMP

Forms of NDMP backup:
Local NDMP: Data is backed up directly from the storage system (a node, in this case, not a cluster) to a
locally attached (or SAN-attached) tape device.
Remote NDMP: Data is transported from the storage system to the data management application server.
The server is attached to a tape device and backs up the data to the tape device.
Three-way NDMP: Data is transported from the storage system to another storage system that has a
locally attached (or SAN-attached) tape device.
Direct Access Recovery (DAR) is the ability of a data management application to restore a selected file or
selected files without the need to sequentially read the entire tape or tapes that are involved in a backup. For
example, if a large backup spans four 100-GB tapes, and the one file that you want to restore is on the fourth
tape, a non-DAR restoration reads the first three tapes in their entirety and then all of the fourth tape up to the
point that it finds the target file. With DAR, the data management application reads some information from
the first tape and then goes directly to the appropriate spot on the fourth tape.
Local, Remote, and Three-Way NDMP
LAN
Remote NDMP
NDMP Backup
Hosts
VERITAS
NetBackup
Server
Local Three-Way
Backup Backup
Tape Drive
Automated Automated
Tape Library Tape Library
LOCAL, REMOTE, AND THREE-WAY NDMP
A Six-Node Cluster with Data-Protection
Mirror Backups
Cluster Network
Compute Farm
SATA Storage
and
Data-Protection
Mirror Copies
Data Network
Tape Library
A SIX-NODE CLUSTER WITH DATA-PROTECTION MIRROR BACKUPS

If a customer can afford to dedicate some nodes to data protection (rather than to serving data to clients), the
customer has an excellent way to protect data. Data-protection nodes are members of the cluster, but their
aggregates consist of slower and denser disks. Data can be mirrored from the other nodes to these nodes and
then backed up from these nodes by using local NDMP, which alleviates the need for tape devices to be
connected to the other nodes. Benefits of this technique include the following:
Data-protection mirror copies dont need to move (or need to move only rarely), which alleviates one
source of headaches for backup administrators.
Fewer NDMP licenses are needed, which can result in considerable cost savings.
Fewer nodes must be backed up, which helps backup administrators.
Nonbackup nodes can use their CPUs and resources for client access.
If these disaster-recovery nodes are not used for client access, they dont need data LIFs and can reassign
those data network ports to be used for cluster network ports for a larger cluster pipe.
Lesson 3
LESSON 3
SnapMirror Data Protection
Create
Intracluster or intercluster replication (TR-4015)
Primary Disaster Recovery

Data Center Data Center
R WAN
R
A B B
dp
Create Volume for Mirror (on disaster recovery Vserver):

volume create vserver vserver -volume datavolume_dp -aggr <aggrname>
-size <equal to datavolume_A size> -type dp
Create Mirror (from disaster recovery site):
snapmirror create -destination-path DR://vserver/datavolume_dp
-source-path PRI://vserver/datavolume_A
NOTE: A cluster peer relationship must be created for intercluster replication.
SNAPMIRROR DATA PROTECTION: CREATE

The -type dp argument automatically puts the volume in restricted mode and is required. You cannot
modify an RW volume type (default) to a DP type. The slides work with either intracluster or intercluster
relationships. The peer relationship, intercluster LIFs (and intercluster ports, if required) must also be created
for intercluster SnapMirror.
Mirrors can be intracluster (within the cluster) or intercluster (to a peer cluster). No replication is currently
available between clustered Data ONTAP and Data ONTAP 7-Mode.
No support for:
Cascading
Vserver level management
Vserver disaster recovery
All data-protection mirrors require licenses on source and destination clusters.
Intercluster SnapMirror requires a peer cluster relations configuration.
Scheduling must also be configured. This is part of the SnapMirror creation wizard when you use System
Manager (recommended).
You can find more details in TR-4015 and the Data Protection Guide.
Because intercluster LIFs are node scoped, intercluster LIFs cannot fail over to other nodes in the cluster. If
you use intercluster ports, then intercluster LIFs can fail over to other intercluster ports on the same node. If
intercluster LIFs are assigned to data ports, then intercluster LIFs can fail over to any data port on the same
node.
Initialize and Update
Primary DR
R WAN R
A B dp B
Initialize Mirror Baseline Transfer (from DR site):

snapmirror initialize -destination-path
DR://vserver/datavolume_dp -source-path
PRI://vserver/datavolume_A
Update Mirror Incremental Transfers (from DR site):
snapmirror update -destination-path
DR://vserver/datavolume_dp -source-path
SNAPMIRROR DATA PROTECTION: INITIALIZE AND UPDATE

Create, initialize, and update separately with data-protection mirrors. Using the initialize command on
a data-protection mirror before creating the mirror will result in a mirror has not been created error.
With LS mirrors, using an update command alone creates the mirror, initializes the mirror (creates a
baseline) and updates the mirror incrementally. Using the initialize command on LS mirrors also
creates the mirror and then initializes it.
The initialize and update commands work differently in LS and data protection. A data-protection
mirror must be created before it can be initialized or updated. The initialize and update commands
must be run from the destination cluster.
Failover Considerations
SnapMirror does not replicate the Vserver namespace
junction path information.
Because the NAS volumes have no junction path,
they will not be accessible after a SnapMirror break
occurs unless they are premounted before failover, or
until they are mounted after failover.
The security style and permissions on the destination
Vserver root volume must be set correctly or the
namespace might be inaccessible after failover.
Use the Cluster Config Dump Tool to collect and
replicate system configuration settings to a disaster
recovery site.
FAILOVER CONSIDERATIONS
Currently, failover is a manual task. If there are multiple volumes in the namespace, failover will have to
repeated for each volume.
The Cluster Config Dump Tool (http://communities.netapp.com/thread/17921) is a Java-based
Windows/Linux/Mac utility that collects configuration information.
The tool stores information that is needed in a disaster recovery scenario:
Volume junction paths
NFS export policies, CIFS shares information
Snapshot and storage efficiency policies
LUN mapping information
Run the tool locally and replicate, or run it remotely.
The tool does not restore a configuration.
Disaster
Primary DR
R
WAN
B R
A B
dp
Break Mirror (from DR) Make destination writeable:

snapmirror break -destination-path
DR://vserver/datavolume_dp
NOTE: Admin must redirect the clients (or host) of the source
volume on the primary site to the new source volume at the DR
site in a disaster situation.
SNAPMIRROR DATA PROTECTION: DISASTER

Breaking the mirror relationship does not delete the relationship; it only stops the updates to the mirror and
makes the mirrored volume writable. You can then redirect the clients to access the disaster-recovery site
mirrored volume.
Disaster Considerations
LUNs should be mapped after failover.

CIFs shares or NFS exports must be created
(or verified).
Assign Snapshot schedules.
Assign storage efficiency policies.
DISASTER CONSIDERATIONS
Currently, breaking the mirror relationship and redirecting clients this is a manual task. If there are multiple
volumes in the namespace, the steps are repeated for each volume.
Recovery Scenario ADev/Test Recovery
Primary DR
R
WAN
R
A B B
dp
Resync Mirror (From DR) Resume relationship:

snapmirror resync -destination-path
NOTE: All new data written to the destination after the break will
be deleted.
SNAPMIRROR DATA PROTECTION:

RECOVERY SCENARIO ADEV/TEST RECOVERY
Recovery Scenario BSource Is Recoverable
Primary DR
R
WAN
R
A B B
dp
Delete Mirror (From DR) Remove Relationship:

snapmirror delete -destination-path
NOTE: You must delete the mirror to change source to

destination for resync.

RECOVERY SCENARIO BSOURCE IS RECOVERABLE
Volumes in clustered Data ONTAP can be only a source or a destination, not both. A source can be a source
for many destinations (fan out), but can not cascade. If the data in the source volume is recoverable, and the
data is still intact, you might have a considerable amount of new data on the destination that you do not want
deleted. To prevent a complete rebaseline (deletion of new data), you must delete the mirror relationship first.
Recovery Scenario BChange Relationship
Primary DR
R
WAN
R
A B B
dp
Create a new relationship with DR as source:

snapmirror create -destination-path
-source-path DR://vserver/datavolume_dp
Resync the mirror (From DR) Copy new data to Primary (no re-
baseline):
snapmirror resync -destination-path
PRI://vserver/datavolume_A -source-path

RECOVERY SCENARIO BCHANGE RELATIONSHIP
In this scenario, no baseline would be required because a common snapshot is available. Only changed blocks
would be sent. To go back to the original relationship, follow steps on the previous slide.
After deleting the original SnapMirror relationship, you can create a new relationship (switching the roles).
The disaster recovery version will now be the source, and the primary will now be the destination. To prevent
a complete baseline, use the resync command instead of the initialize command. This will find the
common snapshot and copy to the primary site only the changes that were made to the disaster-recovery site.
You can keep this relationship if desired. Otherwise, you will need to follow steps outlined in the Data
Protection Guide.
Recovery Scenario CSource Is Unrecoverable
Primary DR
R
WAN
R
A B B
dp
Create a new volume on the source

Create a new relationship with disaster recovery as the source
Initialize the mirror (rebaseline required)

RECOVERY SCENARIO CSOURCE IS UNRECOVERABLE
If the volume is corrupted or unrecoverable, you must create a new volume and relationship. You must re-
initialize the mirror, which will require a complete rebaseline.
As with Scenario B, you can keep this relationship if desired. Otherwise, follow the steps outlined in the Data
Protection Guide.
RDB Disaster Recovery
The replicated database (RDB) cannot be

backed up.
As long as one node survives, a copy of the
RDB is available and propagates itself to
reconstructed nodes.
If all nodes are destroyed, the RDB data is
destroyed.
RDB DISASTER RECOVERY
Module Summary

should be able to:
Configure and replicate to SnapVault backups
Answer questions about NDMP backup
Discuss disaster recovery in a clustered
environment
MODULE SUMMARY
Exercise
Module 13: Data Protection:
Backups and Disaster Recovery
EXERCISE
Module 15
Recommended Practices
MODULE 15: RECOMMENDED PRACTICES
15-1 Clustered Data ONTAP Administration: Recommended Practices
Module Objectives

Identify some recommended practices for
clustered Data ONTAP high availability
List some of the ways to balance clustered
Data ONTAP resources
Describe some of the methods for managing a
cluster
Enumerate some of the ways to protect and
recover a cluster from a disaster
MODULE OBJECTIVES
Recommended Practices for Nondisruptive
Operations (NDO) RAID-DP Technology
Create a dedicated, three-disk RAID-DP root
aggregate on each node.
Best practices for ONTAP 7G and Data ONTAP 7-
Mode RAID and storage still apply.
Use RAID-DP technology for all user-data
aggregates.
Use RAID-DP technology to enable online disk
firmware upgrades.
Maintain two spare disks per disk type to allow for
disk maintenance center and NDU of disk firmware.
RECOMMENDED PRACTICES FOR NONDISRUPTIVE OPERATIONS (NDO):

RAID-DP TECHNOLOGY
Below are notes about node root aggregates (of which there is one per node):
A nodes vol0 resides (permanently) on the root aggregate.
Only vol0 should be placed on root aggregates.
Root aggregates improve resiliency.
Root aggregates speed up takeover and giveback.
If you have few available disks, you can use two-disk RAID 4 instead of three-disk RAID-DP
technology.
Recommended Practices for NDO
SFO
Enable storage failover (SFO).
Reboot the high-availability (HA) pair after
enabling SFO for the first time.
Enable two-node high availability for clusters
that contain only two nodes.
Consider the advantages and disadvantages
of automatic giveback.
RECOMMENDED PRACTICES FOR NDO: SFO
Nondisruptive Upgrade (NDU)
You can perform upgrades in stages.
Rolling upgrades are becoming the norm.
You can reboot multiple nodes in parallel, depending
on number of nodes in the cluster.
You should use an HTTP or FTP server as your
primary means of performing package downloads.
Remember to revert the logical interfaces (LIFs) back
to their home ports after you boot the LIFs (or set the
automatic reversion option).
RECOMMENDED PRACTICES FOR NDO: NONDISRUPTIVE UPGRADE (NDU)

You can perform upgrades in stages by leaving the old image as the default, so that booting does not bring up
the upgraded image.
Rolling upgrades of a high-availability (HA) pair are faster than parallel reboots.
You can boot multiple nodes in parallel (though only one per HA pair), but remember that quorum rules
demand that fewer than half of the nodes in a cluster be down or booting at any given time. Also, be aware of
the logical interface (LIF) failover rules to guarantee that the data LIFs are not all failing over to nodes that
are also being rebooted.
Mixed-Version Clusters and LIF Failover
Mixed-version clusters are supported with
caveats that are specific to each version.
You should use the default configuration of
LIF failover and manually assign policies for
any exceptions:
First-level failover: same node, different
network interface card (NIC)
Second-level failover: different node (not the
HA partner)
RECOMMENDED PRACTICES FOR NDO:

MIXED-VERSION CLUSTERS AND LIF FAILOVER
Load-Sharing Mirror Copies
Place load-sharing mirror copies of the virtual
server (Vserver) root volume onto all nodes or
at least onto one node of each HA pair:
This configuration enables continuous access,
even if the node with the Vserver root volume
is down.
Because default access is to a load-sharing
mirror copy (a read-only volume), this
configuration prevents the root volume from
filling up accidentally.
RECOMMENDED PRACTICES FOR NDO: LOAD-SHARING MIRROR COPIES
Servers and Locality
Where possible, configure multiple Domain
Name System (DNS), Network Information
Service (NIS), Lightweight Directory Access
Protocol (LDAP), and Network Time Protocol
(NTP) servers.
Time zone settings should be the same
across all nodes.
Language settings should be consistent
among Vservers and volumes.
RECOMMENDED PRACTICES FOR NDO: SERVERS AND LOCALITY

Certain management operations (like a Network Information ServiceNISlookup) happen over the
management network port, which can be a single point of failure. The cluster management LIF can use LIF
failover functionality.
Recommended Practices for Resource
Balancing 1 of 2
Balance resources across the cluster:
Data and cluster interfaces
Flexible volumes of a namespace
Load-sharing mirror copies
Maintain a junction-only Vserver root volume
with a low change rate, and create multiple
load-sharing mirror copies of the volume.
RECOMMENDED PRACTICES FOR RESOURCE BALANCING: 1 OF 2

Note the maximum number of volumes that are allowed per controller.
Balance the number of volumes per controller across the cluster. When you create volumes, distribute the
volumes evenly across the cluster. When the cluster becomes unbalanced (for example, because of volume
deletions or volume size changes), use the volume move capability to redistribute volumes accordingly.
Recommended Practices for Resource
Balancing 2 of 2
Use built-in DNS load balancing to balance
NAS client connections across network
interfaces:
Create many data LIFs for the cluster.
Consider creating dedicated LIFs for NFS and
SMB protocols respectively.
Assign LIFs evenly to available network ports.
Monitor network use levels and migrate LIFs to
different ports as needed to rebalance the load.
When many clients are attached, the clients are
evenly spread across the system.
RECOMMENDED PRACTICES FOR RESOURCE BALANCING: 2 OF 2
Recommended Practices for
Load-Sharing Mirror Copies
Use load-sharing mirror copies:
For read-only or mostly read-only data
When data is updated only by a few authorized
individuals or applications
When the data set is relatively small, or the cost of
the mirror copies in disk space is justified
To netboot many clients at the same time (which is
a read-only operation and a popular use of load-
sharing mirror copies)
Schedule load-sharing mirror copies to be
automatically replicated every hour.
RECOMMENDED PRACTICES FOR LOAD-SHARING MIRROR COPIES
Intercluster Mirror Copies
A full mesh intercluster network supports node
failover and volume moves of the source or
destination volumes.
Intercluster LIFs can be created on ports that
have an intercluster role or a data role
(through the CLI).
RECOMMENDED PRACTICES FOR INTERCLUSTER MIRROR COPIES
Manageability Granularity
When you are deciding whether to create a volume, a
directory, or a qtree, ask these questions:
Will this element benefit from being managed or
protected separately?
How large will this element get?
Greater volume granularity is beneficial for many
workflows and enables movement of volumes and
resource distribution.
Larger volumes tend to yield better compression and
dedup ratios.
RECOMMENDED PRACTICES FOR MANAGEABILITY: GRANULARITY
Manageability Volume Naming Conventions
Volume names and junction names are
distinct.
Each volume name must be unique within the
Vserver.
Volume names should be wildcard-friendly.
Volumes can be grouped by name (in
alphanumeric order).
Volume names should be consistent with case
usage (all lower case or all upper case)
RECOMMENDED PRACTICES FOR MANAGEABILITY:

VOLUME NAMING CONVENTIONS
Recommended Practices for Networking
Modify port roles and create all additional

cluster LIFs before you create any data LIFs.
(Cluster LIFs are necessary to prevent cluster
ports from being automatically added to data
LIF failover rules.)
Enable the PortFast option on data switch
ports for fast LIF failover and migration.
RECOMMENDED PRACTICES FOR NETWORKING

Refer to TR-3802 Ethernet Storage Best Practices
Recommended Practices for Disaster
Recovery 1 of 2
Enable Snapshot copies and data-protection
mirror copies for critical volumes.
Consider putting data-protection mirror copies
on SATA disks:
The use of data-protection mirror copies on
SATA disks is a disk-based backup solution.
Intercluster data-protection mirror copies can
be used for off-site backups.
RECOMMENDED PRACTICES FOR DISASTER RECOVERY: 1 OF 2
Recommended Practices for Disaster
Recovery 2 of 2
Plan disaster-recovery implementations
carefully by considering taking quorum and
majority rules. (You can recover an out-of-
quorum site, but doing so is not customer-
friendly.)
Use NDMP to back up important volumes to
tape.
Have a policy for rotating backups off-site for
disaster recovery.
RECOMMENDED PRACTICES FOR DISASTER RECOVERY: 2 OF 2
Module Summary

should be able to:
Identify some recommended practices for
clustered Data ONTAP high availability
List some of the ways to balance clustered
Data ONTAP resources
Describe some of the methods for managing a
cluster
Enumerate some of the ways to protect and
recover a cluster from a disaster
MODULE SUMMARY
Course Summary
1 of 2
Now that you have completed this course, you should be
able to:
Explain the primary benefits of a Data ONTAP cluster
Create a cluster
Implement role-based administration
Manage the physical and logical resources within a cluster
Manage features to guarantee nondisruptive operations
Create aggregates
List the steps that are required to enable storage failover
(SFO)
COURSE SUMMARY: 1 OF 2
Course Summary
2 of 2
Create a Flash Pool
Build a namespace using multiple volumes
Configure FlexCache
Create an Infinite Volume
Set up and configure SAN and NAS protocols
Configure the storage-efficiency features
Administer mirroring technology and data protection
Explain the notification capabilities of a cluster
Scale a cluster horizontally
Configure the storage QoS feature
COURSE SUMMARY: 2 OF 2
THANK YOU
Appendix A
Technical Reports and
Knowledge Base Articles
APPENDIX A: TECHNICAL REPORTS AND KNOWLEDGE BASE ARTICLES
A-1 Clustered Data ONTAP Administration: Appendix: Technical Reports and Knowledge Base Articles
Technical Reports
TR-3450: High-Availability Overview and Best Practices
TR-3802 Ethernet Storage Best Practices
TR-3832: Flash Cache Best Practices Guide
TR-3967: Deployment and Best Practices Guide for Clustered Data ONTAP Windows File
Services
TR-3982: Clustered Data ONTAP 8.2: An Introduction
TR-3966: Compression and Deduplication for Clustered Data ONTAP
TR-4015: SnapMirror Configuration and Best Practices for Clustered Data ONTAP
TR-4037: Introduction to NetApp Infinite Volume
TR-4067: Clustered Data ONTAP NFS Implementation Guide
TR-4070: NetApp Flash Pool Design and Implementation Guide
TR-4078: Infinite Volume Technical FAQ
TR-4080: Best Practices for Scalable SAN in Clustered Data ONTAP
TR-4129: Namespaces in Clustered Data ONTAP
TR-4183: SnapVault Best Practices Guide for Clustered Data ONTAP
TR-4182: Best Practices for Clustered Data ONTAP Network Configurations
TR-4186: Nondisruptive Operations (NDO) Overview
TECHNICAL REPORTS
Knowledge Base Articles
KB-1013801: How to set up DNS load balancing in
KB-1013831: How to create and understand Vserver
name-mapping rules in Clustered Data ONTAP
KNOWLEDGE BASE ARTICLES

Clustered ONTAP Admin 8.2 - Student Guide (STRSW-ILT-D8CADM-REV03-SG) PDF

Cargado por

Información del documento

Título original

Derechos de autor

Formatos disponibles

Compartir este documento

Compartir o incrustar documentos

Opciones para compartir

¿Le pareció útil este documento?

¿Este contenido es inapropiado?

Copyright:

Formatos disponibles

Clustered ONTAP Admin 8.2 - Student Guide (STRSW-ILT-D8CADM-REV03-SG) PDF

Cargado por

Copyright:

Formatos disponibles

NETAPP UNIVERSITY

Clustered Data ONTAP Administration

U.S. GOVERNMENT RIGHTS

2 Clustered Data ONTAP Administration: Welcome

3 Clustered Data ONTAP Administration: Welcome

CLUSTERED DATA ONTAP ADMINISTRATION

4 Clustered Data ONTAP Administration: Welcome

5 Clustered Data ONTAP Administration: Welcome

6 Clustered Data ONTAP Administration: Welcome

7 Clustered Data ONTAP Administration: Welcome

COURSE AGENDA: DAY 1

8 Clustered Data ONTAP Administration: Welcome

COURSE AGENDA: DAY 2

9 Clustered Data ONTAP Administration: Welcome

COURSE AGENDA: DAY 3

10 Clustered Data ONTAP Administration: Welcome

COURSE AGENDA: DAY 4

11 Clustered Data ONTAP Administration: Welcome

COURSE AGENDA: DAY 5

12 Clustered Data ONTAP Administration: Welcome

NetApp University Support

NETAPP UNIVERSITY INFORMATION SOURCES

13 Clustered Data ONTAP Administration: Welcome

1-1 Clustered Data ONTAP Administration: Overview

After this module, you should be able to:

1-2 Clustered Data ONTAP Administration: Overview

A Reliable, Optimized Computing

Simplified End-User Experience

CLUSTERED DATA ONTAP

1-3 Clustered Data ONTAP Administration: Overview

Protocols Scalability Storage Efficiency

CLUSTERED DATA ONTAP HIGHLIGHTS

1-4 Clustered Data ONTAP Administration: Overview

PRIMARY REASONS TO USE CLUSTERED DATA ONTAP

1-5 Clustered Data ONTAP Administration: Overview

A single system image for 1 to 24 nodes

Note: Not all platform combinations can scale up to 24 nodes.

1-6 Clustered Data ONTAP Administration: Overview

R A Six-Node Data ONTAP Cluster

Linearly scale read performance

SCALABILITY: PERFORMANCE (NAS)

1-7 Clustered Data ONTAP Administration: Overview

Movement that is transparent

Offload (move) volumes to newly added storage.

1-8 Clustered Data ONTAP Administration: Overview

FLEXIBILITY: THE VIRTUAL STORAGE TIER

1-9 Clustered Data ONTAP Administration: Overview

Project A gets dedicated resources.

TRANSPARENCY: LOAD OPTIMIZATION

1-10 Clustered Data ONTAP Administration: Overview

DataMotion for Volumes

1-11 Clustered Data ONTAP Administration: Overview

VIRTUAL STORAGE SERVERS

1-12 Clustered Data ONTAP Administration: Overview

1-13 Clustered Data ONTAP Administration: Overview

PHYSICAL AND LOGICAL ELEMENTS

1-14 Clustered Data ONTAP Administration: Overview

4 entries were displayed.

VSERVER SHOW (SUMMARY VIEW)

1-15 Clustered Data ONTAP Administration: Overview

Administrative Vserver (Physical Cluster)

1-16 Clustered Data ONTAP Administration: Overview

1-17 Clustered Data ONTAP Administration: Overview