All Posts By

Jon Austin

Make Your Analytic Environment More Effective and Efficient With Optimized Infrastructure

By | Analytics, Storage | No Comments

Analytics and Big Data are at a crossroads: effectively leveraging Hadoop map-reduce on larger and larger datasets is getting expensive to store and protect.

The traditional deployment model consists of name nodes and data node services running on the same hardware as compute layer services (job scheduling and execution). Hadoop File System (HDFS) data is protected at the server level and drive level by replication to another node through the protocol. The number of copies is a tunable parameter; however, best practices recommends 3 copies.

As systems scale to petabytes and beyond, the storage requirements to sustain 3 copies becomes astronomical.

Another key feature of HDFS is that the objects that are stored are generally larger blocks. These blocks are written sequentially and are never updated or overwritten. Essentially, they are WORM (Write Once, Read Many) objects until their relevance expires and they are deleted.

One of the tenants of traditional HDFS is that “Moving computation is cheaper than moving data.” As a storage guy, I would like to rewrite this tenant as “Let’s leverage computation for computation and optimize Data infrastructure to best serve the application’s data requirements.” I know, it’s a bit wordy, but it makes the point. There are a number of technologies that added the HDFS protocols.


Isilon added HDFS support to its OneFS code base with the 7.0 release. An Isilon cluster can scale from 3 nodes up to 144 nodes. These nodes can be one of 3 tiers:

  1. SAS (Serial Attach SCSI) and SSD (Solid State Drive) based S Nodes for extreme performance
  2. SATA (Serial ATA drive interface) and SSD based X Nodes for typical medium performance workloads
  3. SATA based NL Nodes for Archive level performance

An Islion cluster can be a combination of these nodes, allowing for tiering of data based on access. The advantage of using Isilon for HDFS is Isilon provides the data protection, so the HDFS copy parameter can be set to a single object.

This reduces nearly in half the amount of storage required to support a Hadoop cluster, while improving the reliability and simplifying the environment.


In very large environments or in environments that require geo-dispersal, Cleversafe can be leveraged to provide storage via the HDFS protocol. Like Isilon, Cleversafe leverages erasure coding techniques to distribute the data across the nodes in its cluster architecture. Cleversafe, however, scales much larger and can be geo-dispersed as it’s cluster interconnect leverages TCP/IP over Ethernet as opposed to Infiniband.

IDS has integrated both the Isilon and Cleversafe technologies in our cloud and has the capacity to support customer analytics environments on this infrastructure.

Our customers can efficiently stand up a Hadoop Ecosystem and produce valuable insights without having to purchase and manage a significant investment in infrastructure.

SMR from Seagate

On a completely separate, but surpisingly related thread: one of the major developments in rotational hard drive technology in the last year have been focused on archival storage. Seagate announced Shingled Magnetic Recording (SMR) with densities up to 1.25TB per platter. SMR drives overlap groups of tracks, leaving valid read tracks inside the boundaries of wider write tracks. SMR drives can store much more data this way, but data re-writes are much slower with SMR than existing perpendicular magnetic recording (PMR) drive technology drives. This is because when a block is updated the entire group of tracks has to be overwritten—much like solid-state page writes. While the only known customers of SMR drives to date are Seagate’s subsidiary E-Vault, this technology would seem to line up well with HDFS workloads.

Photo credit: _cheryl via Flickr

Cloud Services, Cloud computing, IaaS, IDS, Integrated Data Storage

Driving Change Through IaaS

By | Cloud Computing, Networking | No Comments

Cloud Services, Cloud computing, IaaS, IDS, Integrated Data Storage

Infrastructures as a Service (IaaS) is transforming the way businesses approach their computing environment. IT executives are improving cost efficiency, improving the quality of service and ultimately providing business agility by embracing this paradigm shift in computing.

On the road to IaaS, the ability to partner with a provider that can customize a hosted Private Cloud is key to providing the flexibility that allows IT decision makers to leverage the consumption-based costs models of the Cloud. With this flexibility it is key that the security, governance, and performance are maintained along with the customization that businesses require. The key to this is that there are multiple options to choose from.

Whether if organizations are looking for a turn-key solution that is fully managed or an Infrastructure where they have full control, IaaS allows for customization and flexibility.

The Cloud Infrastructure Service from IDS brings Enterprise-class, best-of-breed architecture to the Cloud. It is an ideal solution for companies that want to leverage cloud computing but need reliable, customizable solutions built by a business partner that understands their needs.

Some of the key benefits from utilizing the IDS Cloud Infrastructure Service include:

1. The IDS Cloud Infrastructure is scalable, expanding and retracting as you need it.
2. Demand for new space can be met in minutes instead of weeks or months.
3. Pay for only the space you need instead of investing in Infrastructure you may never use.
4. All IDS Cloud systems are guaranteed and proven to be as much as 99.999% reliable.
5. IDS Cloud can meet key governance standards and regulations on a client by client basis.
6. Data will be protected by 24×7 surveillance, man traps, biometric readers, key cards and multifactor authentication.

IDS Cloud Infrastructure Service is a standardized, highly automated offering, where compute resources, complemented by storage and networking capabilities, are offered to customers on-demand.

Photo by Hugh Llewelyn

Avamar, Cloud Infrastructure, eSata card, Segate GoFlex, Iomega, NAS, Ethernet, Xcopy, Windows, Linux box, grid, Jon Austin,

Speed Up First-time Avamar Backup by “Seeding” the Server

By | Avamar, Backup, How To | No Comments

avamar seeding blog header 1200Avamar is a great tool to backup remote office file servers to a private or hybrid cloud infrastructure. However, performing an initial backup can be a challenge if the server is more than a few GB and the connection to the remote office is less than 100Mb.

In this scenario, the recommended process is to “seed” the Avamar server with the data from the remote server. There are a number of devices that can be used to accomplish this: USB hard drives are the most often used; however, they can be painfully slow, as most modern servers only have USB 2.0 ports that can only transfer around 60MB/sec and are limited to 3-4TB in size. In order to copy 3TB to a USB 2.0 drive, it will typically take 12-16 hours. Not unbearable, but quite a while.

Another option would be to install a USB 3.0 adapter card or eSata card—but that requires shutting down the server and installing drivers, etc. An alternative that I have had a good deal of success with is using a portable NAS device like the Seagate GoFlex drives or, for larger systems, the Iomega/LenovoEMC StorCenter px4-300d. The px4-300d has an added feature that I will touch on later. These NAS devices leverage Gigabit Ethernet and can roughly double the transfer rate of USB 2.0.

portable nas storage devices

Moving the data to these “seeding” targets can be as simple as drag-and-drop or using a command line utility like Xcopy from Windows or Rsync from a Linux box once you plug in the USB device or mount a share from the NAS drives. When the data copy is complete, eject the USB drive or unmount the share, power down the unit, package the drive for shipping and send it back to the site where the Avamar grid lives.

At the target site attach the portable storage device to a client locally and configure a one-time backup of this data. With the Iomega device, it includes a pre-installed Avamar client that can be activated to the grid and backed up without having to go through an intermediary server.

Once you get this copy of the backup into the Avamar grid, activate and start the backup of the remote client. The client will hash it’s local data and compare to what is on the gridfinding that the bulk of the unique data is already populated – reducing the amount of data required to transfer to the data that has changed or added since the “seed”.

Photo credit: Macomb Paynes via Flickr

Letting Cache Acceleration Cards Do The Heavy Lifiting

By | EMC, How To, Log Management, Networking, Storage, VMware | No Comments

Up until now there has not been a great deal of intelligence around SSD Cache cards and flash arrays because they have primarily been configrued as DAS (Direct Attach Storage). By moving read intensive workload up to the server off of a storage array, both individual application performance as well as overall storage performance can be enhanced. There are great benefits to using SSD Cache cards in new ways yet before exploring new capabilities it is important to remember the history of the products.

The biggest problem with hard drives either local or SAN based is that they have not been able to keep up with Moore’s Law of Transistor Density. In 1965 Gordon Moore, a co-founder of Intel, made the observation that the number of components in integrated circuits doubled every year – he later (in 1975) adjusted that prediction to doubling every two years. So, system processors (CPUs), memory (DRAM), system busses, and hard drive capacity have been doubling in speed every two years, but hard drives performance has stagnated because of mechanical limitations. (mostly heat, stability, and signaling reliability from increasing spindle speeds) This effectively limits individual hard drives to 180 IOPs or 45MB/sec under typical random workloads depending on block sizes.

The next challenge is that in an effort to consolidate storage, increase the number of spindles, availability and efficiency we have pulled the storage out of our servers and placed that data on SAN arrays. There is tremendous benefit to this, however doing this introduces new considerations. The network bandwidth is 1/10th of the system bus interconnect (8Gb FC = 1GB/sec vs PCIe 3.0 x16 = 16GB/sec). An array may have 8 or 16 front-end connections yielding and aggregate of 8-16GB/sec where a single PCIe slot has the same amount of bandwidth. The difference is the array and multiple servers share its resources and each can potentially impact the other.

Cache acceleration cards address both the mechanical limitations of hard drives and the shared-resource conflict of storage networks for a specific subset of data. These cards utilize NAND flash (either SLC or MLC, but more on that later) memory packaged on a PCIe card with an interface controller to provide high bandwidth and throughput for read intensive workloads on small datasets of ephemeral data.

[framed_box bgColor=”#F0F0F0″ textColor=”undefined” rounded=”true”] I realize there was a lot of qualification statements there so lets break it down…

  • Why read intensive? As compared to SLC NAND flash, MLC NAND flash has a much higher write penalty making writes more costly in terms of time and overall life expectancy of a drive/card.
  •  Why small datasets? Most Cache acceleration cards are fairly small in comparison to hard drives. The largest top out at ~3TB (typical sizes are 300-700GB) and the cost per GB is much much higher than comparable hard drive storage.
  •  Why ephemeral data and what does that mean? Ephemeral data is data that is temporary, transient, or in process. Things like page files, SQL server TEMPDB, or spool directories.
[/framed_box] Cache acceleration cards address the shared-resource conflict by pulling resource intense activities back onto the server and off of the SAN arrays. How this is accomplished is the key differentiator of the products available today.

SSD Caching , EMC, VFCache, FusionIO, VMWare, SLC, MLC NAND Flash, Gordon Moore, Intel, processors, CPU's, DRAM

FusionIO is one of the companies that has made a name for themselves early in the enterprise PCI and PCIe Flash cache acceleration market. Their solutions have been primarily DAS(Direct Attach Storage) solutions based on SLC and MLC NAND Flash. In early 2011 FusionIO released write-through caching to their SSD cards with their acquisition of ioTurbine software to accelerate VMWare guest performance. More recently – Mid-2012 – FusionIO released their ION enterprise flash array – which consists of a chassis containing several of their PCIe cards. They leverage RAID protection across these cards for availability. Available interconnects include 8Gb FC and Infiniband. EMC release VFCache in 2012 and has subsequently released two additional updates.

The EMC VFCache is a re-packaged Micron P320h or LSI WarpDrive PCIe SSD with a write-through caching driver targeted primarily at read intensive workloads. In the subsequent releases they have enhanced VMWare functionality and added the ability to run in “split-card” mode with half the card utilized for read caching and the other half as DAS. EMC’s worst kept secret is their “Project Thunder” release of the XTremIO acquisition. “Project Thunder” is an all SSD array that will support both read and write workloads similar to the FusionIO ION array.

SSD Caching solutions are an extremely powerful solution to very specific workloads. By moving read intensive workload up to the server off of a storage array, both individual application performance as well as overall storage performance can be enhanced. The key to determining whether or not these tools will help is careful analysis around reads vs writes, and the locality of reference of active data. If random write performance is required consider SLC based cards or caching arrays over MLC.


Images courtsey of “the register” and “IRRI images

Protecting Exchange 2010 with EMC RecoverPoint and Replication Manager

By | Backup, Deduplication, Disaster Recovery, EMC, Replication, Storage | No Comments

Regular database backups of Microsoft Exchange environments are critical to maintaining the health and stability of the databases. Performing full backups of Exchange provides a database integrity checkpoint and commits transaction logs. There are many tools which can be leveraged to protect Microsoft Exchange environments, but one of the key challenges with traditional backups is the length of time that it takes to back up prior to committing the transaction logs.

Additionally, the database integrity should always be checked prior to backing up: to ensure the data being backed up is valid. This extended time often can interfere with daily activities – so it usually must be scheduled around other maintenance activities, such as daily defragmentation. What if you could eliminate the backup window time?

EMC RecoverPoint in conjunction with EMC Replication Manager can create application consistent replicas with next to zero impact, that can be used for staging to tape, direct recovery, or object level recovery with Recovery Storage Groups or third party applications. These replicas leverage Microsoft VSS technology to freeze the database, RecoverPoint bookmark technology to mark the image  time in the journal volume, and then thaw the database in a matter of less then thirty seconds – often in less than five seconds.

EMC Replication Manager is aware of all of the database server roles in the Microsoft Exchange 2010 Database Availability Group (DAG) infrastructure and can leverage any of the members (Primary, Local Replica, or Remote Replica) to be a replication source.

EMC Replication Manager automatically mounts the bookmarked replica images to a mount host running the Microsoft Exchange tools role and the EMC Replication Manager agent. The database and transaction logs are then verified using the essentials utility provided with the Microsoft Exchange tools. This ensures that the replica is a valid, recoverable copy of the database. The validation of the databases can take from a few minutes to several hours, depending on the number and size of databases and transaction log files. The key is: the load from this process does not impact the production database servers. Once the verification completes, EMC Replication Manager calls back to the production database to commit and delete the transaction logs.

Once the Microsoft Exchange database and transaction logs are validated, the files can be spun off to tape from the mount host, or depending on the retention requirement – you could eliminate tape backups of the Microsoft Exchange environment completely. Depending on the write load on the Microsoft Exchange server and how large the journal volumes for RecoverPoint are, you can maintain days or even weeks of retention/recovery images in a fairly small footprint – as compared to disk or tape based backup.

There are a number of recovery scenarios that are available from a solution based on RecoverPoint and Replication Manager. The images can be reversed synchronized to the source – this is a fast delta-based copy, but is data destructive. Alternatively, the database files could be copied from the mount host to a new drive and mounted as a recovery storage group on the Microsoft Exchange server. The database and log files can also be opened on the mount host directly with tools such as Kroll OnTrack for mailbox and message-level recovery.

Photo Credit: pinoldy

Fun With DART’s & Replication Between A New VNXe And A Celerra NS-120

By | EMC, How To, Replication | No Comments

A couple of weeks ago I had some fun configuring a new VNXe to replicate with a Celerra NS-120. Here is a transcript of how I got it to work and some of the oddities I encountered:

1. I started out by checking the ESM whose configuration is supported according to the Support Matrix, as long as the Celerra is running DART 6.0. I then upgraded the NS20 (CX3-10 back-end running latest [and last] R26 code) to 6.0.41.

2. Moving along, I set up the interconnects using the “Unisphere” Hosts Replication Connection wizard. I validated the interconnects using nas_cel -interconnect -list on the NS20.

3. I had some initial issues with routing that were quickly resolved and the interconnects looked good to go.

4. This is where it gets dicey: I started out using the wizard on the NS20 Unisphere to replicate a filesystem. Apparently, the NS20 can’t set up replication destination storage and doesn’t seem to be able to enumerate/read the remote filesystem names.

I was able to see a list of remote filesystem IDs though, so this started me thinking : what if I could login to the remote “Celerra” (read, DART instance on the VNXe) to decode which filesystem correlated to which ID, i.e. run nas_fs -list ?

I tried SSH’ing to the VNXe and saw that it was shut down, so I started poking around in the service options and realized that I could enable SSH. I did that – SSH’ed to the box and logged in as “service”, because admin didn’t work. From there, I SU’d to nasadmin and was prompted for the root password. I tried nasadmin, the service password and a couple other passwords I knew of, but it timed out after three tries. However, I was in a nasadmin context so I ran the nas_fs -list command and got precisely what I was looking for – the list of filesystem ID’s to filesystem names.

5. Time services – for replication to work, you have to be within ten minutes (preferably five) of the respective datamovers. I thought I would proactively double-check and set the NTP on the VNXe “server_2” – however I was shut down, because that requires root permissions. Luckily time was pretty close so I was good there (NOTICE: the datamover was set to UTC – probably by design, but required conversion to local time).

6. By this time I realized that using the Celerra Manager/Unisphere/Wizards were not likely to work, so I logged on to the NS20 and ran the nas_replicate -create FS_Name -source -fs -id=552 -destination -fs id=30 -interconnect id=20003 -max_time_out_of_sync 10 command which erred out after a few minutes.

I did some digging on iView and found the primus article emc263903, which referenced logging in as root to run the command. Ok, I have the NS20 root password, so I did that and got the error message “destination filesystem not read-only”. I had created the “Shared Folder” (otherwise known to us old timers as a “File System”) as a replication target – don’t you think that if you are creating a replication target that the wizard would mount it as read-only?

7. Ok, back on the VNXe through SSH as nas admin: I run server_unmount and am prompted for the root password again, it errors out three times and then check – it’s unmounted! I run server_mount with -o ro, get prompted for the root password and error out three more times.

8. Back on the NS20, I re-run the nas_replicate command and it errors, this time with a “destination not empty” message. I used the -overwrite option, because when I provisioned the destination filesystem the minimum size that the wizard presented for the save was the same size as the destination filesystem …

Finally success: the filesystem is replicating!

Photo Credit: calypso_dayz

What Happens When You Poke A Large Bear (NetApp SnapMirror) And An Aggressive Wolf (EMC RecoverPoint)?

By | Backup, Clariion, Data Loss Prevention, Deduplication, Disaster Recovery, EMC, NetApp, Replication, Security, Storage | No Comments

This month I will take an objective look at two competitive data replication technologies – NetApp SnapMirror and EMC RecoverPoint. My intent is not to create a technology war, but I do realize that I am poking a rather large bear and an aggressive wolf with a sharp stick.

A quick review of both technologies:


  • NetApp’s controller based replication technology.
  • Leverages the snapshot technology that is fundamentally part of the WAFL file system.
  • Establishes a baseline image, copies it to a remote (or partner local) filer and then updates it incrementally in a semi-synchronous or asynchronous (scheduled) fashion.


  • EMC’s heterogeneous fabric layer journaled replication technology.
  • Leverages a splitter driver at the array controller, fabric switch, and/or host layer to split writes from a LUN or group of LUNs to a replication appliance cluster.
  • The split writes are written to a journal and then applied to the target volume(s) while preserving write order fidelity.

SnapMirror consistency is based on the volume or qtree being replicated. If the volume contains multiple qtrees or LUNs, those will be replicated in a consistent fashion. In order to get multiple volumes replicated in a consistent fashion, you will need to quiesce the applications or hosts accessing each of the volumes and then take snapshots of all the volumes and then SnapMirror those snapshots. An effective way to automate this process is leveraging SnapManager.

After the initial synchronization SnapMirror targets are accessible as read-only. This provides an effective source volume for backups to disk (SnapVault) or tape. The targets are not read/write accessible though, unless the SnapMirror relationship is broken or FlexClone is leveraged to make a read/write copy of the target. The granularity of the replication and recovery is based off a schedule (standard SnapMirror) or in a semi-synchronous continual replication.

When failing over, the SnapMirror relationship is simply broken and the volume is brought online. This makes DR failover testing and even site-to-site migrations a fairly simple task. I’ve found that many people use this functionality as much for migration as data protection or Disaster Recovery. Failing back to a production site is simply a matter of off-lining the original source, reversing the replication, and then failing it back once complete.

In terms of interface, SnapMirror is traditionally managed through configuration files and the CLI. However, the latest version of ONCommand System Manager includes an intuitive easy to use interface for setting up and managing SnapMirror Connections and relationships.

RecoverPoint is like TIVO® for block storage. It continuously records incoming write changes to individual LUNs or groups of LUNs in a logical container aptly called a consistency group. The writes are tracked by a splitter driver that can exist on the source host, in the fabric switch or on a Clariion (VNX) or Symmetrix (VMAXe only today) array. The host splitter driver enables replication between non-EMC and EMC arrays (Check ESM for latest support notes).

The split write IO with RecoverPoint is sent to a cluster of appliances that package, compress and de-duplicate the data, then sends it over a WAN IP link or local fibre channel link. The target RecoverPoint Appliance then writes the data to the journal. The journaled writes are applied to the target volume as time and system resources permit and are retained as long as there is capacity in the journal volume in order to be able to rewind the LUN(s) in the consistency group to any point in time retained.

In addition to remote replication, RecoverPoint can also replicate to local storage. This option is available as a standalone feature or in conjunction with remote replication.

RecoverPoint has a standalone Java application that can be used to manage all of the configuration and operational features. There is also integration for management of consistency groups by Microsoft Cluster Services and VMWare Site Recovery Manager. For application consistent “snapshots” (RecoverPoint calls them “bookmarks”) EMC Replication Manager or the KVSS command line utilities can be leveraged. Recently a “light” version of the management tool has been integrated into the Clariion/VNX Unisphere management suite.

So, sharpening up the stick … NetApp SnapMirror is a simple to use tool that leverages the strengths of the WAFL architecture to replicate NetApp volumes (file systems) and update them either continuously or on a scheduled basis using the built-in snapshot technology. Recent enhancements to the System Manager have made it much simpler to use, but it is limited to NetApp controllers. It can replicate SAN volumes (iSCSI or FC LUNs) in NetApp environments – as they are essentially single files within a Volume or qtree.

RecoverPoint is a block-based SAN replication tool that splits writes and can recover to any point in time which exists in the journal volume. It is not built into the array, but is a separate appliance that exists in the fabric and leverages array, and fabric or host based splitters. I would make the case that RecoverPoint is a much more sophisticated block-based replication tool that provides a finer level of recoverable granularity, at the expense of being more complicated.

 Photo Credit: madcowk

Isilon Storage (Still) Supports “Block Headed” IT

By | EMC, Isilon, Storage | No Comments

Call me crazy, but imagine this: a high performance, highly available storage system that uses off-the-shelf components and no RAID. And add to that ease of use and the ability to scale to petabytes in minutes. On top of that, throw in some features like snapshots, site-to-site replication, and intelligent auto-tiering. Seeing it yet?

The picture I’m painting is a system that has NetApp throwing stones and EMC wondering what’s next for Celerra, now that it has spent $2.25 billion on keeping it out of NetApp’s hands. If you haven’t figured it out yet, I am thoroughly enamored with our friends at Isilon:)

Okay, so I asked you to imagine a highly available system without RAID—yet, not without data protection. Isilon’s scale-out architecture distributes incoming writes across nodes mirroring what they deem small files (less than 128KB) and striping files 128KB and larger in 16KB chunks across each node in the grid with parity.

Here is where it gets interesting, though: Protection can be set on a block and node protection basis, so you can define how many drives or nodes you want to be able to survive losing. When you lose a drive or a node, the grid rebuilds from parity across free space on the remaining nodes.

But that’s only the first reason for my enamoredness (yep, it’s a word) …

I’m a dyed-in-the wool “Block Head,” and I know Isilon is a NAS platform, but they really got my attention with the OneFS OS. They handle all the major protocols: NFSv2, v3, and v4, pnfs, FTP, HTTP, and CIFS/SMB. Authentication support for LDAP, NIS, Active Directory and local users and groups are all supported. For backup: NDMP is supported, directly to tape through a backup accelerator node or across the front-end Ethernet via 3-Way NDMP.

For us “Block Heads,” Isilon supports iSCSI—after all, a LUN is just a file, right?


144 Node Isilon Cluster

Photo Credit: Paul Stevenson

How To: Migrating VMware ESX 2.5 Datastores with EMC SANCopy #fromthefield

By | EMC, How To, Storage, VMware | No Comments

I know this is a little old-school, but since I didn’t find a good reference anywhere online … I am going to cover migrating VMWare ESX 2.5 datastores with EMC SANcopy in this month’s installment. If you find yourself in this situation—you will know that it works and how it’s done. I recently ran through this migrating from an EMC Clariion CX-300 to a new VNX5300.

High-level, here are the steps to migrate an ESX 2.5 Datastore with SANCopy
1. Document, Document, Document
2. Setup SANCopy
3. Shut down Guests and Host
4. Start Copy Sessions
5. Reconfigure host to access new array
6. Restart Host and then Guest

This how-to makes some assumptions.
– You know your way around Navisphere and Unisphere
– You know your way around the VMWare ESX 2.5 management interfaces

1. Document, document, document. I can’t stress this enough; it’s key to know exactly how things are set up, because it’s essential that we make it look the same on the target system. It is invaluable to have an up to date CAP report or Array Configuration report. The goal is to make as little change for the ESX 2.5 host as possible and the more you record upfront, the easier your life will be. The key things to record for each of the attached Datastores are the following:
– Source LUN ID
– Host LUN Id (this is configured and shown in the Storage Group LUN properties)
– Owning SP (this is in LUN properties)
– Size in GB (If this is not an integer value record the number of blocks in LUN properties)
– LUN WWN (this is in LUN properties, but the best place to get this is from a CAP report – because you can copy and paste)

Here’s a sample table you can use to record the data:

2. Setup SANCopy. In order to run SANCopy at least one of the arrays needs to have the SANCopy license enabler installed. It makes the process significantly easier if the enabler is installed on both arrays, but at a minimum it should be installed on one array, preferably the target array. If the enabler is installed on both arrays, both arrays will need to be in the same Navisphere management domain. Once the enabler(s) are installed, zone the front end ports on each array to the front end ports on the partner array (i.e. SPA0 on source to SPA0 and SPB1 on target, and so forth). When the FE ports are zoned, create a SANCopy Storage Group on the source array and update the SANCopy connections on both arrays. The initiators should show up and register automatically on the source array.

The next part is to setup the SANCopy sessions. Put each of the source LUNs to be replicated into the SANCopy Storage group and then run the SANCopy wizard in order to create a session for each LUN. The wizard walks you through selecting the SANCopy array (target array), Source LUN (use the enter WWN and paste the LUN WWN in the entry field), Target storage device (right-click and select target storage), and the session details. I find it valuable to set the name of the session to something descriptive, like the source LUN name, size, etc. If both arrays have the SANCopy enabler installed the selection process is a little more intuitive as you can see the LUN names, size, etc.

3. Shut down Guests and VMWare Host. SANCopy is an array-based block copy that copies all blocks in a LUN sequentially to a target LUN. In order to get a consistent useable copy the host cannot access the source or target for the duration of the copy. Ideally, when shutting down the VMWare guest machines, set the automatic restart value to disabled so they don’t try to start when the new target datastores are mounted when VMWare starts.

4. Start SANCopy Sessions. By default SANCopy sessions have a throttle value of 6—I typically change this to 8 or 10 in a migration effort for best throughput. This is equivalent to setting a LUN migration to High. If you are going to start multiple simultaneous migrations, you may need to adjust the Active Sessions per SP from 2 to 4. Start the sessions and monitor until all sessions complete.

5. Reconfigure host to access new array. Zone the host to the new array making sure to disable or delete the zones to the old array. Power on the host and interrupt the boot post cycle at the HBA BIOS screen. At this point the HBA should have logged in to the target array—if it has not, re-check zoning and/or use the HBA BIOS utility to rescan targets. Once the HBA is logged in to the array, use the Connectivity Status tool to manually register the host connections. When the host HBA connections are registered, create a Storage Group adding the LUNs and paying careful attention to setting host LUN id values to match the source array configuration. Add the host to the storage group and exit the HBA BIOS utility and reboot.

6. Restart Host and then Guests. When the host powers up verify that the datastores are visible and browsable using the VMWare Client utility. If everything looks good power on the guests and you are done.

A couple of thoughts … this example was migrating between two Clariion arrays, but SANCopy also works for migrating data from HDS arrays, HPQ arrays and other SAN platforms to a Clariion—as long as the LUNs are identifiable by LUN WWN or storage array Port WWN and LUN id.

Photo credit: alex.ragone via Flickr