All Posts By

Humayun Khan

Data classification

Data Classification: Creating Business Value

By | Data Center, Storage | No Comments

As we discussed in my previous article, “Data Retention: In IT We Trust,” data retention is a necessary component of a comprehensive information lifecycle management (ILM) strategy. In this article we will turn our attention to data classification.

Data classification is the essential step towards ILM. Data classification helps organizations know what data they have, where the data is located and how they can access it (if at all). This becomes increasingly important with the uncontrollable growth in unstructured data that appears to push the infrastructure limits to new heights with every technological advance. Read More

Data retention

Data Retention: In IT We Trust

By | Archiving, Backup | No Comments

While discussing backups and recovery validation in my first blog of this series, “The Case for Disaster Recovery Validation“, I cited “…The secondary purpose of backups is to recover data from an earlier time, according to a user-defined data retention policy…” [Wikipedia]. In this blog, I will review data loss/retention in regard to backups and archives, and the difficulties inherent in specifying retention times.

The burgeoning cost of storing increased amounts of data may have put an unfair burden on most IT organizations because the business and application “users” are failing to specify the retention needs of the data created. Now, more than ever before, IT has to balance budgets with regulatory compliance, industry standards and company constraints. This may vary from five days to fifty years, depending on the purpose of data retention strategy, the type of data and the functional use of the data: financial, health, education, research, government, etc. Further, retention policies apply to one or both systems of data retention: backup and archiving. Read More

The Case for Disaster Recovery Validation

The Case for Disaster Recovery Validation

By | Backup, Disaster Recovery, How To | No Comments

Disaster Recovery Planning (DRP) has gotten much attention in the wake of natural and man-made disasters in the recent years. But Executives continue to doubt the ability of IT to restore business IT infrastructure after a serious disaster. And this does not even include the increasing number of security breaches worldwide. By many reports, the confidence level in IT recovery processes is less than 30%, bringing to question the vast amounts of investment poured into recovery practices and recovery products. Clearly, backup vendors are busy – see compiled list of backup products and services at the end of this article (errors and omissions regretted). Read More

How to Create a Technology Roadmap

Developing Successful Technology Roadmaps for Your Organization

By | Data Center, Design & Architecture, How To, Strategy | No Comments

This is the second of a two-part series on Technology Roadmaps. Previously we explained “The Concepts behind a Technology Roadmap,” and here we explain how to develop one. 

Technology roadmaps begin with a “handshake” between IT and the business. Knowing future business plans allows IT to determine the focus area(s). As businesses evolve and new technologies emerge, IT is challenged with constant change. Developing roadmaps helps IT to be prepared for the change and manage the associated risks.

How Do You Create a Technology Roadmap?

  1. Collect Data. Take the time to gather preliminary information about products, people and processes. Understand current implementations and directions.
  2. Hold Interviews. Identify key stakeholders and gain different perspectives. Meet individually or in groups, and be sure to cover topics like resources, costs, risk, compliance, growth, skills, support and management.
  3. Create technology baselines. Document the essentials and highlight the constraints. Stay sufficiently high-level, but acknowledge the details around recent changes.
  4. Analyze focus areas. Use a structured method for the analysis. One of the most widely used framework in business analysis is the SWOT (Strength-Weakness-Opportunities-Threats) model. Since opportunities and threats relate to the industry at large, it is important to have subject matter experts (SMEs) provide input at this stage.
  5. Construct technology roadmaps. This is a collaborative exercise incorporating the inclusion of emerging technologies over several years. This does not always have to be a chart or a graph. It can be as simple as an enumeration of important technology adoptions in stages. For best results, use a backward sweep starting from end objectives, and then a forward sweep showing how adopting a technology at each stage can lead to the end objective. Continue this same pattern until you get it just right.
  6. Present recommendations. Knowing the roadmaps enables you to enumerate the IT projects that need attention in the coming months. There should also be clarity on the investment needed in terms of budget, time and resources.
  7. Host a workshop. Facilitate a workshop where key stakeholders meet again to review the results. This is a necessary touch point to discuss the project-based initiatives and make any final adjustments to the course.

How effective are Technology Roadmaps?

It all depends on the people and the effort put into the exercise. As indicated in the first part of this two-part series, technology roadmaps bring consensus and improved planning, budgeting & coordination. It is critical that organizations treat this as a project in itself, and provide the necessary funds and resources.

While an internal committee may be established to execute such a project, the benefits of technology roadmaps multiply exponentially when an external partner, like IDS, is involved. IDS guarantees a proven process with expert methodology, and key insight on the final deliverable. A partner like IDS can pre-empt much of the struggle by bringing SMEs to the table and a fresh external perspective.

And remember: As businesses and technologies evolve, so will the roadmaps. So, review them often.

Learn more by reading the first part of this two-part series, “The Concepts Behind a Technology Roadmap.”

Creating a Technology Roadmap for your Organization

The Concepts Behind a Technology Roadmap

By | Design & Architecture, Project Management, Strategy | No Comments

Information Technology is critical to an organization’s success in modern times. Yet, too often we tend to get comfortable with what we have inherited or have in place today. Yes, IT is concerned with growth, and costs and will “shrink” on demand, but does anyone know where IT is headed? The costs question is directly related to at least two other questions that not every IT department asks:

  • Where do we [IT] want to be in 3 years? In 5 years? In 10 years?
  • If we continue to do what we do today, will we get there?

The good news is, IT is getting wiser. IT knows the importance of strategy. Strategic thinking has made its way from textbooks to the real world. Today, IT leaders work with businesses to provide direction that translates to a strategy, which leads to a plan. A technology roadmap is a special kind of plan. Here is the Wikipedia definition:

“A technology roadmap is a plan that matches short-term and long-term goals with specific technology solutions to help meet those goals. It is a plan that applies to a new product or process, or to an emerging technology.”

Some will differentiate between product roadmaps and technology roadmaps. For our purposes, we will stick to implementation of IT technologies in a typical organization.

What Drives a Technology Roadmap?

Technology roadmaps are only plausible when the goals are clear. The roadmaps are only as valuable as the areas of interest in the business. So, from an IT perspective, the rubber meets the road when we know:

  • How business applications are preparing for tomorrow’s challenges?
  • Which infrastructure changes will maximize the value to the business?

This gives a roadmap its purpose.

From there on, it is a matter of gaining a good understanding of “what is” and “what can be.” In each focus area, the IT team must evaluate the technology trends and uncertainties, relating that back to the current state and skills in the organization.

  • Do we know what else is out there?
  • Do we have what it takes to get there?

This gives a roadmap its framework.

What Can Happen Without a Technology Roadmap?

Without a technology roadmap, organizations carry unaddressed costs and risks due to outdated strategies and quick fixes that resemble patches in a quilt. Technology roadmaps bring consensus and improved planning, budgeting & coordination.

As technology evolves, so does the roadmap. An outdated technology roadmap can be almost as harmful as not having one at all. Sedentary strategies mean organizations are likely to fall victim to unplanned costs and reduced value to the business. It is, therefore, critical to setup a recurring review period where key stakeholders refresh and revise the map as business requirements continue to transform.

Stay tuned for the next part of this two-part series, when we dive into the steps needed to create your own technology roadmap.

The Many Faces of Archiving (And Why It Isn’t Backup)

By | Archiving, Backup, Disaster Recovery | No Comments

If archiving is defined as intelligent data management, then neither Backup Technologies, nor Hierarchical Storage Management (HSM) techniques, nor Storage Resource Management (SRM) tools qualify; however, these continue to be leveraged for archiving as substitute products. Even “Information Lifecycle Management” that would benefit from archiving is now equated with archiving. This has led to a proliferation of archiving products that tend to serve different purposes for different organizations.

Background

IT organizations have long valued the notion of preserving copies of data in case “work” got lost. In fact, with every occurrence of data disaster, the role of data backup operations has strengthened and no company can do without a strategy in place. Since 1951, when Mauchly and Eckert ushered in the era of digital computing with the construction of UNIVAC, the computing industry has seen all kinds of media in which storage could be kept for later recall: punch cards, magnetic tapes, floppy disks, hard drives, CD-R/RW, flash drives, DVD, Blue-ray and HD-DVD to name a few. And the varying formats and delivery methods have helped create generations of vendors with competing technologies.

Backups had come of age … but also became increasingly costly and hard to manage with data complexity, growth and retention.

Backups had come of age, cloaked and dressed with a respectable name “data protection”—the magic wand that was insurance for “data loss.” But, it also became increasingly costly and hard to manage with data complexity, growth and retention. Thus came about the concept of “archiving,” defined simply as “long term data.” That, coupled with another smart idea for moving data to less expensive storage (tier), helped IT organizations to reduce costs. The HSM technique dovetails into tiered storage management, as it is really a method to move data that is not changing or not being accessed frequently. HSM was first implemented by IBM and also by DEC VAX/VMS systems. In practice, HSM is typically performed by dedicated software, such as IBM Tivoli Storage Manager, Oracle’s SAM-QFS, Quantum SGI DMF, StorNext or EMC Legato OTG DiskXtender.

On the other hand, SRM tools evolved as quota management tools for companies trying to deal with hard-to-control data growth, and now include SAN management functions. Many of the HSM players sell tools in this space as well: IBM Tivoli Storage Productivity Center, Quantum Vision, EMC Storage Resource management Suite, HP Storage Essentials, HDS Storage Services Manager (Aptare) and NetApp SANscreen (Onaro). Other SRM products include Quest Storage Horizon (Monosphere), SolarWinds Storage Profiler (Tek-Tools) and CA Storage Resource Manager. Such tools are able to provide analysis, create reports and target inefficiencies in the system, creating a “containment” approach to archiving.

Almost as old as the HSM technique is the concept of Information Lifecycle Management (ILM). ILM recognizes archiving as an important function distinct from backup. In 2004, SNIA gave ILM a broader definition by aligning it with business processes and value, while associating it with five functional phases: Creation and Receipt; Distribution; Use; Maintenance; Disposition. Storage and Backup vendors embraced the ILM “buzzword” and re-packaged their products as ILM solutions, cleverly embedding HSM tools in “policy engines.” And so, with these varied implementations of “archiving tools,” businesses have come to realize different levels of satisfaction.

Discussion

Kelly J. Lipp, who today evaluates products from the Active Archive Alliance members, wrote (in 1999) the paper entitled “Why archive is archive, backup is backup and backup ain’t archive.” Kelly wrote this simple definition: “Backup is short term and archive is long term.” He then ended the paper with this profound statement: “We can’t possibly archive all of the data, and we don’t need to. Use your unique business requirements and the proper tools to solve your backup and archive issues.”

Backup is short term and archive is long term.
— Kelly Lipp

However, the Active Archive Alliance promotes a “combined solution of open systems applications, disk, and tape hardware that gives users an effortless means to store and manage ALL their data.” ALL their data? Yes, say many of the pundits who rely on search engines to “mine” for hidden nuggets of information.

Exponential data growth is pushing all existing data management technologies to their limits, and newer locations for storing data—the latest being “storage clouds”—attempt to solve the management dilemma. But the realization that for the bulk of data that is “unstructured,” there is no orderly process to bring back information that is of value to the business brings increasing concern.

Similarly, though, to the clutter stored in our basement, data that collects meaninglessly may become “data blot.”

Although businesses rely on IT to safeguard data, the value of the information contained therein is not always known to IT. Working with available tools, IT chooses attributes such as age and size and location to measure the worth, and then executes “archiving” to move this data out, so that computing systems may perform adequately. Similarly, though, to the clutter stored in our basement, data that collects meaninglessly may become “data blot.” Data survival then depends on proper information classification and organization.

Traditionally, data has seen formal organization in the form of databases—all variations of SQL, email and document management included. With the advent of Big Data and the use of ecosystems such as “Hadoop,” large databases now leverage flat file systems that are better suited for mapping search algorithms. And this may be considered as yet another form of archiving because data stored here is “immutable” anyway. All of these databases (and the many related applications) tend to have more formal archiving processes, but little visibility into the underlying storage. Newer legal and security requirements tend to focus on such databases, leading to the rise of “archiving” for compliance.

That brings us back full circle. While security and legality play a lot in today’s archiving world, one could argue that these tend to create “pseudo archives” that can be removed (deleted) when the stipulated time has passed. In contrast, a book or film on digital media adds to the important assets of a company that become the basis for its valuation and for future ideas. If one were to create a literature masterpiece, the file security surrounding the digitized asset is less consequential than the fact that 100 years later those files would still be valuable to the organization that owns it.

Archiving … is the preservation of a business’s digital assets: information that is valuable irrespective of time and needed when needed.

The meaning of archiving becomes clearer when viewed as distinctly different from backup. It is widely accepted that purpose of a backup is to restore lost data. Thus, backup is preservation of “work in progress”: data that does not have to be understood, but resurrected as-is when needed. Archiving, on the other hand, is the preservation of a business’s digital assets: information that is valuable irrespective of time and needed when needed. The purpose of archiving is to hold assets in a meaningful way for later recall.

Backup is a simple IT process. Archiving is tied to business flow.

This suggests that archiving does not need “policy engines” and “security strongholds,” but rather information grouping, classification, search and association. Because these tend to be industry-specific, “knowledge engines” would be more appropriate for archiving tools. Increasingly, IT professional services are now working with businesses and vendors alike to bridge the gaps and bring about dramatic industry transformations through the implementation of intelligent archiving.

Conclusion

Backups have grown in importance since the days of early computing, and as technology has changed, so has the costs for preserving the data in different storage media. Backup technologies also have become substitute tools for archives by choosing long-term retention for those data.

With a plethora of tools and techniques developed to manage the storage growth, and contain the storage costs (the HSM techniques and the SRM tools), archiving has been implemented in different organizations for different purposes and with different meaning.

In defining Information Lifecycle Management, SNIA has elevated the importance of archiving, and thereby encouraged vendors to re-package HSM tools in policy engines. On the other hand, databases for SQL and email—and even Big Data ecosystems—have implemented archiving without visibility into the underlying storage.

As archiving tools continue to evolve, it is now considered distinctly different from backup. While backup protects “work in progress,” archiving preserves valuable business information. Unlike backups that need “policy engines,” archiving requires “knowledge engines” which may be industry-specific. IT professional services have stepped in to bridge the gaps and bring about transformations through the implementation of intelligent archiving.

References

1. “Why archive is archive, backup is backup and backup ain’t archive” by Kelly J. Lipp, 1999
2. http://en.wikipedia.org/wiki/Backup
3. http://en.wikipedia.org/wiki/Archive
4. http://www.backuphistory.com/
5. http://www.activearchive.com/blog/19
6. http://www.it-sense.org/webportal/index.php?option=com_content&view=article&id=60&Itemid=67

Photo credit: Antaratma via Flickr

float(1)