Understanding cloud storage - Is your organization ready for it
There’s a great deal of hype surrounding cloud delivery models of all flavors, with the assumption that it will usher in a new era of cheaper, better IT. But most organizations struggle to compare the capital, operational, and staffing cost differences between internal and cloud storage. In this post, I try to unravel the cost differences of a common workload - file storage - consumed through the public cloud versus built and deployed in-house.
Cloud Storage Is Workload-Specific
Cloud, in general, is a way of delivering IT services with its own limitations. Having said that, Cloud is a viable platform for specific workloads where it can deliver performance and security experiences that is as good as or better than what can be deployed in-house. In the world of Storage, latency is a big problem as the public cloud is far-away from the servers in the physical data centers. Therefore, customers should do due diligence on latency-sensitive workloads before diving into the Cloud experience
Typical Cloud Storage Use-Cases
- Whole in-cloud applications with their own storage
- Backup and archiving to Cloud
- File storage in the Cloud
It is imperative that customers need to evaluate the underlying facilities, infrastructure and storage management applications put forth by the Storage Service Provider before latching onto Storage on the Cloud bandwagon.
Evaluating Cloud Based Storage
In this section, I try to provide a set of guidelines for CIOs & IT Managers on the evaluation methods for Cloud Storage
- Applications that are storage-centric : Jumping onto a Software-as-a-Service (SaaS) environment applications. In such situations, as both servers and storage are colocated in the cloud, the latency of cloud delivery is typically limited to client-to-host traffic and not host-to-storage. So almost all SaaS vendors tend to prefer this deployment model to limit their latency exposure. For example - CRM-on-the-cloud, ERP-on-the-cloud
- Less latency sensitive data such as Backup and archive data : As backup is a secondary copy of the data and archive data are infrequently accessed, these workloads are latency insensitive. Incremental data sent over WAN can happen throughout the day without the negative impact to applications using primary database. Due to the aforesaid, it can be easier to archive/ backup on the cloud, but restoring and entire data-set back may be a challenge over the WAN. Hence, it is advised that the IT teams set appropriate Service Levels to business for recovery
- Files that are generally less performance critical : Files, aka "unstructured data" which are not needed for application to make sense of information. Files have their own metadata which can be read anywhere, regardless of who created it. Departmental file shares wherein performance is not critical makes it a viable cloud storage evaluation use-case
Computing And Comparing Costs Of In-House And Cloud Storage
Many people tend to make the mistake of comparing cost of buying storage and cost of procuring public cloud storage. A simple calculation that CIOs tend to make is - to estimate $/GB of procurement and annualize them over the life expectancy of of the storage that is being procured and compare with the price of the cloud based storage for a year at the prevailing prices. This is not the right comparison.
This method of comparison leaves out significant amount of costs on either side of the equation. A detailed business case needs to include
- You need more capacity than data : In an in-house environment, one has to pay for operations and the redundancy, which is included in Cloud based storage offerings. Typically, operational expenses for storage is estimated to be 100% of the annualized cost of buying the storage. You will also have to consider much more raw storage capacity to deliver enterprise-class reliability and performance and growth buffers
- Cloud Storage includes all, except WAN and transaction charges : A Cloud storageoffering includes the operating costs and capacity redundancy which makes measuring costs easier. The elements that are missed out in a cost-calculation of such an offering should include
- Impact on the network bandwidth from sending data across WAN
- Any transactional charges should there been additional storage capacity than the contracted size
- Performance capabilities of specific application requirements
- Does Cloud options have native support for data access :Cloud storage repositories are built on object repositories which allows for massive scalability, custom metadata, geographical spread and low cost. But this will involve modification of existing data or business processes or code to vendor's Application Programming Interfaces (APIs) to use object stores directly. In such case, the cost of this custom integration needs to be added to the base cloud pricing tag.
An Accurate Cost Comparison Model Requires A Detailed Model
Cost of storing 100 TB of data internally is way higher than presented above - Some of the key factors in determining the cost of internal storage include
- Varies based on number of years of internal storage : Organizations have different policies pertaining to the service tenure of storage arrays. It can widely vary from organizations that lease storage and hence has a new storage environment almost every 3 years, while other set of organizations run storage till the time the vendor discontinues support for that model. Since we use an annualized model in our calculations for internal storage, this can have a vast impact on the cost calculations
- Acquisition cost drives a significant portion of internal storage model : Depending on the size of storage that is being planned the organizations negotiate various acquisition prices with the storage vendors. While at the same time, the type of storage (NAS/ SAN) could also have a major impact in the cost calculation numbers provided below. In this calculation, it is assumed that the price used includes the RAID protection overhead and system resource overheads.
- Number of redundancy copies : When you send 100 TB of data on the cloud, you pay for 100 TB. However, when you store 100 TB of data internally, the capacity of storage that needs to be procured is much higher. The number of redundancy copies of data can be higher or lower based on availability and other requirements but three (3) is a reasonably conservative copy count, which is what is used in the calculations below
- Low utilization of storage : The burden of how much raw capacity to provide to availability needs in a Cloud Storage model falls on the Service Provider. But in the case of internal storage, organizations struggle with capacity management and have low utilization or raw storage as a result. Typical storage environments see raw capacity utilization of 20% to 40% due to RAID protection, system resource capacity, growth buffers, resulting in much bigger allocations. Since in this calculation, the usable capacity is being used - system overhead and RAID protection is already factored in. Hence the usable capacity in this model is kept at 60%. (To address the mindset of traditional in-house storage environments :) )
- Staff costs : The staff costs for in-house storage environments are typically niche' and tend to be expensive. On the contrary, the Cloud storage environment includes the staff costs that are needed for delivering the committed SLAs.It might be a safe assumption to have one FTE per 50 TB of raw storage and assuming organizations require such FTEs to be available 16 hours in a working day thereby making it 2 FTEs per 50 TB.
- Facilities and Power costs : While it is difficult to ascertain the power and facility cost for the Storage environment alone, in this calculation it is assumed at a conservative value of 5% of the total acquisition costs for power as well as cooling requirements of the Storage environment
- Maintenance costs : As majority of the storage vendors include a 3 year support and maintenance (which is typically 15% of storage costs of the acquisition cost) and considering that the storage environment will be refreshed in 4 years, it is taken as 30% addition to acquisition cost (adjust 15% for the year beyond the 3 years support and maintenance included with the purchase) over a 4 year term for calculation
- Data migration costs : Not only do you have to buy, run, maintain, and power internal storage, but you also have to migrate it from old equipment to new every few years when it’s time for refresh. This process is onerous, risk-laden, and the bane of existence for many storage directors. When you get storage from the cloud, you get perpetual storage that doesn’t have the&
Author : Govind Desikan
Govind Desikan is the Business Development Head for Cloud Services, Netmagic, responsible in evangelising Cloud initiatives and to engage with customers deeply in preparing a Cloud blue-print for successful Cloud roll-outs. He has been associated with IT industry for close to 20 years with a wide ranging experiences from large Enterprises, working with Software Vendors, building Datacenter services grounds-up as well as in architecting large system roll-outs including elastic and adaptable architecture. Conversant with most software technologies, he is a passionate and vivid believer in simplifying technology to make it relevant for Business Decision Maker. In his past, he has worked with popular software OEM brands such as VMware, Microsoft, Sun, Oracle etc. Apart from being an Computer Science Engineer, he is also a Cost Accountant and an ISO Lead Auditor. He is well known with his customers who fondly recognize him as "one of the best consultative solution seller".