Skip navigation

StorageIO industry trends cloud, virtualization and big data


It is late in the day December 12, 2012 and best I can tell, we are still here, and for some, by time you read this it will be a few days or weeks later which means that either the Mayan calendar had it wrong, or we misinterpret it. Some would say that December 12, 2012 is not the important date, that it is really December 21, 2012 that the world will end, ok, lets wait and see what happens in a few more days.


Image of Becán Mayan ruins in southern Campeche, Mexico


However taking a step back from the Mayan calendar it dawned on me that some predictions such today's Mayan calendar forecast is similar to others that happen around this time of the year. That is the annual information technology or IT related predictions made by pundits or anybody else with an opinion, most of which in theory their concepts are not even close. Granted many predictions make good press and media things to read or listen to for entertainment. In some cases, these predictions are variations of what we're predicted last year in 2011 and the year before in 2010 and they year before that and so forth.


StorageIO industry trends cloud, virtualization and big data


I'm still working on my predictions for 2013 and forward-looking into 2014, however I keep getting interrupted fending off vendors and their PR surrogates calling or emailing asking me if they can make contributions, or write my list for me (how thoughtful of them ). For now one of my predictions is that I hope to get my predictions for 2013 done before 2013, however if you need something to hold you over, check this out from last year, or this from a few months ago.


I will also say that for 2013, those who see or view cloud, virtualization, big data (and little data) in pragmatic terms will be very prosperous. On the other hand, those who have narrow or constrained views will be envious of the others. Likewise plenty of new additions to the buzzword bingo line up with software defined having strong representation.


StorageIO industry trends cloud, virtualization and big data


Like the Mayan calendar predictions, with annual technology predictions, are we reading them wrong, or are they simply wrong and who if anybody cares, or are they just garbage in and garbage out, or big data garbage in, big data garbage out results?


In the meantime, I need to check that my local and cloud backups are working, try a restore test, have plenty of cash on hand, gas tanks full, cerveza in the fridge, propane for the generator and other things ready if the Mayans had it right, just off by a few days ;) .


Ok, nuff said (for now).

Cheers gs

StorageIO industry trends cloud, virtualization and big data


When I was in Europe presenting some sessions at conferences and doing some seminars last month I meet and spoke with one of the attendees at the StorageExpo Holland event. The persons name (Han Breemer) came up to visit with me after one of my presentations that include SSD is in your future: When, where, with what and how, and Cloud and Virtual Data Storage Networking industry trends and perspectives. Note you can find additional material from various conferences and events on the Downloads page accessible via the resources menu on the StorageIO web site.


As I always do, I invite attendees to feel free and follow-up via email, twitter, Linked In, Google+ or other venue with questions, comments, discussions and what they are seeing or running into in their environments.


Some of the many different items discussed during  my StorageExpo presentations included:


Recently Hans followed up and sent me some comments and asked if I would be willing to share them with others such as who ever happens to read this. I also suggested to Hans that he also start a blog (here is link to his new blog), and that I would be happy to post his comments for others to see and join in the conversation which are shown below.



Hans Breemer wrote:


Hi Greg,             


we met each other recently at the Dutch Storage Expo after one of your sessions. We briefly discussed the current trends in the storage market, and the "risks" or "threats" (read: challenges) it means to "us", the storage guys. Often neglected by the sales guys...    


Please allow me a few lines to elaborate a bit more and share some thoughts from the field. :-)


1. Bigger is not better?


Each iteration in the new disk technologies (SATA or SAS) means we get less IOPS for the bucks. Pound for pound that is. Of course the absolute amount of IOPS we can get from a HDD increases all the time. where 175 IOPS was top speed a few years ago, we sometimes see figures close to 220 IOPS per physical drive now. This looks good in the brochure, just as the increased capacity does. However, what the brochure doesn't tell us that if we look at the IOPS/capacity ratio, we're walking backwards. a few years ago we could easily sell over 1000 IOPS/TB. Currently we can't anymore. We're happy to reach 500 IOPS/TB. I know this has always been like that. However with the introduction of SATA in the enterprise storage world, I feel things have gotten even worse.


2. But how about SSD's then?    


True and agree. In the world of HDD's growing bigger and bigger, we actually need SSD's, and this technology is the way forward in an IOPS perspective. SSD's have a great future ahead of them (despite being with us already for  some time). I do doubt that at the moment SSD's already have the economical ability to fill the gap though. They offer many of thousands of IOPS, and for dedicated high-end solutions they offer what we weren't able to deliver for decades. More IOPS than you need! But what about the "1000 IOPS/TB" market? Let's call it the middle market.


3. SSD's as a lubricant?


You must have heard every vendor about Adaptive Storage Tiering, Auto Tiering etc. All based on the theorem that most of our IO's come from a relative small disk section. Thus we can improve the total performance of our array by only adding a few percent of SSD. Smart technology identifies the hot tracks on our disks, and promotes these to SSD's. We can even demote cold tracks to big SATA drives. Think green, think ecological footprint, etc. For many applications this works  well. Regular Windows server, file servers, VMWare ESX server actually seems to like adaptive storage tiering ,and I think I know why, a positive tradeoff of using VMDK's. (I might share a few lines about FAST VP do's and dont's next time if you don't mind)


4. How about the middle market them you might ask? or, SSD's as a band-aid?


For the middle market, the above developments is sort of disaster. Think SAP running on Sun Solaris, think the average Microsoft SQL Server, think Oracle databases. These are the typical applications that need "middle market" IOPS. Many of these applications have a freakish IO pattern. OLTP during daytime, backup in the evening and batch jobs at night. Not to mention end of month runs, DTA (Dev-Test-Acceptance) streets that sleep for two weeks or are constantly upgraded or restored. These applications hardly benefit from "smart technologies". The IO behavior is too random, too unpredictable leading to saturated SATA pools, and EFD's that are hardly doing more IO's than the FC drives they're supposed to relief. Add more SSD's we're told. Use less SATA we're told. but it hardly works. Recently we acquired a few new Vmax arrays without EFD or FASTVP, for the sole purpose of hosting these typical middle market applications. Affordable, predictable performance. But then again, our existing Vmax 20k had full size 600GB 15rpm drives, with the Vmax 40k we're "encouraged" to use small form factor 600GB 10krpm drives. Again a small step backwards?


5. The storage tiering debacle.


Last but not least, some words I'd like to share with you about storage tiering. We're encouraged (again) to sell storage in different tiers. Makes sense. To some extent it does yes. Host you most IO eager application on expensive, SSD based storage. And host your DTA or other less business critical application on FC or SATA quality HDD's. But what if the less business critical application needs to be backed up in the evening, and while doing so completely saturates your SATA pool? Or what if the Dev server creates just as many IO's as the Prod environment does? People don't seem to care it seems. To have people realize how much IO's they actually need and use, we are reporting IO graphs for all servers in our environment. Our tiering model is based on IOPS/TB and IO response time.    

Tier X would be expensive, offering 800 IOPS/TB @ avg 10ms
Tier Y would be the cheaper option offering 400 IOPS/TB @ avg 15 ms

The next step will be to implement front end controls an actually limit a host to some ceiling. for instance, 2 times the limit described in the tier description. thus allowing for peak loads and backups.    


Do we need to? I think so...


Greg, this small message is slowly turning into a plea. And that is actually what it is, a plea to our storage vendors, and to our evangelists. If they want us to deliver, I feel they should talk to us, and listen to us (and you!).



Hans Breemer


ps, I love my job, this world and my role to translate promises and demands into solutions that work for my customers. I do take care though not to create solution that will not work, despite what the brochure said.

pps, please feel free to share the above if needed.



Here is my response to Hans:


Hello Hans good to hear from you and thanks for the comments.               


Great perspectives and in the course of talking with your peers around the world, you are not alone in your thinking.    


Often I see  disconnects between customers and vendors. Vendors (often driven by their market research) they know what the customer needs and issues are, and many actually do. However I often see a reliance on market research data with many degrees of separation as opposed to direct and candied insight. Likewise some vendors spend more time talking about how they listen to the customer vs. how time they actually do so.


On the other hand, I routinely see customers  fall into the trap of communicating wants (nice to haves) instead of articulating needs (what is required). Then there is confusing industry adoption with customer deployment, not to mention concerns over vendor, technology or services lock-in.


Hope all else is well.    


Cheers       gs


Check out Hans new blog and feel free to leave your comments and perspectives here or via other venues.


Ok, nuff said.

Cheers gs

HP has been talking and promoting for several weeks (ok, months) their upcoming December 3rd storage announcements from the HP discovery event in Frankfurt Germany.


Image of HP discover day


The December 3rd embargos have been lifted so I can now talk about what HP shared before announcements. Basically what I received was a series of press releases as well link to their updated web site providing information about todays announcements.



HP has enhanced the 3PAR aka P10000 with new models including for entry-level, as well as for higher performance enterprises needs. This also should beg the question for many longtime EVA (excuse me, P6000) customers, have they hit the end of the line? For scale out storage, HP has the StoreAll solutions (think about products formerly marketed as certain X9000 models based on Ibrix) with enhancements for analytics, bulk and various types of big data. In addition HP has enhanced its backup and recovery capabilities and Dedupe products including integration with Autonomy (here and here) along with capacity on demand services.


HP 3par storage products image

New 3PAR (P10000 models)


New Store All (formerly known as IBRIX based)
New StoreAll storage system


From the surface and what I have been able to see so far, looks like a good set of incremental enhancements from HP. Not much else to say until I can get some time to dig around deep to see what can be found on more details, however  check out Calvin Zito (aka @hpstorageguy) the HP storage blogger who should have more information from HP.


Ok, nuff said (for now).


Cheers gs

StorageIO industry trends cloud, virtualization and big data


I am surprised nobody has figured out how to  use the term valueware to describe their hardware, software or services solutions,  particular around cloud, big data, little data, converged solution stacks or bundles, virtualization  and related themes.


Cloud virtualization storage and networking building blocks image
Cloud and virtualization building blocks transformed into Valueware


Note that I'm referring  to IT hardware and not what you would usually find at a TrueValue hardware store (disclosure, I like to shop there for things  to innovate with and address the non IT to do project list).


Image for truevalue hardware stores


Instead of value add software or what might  otherwise be called an operating system (OS), or middleware, glue, hypervisor,  shims or agents, I wonder who will be first to use valueware? Or who will be  the first to say they were the first to articulate the value of their industry  unique and revolutionary solution using valueware?


Cloud and convergence stack image from Cloud and Virtual Data Storage Networking Book


For those not familiar, converged solution  stack bundles combine server, storage and networking hardware along with  management software and other tools in a prepackaged solution from the same or multiple  vendors. Examples include Dell  VIS (not to be confused with their reference architectures or fish in Dutch), VCE or EMC vBlocks, IBM Puresystems, NetApp FlexPods and Oracle Exaboxes among others.


Converged solution or cloud bundle image from Cloud and Virtual Data Storage Networking Book


Why is it that the IT or ICT (for my  European friends) industries are not using valueware?


Is Valueware not being used because it has  not been brought to their attention yet or part of anybody’s buzzword bingo list or read about  in an industry trade rag (publication) or blog (other than here) or on twitter?


Buzzword bingo image


Is it because the term value in some marketers  opinion or view their research focus groups associate with being cheap or low-cost? If that is the case, I wonder how many of those marketing focus groups  actually include active IT or ICT professionals. If those research marketing  focus groups contact practicing IT or ICT pros, then there would be a lower degree of separation to the  information, vs. professional focus group or survey participants who may  have a larger degree of separation from practioneers.


Degrees of seperation image


Depending on who uses valueware first and  how used, if it becomes popular or trendy, rest assured there would be  bandwagon racing to the train station to jump on board the marketing  innovation train.


Image and video with audio of train going down the tracks


On the other hand, using valueware could be an  innovative way to help articulate soft  product value (read more about hard  and soft product here). For those not familiar, hard product does not simply mean  hardware, it includes many technologies (including hardware, software, networks, services)  that combined with best practices and other things to create a soft product (solution experience).


Whatever the reason, I am assuming that  valueware is not going to be used by creative marketers so let us have some fun  with it instead.


Let me rephrase that, let us leave valueware  alone, instead look at the esteemed company it is in or with (some are for fun,  some are for real).

  • APIware (having some fun with those who see  the world via APIs) 
  • Cloudware (not to be confused with cloud  washing)
  • Firmware (software tied to hardware, is it  hardware or software? )
  • Hardware (something software, virtualization  and clouds run on)
  • Innovationware (not to be confused with a data  protection company called Innovation)
  • Larryware (anything Uncle Larry wants it to  be)

Image of uncle larry aka Larry Elison taking on whomever or whatever


  • Marketware (related to marketecture)
  • Middleware (software to add value or glue  other software together)
  • Netware (RIP Ray Noorda)
  • Peopleware (those who use or support IT and  cloud services)
  • Santaware (come on, tis the season right)
  • Sleepware (disks and servers spin down to sleep  using IPM techniques)
  • Slideware (software defined marketing  presentations) 
  • Software (something that runs on hardware)
  • Solutionware (could be a variation of implementation  of soft product)
  • Stackware (something that can also be done  with Tupperware)
  • Tupperware (something that can be used for  food storage)
  • Valueware ( points to this page, unless somebody wants to buy or rent it )
  • Vaporware (does vaporware actually exist?)


More variations can be added to the above  list, for example substituting ware for wear. However, I will leave that up to  your own creativity and innovation skills.


Let’s see if anybody starts to use Valueware  as part of their marketware or value proposition slideware pitches, and if you  do use it, let me know, be happy to give you a shout out.


Ok, nuff said.

Cheers gs