Did HP StorageWorks Invent EMC FAST and FAST VP?

While recently in Las Vegas for HP Discover 2011, I had a really interesting conversation with the guys at the HP P9500 booth.  For those, like me, who despise the HP naming convention, The P9500 is the latest and greatest in what used to be the XP product line – you know the high end storage array that HP OEM and apparently joint-develop with Hitachi Ltd. of Japan.

As you might expect, the HP guys were top notch engineers, the type I could spend all day talking to.  While we were chatting, they were obviously passionate about the involvement HP engineers and architects in the design of the XP/P9500 (HDS VSP).  However, they also threw out some shots at EMC.  And not wanting to simply swallow what they feed me, I thought I’d open up the debate on here in case anybody has any opinions…..

First up, a quick picture of the P9500 that HP had on display on the show floor –

 p9500

EMC FAST is Apparently Based on a an HP Research Paper

Ayman Abouelwafa, Array Lead Architect for the XP and P9000 system at HP Storage told me about a paper he wrote and published back in 2007.  The paper was posted on Research Disclosure website in 2008 and outlines at a high level what we generally refer to as sub-LUN tiering today.  Such technologies can be found in most major storage arrays on the market today, but not back then. 

NOTE: Apparently all major technology companies have free logins to this website, I don’t but I have managed to obtain a copy of the paper, it is titled Thin Provisioning with Native Hierarchical Storage Management and the link to a copy I’m hosting on my website can be found here

Apparently due to some political reasons within HP at the time (cutting back on R&D and investment in such things etc) HP never did anything with the work.  However, and this is the interesting part…… Ayman firmly believes that EMC took this paper and based their FAST/FAST VP technology on it.  A bold claim I know, but Ayman is adamant.  And anyone who knows Ayman knows that while he is technically as sharp as just about anyone, he is also passionate and doesn’t hold back with his opinions.

Now I personally have no idea if this happened, however, the paper is a short but interesting read, and I would recommend it to anyone specialising in the HP XP/P9500 or HDS VSP.

After finishing up at HP Discover (which by the way was an excellent event) it was time to grab a taxi and head to the airport to fly home.

Taxi!

HP Discover taxi

10 comments for “Did HP StorageWorks Invent EMC FAST and FAST VP?

  1. David
    June 19, 2011 at 8:40 pm

    I just wandered through this thread.  As a lay-observer Ayman's assertion isn't material.  You put a high-level paper out for all to view, someone else implements it [always the hardest part] while it founders at your own company, then you claim some conceptual fame?  Cheeky, if you ask me.

  2. June 20, 2011 at 4:07 pm

    comical to say the least. either way sub-lun tiering does work and can save companies money and resources. i just have a hard time listening to hp storage guys talk about the XP/P series like they designed the whole thing.

  3. June 21, 2011 at 10:48 am

    They might want to chat to Barry Whyte as well as he (and IBM) hold a number of patents in this space. Lots of people were talking about sub-LUN even back then, it was a fairly obvious progression really.

  4. June 21, 2011 at 3:38 pm

    Hey Nigel – I wasn't directly involved with this but had this conversation with Ayman last summer when we were preparing to announce the P9500.  He is passionate, isn't he!  One clarification to what you stated.  HP absolutely pursued the work in the paper.  HP decided to publish the paper instead of pursuing a patent.  Whether a patent would have been awarded or not is speculation.  However, Ayman and his team worked with our partner Hitachi Japan to implement Smart Tiers in the P9500. 
    Steven – HP hasn't ever claimed to have "designed the whole thing" but the fact is HP did have a big role in the latest generation.  HDS doesn't like that fact to be out there so they spread FUD – going so far as to say HP has discontinued our relationship with Japan.  Those are the facts.  Hitachi Japan engineers sit with our P9500 team in Roseville and HP was directly involved in the design.  Here's a video I helped drive with Hitachi Japan talking about exactly that: http://www.youtube.com/watch?v=qekp642K0vQ.
    For those that don't know, I work for HP Storage.

  5. June 22, 2011 at 7:29 am

    as Calvin stated there were several companies working on that in those times. Compellent even had a patent for a similar technology (Data Progression) in that same july 2008 (http://www.freepatentsonline.com/7398418.pdf) and I suppose 3PAR was going that way too in that period (should ask Craig Nunes to confirm).
    @Calvin: For those that don't know, I work for HP Storage. > who doesnt?  LOL

  6. June 23, 2011 at 11:46 am

    As noted, the notion of sub-LUN tiering existed long before this paper was published.
    And fortunately, EMC engineers DIDN'T copy the architecture described in that paper (and later implemented in the VSP's Dynamic Tiering).
    FAST VP is far more innovative and efficient. Unlike DT's developers, who were hampered by the cumbersome and bloated 42MB page size of the VSP, EMC engineers first re-architected Enginuity on VMAX to support much more efficient 7.5MB FAST VP extents, along with a dynamic metadata infrastructure to track extents utilization, demand and decay. FAST VP incorporates heuristics and algorithms based upon patterns gleaned from traces of thousands of production workloads and 10s of millions of I/Os. The result is a solution that support individual application policies, reacts sooner to workload changes, and moves less data to optimize performance and utilization. By moving smaller increments of data, less internal bandwidth is wasted moving data that isn't hot, and more of the flash is utilized to accelerate I/O instead of holding "cold" data.
    Not only does the result bear no resemblance to that described in the paper, I can attest personally that neither the design nor even the very idea was based upon this paper – couldn't have, since the paper was published well after FAST VP development began.
    If anything, the paper was perhaps published by HP in a defensive move, as an attempt to put the notion into public domain before it could be patented. As noted by other writers, even this would seem to have been an ineffective strategy, as patents protect not the idea, but the actual methodology. And FAST VP's methodology is radically different than that described.

  7. June 23, 2011 at 2:15 pm

    by no means, am i discounting the fact that HP does a lot of great work with Hitachi Japan. my point was it is comical that this paper was the sole driving force around sub-lun tiering on the VSP or in general. it's also very possible that i complete mistook why this document was released. either way (as i did say in the OP) sub-lun tiering does work very well. for the record, i don't work for any storage vendor or partner.

  8. John DeFrees
    June 24, 2011 at 2:50 am

    Back before HP had to remind you that they do indeed do some inventing from time to time they came up with the Virtual Array 7000 series which went EOL around ’04.  This FC array had pools of RAID5 and RAID10 blocks.  You created a virtual vol which was RAID5 and the controller migrated the hot blocks to RAID10 thus creating a LUN which consisted of a mix of RAID5 & RAID10 blocks at the sub-LUN level.  The VA (not to be confused with EVA) came in two models (7100 & 7400 if memory serves) could have around 100 to 200 spindles and had snapshots. 
    The VA family lost out to EVA which had better OS support, remote replication and an installed base. Development of this technology was soon put out to pasture like so many others.  I’ve worked with 7 or 8 different arrays over the years from DEC, Compaq, HP, LeftHand, Hitachi, EMC and EqualLogic and VA was one of the easiest to use of the lot.

  9. Pingback: Snap Products Blog

Leave a Reply

Your email address will not be published. Required fields are marked *


*

You can add images to your comment by clicking here.