Considering the number of searches for “VSP vs VMAX” that land people at my site, I figured it was about time I wrote something on the topic so that all those people who end up here from that search have something worth reading. So here we go…..
EMC Symmetrix VMAX vs Hitachi Virtual Storage Platform (VSP)
I do not work for EMC or Hitachi, or even HP….. I am not an official authority on either VMAX or VSP and any comments and opinions are my own personal opinions and not the opinions of any of my employers, past, present or future. This is a point in time article written in January 2011, so when you read this some of the information might be out of date. Do not take the information in this article on face value, speak with your vendor for clarification and the latest information.
So…. at first glance it may appear that VMAX and VSP are very similar, and in some respects they are. However, when you get under the hood there are some fundamental and potentially important architectural differences. The goal of this post is to discuss some of these differences and hopefully generate some discussion and opinion in the comments section at the end.
So, lets start…….
Custom vs Commodity
I have to mention this first as it provides the foundation for much of what we will discuss further on.
When it comes to the use of commodity off-the-shelf hardware versus design and development of custom silicon, such as ASICs, VMAX and VSP (actually EMC and Hitachi) find themselves staring at each other from opposite sides of the fence.
EMC – EMC have pitched their flag firmly in the commodity camp. It’s fair to say that EMC have recently positioned themselves up close and personal with Intel. In line with this, EMC have former Intel legend Pat Gelsinger as a direct report to Tucci. I’m led to believe that Pat maintains a very close relationship with his former colleagues at Intel and is key in facilitating a very tight relationship between EMC and Intel – a relationship that they hope will allow VMAX and its siblings to be ahead of the curve when it comes to exploiting the latest and greatest from Intel. In fact I believe I’m right in saying that EMC committed to utilising and exploiting the latest and greatest Intel chips as soon as is reasonably possible.
On Symmetrix VMAX, FA ports (front end ports) and DA ports (back end ports) are all tied to and controlled by commodity off-the-shelf Intel processors. The commodity Intel chips process and handle all in and outbound I/O, all RAID functions, all fetch instructions, as well as all higher level software functions including Virtual Provisioning, Thin Provisioning, TimeFinder, RDF… For example, RDF executes on cores that are tied to front end ports and TimeFinder executes on cores that are tied to back end ports. In the current version of VMAX these higher level software tasks are rigidly locked to cores.
The general opinion around the benefits of the commodity approach and aligning with Intel include leveraging Intel’s massive R&D and riding the price/performance curve.
Hitachi – Hitachi on the other hand have a different approach, I suppose you could call it more of a hybrid approach. In a Hitachi VSP you’ll find a far more balanced mix of commodity Intel processors as well as Hitachi designed proprietary ASICs.
In a VSP, front and back end ports are controlled by Hitachi ASICs that are designed specifically for I/O processing, whereas higher level software functions such as Dynamic Provisioning, Thin Provisioning, ShadowImage, TrueCopy… are all offloaded to a flexible pool of commodity Intel processors.
Hitachi calls this ASIC "Data Accelerator" and it is a dual core ASIC which Hitachi says is designed specifically for I/O processing.
It is the Hitachi belief that this approach yields superior performance while also being more efficient when it comes to power and cooling etc. While architecturally this sounds great and may well be true, is this sustainable from and R&D perspective?. And how much how much of a difference does it really make?
Some of the challenges around designing and making your own custom silicon (ASICs) is that they cost a lot to design and make, not only in monetary terms but in terms of time. Is this part of the reason that VSP felt like it was late to market? After all, it didn’t appear until ~18 months after VMAX made its debut, but still based on the same processor and PCIe gen1 architecture as VMAX…..:-S ??
Putting a positive spin on things, at least you are not wholly reliant on somebody else’s roadmap – somebody who is admittedly driven by the gaming industry rather than the storage industry. But then again, as Hitachi does leverage Intel inside (pun intended) they are somewhat reliant on Intel’s roadmap. And as they are not as tightly aligned to Intel will they always be behind EMC in the exploitation of the Intel advances – how much coding on VSP will have to change when we see the likes of Sandy Bridge and Ivy Bridge processors arrive with their new memory controller architecture etc?
Point being, there are clear and fundamental differences that make for great technical debate.
Both arrays now do Sub-LUN auto-tiering, but interestingly take slightly different approaches.
EMC. In VMAX world, sub-LUN auto-tiering is marketed as FAST VP (Fully Automated Tiering with Virtual Pools).
If you want multiple tiers of storage in VMAX you have to manage multiple Pools (VP Pools) of storage. Your config might look like this -
- Tier 0 = VP Pool comprising 400GB SSD 3RAID5. You might call this pool PoolA
- Tier 1 = VP Pool comprising 600GB 10K FC RAID 0. You might call this pool PoolB
- Tier 2 = VP Pool comprising 2TB SATAII 7.2K 14RAID6 You might call this pool PoolC
For the record, if you did name your pools like this you should be shot!
There are constraints around mixing drive geometries and RAID configs within a single VP Pool etc. If you want 3 tiers, you have a minimum of 3 tiers and 3 VP Pools to manage.
Hitachi. In VSP land sub-LUN auto-tiering is marketed as HDT (Hitachi Dynamic Tiering).
Implementation wise, HDT approaches things slightly differently. In VSP you can have multiple tiers within a single HDP Pool, referred to as an HDT Pool. So a single pool can have up to three tiers (drive geometries and RAID levels).
Strangely the VSP approach looks very similar to how EMC do things with CLARiiON. So VSP is pretty similar to CLARiiON, but both are different to VMAX :-S
While on this topic it’s worth mentioning Solid State Drives (EMC calls them Enterprise Flash Drives).
Both companies source ZeusIOPS drives from STEC Inc and both arrays put the drives on their standard backend loops where they cannot achieve their full potential of IOPs and throughput. Both arrays also utilise SSD as a discrete tier of storage and not as a form of level 2 cache. The drives in the VMAX have an FC interface whereas the drives in VSP have a SAS interface and VSP supports a 2.5" form factor SSD.
When it comes to the granularity of sub-LUN tiering, VMAX moves in units of either 7.6MB or 360MB whereas VSP moves in units of 42MB. See here for more detail.
While strides have been made by both companies with advances such as; hot and cold isle designs, power feeds from below and above, and even industry standard 19” racks for VSP, both VMAX and VSP require bespoke custom cabinets and would be an eye sore in well designed Data Centres such as those pictured below -
I suppose at least VMAX will look good with its blue neon light on the front.
While I understand why storage vendors only certify their kit in their own racks, it is not beyond the realms of possibility for vendors to make it happen when it comes to having their arrays installable in customer cabinets.
While on the topic of Racks, lets mention 2 things -
- VMAX is sexy whereas VSP is ugly
- Off the shelf configs of VMAX allow for up to 10 storage bays, whereas VSP allows for up to 4 storage bays.
The above means that off the shelf (not requiring any form of RPQ) VMAX confgis can have more disks than VSP, although VSP does support more disks in a smaller footprint by supporting 2.5” drives whereas VMAX only support 3.5” drives. However……. can you imagine trying to get approval for a microcode upgrade on a box that size as it is so big it will no doubt have 10,000 applications running on it! Still, if you want to, you can.
Both arrays are predominantly FC arrays, with both VMAX and VSP also support FICON for Mainframe attach. VMAX also supports iSCSI and IP ports for replication. VSP does not. Neither currently support FCoE but both will add this support in the very near future.
From an FC perspective, both arrays can have a maximum of 128 FC ports, although you can configure a VSP with no backend connectivity, or half backend connectivity which allows for more front end ports, up to 192 FC ports. Not sure how may people make use of this?
As mentioned when we talked about commodity vs custom, Hitachi have developed a custom ASIC to handle I/O on the front end and back end ports, whereas EMC have commodity Intel processors handling front and back end I/O.
VMAX and VSP both offer extremely large caches. A significant architecral difference being that VSP separates control data and real data. Control data has dedicated paths and dedicated DIMMs. A fully configured VSP has 32GB of dedciated Control Data. Another example of the Hitachi philosophy of specialisation (specialised ASICs, specialised memory areas and memory interconnects….).
VMAX implements somewhat of a distributed cache. Each Engine has a large local cache. However, thanks to the RapidIO fabric, all cache (even cache not sitting on the motherboard of the Engine – Director actually) is still directly addressable by every CPU/core in the system – think of it as globally addressable local memory. This approach is a step away from the way things were with Symmetrix DMX and overhead when a core from Engine A needs to access memory in Engine B is very low due to the RapidIO fabric and custom EMC ASIC and RapidIO driver.
The VSP cache is more of a monolithic cache – you get a very large single image cache with a single cache directory and no requirement for a Store and Forward function. Accessing cache slot 0 should be exactly the same as accessing slot 65536 (for example).
Talk about separation of control data and real data and line this with Hitachi’s ethos of separation and specialisation.
EMC shares user data and metadata on the same cache boards over the same interconnects. EMC cache and disk affiniation to Engines. Globally addressable local cache?
Engines and Interconnects
VMAX is radically different to it’s predecessor DMX when it comes to its caching, interconnect and controller design. VMAX is now based on Engines as processing building blocks. An Engine has it’s own; FA ports, DA ports, CPU cores, cache, backend disk loops (switched actually) and attached disk drives etc. A VMAX can have between 2 and 8 Engines.
Within an Engine there are two directors and these directors communicate over PCIe. But inter-Engine communication is over RadIO. You could say that the Engines are loosely coupled and give VMAX more of a modular look and feel. Although each Engine has its own directly addressable cache (128GB at time of writing) it is able to transparently address and access cache in every other Engine over the RapidIO fabric.
Hang on…it’s got an ASIC!!! While EMC are firmly sitting in the commodity Intel camp, it is worth pointing out that the chip that handles the RapidIO protocol and interface is an EMC designed ASIC.
Oh and while were being picky, current Data At Rest Encryption is implemented by an ASIC that is built in to the Tachyon processor on the new backend SLICs. However, this is not an EMC design and therefore you could argue is quasi-commodity :-S
Personally both of the above do not take away from EMCs drive to exploit commodity Intel. Make no mistake, EMC re-engineered Symmetrix, almost from the ground up, in order to exploit the Intel architecture. Oh and two ASICs is still less than half the ASICs in VSP.
VSP on the other hand looks and feels less modular in its design. In fact the VSP design looks more like a traditional monolithic array with a single large shared global cache. VSP design is a tightly coupled two controller fully active-active design with communication between the two controllers being over PCIe. So I suppose you could say that it looks a little like a very large VMAX Engine with two very large Directors…..No?
The VMAX backend is still Fibre Channel whereas VSP has made the jump to SAS. While talking about the backend, VMAX supports only 3.5” drives while VSP can have either 3.5” or 2.5”. Both support pretty much the full spectrum of drives, ranging from high capacity 7.2K RPM SATAII drives through the usual 10K and 15K all the way up the to high performance Flash drives.
Also, each VSP Back End Director (BED for short) has two of the Data Accelerator ASICS which performs parity calcs, drive rebuilds and encryption. VMAX only uses an ASIC on the backend for encryption, and even then its not an EMC designed ASIC, so you could say its off-the-shelf (commodity) built in to the Tachyon chip.
However, the major difference on the backend is the capability to virtualise 3rd party arrays, which VSP does and VMAX does not. This is an area in which I believe EMC and Hitachi differ philosophically. For example, I’m sure EMC could have implemented VSP style virtualisation within VMAX but have chosen not to. Either way, if you want to do it today, you need a VSP, as VMAX cannot do it.
Of course technically speaking it is the front end ports and not the back end ports on VSP that do the 3rd party virtualisation. They do this by flipping the mode of the front end FC port from target to initiator and having it pretend to be a Windows host – scary thought, although it does work
Oh and let’s not forget about VPLEX from EMC…. although for now that’s another conversation.
Both VMAX and VSP have very traditional RAID architectures that have barely changed over the last 10+ years. Both support RAID1, RAID5 and RAID6 in typical formations, although VSP implements RAID10 rather than RAID1. Importantly, neither have parallel, declustered RAID architectures or anything that is specifically designed for todays demands – in fact both have RAID implementations that were designed with yesterday demands in mind. Having said that, both generally still cope well but are in need of modernisation. Fingers crossed that both are working on such modernisations.
Oh yes…. VSP performs RAID XOR calcs etc in a custom ASIC whereas VMAX performs such calcs in commodity Intel chips.
If I was choosing one myself, which would I choose…… clearly a difficult choice, but if I were pushed it could potentially be III16118
Feel free to cast your vote on the enterprise storage array poll Ive just posted over on the right hand side of the site!
If there is enough interest I will consider doing more of a software comparison with things such as LUNs, management software, replication etc……
Constructive comments welcome (please disclose if you work for a vendor). Oh and let me know if there are any areas you think I’ve missed.
Disclaimer: And please remember…. I do not work for EMC or Hitachi, or even HP….. I am not an official authority on either VMAX or VSP and any comments and opinions are my own personal opinions and not the opinions of any of my employers, past, present or future. This is a point in time article written in January 2011, so when you read this some of the information might be out of date. Do not take the information in this article on face value, speak with your vendor for clarification and the latest information. Sigh….
You can follow me on Twitter and join me in putting the the technology world right ;-) My Twitter handle is @nigelpoulton