SSD will change everything

By | March 10, 2008

So the imminent arrival of SSD in enterprise storage has got the old grey matter heating up somewhat.  Below are a couple of things that I’ve come across in the last week that will be affected by the arrival of Flash based SSD in enterprise storage.  

This list will no doubt grow……

Cutting Disk

As anybody who has ever needed to quickly cut some new LDEVs from existing internal RAID groups on Hitachi storage will already know – it takes an age!  

•    The process runs as a background task.
•    All tracks are written with the new layout.  (has anyone ever tried to recover data from an LDEV that was moved to space and then re-cut!!!  Don’t waste your time)

So I was thinking, how will cutting LDEVs be affected by Solid State Disk?

Yes unfortunately I do know that Hitachi storage has no support for SSD, but Im hoping that will change sooner rather than later.

So…… considering the use of ‘wear levelling’ algorithms in SSD – one cannot be certain that when overwriting a track cell they are actually overwriting the original cell, or if the drives internal algorithms have decided to target your write operation to a remapped cell – there seems no point writing to every cell when cutting your new LDEV if you cant be sure your future writes will actually be directed to the cells your are now formatting.

And then there’s the fact that unnecessary overwriting of storage cells is only going to reduce the life of your drive – and if the vendors will be replacing them under warranty then you can guarantee they wont be writing over cells unless its absolutely necessary.

So with these two points in mind, I see the arrival of SSD improving LDEV cutting times, bringing them in line with cutting LDEVs on external storage.  

OK so its not exactly the main reason for implementing SSD, but will certainly make many a storage admins life easier.

Data Shredding

On a similar point, last week a colleague asked me to show him how to use the Data Shredding program on a USP.  After showing him I wondered how this too might be affected by SSD.  

Basically, the Hitachi Data Shredding (HDS Tongue out) program allows you to overwrite every track of an LDEV several times (up to 8 last time I used it) with various patterns of data.  The point being to guarantee that none of your customers data remains on the physical disk when its disposed of…..

Again, how would this be affected by the arrival of SSD, considering the fact that writing to every single track of an LDEV would –

1.    Have an effect on the life of the media (especially if you choose to overwrite 8 times)

2.    Not guarantee to overwrite the original data (due to wear levelling/remapping writes)

Granted I have only had the need to use the Data Shredding program twice in my life.  The first time I used it, is took so long it took was ridiculous (if you thought cutting an LDEV took a long time, then you have never used the data shredder).  So ridiculous that the second time it was decided not to use it because of time limitations.


Then there’s the occasional need for smaller faster drives (apologies to anyone who regularly reads this blog and is sick of me moaning about this).  I have shook my head many a time over the one track (no pun intended) mind of the hard disk industry.  Bigger bigger bigger! 

Every once in a while I come across a situation where dedicated 73GB 15K disks would be ideal.  But it’s just so hard to get them these days.  While moaning about this in the past, rarely did I imagine I would one day be blessed with not only smaller capacity drives, but solid state!!  Not only are the current generation of flash based single cell solid state disks smaller in capacity than traditional spinning disks, they are also faster.  Bring it on!

And finally

You have to wonder what the disk manufacturers and array vendors think of the possible explosion of SSD into the mainstream?  Are they eyeing up the current crop of SSD companies with the view to buying some of them up?  Surely Seagate will have half an eye on this!  

May be now is a good time to buy shares in some of these SSD players!?

So I guess now that Ive said all of that, only one small thing remains – Hitachi to announce support for SSD in the USP-V.  Otherwise I will just have watch the rest with wanton eyes.


10 thoughts on “SSD will change everything

  1. Barry Whyte

    Interesting points, on the ‘secure erase’ as we know it, since you have to erase a 128K block in flash before you can write to it – then a single task to erase all blocks just once would probably be enough. I don’t think that Flash suffers the same ‘residual’ charge that magnetic mediums do. Since there are multiple channels per device, it should be a lot quicker. However its the erase that lets the write performance down, and the usual trick is to ‘write to a new pre-erased track’ so that you don’t suffer this penalty when writing I/O – so you would be back to HDD IOPs for such a task.

  2. Nigel Poulton

    Hi Barry,

    Thanks for your comments.

    Im not an expert (yet  ) on all things flash and SSD….. but I understand that  previous stored data can be extracted from flash memory if you have the right kit.  How easy or hard this is in comparison to hard disks, and how realistic it is to expect anybody to be able to do this I do not know.

    Ive known of a company who tried to recover some data from a RAID group that was wiped and then reformatted on a Hitachi array.  They sent the disks off to a recovery specialist who sent them back saying no data was recoverable.  Im guessing the combination of proprietary RAID controllers, multiple disks that would need to be put back together in the correct order, and the fact that they were overwritten at least once was too much for the recovery experts (who charge a lot of money).   Sooooooo Ive always wondered how relevant these data shredding type programs are???

  3. the storage anarchist

    First, initializing/VTOC'ing the data space of a RAID 3/4/5/6 group is pretty much mandatory in order to make future XORs' work when calculating the RAID Parity for writes – you need to XOR over zero's to calc the parity properly. Now, of course, the drives don't HAVE to have actual zeros on them for this to work – you can accomplish the same thing by logically keeping track of blocks that "should be zero" or that have "never been written" and effecting the logical XOR outside the drive. But even if you do write the zero's to the device (which you have to do if you're having the drive calculate the XOR for you, as an example), it is really only one write to all the included "usable" blocks of cells. Since VTOC is a very infrequent operation to create/reform devices, it's not really that big of a deal.

    As to secure erase, SSDs have the same problems as disk drives – you generally cannot erase stuff on blocks that have been mapped out of service due to a write failure. BUT, these blocks are probably still quite readable. On the STEC drives, the reserved capacity is pretty large, so there could be a LOT more old data on them – even if you just wrote 128GB of zeros to the 73GB drives, there'll be lots of readable-but-unwritable blocks inside the device. DoD Secure Erase requirements further complicate things – it's not enough to simply "erase" every block in the device…you actually have to write specific patterns of data over the device – at least 3 and as many as 24 times.

    It is not clear that Flash drives are susceptible to the same forensic reclamation of overwritten data as disk drives. Magnetic media leaves "shadows" of prior writes because head positioning is never 100% accurate and the bits tend to "bleed" substrate over the substrate. Flash drives may have a similar "shadow", although the forensics would be different. However, government requirements don't care whether they do or not – all storage media are required to be repeatedly overwritten according to specific patterns and repititions. And if you can't do that reliably, then grind them into dust!

    Contrary to popular believe, encrypting the data in/on the drive is not (yet) accepted as a secure means of "erasing" the content of a drive, for a variety of reasons.

  4. Matt

    I'm sure you've covered this one off in the past but I'm going to ask anyway 🙂

    Why not just buy the 146s (or whatever) and short stroke them? That should give you (slightly) better overall performance than the 73GB no? Over time, you would expect that the manufacturing cost of technology x (think 146GB) will approach that of technology w (think 73GB). In fact, as the manufacture of tech w drops off, likely as not, the cost of manufacture for that tech will actually overtake that of it's daughter tech x. Think of drive techs as long tailed 'u' shapes on a cost\time graph.

    Or not 🙂

  5. Pete Steege

    I think the overwrite/security challenge you point out will be addressed more thoroughly with encryptiion on either SSD or disk.  SSD is bringing lots of great benefits to the table, but in and of itself, security is not one of them.

  6. Nigel Poulton


    Thanks for your comments.  While I agree with the principals behind short stroking disks – I suppose you may even get better performance out of short stroking a 146GB vs full stroking a 73GB if you only use the outer tracks of the 146GB – the reality of implementing this is more difficult.  Im thinking especially when the folks with the purse strings realise they have paid for space they are not using and then decide to start using it.

    Barry and Barry,

    Re the persistence of residual data on flash drives after the standard block erase procedure…… I did a little reading on this today while in a rather long and boring client meeting.  Although I was only able to have half an eye and half my brain on the document, and the other half on the meeting, the paper suggests that it is possible to restore information from erased memory.  The paper is titled “Data Remanence in Flash Memory Devices” by Sergei Skorobogatov.  It might be worth a read for some of us as we skill up on the whys and wherefores of flash memory.  All suggesting that Data Shredder style programs may still be required if you cant physically jump up and down on the drive.

    As for encrypting flash drives – that’s an interesting one.  How does that work with wear-levelling and write endurance?  Overwriting data with its encrypted replacement will effect the life of your flash drive (although I don’t expect this to be an issue for long) but without an intimate relationship with the drives internal wear-lelvelling etc you cannot guarantee to overwrite the unencrypted data.

    Man Im gunna lie awake in bed for weeks on end while I think about the remifications of SSD!

    Oh and sorry about the crappy software sometimes not handling text formatting and the likes – appreciate your preseverance  😉

  7. snig

    Please send me an email and let me know what you’d like to see for comment formatting.

  8. the storage anarchist

    Embedded encryption is an interesting alternative, but there are still many hurdles to be overcome – even BEFORE dealing with the differences of Flash vs. Hard drives.
    The biggest is key management. The achille’s heel of embedded encryption approaches on disk drives (to date) has been that the actual encryption key is stored INSIDE the device. This violates just about every rule about cryptology that you can imagine. Even though the keys aren’t "enabled" until you enter the appropriate "unlock code", these unlock codes aren’t reliably protected (nor are they necessarily all that strong).
    Some (including those that make purchasing rules for the US govt) prohibit the use of in-device encryption as a means of securely erasing data for just this reason – AND because it is impossible to verify that you’ve actually deleted all the copies of a key (like trying to prove a negative).
    Still, many talk about encryption as the answer – so perhaps one day these issues will be resolved sufficiently to meet the stringent requirements placed on governments, medical records and anything remotely related to international currency exchange :).

  9. Stephen2615

    As most customers who are likey to have large systems have sensitive data and having working in some sensitive areas, encryption at this level is a waste of time in my opinion.  Its only there for some feel good solution for customers who can’t or wont take security seriously.  I don’t see many organisations lining up to get government certification on encryption.  Why, because it costs a fortune and it makes no real difference and it takes a real long time.  It’s generally left up to the organisation to protect its own stuff.  Besides no one really trusts encryption.  Same goes for disks.  Destruction is the only real way to ensure data protection.

Leave a Reply

Your email address will not be published. Required fields are marked *


You can add images to your comment by clicking here.