Comments on: Spread it and forget http://blog.nigelpoulton.com/spread-it-and-forget/#utm_source=feed&utm_medium=feed&utm_campaign=feed with nigel poulton Tue, 20 Dec 2011 19:37:54 +0000 http://wordpress.org/?v=2.8.6 hourly 1 By: Cleanur http://blog.nigelpoulton.com/spread-it-and-forget/comment-page-1/#comment-105 Cleanur Mon, 08 Jan 2007 12:32:52 +0000 http://blogs.rupturedmonkey.com/?p=56#comment-105 On the EVA chunk size is fixed at 128KB (256 blocks of 512 bytes each). This is often confused with the minimum PSEG size of 2MB. A PSEG is the minumu space the EVA can allocate internally when creating VDisks. Each PSeg being equal to 16 chunks (16 X 128KB = 2MB). On the EVA chunk size is fixed at 128KB (256 blocks of 512 bytes each). This is often confused with the minimum PSEG size of 2MB. A PSEG is the minumu space the EVA can allocate internally when creating VDisks. Each PSeg being equal to 16 chunks (16 X 128KB = 2MB).

]]> By: Jesse (SanGod) http://blog.nigelpoulton.com/spread-it-and-forget/comment-page-1/#comment-104 Jesse (SanGod) Sat, 09 Dec 2006 04:18:02 +0000 http://blogs.rupturedmonkey.com/?p=56#comment-104 You know it's hilarious - I've heard this story time again and I think it all comes down to one basic premise. People just *LOVE* to overmanage their disks. In the EMC world, people talk about wanting to know what physical disk each hypervolume is on, then when you start working on 8 and 12-way metavolumes, you're talking 16-24 back end disks for each lun presented forward. (In a Raid-1 configuration that is, actually more if you're talking Raid-5) Either way, in a cached array the only time you're dependant on back-end performance is A> Random Reads - unfortunately there is no way around a random-read. It's going to wait for the disk. But any intelligent array can pick and choose which disks are being read from based on activity level of the mirror pairs. B> Cache Saturation. Yes, it does happen, even in boxes with 48-64G of cache. (Though I don't see it happen that often) In the EMC world, and I'm guessing this applies to all high-end arrays, the write has been acknowledged back to the host long before the physical drive enters into the picture. *THIS* is what actually makes a cached array faster than JBOD. When write speeds are measured in nano-seconds and not miliseconds, all is good. Most arrays also have some form of predictive read-ahead, based on odds that once you've read X number of tracks, you're more likely to read the next Y tracks in series. But yet, (and back to the point of the response) I still see people trying desparately to micro-manage their disks. Even in my company I have daily fights with our DBA because he wants to see two dozen 8G volumes presented instead of the 200G volume spread across many spindles on the back-end, despite the fact that the 8G will end up being slower because you can't use a 'scatter-gather' read on a single volume. In the EMC world the rule is to use sequential symmetrix device ID's for volumes which, by virtue of the way the Enginuity software lays out the disks by default, guarantees that you won't step on the same physicals. You know it’s hilarious – I’ve heard this story time again and I think it all comes down to one basic premise.

People just *LOVE* to overmanage their disks.

In the EMC world, people talk about wanting to know what physical disk each hypervolume is on, then when you start working on 8 and 12-way metavolumes, you’re talking 16-24 back end disks for each lun presented forward. (In a Raid-1 configuration that is, actually more if you’re talking Raid-5)

Either way, in a cached array the only time you’re dependant on back-end performance is

A> Random Reads – unfortunately there is no way around a random-read. It’s going to wait for the disk. But any intelligent array can pick and choose which disks are being read from based on activity level of the mirror pairs.

B> Cache Saturation. Yes, it does happen, even in boxes with 48-64G of cache. (Though I don’t see it happen that often)

In the EMC world, and I’m guessing this applies to all high-end arrays, the write has been acknowledged back to the host long before the physical drive enters into the picture. *THIS* is what actually makes a cached array faster than JBOD. When write speeds are measured in nano-seconds and not miliseconds, all is good.

Most arrays also have some form of predictive read-ahead, based on odds that once you’ve read X number of tracks, you’re more likely to read the next Y tracks in series.

But yet, (and back to the point of the response) I still see people trying desparately to micro-manage their disks. Even in my company I have daily fights with our DBA because he wants to see two dozen 8G volumes presented instead of the 200G volume spread across many spindles on the back-end, despite the fact that the 8G will end up being slower because you can’t use a ’scatter-gather’ read on a single volume.

In the EMC world the rule is to use sequential symmetrix device ID’s for volumes which, by virtue of the way the Enginuity software lays out the disks by default, guarantees that you won’t step on the same physicals.

]]> By: Nigel Poulton http://blog.nigelpoulton.com/spread-it-and-forget/comment-page-1/#comment-103 Nigel Poulton Thu, 07 Dec 2006 20:35:32 +0000 http://blogs.rupturedmonkey.com/?p=56#comment-103 Thanks for the great comments guys. JM I agree that the isolation game is a non starter in large environments. Many of the environments Ive worked in recently are very large and one of the commonalities in these environments is that nobody seems to have a clue what storage requirements are around the corner – you can turn up at work one day and find a request for adding 10 new server with 800GB each that needed to be done yesterday ;-) That’s one of the things I was getting at when I said “this approach makes it much simpler to manage storage especially in dynamic environments”. Ive also worked quite a lot with EVAs in the past and initially did try to separate workloads to different spindles – one big disk group for random and a few smaller 8 disk (single RSS) disk groups for sequential. But because of the limited architecture on the controllers I soon stopped doing this and just went for the single large disk group approach. I remember first using a Compaq EVA and thinking that it was a breath of fresh air and ahead of its time with some of its technology. Thanks for the great comments guys.

JM I agree that the isolation game is a non starter in large environments. Many of the environments Ive worked in recently are very large and one of the commonalities in these environments is that nobody seems to have a clue what storage requirements are around the corner – you can turn up at work one day and find a request for adding 10 new server with 800GB each that needed to be done yesterday ;-) That’s one of the things I was getting at when I said “this approach makes it much simpler to manage storage especially in dynamic environments”.

Ive also worked quite a lot with EVAs in the past and initially did try to separate workloads to different spindles – one big disk group for random and a few smaller 8 disk (single RSS) disk groups for sequential. But because of the limited architecture on the controllers I soon stopped doing this and just went for the single large disk group approach.

I remember first using a Compaq EVA and thinking that it was a breath of fresh air and ahead of its time with some of its technology.

]]> By: Liho http://blog.nigelpoulton.com/spread-it-and-forget/comment-page-1/#comment-102 Liho Thu, 07 Dec 2006 18:11:49 +0000 http://blogs.rupturedmonkey.com/?p=56#comment-102 It’s possible to create disk groups in EVAs to spread data over all disks in dedicated disk group. Minimal number of disks in such group is 8. HP engineer told me that customers are using one or two groups only. Thus it’s possible to isolate workloads but nobody does it. I also like this idea to spread similar workload over common disks, but here is one more issue. Chunk size is 2MB in case of EVAs. Thus such striping is good for Oracle and similar databases only. Remember Oracle’s SAME strategy (Stripe And Mirror Everything), where recommended chuck size was 1MB... I’ve heard that other applications became slower after migration to EVA. EVAs as another modular storages don’t have a lot of opportunities to isolate workloads: use different controllers and different disks groups only. It’s possible to create disk groups in EVAs to spread data over all disks in dedicated disk group. Minimal number of disks in such group is 8. HP engineer told me that customers are using one or two groups only. Thus it’s possible to isolate workloads but nobody does it.

I also like this idea to spread similar workload over common disks, but here is one more issue. Chunk size is 2MB in case of EVAs. Thus such striping is good for Oracle and similar databases only. Remember Oracle’s SAME strategy (Stripe And Mirror Everything), where recommended chuck size was 1MB… I’ve heard that other applications became slower after migration to EVA.

EVAs as another modular storages don’t have a lot of opportunities to isolate workloads: use different controllers and different disks groups only.

]]> By: JM http://blog.nigelpoulton.com/spread-it-and-forget/comment-page-1/#comment-101 JM Thu, 07 Dec 2006 16:10:19 +0000 http://blogs.rupturedmonkey.com/?p=56#comment-101 I agree that ideally there's a balance to be struck between spreading and isolating workload. However, I would argue that at some point, when you end up managing enough storage it becomes impractical (and expensive!) to play the isolation game. Everyone thinks their application is more critical than the next and should be isolated, DBAs want separate spindles for everything, etc. If you're dealing with a single application or just a handful of applications and one or two storage arrays, fine tuning is probably worth your time, but when you move into larger environments with hundreds of applications the "remove your brain" method of managing storage becomes a lot more attractive. Array vendors also seem to be heading this direction. Look at 3Par who spreads LUNs of differing RAID protection across the same spindles. HP EVA spreads all LUNs of the same RAID level across as many disks as you ask it to. Pillar will not only spread your IO across as many disks as it can, but it'll try to optimize your IO by placing high QoS LUNs on outer tracks of disks. They're all trying to do this work for us, and I for one appreciate it. The geek in me wants to tune every app by hand, but it can be a monumental task. There will always be exceptions for highly critical or IO intensive apps, but in general I'm a proponent of spreading IO as far and wide as possible and letting your arrays do the best job they can. I agree that ideally there’s a balance to be struck between spreading and isolating workload. However, I would argue that at some point, when you end up managing enough storage it becomes impractical (and expensive!) to play the isolation game. Everyone thinks their application is more critical than the next and should be isolated, DBAs want separate spindles for everything, etc. If you’re dealing with a single application or just a handful of applications and one or two storage arrays, fine tuning is probably worth your time, but when you move into larger environments with hundreds of applications the “remove your brain” method of managing storage becomes a lot more attractive.

Array vendors also seem to be heading this direction. Look at 3Par who spreads LUNs of differing RAID protection across the same spindles. HP EVA spreads all LUNs of the same RAID level across as many disks as you ask it to. Pillar will not only spread your IO across as many disks as it can, but it’ll try to optimize your IO by placing high QoS LUNs on outer tracks of disks. They’re all trying to do this work for us, and I for one appreciate it. The geek in me wants to tune every app by hand, but it can be a monumental task. There will always be exceptions for highly critical or IO intensive apps, but in general I’m a proponent of spreading IO as far and wide as possible and letting your arrays do the best job they can.

]]>