The Ongoing Need for Tiering

Its always amusing to be pinged by several people asking for comment on the same topic.

This time it seems that Pure are trying to use their lack of function (there are several) and sell it as an advantage, while attacking the competition that do have it.

I know reverse psychology worked on my kids when they were younger, but even they have got too smart for that, so don’t let yourself drink too much of the orange kool aid!

A Brief History of Tiering

Back in 2008 Enterprise class flash drives were a massive game changer when first introduced in external storage controllers. We all take it for granted today, the fact that demand has brought the price to a place where we can afford to put in all flash solutions. This was not the case 10 plus years ago, and so Tiering was added to storage controllers to enable a small amount of the then expensive Flash capacity to accelerate the most demanding workloads.

Over the last 10 years the ratios and tiers have changed, but the same basic ethos prevails.

In almost all typical workloads, running on dedicated external storage controllers, there is a very clear workload skew. By skew we mean what percentage of the capacity provides what percentage of the I/O demands.

In the majority of cases this is less than 10% of the capacity.

Workloads don’t magically change in skew, its a function of the workload type itself. New workloads come along and may have unique skews. Sometimes there are workloads with almost no skew, but those are rare or unique and have unique requirements that can be met by the right solution. For example some AI workloads or Data Warehouse, where something like Spectrum Scale and very high GB/s are needed, very different from the low latency high IOPs workloads that typically benefit from lots of Flash.

So Why Do We Still Need Tiering?

Tiering is as much about price as it is performance. If money was not a concern we would all be filling every box with Storage Class Memory devices and be done with it.

So there is the first thing, NAND based Solid State devices are no longer top dog. Yes we’ve had huge improvements in Flash as a storage technology in the last decade. In addition, price has dramatically reduced and most people can now afford all Flash, but as we march on, SCM is now the lowest latency, highest IOPs technology, and of course that comes at a price.

The exact same value proposition from 2008 still applies today but instead of Flash and HDD the story is now SCM and Flash…. and HDD. Never mind NVMe and SAS as well.

A typical config today may include multiple tiers, as we did with 15K, 10K, 7.2K HDD. We have multiple tiers to choose from within the Solid State technology realm. I think our EasyTier(TM) technology now caters for 7 different tier types. However, it really only makes sense to use 3 active tiers in any given config. One hot, one cold, and everything else in the middle. Most people these days are just using two. Flash and “trash” as our American friends like to call it. Its early days for SCM, but I would expect to see that moving us back to 3 tiers as it becomes more widespread in its adoption.

So What Do We Use for the Lowest Tier?

Well some of that depends on your workload, and of course your architectural decisions.

For example, some clients have made the decision to drop HDD altogether. Others maybe more cost conscious and the NL-SAS large capacity HDD are still the most cost effective in $/GB terms (for online storage).

For those going all Flash you can still tier. We can have our Tier1 Flash drives. High capacity, SAS attached still getting low latency and respectable throughput, but more cost conscious than NVMe attached FlashCore Modules (FCM) or traditional OEM NVMe Flash drives.

But SAS, I hear you ask? NVMe is better… I hear you kool aid drinkers. True, NVMe has given us a closer link to the CPU inside the storage controller. Its also given us a user space driver with Direct Memory Access (DMA) capabilities.

Well, we had that for many years. Since 2003, the Spectrum Virtualize internal drivers have all been user space, using a polling loop and DMA. So actually our SAS driver was doing what NVMe does since 2010! The only difference is that NVMe removes the middle man, the SAS HBA and so can increase bandwidth beyond the 8 or 16 lanes of PCIe. But latency, and CPU efficiency, there is very little difference in our boxes with either protocol.

This brings me back to my original point. It seems Pure are attacking our SAS expansion and saying they are better as they are all NVMe. Well yes, they are all NVMe, but they are also single tier. All the advantages I describe above… saving money being the biggest one... they can’t achieve. Nor can they offer a small amount of SCM as a top tier and do reads and writes to it as primary storage. We’ve been here before, NetApp flash cache anyone. You only use it solely as a cache because you can’t use it as real tiered storage capacity.

Its also clear that those wanting large capacity don’t even need to scale up to SAS with our FlashSystems. With 88TB per FCM and 24x per controller, that’s 1.5PB of capacity in 2U… if you really drive that to its maximum, you probably don’t want to add any more capacity, rather you cluster and scale out to linearly increase the capacity AND the processing power.

But if you do want to tier, and you have a good view of your skew, 8.3.1 added the skew graphs to the GUI, then by all means look to add SAS Flash drives or even HDD as your lower tier and reap the cost savings.

So there we have it, Tiering is here to stay, we will have the next great innovation in storage media soon I’m sure, which will relegate SCM down a tier, and the whole story will repeat.

Tiering is an essential capability in a modern storage controller. One size doesn’t fit all. Don’t let yourselves be treated as a child, break the reverse psychology and beware of drinking the orange kool aid.

One response to “The Ongoing Need for Tiering”

  1. Great post Barry! Had a web meeting this afternoon with a service provider and we discussed the points raised here – they were very receptive to the message and surprisingly interested in Storage Class Memory.

    That discussion aside, in my experience, too many people look at capabilities they think they understand only from the perspective of what they would have used them for historically, not understanding them in the context of what is coming – in car analogy terms, the equivalent of driving using only a rear view mirror…..

    As storage professionals, it’s our job to highlight how the capabilities we build into our solutions will be valuable going forwards or have non-obvious benefits outside of storage (I’m talking NVMe-oF).

    Your blog is a great way of doing this, so please keep up the great work!

    Liked by 1 person

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: