ORIGINALLY POSTED January 10th 2008
10,649 views on developerwortks
I’d mentioned we (the SVC performance team) have a few bits of fine tuning and investigations to perform for the next (4.3.0) release of SVC. This is the part of my job that I enjoy the most, getting the latest builds from development and putting them through their paces, drilling into the running software to find where those extra few tweaks will give just that bit more performance, or even determine where things can be reworked to provide real life improvements. I even managed to write some code at the tail end of last year that has made some noticeable changes to certain mixed and over stressful workloads, helping protect from runaway/rogue applications.
As you can imagine the set of tools I use to generate the I/O load that saturate the node hardware have grown and expanded over the years into a large suite of configuration, benchmarking and analsysis tools. However one of the most useful high level analysis tools I use is the same tool we provide to IBM storage customers, that is, TotalStorage Productivity Center. The Productivity Center is a collection of modules that snap into a single framework, allowing Fabric, Data, Disk, Tape, File and Replication Management. I must confess that I haven’t had the time to play with all of these, and primarily I concentrate on the Disk management tools in my day to day work. I guess its a major plus for the software to say that I couldn’t do my job without it.
Last year, SSPC (System Storage Productivity Center) was released which is supplied on a single master console which can be used to manage IBM’s storage products, primarily DS8000 and SVC. This comes with a limited license version of Productivity Center which can be upgraded to the full version (adding whichever modules you require). The DS4/3000 management software can of course be installed on this box and provide a single console for all your IBM storage.
Productivity Center makes use of the SNIA SMI-S compliant nature of IBM’s storage management interfaces, discovering CIMOM’s on the network and any managed devices under those CIMOM’s. Most of the time I’m using the SVC CLI (Command Line Interface) through an SSH connection, however in order to utilise Productivity Center and its performance monitoring aspects I have the SVC GUI (CIMOM and client) installed on a Windows server which my Productivity Center instance talks. With little enough hours in the day, my ‘if it aint broke don’t fix it’ attitude meant I was running a strange (unsupported) combination of levels. 4.2.1 SVC code, with an old 4.2.0 GUI and the earlier Productivity Center 3.1.3 – However the time had come, by moving up to the latest development 4.3.0 build, that I had to upgrade my GUI code to cope with the new features. At the same time I upgraded to the latest Productivity Center 3.3 release.
This image shows the tree structure of Productivity Center 3.3.0 with the ‘Launched’ SVC ‘Element Manager’ GUI showing my one rather large Managed Disk Group. Click to view full size.
I’m glad to see that a few of the ‘wouldnt it be nice if’ type features have been added and the performance monitoring / analysis areas have been enhanced. So now I’m all set and the database is filling with lots and lots of performance metrics. At the moment I’m running a pathalogical test case with over 80 mdisks (6 disk RAID-10 arrays) in a single managed disk group. These are all 15K RPM drives, and so the performance from striping across around 240 spindles means I can eliminate the backend disks from my equations, and concentrate on the SVC performance. This got me thinking about XIV and its massively striped array capability.
There have been several questions posed out there, and Tony did a great job of summarizing the answers. I’ve a couple of animated Power Point slides that shows this in action (someone somewhere has too much time on their hands) however the linking inside this site to a ppt seems to be failing (will see if I can get it fixed), this answers the questions about re-striping when adding and removing storage. At all times Nextra will try to keep even utilisation across all disks, using a pseudo-random allocation to avoid hot-spotting. The idea of equilibrium is neat. Its one of the things we provide with SVC however we don’t autonomically re-stripe, over time you may need to make use of the re-stripe facility we provide to set off an internal migration of the stripe set in your managed disk group.
The main area of Productivity Center that I use on a daily basis is the Performance Reporting section. The graph opposite was generated from data collected from SVC during a ‘response curve’ test where increasing write data rates are applied until such time as the response time goes above an acceptable level. This section shows various response time metrics over a series of measurements with one, two, four and six vdisks.
I’d highly recommend Productivity Center to all IBM storage users, not just because its another IBM product, but because of the ability it gives users to self monitor, setup alert thresholds and not only for DS8000 and SVC, but also the backend storage itself. In an SVC environment remember that SVC monitors the backend storage performance, so not only will Productivity Center provide virtual disk, node and cluster wide statistics, but also the metrics of your heterogeneous storage environment. Truely a single pane solution that can provide the complete SAN disk performance picture. I’ve only scratched the surface of what Productivity Center can provide here, I’ll try and cover more aspects over the coming months.