Tips and tricks using the Spectrum Virtualize REST API

Hi all,

I’ve been getting some questions recently about using the Spectrum Virtualize REST API, and I noticed that the documentation is missing some useful information.

So I thought I’d put together some tips based on what I have been able to find out so far. If you have questions or your own tips – please ask in the comments and I’ll do my best to answer and update this page for the benefit of others.

There are many ways that you can send commands using REST APIs – I’m going to do almost all my examples using CURL – but the concepts should be the same.

This is my attempt at compiling a useful walkthrough. The design of the REST API will almost certainly change over time, and I almost certainly have some mistakes in here somewhere. Please comment if you spot any mistakes.

Many thanks to Laura Tuller and Jason Heller who reviewed this to make sure I was explaining things clearly enough.

Basics

Lets start with the simple stuff

The Minimum code level that support the REST API is V8.1.3.0
The REST API only supports HTTPS POST requests
- Don’t worry if you don’t know what this means.
- The main impact of this is that you can’t use the REST API by typing fields directly into the address bar of your web browser, although there are plugins for web browsers that will let you do this
There is currently a restriction that REST API queries that return more than 2000 results shouldn’t be used
- Basically this means that if you have more than 2000 vdisks, you technically shouldn’t run lsvdisk using the REST API
- This restriction is not policed – but the REST API service may restart (although not always) if you do large queries
- The restriction is documented in the configuration limits and restrictions page. Here’s an arbitrary example page
It is not possible to access the REST API using a cluster’s IPv6 address

The output from the REST API is always a data format called json. This is a data format that’s very easy for computers to read, and there are lots of libraries out there that can convert the json into a datastructure.

Its actually fairly easy for humans to read too, but often the json output is all on a very long line – making it harder to read as a human. But this is easily fixed – many json libraries have the concept of “pretty print” which put lots of spaces and newlines into the text to make it easier for humans to read.

The easiest tool I’ve come across for taking json and making it quickly and easily human readable is to pipe the output to python -m json.tool. I’ve included an example of what a big difference this makes below.

A quick example

First Authenticate:

curl -k -X POST -H 'Content-Type:application/json' -H 'X-Auth-Username: andrew' -H 'X-Auth-Password: bananarama' https://9.71.57.16:7443/rest/auth

{"token": "3b8916b21038d52187623ffa1c33f5f01cefe7fc9881d7afb8c105142a40440b"}

The authentication step returns a token. This is a long hexadecimal string that must be used to authenticate all of the following commands.

Then run the command – I’ve replaced the Username/Password combination with the token that I got back from the previous authentication:

curl -k -X POST -H 'Content-Type:application/json' -H 'X-Auth-Token: 3b8916b21038d52187623ffa1c33f5f01cefe7fc9881d7afb8c105142a40440b' https://9.71.57.16:7443/rest/lsmdisk

[{ "id": "0", "name": "mdisk0", "status": "online", "mode": "array", "mdisk_grp_id": "0", "mdisk_grp_name": "Pool0", "capacity": "21.7TB", "ctrl_LUN_#": "", "controller_name"
: "", "UID": "", "tier": "tier1_flash", "encrypt": "no", "site_id": "", "site_name": "", "distributed": "yes", "dedupe": "no", "over_provisioned": "no", "supports_unmap": "ye
s" }]

Here’s the same command but using python to make the output more human readable – see what a big difference it makes!

curl -k -X POST -H 'Content-Type:application/json' -H 'X-Auth-Token: a3517879443c9010ab2313fa9aa94a574aa34577b1fdb5c7cb185a45ac0d9e57' https://9.71.57.16:7443/rest/lsmdisk | python -m json.tool

[
    {
        "UID": "",
        "capacity": "21.7TB",
        "controller_name": "",
        "ctrl_LUN_#": "",
        "dedupe": "no",
        "distributed": "yes",
        "encrypt": "no",
        "id": "0",
        "mdisk_grp_id": "0",
        "mdisk_grp_name": "Pool0",
        "mode": "array",
        "name": "mdisk0",
        "over_provisioned": "no",
        "site_id": "",
        "site_name": "",
        "status": "online",
        "supports_unmap": "yes",
        "tier": "tier1_flash"
    }
]

Note: I had a lot of problems copying and pasting commands into my cygwin command line. I’ve put more details below – but be aware that when you copy and paste – you may get the wrong type of hyphen (-) or the wrong type of single quote ( ‘ ) – and that can make curl do REALLY weird looking things

What are all the random words that were used in the example above?

So now you’ve seen a worked example – what does all that magic stuff mean

Command option	Description	Notes
curl	This is the executable that is sending the request to the server
-k	This tells curl to perform the request even if the SSL authentication fails.	If you are using signed SSL certificates – you shouldn’t need this
-H ‘Key:Value’	Send the information in the quote as a header Key is the name of the header – describing what specific header is being sent Value is the …. well …. value
X-Auth-Username	The username that you use to log in	Only used for initial authentication
X-Auth-Password	The password that you use to log in	Only used for initial authentication
X-Auth-Token	The authentication token that is used to authenticate the REST calls after authentication is complete	Only used for running commands – not for the authentication step
Content-Type:application/json	Tells the server to send the result back in json format	I believe that json is the only supported format
`https://<cluster IP address or DNS name>:7443/rest/auth`	The URI that you send an authentication request to
`https://<cluster ID or DNS name>:7443/rest/<cli command>`	The URI that you send a CLI command to
-d ‘<DATA>’	The -d flag is used to send the CLI options, encoded in json	More details on this later in the document

Authentication tips

So here are some things I have learned about authentication:

Each user can only have a single valid token at any time. The token lasts for 2 hours if you are using it, but will expire after 30 minutes after the last use.

If you send an authentication request for user “andrew” – then the previous token will be invalidated and replaced with a new one.

IMPORTANT If you are doing automation, I recommend you create a different user account for each piece of software that is connecting to the REST API.

CURL is not secure – your password may be at risk

You probably noticed that all my examples here have the password in plain text on the command line. That means that if you use CURL – anyone who can read your command line history can also read your password. Also – anyone that happens to run “ps” on your machine when you are in the middle of authentication will be able to see your password.

The password will be encrypted when sent over the network, but may be easy for another user on the same server to discover. You will have to worry about how much this worries you. The right solution is probably to not use CURL for anything other than testing with an unpriviliged user.

Which basically means that curl isn’t a great solution for the “real world”, and if you are going to do this properly – you should write a script or use a proper tool that has ways of handling these types of problems for you.

Running a real command

The REST API uses exactly the same syntax as the command line interface, so you can (and should) use the command line guide as the reference for how to run commands.

You may even find some useful tips here: https://barrywhytestorage.blog/2020/01/09/how-to-script-on-spectrum-virtualize-tips-and-tricks/

But once you’ve worked out what CLI command to run – how do you create the “proper” REST API string.

Simple information command with no parameters

We saw earlier that if I want to run a really simple information command without any parameters – we just change the URL.

https://9.71.57.16:7443/rest/lsmdisk

Detailed information command – getting all the properties of a single object

This is as simple as adding a foward slash and then the object ID or name to the end of the command. Just like the way you add the object ID or name to the end of the CLI command. So

lsmdisk mdisk0 
becomes
https://9.71.57.16:7443/rest/lsmdisk/mdisk0

lsmdisk 0
becomes
https://9.71.57.16:7443/rest/lsmdisk/0

If you want to change or delete an object – you use the same approach.

https://9.71.57.16:7443/rest/rmvdisk/0

Note: The examples in the documentation talks about putting the object ID inside the block (covered below). I’m almost certain that this won’t work – because the examples are using invalid JSON.

Using command line parameters and flags

The next step in the process is to start adding command parameters and flags.

Note: The CLI guide doesn’t differentiate between parameters and flags – they call them all parameters. However I’ve given them different names here so that it’s easy to distinguish between the different types of parameters and how to encode them on the REST API.

Here are my definitions of parameters versus flags, and how to encode them.

A parameter is something with a value e.g. -unit tb or -size 5
- This is encoded as a json key/value pair
- { "unit" : "tb" } or { "size" : 5 }
- Keys should always be specified in quotes ("key")
- Values should be put in quotes if they are a string ("value_string")
- Values should not be put in quotes if they are numbers (42)
A flag is something without a value e.g. -bytes or -force
- This is encoded as a json key/value pair with the value set to true
- {"bytes" : true } or {"force" : true }
- Do not put true in quotes

If you need multiple flags or parameters for a single command – just put commas between them
- {"size" : 5, "unit" : "tb", "force" : true }
- Generally you can have as many or as few spaces in json as you like – but not within the quotes

IMPORTANT: When specifying the json data on the command line – it’s very important that you use different quotes (single quote ‘ vs double quotes “) inside the json versus around it. In the code examples below I’ve used single quote around the data block and double quotes inside. This is a good convention – but the other way around should work too.

Here’s how to send that in curl. This example uses the object ID 0. But it would work just as well for lsmdisk.

 curl  -k -X POST -H 'Content-Type:application/json' -H 'X-Auth-Token: a3517879443c9010ab2313fa9aa94a574aa34577b1fdb5c7cb185a45ac0d9e57' -d '{"bytes" : true }' https://9.71.57.16:7443/rest/lsmdisk/0

[{ "id": "0", "name": "mdisk0", "status": "online", "mode": "array", "mdisk_grp_id": "0", "mdisk_grp_name": "Pool0", "capacity": "23873716224000", "ctrl_LUN_#": "", "controll
er_name": "", "UID": "", "tier": "tier1_flash", "encrypt": "no", "site_id": "", "site_name": "", "distributed": "yes", "dedupe": "no", "over_provisioned": "no", "supports_unm
ap": "yes" }]

That’s basically it

We covered above:

How to run a single command with no object
How to run a command that acts on a single object
How to add flags and parameters

I think that this is all you need to know to run any commands. I’m sure you’ll let me know in the comments if I missed anything.

However I haven’t done any task commands yet…. so just for completeness, here are a couple of task commands.

Taking this example from the mkvdisk knowledgecenter – lets convert it into a REST API CLI mkvdisk -mdiskgrp Group0 -iogrp 0 -vtype striped -size 10 -unit gb -rsize 20% -autoexpand -grainsize 32

I do have to change “Group0” to “Pool0” to match my configuration – but other than that is’ simply a job of converting the flags into json one at a time.

 curl -k -X POST -H 'Content-Type:application/json' -H 'X-Auth-Token: a3517879443c9010ab2313fa9aa94a574aa34577b1fdb5c7cb185a45ac0d9e57' -d '{ "mdiskgrp" : "Pool0", "iogrp" : 0, "vtype" : "striped", "size" : 10 , "unit": "gb", "rsize" : "20%", "autoexpand":true, "grainsize":32 }' https://9.71.57.16:7443/rest/mkvdisk

{"message": "Volume, id [11], successfully created", "id": "11"}

Here’s another example expandvdisksize -size 20 -unit gb vdisk11

curl -k -X POST -H 'Content-Type:application/json' -H 'X-Auth-Token: a3517879443c9010ab2313fa9aa94a574aa34577b1fdb5c7cb185a45ac0d9e57' -d '{ "size" : 20 , "unit": "gb" }' https://9.71.57.16:7443/rest/expandvdisksize/vdisk11

""

A quick perl example

Here’s a quick script that will send commands to the REST API.

use strict;
use warnings;
use HTTP::Request; 
use HTTP::Headers; 
use LWP::UserAgent; 
use IO::Socket::SSL;
use JSON;

my $MGMT_IP = "9.71.57.16";
my $USERNAME = "andrew";
my $PASSWORD = "bananarama";


my $debug=0;

my $command_to_run = "lsmdisk";
my $options_for_command = { 'bytes' => JSON::true};
my $object_for_detailed_command = 0;

#IMPORTANT:  I've disabled SSL verification of the hostname because I'm using self signed certificates here
# I don't recommend you do this in production scripts
my $ua = LWP::UserAgent->new(ssl_opts => { verify_hostname => 0 });

#First authenticate
print "Authenticating\n";
my $auth_response = send_request($ua, "auth", undef, { "X-Auth-Password" => $PASSWORD,  "X-Auth-Username" => $USERNAME}, {} );

if (!defined $auth_response->{token}) {
	die "Missing token from authentication\n" . to_json($auth_response);
}
my $token = $auth_response->{token};
print "Token is $token\n";

print "Sending summary command $command_to_run\n";
my $cli_response = send_request($ua, $command_to_run, undef, { "X-Auth-Token" => $token}, $options_for_command );

print "Response was : \n" . to_json($cli_response, {pretty => 1}); 

print "Sending detailed command $command_to_run $object_for_detailed_command\n";
my $detailed_cli_response = send_request($ua, $command_to_run, $object_for_detailed_command, { "X-Auth-Token" => $token}, $options_for_command );

print "Response was : \n" . to_json($detailed_cli_response, {pretty => 1}); 



sub send_request{
	my($ua, $cmd, $object,  $header_ref, $data_ref) = @_;
	my $data_json = %{$data_ref} ? encode_json($data_ref) : "";
	my $url ="https://$MGMT_IP:7443/rest/$cmd"; 
	if (defined $object && $object ne "") {
		$url .= "/$object";
	}
	my $request = HTTP::Request->new("POST"  => $url, HTTP::Headers->new(%{$header_ref}), $data_json);
	$request->header(Content_Type => 'application/json');
	
	#Debug code in case something isn't going quite right
	if ($debug) {
		print "Request being sent is :\n" . $request->as_string();
	}
	
	my $response = $ua->request($request);
	my $json_response ;
	if ($response->is_success ) {
		
		 $json_response = decode_json($response->content);
	} else {
		die "Request $cmd failed with code " . $response->code . " and return " . $response->content . "\n";
	}
	return $json_response;
}

And here’s what the output looks like:

Authenticating
Token is 0343ad57bd00790493df81dc1f41f05d13abd813b224fcb747fce7002d4f052b
Sending summary command lsmdisk
Response was : 
[
   {
      "over_provisioned" : "no",
      "tier" : "tier1_flash",
      "capacity" : "23873716224000",
      "controller_name" : "",
      "name" : "mdisk0",
      "distributed" : "yes",
      "id" : "0",
      "mode" : "array",
      "site_name" : "",
      "mdisk_grp_id" : "0",
      "mdisk_grp_name" : "Pool0",
      "site_id" : "",
      "ctrl_LUN_#" : "",
      "UID" : "",
      "dedupe" : "no",
      "encrypt" : "no",
      "supports_unmap" : "yes",
      "status" : "online"
   }
]
Sending detailed command lsmdisk 0
Response was : 
{
   "supports_unmap" : "yes",
   "replacement_date" : "",
   "controller_id" : "",
   "encrypt" : "no",
   "strip_size" : "256",
   "rebuild_areas_goal" : "1",
   "balanced" : "exact",
   "status" : "online",
   "easy_tier_load" : "",
   "physical_free_capacity" : "12843025956864",
   "slow_write_priority" : "latency",
   "raid_level" : "raid6",
   "dedupe" : "no",
   "redundancy" : "2",
   "raid_status" : "online",
   "mode" : "array",
   "write_protected" : "no",
   "preferred_WWPN" : "",
   "stripe_width" : "12",
   "distributed" : "yes",
   "id" : "0",
   "rebuild_areas_available" : "1",
   "tier" : "tier1_flash",
   "ctrl_WWNN" : "",
   "block_size" : "",
   "drive_class_id" : "0",
   "capacity" : "23873716224000",
   "active_iscsi_port_id" : "",
   "max_path_count" : "",
   "quorum_index" : "",
   "site_id" : "",
   "rebuild_areas_total" : "1",
   "mdisk_grp_name" : "Pool0",
   "spare_protection_min" : "",
   "ctrl_LUN_#" : "",
   "UID" : "",
   "provisioning_group_id" : "",
   "preferred_iscsi_port_id" : "",
   "fabric_type" : "",
   "active_WWPN" : "",
   "ctrl_type" : "",
   "allocated_capacity" : "11001558728704",
   "site_name" : "",
   "spare_goal" : "",
   "name" : "mdisk0",
   "effective_used_capacity" : "0",
   "mdisk_grp_id" : "0",
   "physical_capacity" : "23844584685568",
   "over_provisioned" : "no",
   "fast_write_state" : "empty",
   "path_count" : "",
   "drive_count" : "16",
   "controller_name" : ""
}

Things I got wrong (In other words learn from my mistakes)

Copy & Paste nightmares

Did you know that there are at least 3 different types of dash (I think they are called hyphen, en dash & em dash). And various editor programs have automatic optimization to help you use the correct one. For example in MS word type a word followed by a space followed by a hyphen followed by a space followed by any letter followed by a space. As soon as you type that final space the hyphen will get longer.

Why am I telling you about this typographic fun? Because I have found that the cygwin command line (and probably others) requires a specific type of “-” when specifying command line arguments. So if you copy and paste from another document (like this page) there’s a chance that you may end up with the wrong type of “-” as part of the “-H”.

Another copy and paste problem that gave me issues was when I had the wrong type of single quote (‘) .

This leads to errors like this:

curl: (6) Could not resolve host: -X
curl: (6) Could not resolve host: POST
curl: (6) Could not resolve host: -H
curl: (3) Port number ended with 'a'
curl: (6) Could not resolve host: -H
curl: (3) Port number ended with 'a'
curl: (6) Could not resolve host: -H
curl: (3) Port number ended with 'b'

If you see errors like this – try replacing all the single quotes and hyphens by typing directly on the command line…..

Wrong Username or password

Here’s what the system returns if you have the wrong username or password.

Note that you can get the same error if the headers aren’t formatted correctly (e.g. they are using the wrong type of dash).

“Invalid or missing username/password”

Tips for helping to understand why the connection failed.

The REST API will give different HTTPS error codes for different types of problems. These HTTPS errror codes are documented in the knowledge center here: https://www.ibm.com/support/knowledgecenter/STPVGU_8.3.1/com.ibm.storage.svc.console.831.doc/rest_api_http_errors.html

If you are using curl – you won’t normally see the HTTPS error code, but if you specify “-f” then curl will give you the error code (although it doesn’t return the error text any more). The example below shows the same command both with and without the -f.


$ curl -k -X POST -H 'Content-Type:application/json' -H 'X-Auth-Token: a3517879443c9010ab2313fa9aa94a574aa34577b1fdb5c7cb185a45ac0d9e57' -d '{ "size" : 20 , "unit": "gb" }' https://9.71.57.16:7443/rest/expandvdisksize/11
"Invalid or missing token"


$ curl -f  -k -X POST -H 'Content-Type:application/json' -H 'X-Auth-Token: a3517879443c9010ab2313fa9aa94a574aa34577b1fdb5c7cb185a45ac0d9e57' -d '{ "size" : 20 , "unit": "gb" }' https://9.71.57.16:7443/rest/expandvdisksize/11
curl: (22) The requested URL returned error: 403 Forbidden

Note: The REST API should automatically restart if it encounters an issue – but we have seen some cases where the restart doesn’t work correctly. If you are seeing error 500 you could try to restart the REST API service with the following command (which must be run as superuser on the config node) satask restartservice -service cfrest

Don’t forget the :7443

If you forget the :7443 in the URL – you will get HTTPS 403 "Invalid or missing token" even if you get everything else right. I wasted about 15 minutes spotting my mistake… hopefully you can solve your problem a bit quicker.

Automated tooling expectation

While all of the REST API commands return valid json – some of them return an array of objects, whereas others return a single object.

I’m told by my co-blogger @barrywhyteibm that this breaks some of the automated tooling that is designed to talk to REST APIs for you.

As far as I know (and I’m not an expert) the return is perfectly valid json – but I think that these tools require a certain type of response data to work “out of the box”.

4 responses to “Tips and tricks using the Spectrum Virtualize REST API”

NAGA SURESH BONDADA

November 24, 2021 at 8:56 pm

How we can collect performance data on spectrum virtualize using restapi?

LikeLike

Reply
1. Andrew Martin
  
  December 6, 2021 at 10:48 am
  
  Hi Naga,
  
  There are two types of performance data. The high level view that you see in the product GUI is available by the CLI commands lssystemstats (and others).
  
  The detailed performance stats are only available as files that have to be copied off the node. I don’t think it’s possible to get these files via the REST API.
  
  LikeLike
  
  Reply
Gabriel

December 5, 2021 at 4:09 am

Great article! I must be doing something wrong though, because when I post auth I keep getting error 503. I’ve tried both with curl and the spectrum virtualize Ansible collection. Do you have any ideas? Thanks a lot!

LikeLike

Reply
1. Andrew Martin
  
  December 6, 2021 at 10:50 am
  
  Hi Gabriel,
  
  503 indicates that the service is not available. I’m not sure what could cause this error – so I don’t think I can help here. I recommend you open a support ticket so we can look at the logs and maybe validate the exact commands you are sending.
  
  LikeLike
  
  Reply

Barry Whyte and Andrew Martin : IBM Storage

Tips and tricks using the Spectrum Virtualize REST API

Basics

A quick example

What are all the random words that were used in the example above?

Authentication tips

CURL is not secure – your password may be at risk

Running a real command

Simple information command with no parameters

Detailed information command – getting all the properties of a single object

Using command line parameters and flags

That’s basically it

A quick perl example

Things I got wrong (In other words learn from my mistakes)

Copy & Paste nightmares

Wrong Username or password

Tips for helping to understand why the connection failed.

Don’t forget the :7443

Automated tooling expectation

Like this:

4 responses to “Tips and tricks using the Spectrum Virtualize REST API”

Leave a Reply Cancel reply

Tips and tricks using the Spectrum Virtualize REST API

Basics

A quick example

What are all the random words that were used in the example above?

Authentication tips

CURL is not secure – your password may be at risk

Running a real command

Simple information command with no parameters

Detailed information command – getting all the properties of a single object

Using command line parameters and flags

That’s basically it

A quick perl example

Things I got wrong (In other words learn from my mistakes)

Copy & Paste nightmares

Wrong Username or password

Tips for helping to understand why the connection failed.

Don’t forget the :7443

Automated tooling expectation

Share this:

Like this:

4 responses to “Tips and tricks using the Spectrum Virtualize REST API”

Leave a Reply Cancel reply