I had to run vSCSIStats for the first time in a while the other day. Normally, trying to display the data that it outputs in a graphical form involves some cutting and pasting and a wrestling match with Excel. I recalled this time that someone had tried to automate the process in the past and a quick google search sent me to Gabe’s site and an article that he wrote back in February. The Excel macro (written by Paul Dunn) that was the subject of the article looked like just the job but I couldn’t get it to work for some reason. I’m not very good with vbscript and so I started to wonder if it might be possible to do it with PowerShell…
Another quick google suggested Microsoft Chart Controls for .NET 3.5 as a potentially useful tool. Originally I had planned to simply write a script that used Excel as I have done in the past. However, that approach might not have been the quickest and easiest thing to do.
Although there are quite a few people using the Chart Controls, most of the content out there focuses on coding / scripting languages other than PowerShell. The best PowerShell example that I found was in this article by Richard Macdonald. It’s a great introduction and well worth a read. In fact, as you’ll below, it formed the foundation of my script.
So, first things first, some pre-requisites:
- Install Microsoft Chart Controls for .NET 3.5 (which requires .NET 3.5 of course)
- Install PowerShell
That’s it. Except of course that we need some data.
The environment that I’m working in uses ESX still rather than ESXi. As such, vSCSIStats is already present and easy to get to using an SSH connection the the server hosting the VM that you’re interested in. Once on the server I had to obtain the worldGroupID of the VM that I wanted to monitor. This was done simply by issuing the following command:
/usr/lib/vmware/bin/vscsiStats -l
Which returned the following output:
Virtual Machine worldGroupID: 6146, Virtual Machine Display Name: XXXXXXX {
Virtual SCSI Disk handleID: 8193
Virtual SCSI Disk handleID: 8194
Virtual SCSI Disk handleID: 8195
Virtual SCSI Disk handleID: 8196
Virtual SCSI Disk handleID: 8197
Virtual SCSI Disk handleID: 8198
}
As you can see, the worldGroupID for the VM is 6146. The disks listed are the Hard Disks presented to the VM in the order in which they are listed in the VM’s settings. A bit detective work is required to work out which disk ID maps to which drive letter in the VM. Fortunately, it was easy enough to discover that the disk that I was most interested in has an ID of 8194.
Now, to run vSCSIStats and collect some data I had to issue the following command:
/usr/lib/vmware/bin/vscsiStats -s -w 6146
And this is the output that I received:
vscsiStats: Starting Vscsi stats collection for worldGroup 6146, handleID 8193 Success. vscsiStats: Starting Vscsi stats collection for worldGroup 6146, handleID 8194 Success. vscsiStats: Starting Vscsi stats collection for worldGroup 6146, handleID 8195 Success. vscsiStats: Starting Vscsi stats collection for worldGroup 6146, handleID 8196 Success. vscsiStats: Starting Vscsi stats collection for worldGroup 6146, handleID 8197 Success. vscsiStats: Starting Vscsi stats collection for worldGroup 6146, handleID 8198 Success.
I left it running for about 30 minutes and then used the following command to export the histogram data to a CSV file:
/usr/lib/vmware/bin/vscsiStats -p all -w 6146 -c > /root/vscsiStats-export.csv
I also used this command to stop the stats collection from continuing:
/usr/lib/vmware/bin/vscsiStats -x
Finally, I used Veeam’s FastSCP tool to copy the CSV file back to my desktop machine.
Looking inside the CSV file using a text editor, I saw this:
Histogram: IO lengths of commands,virtual machine worldGroupID,6146,virtual disk handleID,8193 min,512 max,65536 mean,4456 count,13257 Frequency,Histogram Bucket Limit 6165,512 75,1024 567,2048 575,4095 2428,4096 72,8191 2270,8192 222,16383 636,16384 110,32768 16,49152 10,65535 111,65536 0,81920 0,131072 0,262144 0,524288 0,524288
This is the data that I will use to create a single histogram. The CSV file will generally contain many more datasets. One for each metric multiplied the the number of disks present. This is why it could be so tedious copying and pasting the data into Excel.
Nothing needs to be changed in the CSV file, the script that I wrote works with it as it is. All I did was place it in the same folder and the script and then executed the script from within PowerShell. There was nothing exciting to see when it ran in the console but within a few seconds a histogram was displayed using the data from above. Each dataset from the CSV file was rendered as a histogram in turn. Simply by closing the graph window, the script moved onto the next dataset and rendered it until I got to the ones that I wanted. Remember the disk handleID from earlier? It was 8194. This is the IO latency histogram for that disk:
Displaying the data and interpreting the data are two different things so I won’t go into it much now. You can see though that disk IO latency varies between 0.5ms and 100ms and mostly sits at 15 – 30ms. That’s what I wanted to know.
So, now for the script. As I mentioned earlier, I based it around the work that Richard Macdonald had already done so you’ll see some sections of code that are very similar looking to snippets from his article. The rest I cobbled together so the whole thing is a bit clunky. I’ll give it a polish sometime soon.
###########################################################
# vSCSIStats-Histograms.ps1
#
# Version: 0.1
# Author: Michael Poore (www.vspecialist.co.uk)
#
# Uses Microsoft Chart Controls to draw charts from CSV data
# outputed by vSCSIStats
###########################################################
$thisXData = ""
$thisYData = ""
$rawData = Import-Csv "vscsistats-export.csv" -Header "Field","Value","worldGroupID","diskHandleIDHeader","diskHandleID"
$dataCollection = @()
foreach( $row in $rawData )
{
$row.Field -match "([a-zA-Z0-9\-\(\)]+)" | Out-Null
switch( $matches[0] )
{
"Histogram" {
if( test-path variable:\thisHistogram )
{
$thisHistogram | Add-Member -Name XData -Value $thisXData -Membertype NoteProperty
$thisHistogram | Add-Member -Name YData -Value $thisYData -Membertype NoteProperty
$dataCollection += $thisHistogram
}
Remove-Variable -Name thisXData
Remove-Variable -Name thisYData
$thisHistogram = New-Object PSObject
$thisHistogram | Add-Member -Name Title -Value $row.Field -Membertype NoteProperty
$thisHistogram | Add-Member -Name worldGroupID -Value $row.worldGroupID -Membertype NoteProperty
$thisHistogram | Add-Member -Name diskHandleID -Value $row.diskHandleID -Membertype NoteProperty
$thisXData = @()
$thisYData = @()
}
"min" {
$thisHistogram | Add-Member -Name Min -Value $row.Value -Membertype NoteProperty
}
"max" {
$thisHistogram | Add-Member -Name Max -Value $row.Value -Membertype NoteProperty
}
"mean" {
$thisHistogram | Add-Member -Name Mean -Value $row.Value -Membertype NoteProperty
}
"count" {
$thisHistogram | Add-Member -Name Count -Value $row.Value -Membertype NoteProperty
}
"Frequency" {
#Do nothing
}
default {
#Assumed to be data
$thisXData += $row.Value
$thisYData += $row.Field
}
}
}
#Load Charting Assemblies
[void][Reflection.Assembly]::LoadWithPartialName("System.Windows.Forms")
[void][Reflection.Assembly]::LoadWithPartialName("System.Windows.Forms.DataVisualization")
foreach( $chart in $dataCollection )
{
$thisTitle = $chart.Title + " (worldGroupID " + $chart.worldGroupID + ", diskHandleID " + $chart.diskHandleID + ")"
#Create Chart Object
$thisChart = New-object System.Windows.Forms.DataVisualization.Charting.Chart
$thisChart.Width = 900
$thisChart.Height = 600
$thisChart.Left = 40
$thisChart.Top = 30
#Create ChartArea
$ChartArea = New-Object System.Windows.Forms.DataVisualization.Charting.ChartArea
$ChartArea.AxisX.Interval = 1
$ChartArea.AxisX.TextOrientation = "Rotated90"
$thisChart.ChartAreas.Add($ChartArea)
[void]$thisChart.Series.Add("Data")
$thisChart.Series["Data"].Points.DataBindXY($chart.XData, $chart.YData)
#Add Title
[void]$thisChart.Titles.Add($thisTitle)
#Change Chart Colour
$thisChart.BackColor = [System.Drawing.Color]::Transparent
#Make Bars Into 3d Cylinders
$thisChart.Series["Data"]["DrawingStyle"] = "Cylinder"
# add a save button
$SaveButton = New-Object Windows.Forms.Button
$SaveButton.Text = "Save"
$SaveButton.Top = 10
$SaveButton.Left = 900
$SaveButton.Anchor = [System.Windows.Forms.AnchorStyles]::Bottom -bor [System.Windows.Forms.AnchorStyles]::Right
$SaveButton.add_click({$thisChart.SaveImage($Env:USERPROFILE + "\Desktop\" + [System.Text.RegularExpressions.Regex]::Replace($thisTitle,"[^1-9a-zA-Z_]","_") +".png", "PNG")})
# display the chart on a form
$thisChart.Anchor = [System.Windows.Forms.AnchorStyles]::Bottom -bor [System.Windows.Forms.AnchorStyles]::Right -bor [System.Windows.Forms.AnchorStyles]::Top -bor [System.Windows.Forms.AnchorStyles]::Left
$thisForm = New-Object Windows.Forms.Form
$thisForm.Text = $thisTitle
$thisForm.Width = 1000
$thisForm.Height = 700
$thisForm.controls.add($SaveButton)
$thisForm.controls.add($thisChart)
$thisForm.Add_Shown({$thisForm.Activate()})
$thisForm.ShowDialog()
}
Subsequently to writing this script and getting the charts and information that I wanted, I discovered that David Owen (@vMackem) has been experimenting with Chart Controls also. We compared notes over a few beers after IP Expo yesterday. His article is a good read and efficiently produces a nice pie chart from his script. I’m looking forward to hearing about his other discoveries with Chart Controls, PowerShell and PowerCLI.
