Check the health of your storage paths

For you vSphere environment you always want to make sure that all paths to your storage system are connected. That is why in our vSphere environment we run a daily morning check that tells us if there are any death paths between hosts and storage. Before upgrades of our SAN or flare code upgrades of our storage arrays, we also make sure there are no death paths in our vSphere environement. Although this looks to be a good practise, we never realised until a few weeks ago, that after rescanning a host with death paths, the rescan can make the path completely dissapear. We did use a script to check a host for number of total paths connected, but that total can sometimes camouflage the loss of an even number of paths.

With the help of Luc Dekens (Thank you Luc!!!) we now have a new script that we run in our vCheck every day. Per host each LUN should be connected over four paths (in our environment) and the script shows LUNs on a per host basis that have less than four paths. If your environment has a different number of paths, you can easily change this in the script.

To be used in Alan Renouf’s vCheck:


# Start of Settings
# Set the Recommended number of paths per LUN
$report= @()
$esxilist = Get-VMHost
$RecLUNPaths = "4"
# End of Settings

foreach( $esxvm in $esxilist){
$esx = Get-VMHost -Name $esxvm
$esxcli = Get-EsxCli -VMHost $esxvm
$hba = Get-VMHostHba -VMHost $esx -Type FibreChannel | Select -ExpandProperty Name

$esxcli.storage.core.path.list() |
Where{$hba -contains $_.Adapter} |
Group-Object -Property Device | %{

if( $_.Group.Count -ne 4 )
{
$row = "" | Select ESXihost, Lun, Datastore, NrPaths
$row.ESXihost = $esxvm.name
$row.NrPaths = $_.Group.Count
$row.Lun = $_.Name

$diskname = $_.Name
$datastore = get-datastore | Where-Object {$_.extensiondata.info.vmfs.extent.diskname -like "$diskname"}
$row.Datastore = $datastore.name

$report += $row
#Write-Host $row
$row
}
}
}

$Title = "Check LUNS have the recommended number of paths"
$Header = "LUNs not having the recommended number of paths ($RecLUNPaths): $(@($missingpaths).count)"
$Comments = "Not enough storage paths can effect storage availability in a FC SAN environment"
$Display = "Table"
$Author = "Gabrie van Zanten, Luc Dekens"
$PluginVersion = 1.0
$PluginCategory = "vSphere"

And to be complete, here is the vCheck plugin for death paths as well:


$deadluns = @()
foreach ($esxhost in ($HostsViews | where {$_.Runtime.ConnectionState -match "Connected|Maintenance"})) {
$esxhost | %{$_.config.storageDevice.multipathInfo.lun} | %{$_.path} | ?{$_.State -eq "Dead"} | %{
$myObj = "" | Select VMHost, Lunpath, State
$myObj.VMHost = $esxhost.Name
$myObj.Lunpath = $_.Name
$myObj.State = $_.state
$deadluns += $myObj
}
}

$deadluns

$Title = "Hosts Dead Lun Path"
$Header = "Dead LunPath : $(@($deadluns).count)"
$Comments = "Dead LUN Paths may cause issues with storage performance or be an indication of loss of redundancy"
$Display = "Table"
$Author = "Alan Renouf, Frederic Martin"
$PluginVersion = 1.1
$PluginCategory = "vSphere"