Script to display NFS Stats per-VMDK

[admin]

I have written a small python script to parse vscsiStats histograms and display the data like the esxtop VM (v) screen. Posting the script here since some other people may find it useful.

Jeremy

[markymark007]

Could somebody walk me through the steps to run this script.....? Please go real slow....

Thank :smileyshocked:

[titaniumlegs]

Sure!

(I'm typing this slowly, per your request.)

1. Download/copy the script to some place on your ESX server where the command path will find it. I typically use /usr/local/bin. There are a couple ways you can ghet it there, but if firewalls (ESX or otherwise) aren't blocking, you can use lwp-download

# lwp-download http://communities.vmware.com/servlet/JiveServlet/download/1511493-37051/nfstop /usr/local/bin

2. Make it executable

# chmod 755 /usr/local/bin/nfstop

3. Run it

# nfstop

4. Enjoy seeing your VM activity, regardless of the datastore in which they reside. And there was much rejoicing.

5. To quit, [c].

6. As protoburger and gicti pointed out, you should ensure vscsiStats isn't still collecting data

# /usr/lib/vmware/bin/vscsiStats -x

Share and enjoy!

[markymark007]

Why thank you sir!!

Would this be a firewall issue?

# lwp-download http://communities.vmware.com/servlet/JiveServlet/download/1511493-37051/nfstop /usr/local/bin

lwp-download: 500 Can't connect to communities.vmware.com:80 (connect: Connection refused)

[titaniumlegs]

Yes sir. Try this:

esxcfg-firewall -o 80,tcp,out,http

[dmalcor]

Thanks! This script got me over a really bad performance problem.

I have been attempting to bend it into a continuous logger so we can have some historical data as well as "IOPS Top Talker" alerts. I have been learning just enough python to be dangerous in order to remove the screen refresh and add time information to each line (then I just output to a dated centralized log file for each ESX server every ten minutes and sort and report on the logs). The problem is that this logger does not seem to react very well to an VM powering off and moving to another ESX server.

I can see this developing into a major programming project where the real answer is that VMware needs to get busy and help everyone who listened to them about how cool NFS was as a datastore. Does anyone see any hope of VMware recognizing NFS IOPS as Disk IOPS anytime soon?

:-Dan

[fletch00]

I am currently out of the office

Checking email only occasionally

[titaniumlegs]

They got busy! They are helping. Yes, there is hope!

Just wait a teeny bit longer....

(That's all - possibly more than - I'm allowed to say about that!)

[Tentakel777]

I have added a function to automatically stop the data collection of vscsiStats after exiting the script with ctrl+c

Just add this lines at the position where the ctrl+c command is handled: (Search for "Interrupted" within the script)

except KeyboardInterrupt:

print 'Interrupted!'

print ' '

print 'Stopping vscsiStats data collection...'

print ' '

os.system("/usr/lib/vmware/bin/vscsiStats -x")

print ' '

print 'vscsiStats data collection stopped.'

return

Maybe you have to modify the path to vscsiStats, in my case (ESX 3.5 U5a) the command is in the path as specified above.

Regards

[fletch00]

Anyone tried vSphere U2 esxtop with NFS stats improvements yet?

I hit 'u' and it now shows the stats for my NFS volume, but how do I expand it to show the per VM NFS stats?

When I hit 'e' (expand) and type in the volname as shown, it does nothing to the display

How do we get the per VM NFS stats (or is this not available in the new esxtop)?

thanks

VCP4

[Tentakel777]

Sorry for this, please delete this post

[Craigh365]

Good day all,

Preface: I am a newbie to unix scripting and light in *nix in general. This is also a TEST/DEMO environment.

I was able to get things sorted and see the script.

THANKS THANKS - we have seen some write latency with our NFS store and had no idea how to manage what the issue was. ( least I know which VM now)

My questions

1. Are we able to concatenate the name when there are spaces in the name of the VMDK?

2. Is there a good way to indicate which of the VMDK files is the culprit when a VM has multiple? ( currently they all show the same name)

3. I tried correlating the GID with something useful and only noted this: GID = (VMID - 1)

My situation is this. ( information altered for sake of ease of reading)

We name most images a with spaces and the vmdks followed suit.

TEST server 1

TEST server 2

show as

TEST

TEST

So I tried to identify them using the GID.

I did a vmware-cmd -l | grep SQL

and got /vmfs/volumes/.../TEST SQL/TEST SQL.vmx

so I did a vm-support -x | grep SQL

and got

vmid= 4444 TEST SQL

followed by vm-support -x | grep 4444

and it returns

vmid=4444 TEST SQL

However, in nfstop it looks like

4443 TEST

4443 TEST

4443 TEST

As mentioned, I have been able to sort things out, but knowing the exact VMDK file when there are multiple files.

It looks as though we could get this from the HANDLE info?

update:

I saw that in ESXTOP the vmware-vmx ID actually does correlate with the GID now, odd that it doesn't from vm-support as ESXTOP doesnt show 4444 at all.

[Tentakel777]

Sorry for this, please delete this post

[nmid]

Hi All

im very keen to get this working and have already found that editing the script in a windows app is a bad idea as you end up with Ctrl characters all over the shop (here is a link to assist with that - )

i have finally got it running however I have 3 questions that somebody may be able to help with:

1) i run nfstop and it shows all VM's on that hosts however after a few seconds it returns to the # prompt and doesnt continuously cycle like esxtop

2) The stats that it does bring up are all either Zero or -0.20 on all hosts and I know my VM's are not that quiet... (especially as im running this tool as im suspecting Disk latency issues on my NFS datastores) - see attached

3) Is there a simple guide anywhere to interpret what each column is and how that affects your NFS performance to assist with troubleshooting

Any help is appreciated

Many Thanks

ESX 4.0U1

[Tentakel777]

Sorry for this, please delete this post

[nmid]

Hi Tentakel777

is it possible you can translate to English? unfortunately my German is not so good

Thank You

[DSTAVERT]

The post is just an out of office reply.

[Tentakel777]

Sorry for the spam, how can i prevent that my out of office-reply is automatically been added as a reply in this thread ?

[dmalcor]

Titaniumlegs,

Can you say any more now? I guess I'm looking to see if the overall "NFS Stats per-VMDK" problem has been solved yet and there was no fanfare....

Is this possible with ESXi (where I cannot cron a script to collect NFS stats).

Thanks

[titaniumlegs]

Whoops, sorry I missed this question.  Hopefully you got your answer from somebody paying more attention!

Yes, in vSphere 4.1 (ESX and ESXi) there are counters built-in to vCenter for NFS datastores and VMDKs on NFS datastores.