jklick's Posts

The best I've seen is the Performance Troubleshooting Guide. Its last iteration was for vSphere 4.1, but it is largely still accurate, particularly the mentioned thresholds. Anything you do not f... See more...
The best I've seen is the Performance Troubleshooting Guide. Its last iteration was for vSphere 4.1, but it is largely still accurate, particularly the mentioned thresholds. Anything you do not find in that guide (or that deviates from the guide) is found by personal experience or amongst the blogs of the many VMware experts out there.
Feedback: CPU Ready - I would not use a ms value for the threshold. The ms thresholds you have listed should get multiplied by the number of vCPUs allocated to the virtual machine. Using 5% a... See more...
Feedback: CPU Ready - I would not use a ms value for the threshold. The ms thresholds you have listed should get multiplied by the number of vCPUs allocated to the virtual machine. Using 5% and 10% is safer, but requires coverting from ms. Memory usage is synonomous with Consumed or Active, depending on the object type you're looking at. Memory usage for a VM is the percentage of allocated memory resources that are "active". Memory usage for a host or cluster is the percerntage of total configured memory that is "consumed". If you're measure disk latency from a VM, I think you're limited to a single metric that is the total roundtrip latency for a disk command. However, at the host level you can break it down to device latency, kernel latency, and queue latency. Device latency is your more typical form of latency and it measures the round trip between the host and the storage. However, kernel latency and queue latency are typically 0 in healthy environments. Anything greater than 0 can indicate a problem. At the VM level, you're only getting the total latency and can be missing part of the story if there is any kernel or queue latency mixed in. Hopefully this helps.
I'm not aware of any disadvantages from using categories/tags (yet). Here are some of the advantages that I'm aware of: Selectable Right now, you can put things like owner and department i... See more...
I'm not aware of any disadvantages from using categories/tags (yet). Here are some of the advantages that I'm aware of: Selectable Right now, you can put things like owner and department into an annotation. However, with tags, you could actually select a particular owner or department and bring up all virtual machines that belong. It's almost like an enhanced version of folders - without the 1:1 limitation. Flexible Categories and tags can be used for more than just virtual machines. As an example, you could group all your datastores into different tiers. Enforceable You can set rules to govern how categories and tags can be used. For example, you wouldn't want someone to have the ability to label a virtual machine as a tier 1 datastore. You can also define what tags are available for each category so that there's consistency in the metadata, whereas an open text field opens the doorway to misspellings, etc. Addtionally, I think tags can be used in conjunction with saved searches, basically giving you the ability to create "smart folders" of items that meet certain criteria and adapt as objects get added or deleted from the environment. Hopefully this has been useful.
This video should be a great starting point for understanding the key vCenter metrics. It tells what the key metrics are, what they mean, and how they'res used. Let me know if it helps or if you ... See more...
This video should be a great starting point for understanding the key vCenter metrics. It tells what the key metrics are, what they mean, and how they'res used. Let me know if it helps or if you have more questions. http://www.youtube.com/watch?v=dR5eyVghOmM
Thanks for the response, so it looks like its a "fractional CPU" maybe bad terminology, but still the right thing. In the example with 2 - 2.0 GHz CPUs and limiting to 3.0 GHz, we are effectivl... See more...
Thanks for the response, so it looks like its a "fractional CPU" maybe bad terminology, but still the right thing. In the example with 2 - 2.0 GHz CPUs and limiting to 3.0 GHz, we are effectivly specifying 1.5 CPUs usage. When you're managing CPU capacity for your hosts, there are two separate kinds: (1) the ratio of virtual CPUs (vCPUs) to physical CPUs (pCPUs) and (2) the amount of GHz available. The first affects the CPU scheduler on the host and helps determine if VMs will contend with each other for physical cores. Note that this sort of contention can (and does) take place even if the amount of GHz being consumed is very small. In the scenario where you have 2 vCPUs allocated with a 3 GHz limit, it would only be effectively 1.5 CPUs in the case of GHz capacity, but it would still be the equivalent of 2 CPUs when it comes to your vCPU:pCPU ratio. Otherwise, I'm not sure why you specify reason # 2 : If you have a VM that starts consuming all its allocated CPU and starts to impact the performance of other virtual machines. CPU limits can be created and removed instantly, without having to power off the VM. because you wouldn't allocate 2 CPUs if you did not intend for the VM to need and be able to use them. I would then expect to use SHARES to determine how the contention should be handled. For the second use case, people allocate more CPU than they need all the time (often by a significant amount); I'm betting your environment has situations like this. Heck, 1 vCPU is often to much for some applications, but it's the least you can do. In some cases, a VM may only need 100 MHz, but performs better with 2 vCPUs because it's a multi-threaded application. Now, let's look at an average scenario where you have 50 vCPUs allocated on a host that only has 24 cores (2x 6 core procs with hyperthreading). This is often a very safe and normal CPU ratio. However, if you have a VM on the same host with 4 vCPUs (no limit) that gets a runaway app causing the CPU usage to jump from 300 MHz to 8 GHz, suddenly 4 of your cores get tied up by that one app. To make matters worse, those 4 cores are hyperthreaded, so you've effectively reduced your number of cores by 8. To do some quick, on-the-fly damage control, you could slap a CPU limit on the VM, throttling it back so it isn't so much a "noisy neighbor" for the other VMs. In regards to shares, they'll do a good job at making sure your VMs are not starved for GHz. However, in 90-95% of cases (including the one above), GHz isn't the bottleneck - it's the vCPU:pCPU ratio that starts causing problems. You'll encounter the same thing with DRS. DRS would look at the host, see it has plenty of GHz available, and not pay attention to the vCPU:pCPU ratio. Hopefully, this helps. Let me know if you have more questions.
2. Although ESXI 4.1 allows you to limit the 1 - 2.5GHz vCPU to 4000GHz, it does not make any sense to put this limit in.     The limit would be only useful where you are tryiing to allow for f... See more...
2. Although ESXI 4.1 allows you to limit the 1 - 2.5GHz vCPU to 4000GHz, it does not make any sense to put this limit in.     The limit would be only useful where you are tryiing to allow for fractional vCPUs - like <1 or 1.5. ?? The first part is correct. Much like memory limits, they only affect the VM if the limit is set lower than the allocation. Otherwise, the allocation already "limits" the VM. I don't think the second part would work like that. Either a VM is using a core or it is not. For example, if you allocated two vCPUs and limited it to 3 GHz, it would still use 2 vCPUs regularly. However, if it attempted to use more than 3 GHz, then it would get limited in its consumption. Because of both these facts, there's only two potential use cases for CPU limits that I can think up at the moment: Testing purposes. If you want to see the impact of CPU contention (CPU ready) on a virtual machine, CPU limits are great for that. If you have a VM that starts consuming all its allocated CPU and starts to impact the performance of other virtual machines. CPU limits can be created and removed instantly, without having to power off the VM.
Besides CPU performance - which appears to be decent - what are you using seeing for memory usage, memory balloon, memory swap, and disk latency? (the complete performance profile)
What he said. Kit is probably the best guy to teach you about how memory works in ESX (and vC Ops) and I highly recommend his VMworld memory presentation(s). However, if you don't have the ... See more...
What he said. Kit is probably the best guy to teach you about how memory works in ESX (and vC Ops) and I highly recommend his VMworld memory presentation(s). However, if you don't have the 90 minutes to spare right now, I also wrote a short blog post to help address this question (because I hear it a lot). Hopefully, it helps: http://www.vkernel.com/reader/items/vsphere-memory-usage-metrics-task-manager Let us know if you have additional questions.
For additional emphasis, I have worked for a vendor for four years where every month I look at dozens of different customer sites. After looking at 100s of virtual environments, I count on seeing... See more...
For additional emphasis, I have worked for a vendor for four years where every month I look at dozens of different customer sites. After looking at 100s of virtual environments, I count on seeing these "stealth" memory limits plaguing at least a few virtual environments every week and it's a part of my regular demos. Admins are regularly stunned and confused when I show this to them and is often a source of performance issues. I'm seriously suprised that there hasn't been enough support tickets to warrant a witch hunt at any point in the past four years.
Yeah, I really don't know where you'd go to discuss alternatives to vC Ops either. Let me know if you find a better place. I work for a competitor (VKernel) of both vC Ops and Solarwinds, s... See more...
Yeah, I really don't know where you'd go to discuss alternatives to vC Ops either. Let me know if you find a better place. I work for a competitor (VKernel) of both vC Ops and Solarwinds, so we'll hope that someone better than me comes along. Until then, here's my feeble attempt at an objective analysis of Solarwinds Virtualization Manager that may help answer your questions: I believe they advertise they can scale to 10000+ VMs, so if you have a few thousand, I don't imagine that being an issue. You can create dashboards for just about anything you want. The downside is creating the dashboards (some people care about this, some don't). Admittedly, I really liked the default "All Alerts" dashboard item and found it useful. Does a decent job of giving problem descriptions and recommendations. Not perfect - I encountered situations where recommendations were not accurate or the root cause was not observed - but it certainly beats vCenter. The way they designed graphs for multiple objects/metrics was nice They won gold in the virtualization management category in 2011 at VMworld. Fairly inexpensive in comparison to most other third-party management tools. In spite of this brief assessment, there's several solutions out there that do virtualization management (*cough, cough* VKernel *cough*) and it really depends on how the strengths/weaknesses of each line up with what is most important to you. It sounds like ease-of-use and scalability are on the list. Do you have any other priorities in mind as you're hunting around? If not, here's a sampling of common use cases to get your wheels turning: Performance monitoring Capacity management VM sizing Dashboards Reporting Alerting Configuration management Cost visibility/showback/chargeback Automation I hope this helps.
A similar question was answered recently and I want to make sure you don't feel neglected. http://communities.vmware.com/thread/422633?tstart=0 Hopefully, this helps. If not, let us know... See more...
A similar question was answered recently and I want to make sure you don't feel neglected. http://communities.vmware.com/thread/422633?tstart=0 Hopefully, this helps. If not, let us know.
Hi Ravi, There are two different variables we need to look at: memory allocated and memory used. How much you can overallocate your host will be somewhat dependant on how accurately the virtua... See more...
Hi Ravi, There are two different variables we need to look at: memory allocated and memory used. How much you can overallocate your host will be somewhat dependant on how accurately the virtual machines are sized. As a couple of examples: In a situation where you consistently provision your VMs with more memory than they need, it's possible to allocate 200% of the physical memory, but never use more thatn 50% of it. This is an extreme, but possible. On the other hand, if your VMs with around what they need what they need, you could find yourself with a 90% allocation of physical resources and 85% utilization of those resources. It really comes down to what your VMs are actually using. To better understand how to measure this, I recently did a blog post that talks a little about the metrics involved: http://www.vkernel.com/reader/items/vsphere-memory-usage-metrics-task-manager Let us know if this helps.
^^ That could be an option. By default, 5 minutes statistics are only kept for 1 day. I think you can bump this up to something like 3 or 5 days, but that's it. I don't know if there's an advance... See more...
^^ That could be an option. By default, 5 minutes statistics are only kept for 1 day. I think you can bump this up to something like 3 or 5 days, but that's it. I don't know if there's an advanced option to make it longer, but it would certainly make the DB fairly large since it's retaining a whole lot more than just CPU and memory.
What are you trying to do with the data? Without additional information, I'm kind of partial to VKernel vOPS. Not only because I'm an employee, but because I've used it for the past four years... See more...
What are you trying to do with the data? Without additional information, I'm kind of partial to VKernel vOPS. Not only because I'm an employee, but because I've used it for the past four years and seen it work wonders. It can not only collect and store the data you want, but analyze it for you, export it to reports, etc. Give a 30-day trial a shot and let me know if it helps or if you have further questions.
Hmm... I think I just responded to you on another thread a moment ago: http://communities.vmware.com/message/2132867#2132867 Let us know if you further questions.
Darn... you beat me to it! I'm glad you figured it out though. Let me see if I can find another way to earn some points... :smileylaugh: Besides the lack of active swapping, there's one other ... See more...
Darn... you beat me to it! I'm glad you figured it out though. Let me see if I can find another way to earn some points... :smileylaugh: Besides the lack of active swapping, there's one other thing I noticed in that esxtop screenshot that I've seen several times before in other environments and will most likely result in some more swapping. Look at the MCTLSZ column. That is the amount of ballooning taking place on the respective virtual machines. In many cases, this is simply caused by memory contention at the hypervisor level. However, in some of the extreme cases I'm seeing, they're usually caused by a configured memory limit. Using the 7th VM in the list as an example, it has 4 GB allocated, 1.2 GB granted, 2.6 GB balooned, 200 MB swapped. My guess is that there's a configured memory limit right around the 1.2 GB mark. By default, a virtual machine is only allowed to balloon out 65% of it's allocated memory - in this case, 2.6 GB. Anything beyond that 65% is forced into host swapping, thus the small amount of swapping to compensate for the 200 MB gap between the configured limit and what ballooning was able to accomplish. As another example, I'm guessing the 5th VM in the list, has a limit configured around 512 MB. I would probably investigate many of the VMs in the list where you see current ballooning. Not to step on the toes of anyone who has mentioned vC Ops already, I highly recommend giving VKernel's vOPS a try. It's built to show you these sorts of issues right out of the box and explain what you should do to fix them, so feel free to abuse a 30-day trial and let it help you clean up your environment. Full disclosure: Since I mentioned a VKernel product in this post, I have to let everyone know I'm a VKernel employee or the powers that be can get cranky with me.
That's a very curious dilemma. What is your admission control configuration?
I highly recommend the aforementioned document to gain an overall better understanding of how memory management works. Hopefully, this blog post will help out as well: http://www.vkernel.com/r... See more...
I highly recommend the aforementioned document to gain an overall better understanding of how memory management works. Hopefully, this blog post will help out as well: http://www.vkernel.com/reader/items/vsphere-memory-usage-metrics-task-manager Using the blog post as a reference, memory active and memory consumed at the host level are mostly an aggregate of the active/consumed memory of VMs on the host; therefore, memory consumed is frequently going to overstate what the VMs need from the host and memory active is going to understate what is needed. Memory consumed is probably the safest of the two; however, you'll find that as you add more VMs, you'll miraculously find more memory as memory consumed gets de-duplicated via transparent page sharing (TPS) and idle/unused memory pages will get ballooned out of the virtual machines that no longer need them (VMs do not innately give back unused/idle memory to the host until there's a need and they're told to). When you begin to see consistent ballooning on a host, you you've most likely crossed a threshold for capacity where further memory consumption could degrade performance. In particular, there's a chance that consistent ballooning on a VM involves swapping inside the OS, so something to watch for. Let us know if this helps or if you have further questions.
Oh, you said "not" extremely CPU intensive. You wrote it correctly, I just misread it. Yeah, so I bet you would be able to get a decent vCPU:pCPU ratio without much worry. You will more than l... See more...
Oh, you said "not" extremely CPU intensive. You wrote it correctly, I just misread it. Yeah, so I bet you would be able to get a decent vCPU:pCPU ratio without much worry. You will more than likely run out of memory long before you run into CPU constraints. For 12 JVMs, you could do fine with one or two physical servers of that size. The reasons you might want two instead of one: To leverage HA (high availability) functionality for business continuity purposes. If you don't have/want this functionality, then don't worry about it. If these JVMs will ever require more than 6 GB of RAM a piece, which doesn't seem to be the case. If you want to perform maintenance on the host without downtime to the VMs, it would be nice to have a second host to temporarily move your VMs to. Otherwise, I'm thinking you'll be fine with 12 cores and 71 GB of RAM.
I didn't realize that 5.x added the ability to get VMs up to 16GB, but I just verfied it on the product page and tried playing around with it a little bit. If it's an existing VM, some good first... See more...
I didn't realize that 5.x added the ability to get VMs up to 16GB, but I just verfied it on the product page and tried playing around with it a little bit. If it's an existing VM, some good first steps are to: Go into VM settings and change the compatibility to Fusion 5 (hardware version 9). Power on or reboot the VM and go through the process of upgrading VMware tools. However, I'm still left with a VM that cannot be configured with more than 8GB for some reason. I even tried creating a fresh VM and no luck. Bug? Or are we missing something? No idea. In any case, the UI prevents us from configuring 16GB, but that doesn't stop us from altering the *.vmx file manually <insert evil, maniacal laughter>. Go to your virtual machine folder in Finder. Right-click and "Show package contents" Scroll down to the *.vmx file and open it with TextEdit. On the sixth line or so, it will have a variable for "memsize". Change this to exactly "16384". Save the edit you did. Power on the VM. You'll get a mean message saying that it's not supported, but ignore it (muahaha). Seems to work fine for me. I don't know why it's not possible via the UI, but hopefully this does the trick for you.