VMotion and RFC (Change Request) Debate

[Jason_Miles]

My organization has just finished deploying and upgrading a total estate of 41 ESX Hosts. The project was immense to say the least and began October 2007.

Today in a meeting with senior managment, it was decided that an RFC would need to raised for each and every manual, operated directed "hot migration" between the ESX hosts.

NOTE: This was decided before my own input. However, may I ask what the status quo is at other organizations.

-

---

From my technical awareness, in Vi3 all that is being moved is the switch port and the RAM of the server, "memory" during a migration or vmotion.

In the case of the switch port, the configs should be replicated.

In the case of the RAM, the vm should be transferred at the final stages from the memory of one host into the memory of another host.

-

---

OK, is it just me or is something not quite right here. Why would an RFC need raising for each and every vmotion attempt. I pointed out that under clustering and HA, DRS that vm now auto-move between hosts depending on resources available, etc... But that is seen as OK since a systems automation process, but it is not OK to vmotion without RFC if a systems administrator is nurturing and watching a vm pass from one host to the next. Odd.

I am looking for some responses here....what does VMware say, what do the partners say and is an RFC really necessary.

If I am to set the story straight I need to produce some official documentation....please let me know your comments and where I can find standard reports. It seems to me inherent in the technology buy-in for vmware esx and vmotion is the flexible estate management on offer which moves us from a 'box' orientated architecture to a 'service' orientated architecture--where the box becomes an appliance only distributing the service.

And furthermore, what kind of tracking system (CMDB) would I need to utilise to track/raise an RFC if vms are being vmotioned around constantly in the background by DRS Clusters. The task would be daunting to say the least.

[aguacero]

From my perspective, an RFC would not be truly needed if you are able to provide them some detail "log" showing when these VMotions took place. You may want to utilize a monitoring system hopefully it's customizable where you can in addition to the monitoring systems own default system, it can monitor "vmotion" instances (successful/failures/warnings, etc) with timestamps. RFC basically is wasting the value of HA/DRS. Do you have a RFC with a clustered firewall (Checkpoint, Sidewinders, etc) or Exchange cluster? Very unlikely you do or will find one in a "clustered" solution. I think RFC may be needed if you were migrating from Cluster 1 connected to SAN 1 to Cluster 2 connected to SAN 2. Other than that, don't see the value of RFC in this situation. RFC can be utilized for ESX upgrades (version upgrade, memory, internal components change, etc.)

If you found this information useful, please consider awarding points for "Correct" or "Helpful". Thanks!!!

[Effective]

Maybe a nice compromis might be to make an informational change request which means the right people get notified so it's traceable, but at least you don't have to wait for approvals from all over.

K.

[lamw]

Agreed with the previous post, an RFC should only be require, least in our environment when a change is made to a system. The VM is still in the same state it was when it was VMotion but its resources are being managed by another ESX Server. The location fo the VM is still on the same LUN and storage unit, it's IP and configuration are the same. Like you said, if you leverage DRS, every recommended migration would require approval, by the time you get it, either the resources have been freed up or your VM has taken an impact. Basically VMotion needs to be explained to your regulations group and once they by off on the technology, then you sould not be required to have an RFC for VMotion, this is a day to day operation and logs are tracked both in Virtual Center and ESX Hosts. Might be interesting to explore Storage VMotion and use the same concept to get that bought off while you're at it. Good luck

[dkfbp]

In our enviroment we have defined a vmotion as below level so we don't have to log as a standard or normal change request.

I understand that RFC is to give better system stability but you really need to get the "below level" stuff sorted out. Our drs cluster

has done more than 6000 vmotions without any problems. That pretty much tells how stable it is.

Best regards

Frank Brix Pedersen

[Subatomic]

All valid comments above. Here's my take on the situation.

Ifyour design encompasses VMotion/DRS then I'd say an RFC is not required. Your design document would explain the technolgy and how it is to apply in your environment. The exception when an RFC maybe is requires is for a scheduled host maintenance. Also note HA is something different and would have to be covered by some sort of incident report. If you treat VI3 as sort of an application, then VMotion is a normal/reasonable activity of the "application" - as per design.

Hope that helps.

[mcowger]

We also do not do a Change Control on vmotions. I will often let my internal customer know that I will move something around, just to be friendly, but its not a requirement.

--Matt

[Jason_Miles]

All these responses have been helpful and everyone will be rewarded in full. Thank you very much.

Just one final question, what is VMware's response on the issue. Is their an official response from VMware? Is an RFC required before manually migrating or vmotioning a vm between hosts?

[Texiwill · Accepted solution]

Hello,

Just to add my two cents..... Can vMotion's fail? Yes. So if they do fail will this cause downtime? Yes. However, in a worst case situation will you go for the RFC or do the work? Case in point. You get several warnings from the local hardware that there is imminent failure, will you not vMotion all the VMs to another host right then or will you wait for the RFC to come through... In the meantime the system has crashed and the 20-30 VMs running on it are no longer functioning.

Using vMotion to balance out systems is a VERY common practice. You will need to get some control over this and define which actions will require Doing before approval and those that require approval.

Another case.... A system is running a Host ragged because of its network or disk IO. Other VMs are not able to run properly. You notice this, will you need to do an RFC to move this one VM to a less used Host? Or wait for the RFC and all the complaints about performance to role in.

What happens if these or more serious issues happen in the dead of night. Will the RFC be required and have to wait until the morning when the approvers are available.

Now, if it was SVMotion, that is a different story.

Now in each of the cases I mentioned it is possible that a vMotion will fail for various reasons, SCSI Reservation Conflicts being the most common culprit. Yo will need a plan just in case a vMotion does fail.

My personal opinion is that vMotion is a common tool to solve important issues and while it should be audited (you can set up alarms to send email for each VM via VC) it is ludicrous to require a long set of approvals to make it happen. Definitely audit, and make notes in the comments that a vMotion took place but other than that, its a normal function of the administrator or DRS to make this happen.

I am not sure you will get an official VMware Response unless you ask them directly. The forums really do not illicit official responses. I would contact your VMware Sales Representative for assistance.

Best regards,

Edward L. Haletky

VMware Communities User Moderator

====

Author of the book 'VMWare ESX Server in the Enterprise: Planning and Securing Virtualization Servers', Copyright 2008 Pearson Education.

CIO Virtualization Blog: http://www.cio.com/blog/index/topic/168354

As well as the Virtualization Wiki at http://www.astroarch.com/wiki/index.php/Virtualization

[mcowger]

Why would vmware have a position on your internal change control strategies?

--Matt

[Jason_Miles]

OK, fair comment Matt. I think the core of asking the vmware question is one of looking for some kind of Official Stance.

I see now, having a day or two to think and digest, the issue is one of CONFIDENCE and my role is to build that CONFIDENCE. In this regard, there is no official CONFIDENCE level, but needs to built and established, on a per org level.

Also and a final note, no manual vmotioning would be attempted unless there was a incident/problem covering the move or the auto-mated background vmotioning discussed, so in all instances the so called "change" would be a known change.

I found this whitepaper on change titled: "Its all change virtually..." Quite a good read.

I will start awarding points now but will be difficult as all contributions have been helpful and good.