VMware Cloud Community
turnbulm
Contributor
Contributor

Failed to deploy template: Operation timed out

I have a VI3 installation with ESX on a Dell 2850 server connected to a NetApp iSCSI datastore. There is a template in the datastore that I'm trying to deploy.

Any time I try to deploy the template I get an error (after about 15 minutes) saying "Failed to deploy template: Operation timed out". There is still activity on the storage at the time this error occurs.

I know the deployment will take time on the storage side as it creates the disk, but surely this shouldn't cause a timeout?

Is there any way to either increase the timout, or something else to make this work?

Thanks!

0 Kudos
49 Replies
raj2989
Contributor
Contributor

I also facing same problem as described above i have IBM x346 server .

Doesn't anybody comeup with the kind of a problem.

Does anyone know how it to be resolved.

Thanx in advance

0 Kudos
jshelly
Enthusiast
Enthusiast

Same problem here.

Anyone else?

0 Kudos
jshelly
Enthusiast
Enthusiast

Wanted to add in the log the error is

"unexpected exception during NfcCopy

0 Kudos
mstanger
Contributor
Contributor

I'm having the same problem. I'm using a locally attached storage array. I also get this error when attempting to make more than one clone of a virtual machine at a time. The status on the task simply sits at "In Progress" for the entire 15 minutes.

0 Kudos
Jamin
Contributor
Contributor

Same deal - DL385 with local storage.

0 Kudos
mstanger
Contributor
Contributor

I've got mine working now! I traced the problem down to my RAID configuration on my locally attached storage. The problem only happened when I tried to deploy the new virtual machine to that particular array. I had set up a RAID 1E0 array-which I think I did incorrectly. I have rebuilt my array now in a RAID 10 configuration with 2 hot spares. I've successfully deployed 3 vm's simultaneously in just under 30 minutes. Hopefully this will help some of you get your problems fixed. To troubleshoot, you may try deploying the vm's to a separate storage location and see if it works that way. The one thing I'm still not sure about is speed. I thought that I would be able to deploy the virtual machines much faster than I am able to.

0 Kudos
Jamin
Contributor
Contributor

thanks for the ray of hope - but no joy still. I have 2 arrays - RAID 5 for the VC and template library (faster read) and RAID 1+0 for the "active vm's". I had been trying to deploy the template from the RAID 5 to the RAID 10, and that was failing. I've just invested another 15 minutes and tried to deploy to the same RAID 5 as the template. No luck - same error.

0 Kudos
turnbulm
Contributor
Contributor

I've just tried this again in the same setup, but using a much smaller template (4GB drive only, instead of 30GB). This time it worked fine.

Still not a great solution, as I need a bigger drive than that in our standard server build, but maybe it points towards some kind of disk timeout issue that's causing the problem?

0 Kudos
eableson
Contributor
Contributor

Well - adding another voice to the choir. I have exactly the same issue with a HP DL360, getting data from a pair of IET iSCSI arrays. Interestingly my first deployment went without a hitch, although the VIC just stated "in progress" until it hit about 85% completion.

I'm thinking that there's a fixed timeout value somewhere that ignore the default server setting of 120 minutes timeout for long operations.

A couple of tidbits from my event sequence :

Deployed a machine properly

Started multiple deployments afterwards - each of these timed out

Subsequent deployment attempts time out

I'm going to reboot and see if that resets the issue, and then avoid multiple concurrent deployments in the future.

Erik

0 Kudos
Jamin
Contributor
Contributor

My issue was resolved by support. Strike another chord for issues with the new licensing model.

For the 3rd time, I re-created the licenses. I stopped the license server and VC, reapplied the licensing, and I can now deploy VM's from the template.

However, it's slow. I found in VI2 (ESX 2.5.2 and VC 1.2?) I could deploy VM's from the RAID 5 array to the RAID 1 mirror on my DL385 in 10 minutes. With VI3, it takes 21 minutes.

0 Kudos
nzsteve
Enthusiast
Enthusiast

Hi,

I was having the same problem. Looks like a time out due to slow disk IO to me.

Im just playing on a test box. Only local storage, 2 x 146 SCSI as RAID1 on a LSI megaRAID controller. All partitions are installed on this single RAID volume, hence the template is on the same VMFS as where I am deploying it to.

I dont have a battery module on the controller, so by default caching is turned off on it. Deploying a template with a 10GB disk I'd hit disk average of around 22000KBbs, and the operation timed out.

I turned caching on in the controller setup and the disk average goes up to 9000KBps, and the deployment completes without error.

Looks to me that like VC is assuming fast disk IO and timing things out as broken even though the op is still running.

Steve

Message was edited by:

nzsteve

0 Kudos
raj2989
Contributor
Contributor

Turnblum

i got a solution to this check out if it could work for you also,

Connect to ESX Server through VI Client -> goto Configuration menu-->

Advanced Settings --> goto SCSI -->

first option SCSI reaborttimeout - change this value form 5000 to 50000

Second option SCSI scantimeout - change this value from 1000 to 10000

0 Kudos
mstanger
Contributor
Contributor

I tried changing my RAID configuration back to a RAID 5EE array and I'm back to the same problem. A RAID 10 array seems to work fine, but I need more disk space than I can get with a RAID 10. I've tried changing the SCSI options in the "Advanced Settings" configuration section in the VI client to no avail. I've also changed my disk cache settings, but that didn't help either. The operation still times out after 15 minutes. I can see that the clone process copies some of the files (including the large vmdk file), but for whatever reason it fails to actually create the VM.

0 Kudos
JP76
Enthusiast
Enthusiast

Same problem here.

I can create VMs from scratch but no clone, no deploy from template.

I have 3 SCSI disks, local storage, no redundancy, connected to a SmartArray in my test system.

I think the problem started around RC2 (I was beta tester). Prior builds were fine, I did a hole bunch of VMs from templates (including 2 or 3 at the same time) then.

I initially thought it was an upgrade problem or traces from earlier beta but after fresh reinstall will format of all drives I then dig the forums and I post where I belong... I get both messages: either the Operation Timed Out, or the Copy Failed (but in fact in both case, the copy operation never starts).

I will try the SCSI parameter and post results. It is definitly a change that occured since RC1...

0 Kudos
JP76
Enthusiast
Enthusiast

OK,

The SCSI parameters did not help.

I decided to rebuild the hole ESX (but not the VC2 Server) after removing it from inventory.

I also decided at last to install a DNS server on my VC Server to handle a mylab.scrap DNS domain and a reverse lookup zone. I made sure both the VC and ESX Servers were registered in the DNS and cleaned all hosts file entries I had used previously. At last I pointed the VC server to itself, regarding Primary DNS client.

I gave the correct FQDN to the ESX Server at installation (esx3tst.mylab.scrap) and point its DNS clients to my VC/DNS server.

For the license, I connected directly to the ESX Server (not VC) andadded a license, using a lic. server and connected to 27000@myvc2.mylab.scrap. I could get my lic. immediatly.

Then I added my ESX 3 to VC2 using the FQDN of the ESX Server (esx3tst.mylab.scrap). From there, I had a very snappy environment and could clone VMs (including several clones at the same time) without a glitch.

So, can you report your DNS configuration of your VC and ESX to see if it helps you too?

0 Kudos
BasisBoy
Contributor
Contributor

I had the same Problem with our VI3 install. After 15 minutes I would get the "Failed to deploy template: Operation timed out".

I found out the other day I could deploy the Template as long as I didn't customize. If I selected the customizing options during the process it would fail after 15 minutes.

Turns out the VMware had changed the spot that you need to place sysprep tools on the Virtual Center Server. It used to be under "C:\Program Files\VMware\VMware VirtualCenter\resources\windows\sysprep". Now its located at "C:\Documents and Settings\All Users\Application Data\VMware\VMware VirtualCenter\sysprep"

After changing the location of these files I was able to delopy a template using Customization with no problems.

See page 325 on the following guide for instructions on how to set it up correctly. http://www.vmware.com/pdf/vi3_admin_guide.pdf

"Don't forget to award Helpful and Correct Answers"

0 Kudos
jfransella
Contributor
Contributor

I'm getting the same timeout problem, but it happens intermittently. It doesn't happen if I don't customize, but it only happens sometimes when i do customize. I can customize a template with no problem, then try to deploy it again later with the same customization and it will fail.

I have the sys prep tools located in the folder per the setup manual.

0 Kudos
Dr_Hofmann
Contributor
Contributor

Same here. We are using x445 with a 4700 SAN. The templete I´m trying to deploy is 10GB. It´s take very long time and when it reached 61% I got time out.

0 Kudos
Dr_Hofmann
Contributor
Contributor

Same here. We are using x445 with a 4700 SAN. The templete I´m trying to deploy is 10GB. It´s take very long time and when it reached 61% I got time out.

0 Kudos