VMware Cloud Community
MKguy
Virtuoso
Virtuoso

Converter Standalone Linux P2V fails to clone /boot

New thread from my post in http://communities.vmware.com/message/1341524#1341524.

I'm stuck with the issue of failed cloning of the /boot partition at 3%.

Source machine is a RHEL 3 box.

The DNS suffixes and servers are configured correctly in the converter task. I confirmed with tcpdump that the helper VM really does establish an SSH connection to the source machine. I also tried using only IP addresses in the task.

There is no firewall blocking communications between the converter server, the helper VM, source machine, vCenter or the ESX Host.

Note that I cloaked some specific information with stuff like or in those logs:

+All Users vmware-converter-server-2.log+

RecordOp ADD: event[35], task-1

HTTP Response: Complete (processed 6072 bytes)

Caught an exception while waiting on the agent task to complete. Gathering agent logs before propogating the exception further. Exception message: converter.fault.CloneFault

All Users vmware-converter-agent-3.log

Volume-based cloning --> updates, state: 1, percentage: 2, xfer rate (Bps): <unknown>

ConvertTask updates, state: 1, percentage: 3, xfer rate (Bps): <unknown>

RecordOp ASSIGN: info, task-1

HTTP Response: Complete (processed 1131 bytes)

updating on event (converter.event.UnixP2VVolumeCloningEvent) {

dynamicType = <unset>,

key = 2,

chainId = 1,

type = "info",

createdTime = "2009-08-19T13:14:32.617527Z",

userName = "",

fullMessage = <unset>,

hostName = "[IP_ADDRESS]",

sourceMountPoint = "/boot",

}

RecordOp ADD: event[8], task-1

User agent is 'VMware-client/4.0.0'

HTTP Response: Client: NeedsContentLength: false UnderstandsChunking: true CanKeepAlive: true (PresetContentLength -1)

Converter Task GetEvent(taskID=task-1)

HTTP Response: Complete (processed 976 bytes)

User agent is 'VMware-client/4.0.0'

HTTP Response: Client: NeedsContentLength: false UnderstandsChunking: true CanKeepAlive: true (PresetContentLength -1)

updating on event (converter.event.UnixP2VVolumeCloneFailedEvent) {

dynamicType = <unset>,

key = 3,

chainId = 1,

type = "error",

createdTime = "2009-08-19T13:14:37.654289Z",

userName = "",

fullMessage = <unset>,

hostName = "[IP_ADDRESS]",

sourceMountPoint = "/boot",

reason = (converter.fault.CloneFault) {

dynamicType = <unset>,

faultCause = (vmodl.MethodFault) null,

description = "'/usr/lib/vmware-converter/copyFileSystem.sh --sshClient /usr/lib/vmware-converter/ssh --user --host --port 22 --sourceMountPoint /boot --targetMountPoint /mnt/p2v-src-root/boot --sshConfigFile /usr/lib/vmware-converter/ssh.conf --sourceTarOption --sparse --useSudo' failed. Return code: 2; message:

/usr/lib/vmware-converter/ssh -z -F /usr/lib/vmware-converter/ssh.conf @[IP_ADDRESS] -p 22 "sudo tar --one-file-system --sparse -C /boot -cf - ." | tar --numeric-owner -C /mnt/p2v-src-root/boot -y -xf -

Warning: Permanently added '[IP_ADDRESS]' (RSA) to the list of known hosts.

xmalloc: zero size

tar: This does not look like a tar archive

tar: Error exit delayed from previous errors

",

msg = "",

},

}

RecordOp ADD: event[9], task-1

Converter Task GetEvent(taskID=task-1)

HTTP Response: Complete (processed 1853 bytes)

updating on taskInfo (converter.task.TaskInfo) {

dynamicType = <unset>,

key = "task-1",

task = 'converter.task.Task:task-1',

name = <unset>,

descriptionId = "",

userName = <unset>,

source = <unset>,

target = <unset>,

state = "error",

cancelled = false,

cancelable = true,

data = <unset>,

error = (converter.fault.CloneFault) {

dynamicType = <unset>,

faultCause = (vmodl.MethodFault) null,

description = "'/usr/lib/vmware-converter/copyFileSystem.sh --sshClient /usr/lib/vmware-converter/ssh --user --host --port 22 --sourceMountPoint /boot --targetMountPoint /mnt/p2v-src-root/boot --sshConfigFile /usr/lib/vmware-converter/ssh.conf --sourceTarOption --sparse --useSudo' failed. Return code: 2; message:

/usr/lib/vmware-converter/ssh -z -F /usr/lib/vmware-converter/ssh.conf @[IP_ADDRESS] -p 22 "sudo tar --one-file-system --sparse -C /boot -cf - ." | tar --numeric-owner -C /mnt/p2v-src-root/boot -y -xf -

Warning: Permanently added '[IP_ADDRESS]' (RSA) to the list of known hosts.

xmalloc: zero size

tar: This does not look like a tar archive

tar: Error exit delayed from previous errors

",

msg = "",

},

result = <unset>,

progress = 2,

estimatedTimeRemaining = <unset>,

transferRate = <unset>,

queueTime = "2009-08-19T13:12:35.448724Z",

startTime = "2009-08-19T13:12:35.448724Z",

completeTime = "2009-08-19T13:14:37.656567Z",

eventChainId = 1,

vcTask = <unset>,

logBundleInfo = (converter.DiagnosticManager.TaskLogBundleInfo) null,

}

User agent is 'VMware-client/4.0.0'

HTTP Response: Client: NeedsContentLength: false UnderstandsChunking: true CanKeepAlive: true (PresetContentLength -1)

Volume-based cloning --> updates, state: 4, percentage: 2, xfer rate (Bps): <unknown>

converter-gui-5.log

"2009-08-19T13:12:37.700078Z",

userName = "[VC_USERNAME]",

fullMessage = "Partitioning the target disks.",

},

(converter.event.UnixP2VDiskFormattingEvent) {

dynamicType = <unset>,

key = 32,

chainId = 23,

type = "info",

createdTime = "2009-08-19T13:12:49.825234Z",

userName = "[VC_USERNAME]",

fullMessage = "Formatting the target partitions.",

},

(converter.event.UnixP2VVolumeCloningEvent) {

dynamicType = <unset>,

key = 33,

chainId = 23,

type = "info",

createdTime = "2009-08-19T13:14:34.826578Z",

userName = "[VC_USERNAME]",

fullMessage = "Starting to clone the volume mounted on '/boot' from '[IP_ADDRESS]'.",

hostName = "[IP_ADDRESS]",

sourceMountPoint = "/boot",

},

(converter.event.UnixP2VVolumeCloneFailedEvent) {

dynamicType = <unset>,

key = 34,

chainId = 23,

type = "error",

createdTime = "2009-08-19T13:14:39.748516Z",

userName = "[VC_USERNAME]",

fullMessage = "Failed to clone the volume mounted on '/boot' from '[IP_ADDRESS]'.",

hostName = "[IP_ADDRESS]",

sourceMountPoint = "/boot",

reason = (converter.fault.CloneFault) {

dynamicType = <unset>,

faultCause = (vmodl.MethodFault) null,

description = "'/usr/lib/vmware-converter/copyFileSystem.sh --sshClient /usr/lib/vmware-converter/ssh --user --host --port 22 --sourceMountPoint /boot --targetMountPoint /mnt/p2v-src-root/boot --sshConfigFile /usr/lib/vmware-converter/ssh.conf --sourceTarOption --sparse --useSudo' failed. Return code: 2; message:

/usr/lib/vmware-converter/ssh -z -F /usr/lib/vmware-converter/ssh.conf @[IP_ADDRESS] -p 22 "sudo tar --one-file-system --sparse -C /boot -cf - ." | tar --numeric-owner -C /mnt/p2v-src-root/boot -y -xf -

Warning: Permanently added '[IP_ADDRESS]' (RSA) to the list of known hosts.

xmalloc: zero size

tar: This does not look like a tar archive

tar: Error exit delayed from previous errors

",

msg = "An error occurred during the conversion.",

},

},

(converter.event.VmRemovedEvent) {

dynamicType = <unset>,

key = 35,

chainId = 23,

type = "info",

createdTime = "2009-08-19T13:15:04.092577Z",

userName = "[VC_USERNAME]",

fullMessage = "Removed the virtual machine '[VM_NAME]'.",

vmName = "[VM_NAME]",

As I see it, it boils down to this line:

tar: This does not look like a tar archive

tar: Error exit delayed from previous errors

Any help on this?

@vmweathers

What exact commands am I supposed to run from another Linux machine to imitate the helper VM? I not entirely sure on what I'm supposed to run from the logs.

The issue in http://communities.vmware.com/message/1228109#1228109 seems not to be related.

What kind of sshd configuration issue do you have in mind?

-- http://alpacapowered.wordpress.com
Reply
0 Kudos
13 Replies
MKguy
Virtuoso
Virtuoso

Ok, so I ran the following from a system on the same network as the source machine, just like it is being logged by the converter:

ssh [USERNAME]@[IP_ADDRESS] "sudo tar --one-file-system --sparse -C /boot -cf - ." | tar --numeric-owner -C /tmp/test/ -y -xf - 

And I get:

tar: invalid option -- y

And the directory /tmp/test, to where the files should have been copied, is empty. I don't see the "This does not look like a tar archive" error though.

If I omit the -y, all files from /boot of the source are copied correctly to /tmp/test. What is this -y switch for? There is no man entry for -y.

Also, I just ran a successful conversion of a RHEL5 (the vMA VM as physical import) without the error above.

What's going wrong with the RHEL3 box and the -y switch?

-- http://alpacapowered.wordpress.com
Reply
0 Kudos
IamTHEvilONE
Immortal
Immortal

MKguy,

I think -y is suppose to auto answer Yes to any questions that may occur during the untar process. Like, are you sure you want to do this? = yes by default.



Regards,

EvilOne

VMware vExpert 2009

5441_5441.jpg

NOTE: If your problem or questions has been resolved, please mark this thread as answered and award points accordingly.

Reply
0 Kudos
vmweathers
Expert
Expert

hi MKguy. Glad to see you figured out what to do.

FYI, -y is a flag we added for passing the credentials through stdin to the ssh client binary. It should not matter that you don't have it in the version of tar you are using.

Can you please try the test from a VM on the same network as the target machine? (Ideally with the same IP address as the helper VM was using)

(If your question has been resolved please mark the answers as "Helpful" or "Correct".)

(If your question has been resolved please mark the answers as "Helpful" or "Correct".)
MKguy
Virtuoso
Virtuoso

Ok, my issue got resolved by using the ssh_config and sshd_config files from the RHEL5 VM that worked earlier already, commenting out some newer options that made the older sshd crap itself all over.

I haven't gotten around to check which option was actually responsible, but it really was the ssh config on the physical server as it seems.

I couldn't find a KB article or a note in the converter guide that there are certain requirements for the ssh config either, but anyways, the machine converted fine after that and is running without major issues in a VM.

-- http://alpacapowered.wordpress.com
Reply
0 Kudos
vmweathers
Expert
Expert

When you do get a chance I'd appreciate if you posted your findings. Maybe just do a "diff old-config new-config" and post the results, or just post the 2 config files.

Knowing which options are problematic would be helpful for others that might encounter this problem in the future.

(If your question has been resolved please mark the answers as "Helpful" or "Correct".)

(If your question has been resolved please mark the answers as "Helpful" or "Correct".)
Reply
0 Kudos
bacownik
Contributor
Contributor

I'm stuck with the same problem during P2V (xmalloc: zero size).

Could you please post your ssh_config and sshd_config?

Reply
0 Kudos
MKguy
Virtuoso
Virtuoso

Oh well, better late than never I guess.

This is the SSH config of an older RHEL2 system with openssh-3.1p1-21 which was successfully P2V'd with converter standalone.

Hope that helps.

-- http://alpacapowered.wordpress.com
Reply
0 Kudos
bacownik
Contributor
Contributor

I'm suprised (or not Smiley Happy )

IT WORKS with your settings!!

Best regards,

Bacownik

Reply
0 Kudos
Catlin
Contributor
Contributor

Pls check the DNS configuration of Help Virtual machine

Reply
0 Kudos
LBNL
Contributor
Contributor

I had the same issue, but I disagree with the gist of what has been said so far. The OP did say he checked SSH from the helper.  But other posts say it boils down to the "tar" with "-y" option causing the final error.  NO.  The final tar error is due to the failure in communication (ssh failure) from the VM Helper (running inside the new VM) trying to communicate back to the original physical machine.

In my case, before I could even start the migration process, I needed to modify /etc/ssh/sshd_config and /etc/hosts.allow.  In /etc/ssh/sshd_config, we had set PermitRootLogin no and generally required regular users to first ssh into the machine, then use su to become root.  So I needed to set PermitRootLogin yes so I could use root credentials for the P2V migration.  For /etc/hosts.allow, we typically restrict ssh access to particular client hosts to reduce the footprint.  So to get started, I added the client machine where I was running VMWare Converter, so I was able to launch the job and everything looked OK - until I hit 3% and the tar error terminated the whole thing.

The trick was to notice in the VM Helper configuration where I configured the job, I let it be "obtain IP address automatically" which uses DHCP - naturally since we have DHCP servers setup to make our life easy. But my /etc/hosts.allow was still configured to restrict ssh access to particular hosts, thereby blocking ssh access from the VM Helper that was using a DHCP address.  So the final answer for me was to add sshd : * to /etc/hosts.allow, followed by service sshd restart.  This allowed the VM Helper using a DHCP configuration, to ssh back to the original physical machine.  With the connection not failing, tar succeeded and the error was gone.

So bottom line, the more restrictive (and safe?) your ssh configuration is on the physical machine, the more likely you are to run into this issue.  Open it up to root ssh from any host and you're less likely to have this problem because the root ssh from the VM Helper will succeed.

Reply
0 Kudos
esxi1979
Expert
Expert

what you mean by "Can you please try the test from a VM on the same network as the target machine?  (Ideally with the same IP address as the helper VM was using)" ?

Reply
0 Kudos
esxi1979
Expert
Expert

I used them but got error

# /etc/init.d/sshd restart

Stopping sshd:                                             [  OK  ]

Starting sshd:/etc/ssh/sshd_config: line 1: Bad configuration option: penBSD:

/etc/ssh/sshd_config: terminating, 1 bad configuration options

                                                           [FAILED]

#

Reply
0 Kudos
esxi1979
Expert
Expert

fixed by commenting that line & yes that solutions worked for me as well !

Reply
0 Kudos