P2V on RHEL 5.3: Unable to SSH to the source machine.

[johnlennon]

I've installed the Gold code of Converter 4.0 (client, server) on a local machine and I'm trying to P2V. I made sure SSH for root is enabled. The source is local, I enter its IP, username (root) and password, after about 10 minutes it tells me:

Unable to SSH to the source machine. Please check if a firewall is blocking access to the SSH daemon on the source machine.

There is no firewall active, I can SSH from the system using its IP directly as root. I did the same on another test system and it worked fine, however this is the production system I need to convert asap.

[vmweathers]

We use putty's plink utility to make the SSH connection from the Converter Server machine to the source machine. It is located in the installation directory of Converter. Well if it's installed on linux then it's in <install dir>/bin/.

Please try using plink manually and see if you can connect to the source. You can look in the converter agent logs (/var/log/vmware-vcenter-converter-standalone/vmware-converter-agent.log) for the command we use to make the connection.

The error you received means that plink received an ETIMEDOUT socket error when running on linux, or WSAETIMEDOUT error when running on windows. This is usually due to a firewall just dropping the requests.

As a guess maybe you are using a different port than 22 for the SSH daemon (e.g., doing thru a NAT/PAT gateway)? If so you can set that in the Converter GUI by appending ":<port>" to the IP.

[johnlennon]

Not sure if you missed the point that it's all installed on the same local system, accessing the same local system. The command it attempts is (IP masked):

/usr/lib/vmware-vcenter-converter-standalone/bin/plink -noprompt -stdin -nokeycheck -P 22 root@129.xxx.xxx.xx cat /tmp/somefile

I let it run for several minutes, it sits there with no output. I tried on the other system from which P2V worked (same kind of install, both server and client local) and I get the same behaviour: no output, just sits there. If I type something, I get a reply: "Access Denied" on both systems.

This is the vmware-converter-agent.log from the system that doesn't work:

cmdArgs[4]: 22

cmdArgs[5]: root@xxx.xxx.xxx.xx

cmdArgs[6]: cat /tmp/.vmware-sysinfo-BNYQGSDVXGQILXOH/vmware-sysinfo.log

while trying to execute "/usr/lib/vmware-vcenter-converter-standalone/bin/plink -noprompt -stdin -nokeych

eck -P 22 root@xxx.xxx.xxx.xx cat /tmp/.vmware-sysinfo-BNYQGSDVXGQILXOH/vmware-sysinfo.log" received error code (1) with result:

MQSI 6.0.0.5

/opt/ibm/mqsi/6.0

cat: /tmp/.vmware-sysinfo-BNYQGSDVXGQILXOH/vmware-sysinfo.log: No such file or directory

received an error code (1) from the ssh client that will be handled later

Converter Agent SysinfoQuery failed to grab log file; return code: 1; result:

MQSI 6.0.0.5

/opt/ibm/mqsi/6.0

cat: /tmp/.vmware-sysinfo-BNYQGSDVXGQILXOH/vmware-sysinfo.log: No such file or directory

Invoking /usr/lib/vmware-vcenter-converter-standalone/bin/plink with the following arguments:

cmdArgs[0]: -noprompt

cmdArgs[1]: -stdin

cmdArgs[2]: -nokeycheck

cmdArgs[3]: -P

cmdArgs[4]: 22

cmdArgs[5]: root@xxx.xxx.xxx.xx

cmdArgs[6]: rm -rf /tmp/.vmware-sysinfo-BNYQGSDVXGQILXOH/

Converter Agent SysinfoQuery deleted temporary files from source; result:

MQSI 6.0.0.5

/opt/ibm/mqsi/6.0

Sysinfo Query failed with error ssh connection timed out

VimConnectionStore stopping keepalive

Scheduled timer canceled, StopKeepAlive succeeds

CloseSession called for session id=520a1c28-cfef-b95f-b379-b990ff341251

Session is closing, releasing cached session objects

CloseSession called for session id=523a8d58-f97d-6180-e3b0-df126a5cf331

[vmweathers]

If you use the "-stdin" option then it expects the password on stdin. You have to type it explicitly. Please try the command both with and without this option. Also add '-v' to get verbose output.

[johnlennon]

Here you go:

<code>

Looking up host "xxx.xxx.xxx.xx"

Connecting to xxx.xxx.xxx.xx port 22

Server version: SSH-2.0-OpenSSH_3.9p1

We claim version: SSH-2.0-PuTTY_Local:_Oct__9_2008_09:40:18

Using SSH protocol version 2

Doing Diffie-Hellman group exchange

Doing Diffie-Hellman key exchange with hash SHA-1

Host key fingerprint is:

ssh-rsa 1024 35:c8:2f:d4:ac:66:0e:8f:d9:d1:43:df:c4:49:67:65

Initialised AES-256 SDCTR client->server encryption

Initialised HMAC-SHA1 client->server MAC algorithm

Initialised AES-256 SDCTR server->client encryption

Initialised HMAC-SHA1 server->client MAC algorithm

Using username "root".

Sent password

Access granted

Opened channel for session

Started a shell/command

MQSI 6.0.0.5

/opt/ibm/mqsi/6.0

Server sent command exit status 0

Disconnected: All channels closed

</code>

Thank you again for looking into this.

[vmweathers]

What command were you executing? This is the "local" machine connecting to itself, right? I don't know why it is showing stuff about the IBM MQSeries in the output, that was also present in all the commands in the partial logs you copied-and-pasted. That is highly suspicious.

I'd be interested in seeing the full log of a failing sysinfo-query (since the one you copied-and-pasted was not complete). I'll send you an email address to which you can send the logs if you are ok with it.

Do you have anything like bastille linux installed that protects the Linux machine?

[johnlennon]

Yes, it's local to local, as server and client are installed here. No firewall on this machine. If you send me an email I can send you full logs.

[vmweathers]

Ok, so the failure actually occurs at the uploading of the vmware-sysinfo tarball to the source machine:

Took too long to read from ssh client.

\[Converter Agent SysinfoQuery] while trying to execute "/usr/lib/vmware-vcenter-converter-standalone/bin/pscp -noprompt -stdin -nokeycheck -P 22 /usr/lib/vmware-vcenter-converter-standalone/pkg/vmware-sysinfo-lin32.tar.gz root@xxx.xxx.xxx.xxx:/tmp/.vmware-sysinfo-FOAYBNOVOCNZPZER/vmware-sysinfo-lin32.tar.gz" received error code (247) with result:

\[Converter Agent SysinfoQuery] ssh connection timed out

Can you please try using pscp yourself on this machine? Similarly please try a normal scp.

FYI, these "MQSI..." messages may prevent Converter from working, since we rely on parsing the XML output of this vmware-sysinfo binary. If these come at the front of the XML then we'll be fine, but if they are intermixed into the output it would cause a failure.

Either way, this MQSI stuff is suspicious, and could indeed be the cause of the failure of pscp. Maybe the ssh daemon is actually MQSI instead of openssh, and pscp is not allowed or even supported with this ssh daemon?

[johnlennon]

Hi,

first of all, scp works as that's how I got the log files from the server and how I put the install binaries there.

I ran the command you listed manually in verbose mode, this is the output:

Server version: SSH-2.0-OpenSSH_3.9p1

We claim version: SSH-2.0-PuTTY_Local:_Oct__9_2008_09:40:18

Using SSH protocol version 2

Doing Diffie-Hellman group exchange

Doing Diffie-Hellman key exchange with hash SHA-1

Host key fingerprint is:

ssh-rsa 1024 35:c8:2f:d4:ac:66:0e:8f:d9:d1:43:df:c4:49:67:65

Initialised AES-256 SDCTR client->server encryption

Initialised HMAC-SHA1 client->server MAC algorithm

Initialised AES-256 SDCTR server->client encryption

Initialised HMAC-SHA1 server->client MAC algorithm

Using username "root".

root@xxx.xxx.xxx.xxx's password:

Sent password

Access granted

Opened channel for session

Started a shell/command

Using SFTP

Connected to xxx.xxx.xxx.xxx

and it sits there....

Question: why do you use a different program than scp? It works just fine and it's always installed on Linux...

Thank you

[johnlennon]

I moved along a bit, I found that commenting out a line from /root/.bashrc would remove the message from a root SSH login. This allowed the file to be transferred and the migration started, I ran into some other issues but I found solutions to it, thank you so much for helping with this, I'm impressed about the fast turnaround

[vmweathers]

Would you be so kind as to tell me what the line is so that I can assist people in the future with the same kind of problem?

[vmweathers]

Question: why do you use a different program than scp? It works just fine and it's always installed on Linux...

2 reasons:

1. We want to keep our windows and linux code the same so we need something that works on linux and windows. Putty was the best choice since it gives us standalone binaries. Openssh would require cygwin on the source, which is not reasonable to require on all systems.

2. We need to be able to send the password into the ssh client, and doing this is non-trivial without making modifications to the ssh client. Again the cleanest choice was to modify the ssh client binaries to read the password from their stdin. So we cannot use the "always installed" scp and ssh binaries that linux has, since they do not have the option to send the password thru stdin.

[johnlennon]

I found a better solution: comment the last 4 lines of /opt/ibm/mqsi/6.0/bin/mqsiprofile:

#echo

#echo MQSI $MQSI_VERSION

#echo $MQSI_FILEPATH

#echo

[vmweathers]

Might be there for a reason with the IBM MQ Series software, that could possibly break stuff. But as a temporary measure to allow Converter to work this seems ok.

[sophearyat]

Yes, Do you have Sample configuration of it?