Holy crap that was a pain to fix...I've been figting with that for the past 12 hours to get it working, but it's finally going. I'll try and describe what I did, hopefully it'll make sense to you if you're a Linux guy at all.
Deploy the vApp
Interrupt the FIRST boot of the UI VM, and boot into single user mode
Move the vaos, vami-*, and vcops* services out of /etc/init.d into a backup folder someplace
Reboot the machine
It'll boot up, and use the blue-screen to configure your networking. Make sure to do IP as the very first thing, don't do gateway or anything else first.
Exit the network config app, then login to the console
Generate your SSH keys
Manually SCP them over to the other server's authorized_keys file
Manually SCP the other server's keys into the UI's authorized_keys file
Edit the /opt/vmware/etc/init.d/vaos file and comment out the two lines that delete and generate the SSL keys again (lines 343 & 344 in mine)
Move all the files from your backup folder (step 3) back into /etc/init.d
Reboot the machine
jbberry posted great solutions on this. I thought i could make it more clear to others who might not know exactly how to proceed with this solution.
1) deploy VCOPS vAPP and start it. Keep an eye when the UI vm will power on, as at this time you should interrupt the booting process by, pressing 'space' key for example. Grub will pause, and you will be able to edit the kernel parameters. Select the first option, and press "e"
Select second line and press "e" again
Type "1" and press enter. This will load the UI vm in single mode
It will ask you for the password...."vmware"
mkdir backup; mv /etc/init.d/vaos /etc/init.d/vami-* /etc/init.d/vcops* /root/backup
then reboot the UI vm
It should power on, and go to blue screen with some red background line that network should be configured. Configure network IP.
Login to shell, root/vmware
ssh-keygen -t rsa to generate new key save it in the default directory
then copy the public key to the analytics box
scp /root/.ssh/id_rsa.pub root@analystics-vm-ip:/root/.ssh/authorized_keys
then copy key from the analytics box
scp root@analystics-vm-ip:/root/.ssh/id_rsa.pub /root/.ssh/authorized_keys
you can press esc and type :set number
to see the line numbers, and go to the lines which jbberry mentioned, and hash 2 mentioned lines
press esc again and type :wq [enter]
And the last step
mv /root/backup/* /etc/init.d/
type exit to see the main menu, do other network settings if needed dns/gw/hostname etc.
reboot the box.
This worked for me , so once again thank you jbberry for your time to figure this out !!!
Other conclusion, this error should not appear if you have configured your network properly i assume. When i was trying to fix it i was deploying vcops several times and noticed that if i give for vapp proper network settings, so it can ping their gateway, this error is not showing up. I had a case where the portgroup where vcops was had wrong vlan, so it could not reach their gateway , but the vms could see eachother(here i had had to do this solution). When i put proper vlan number, vcops was installed without this error. Any ideas why this might be happening ?