VMware Cloud Community
lch12mc
Contributor
Contributor

Failed to deploy VIO2.0 with NSX

I have setup NSX and trying to deploy VIO2.0 and integrate with NSX.

Deployment wizard was run without issue but error shown in the middle of the deployment.

2016-03-23_112729.png

Checking the log files on VIO management node, I found the below errors in ansible.log

2016-03-22 18:29:37,689 p=366 u=jarvis |  TASK: [config-controller | update neutron lbaas configuration] ****************

2016-03-22 18:29:38,024 p=366 u=jarvis |  changed: [172.31.17.157]

2016-03-22 18:29:38,039 p=366 u=jarvis |  changed: [172.31.17.158]

2016-03-22 18:29:38,054 p=366 u=jarvis |  TASK: [config-controller | initialize neutron database] ***********************

2016-03-22 18:31:20,877 p=366 u=jarvis |  failed: [172.31.17.157] => {"changed": true, "cmd": ["neutron-db-manage", "--config-file", "/etc/neutron/neutron.conf", "--config-file", "/etc/neutron/plugins/vmware/nsxv.ini", "upgrade", "head"], "delta": "0:01:42.597801", "end": "2016-03-22 10:31:20.839305", "rc": 1, "start": "2016-03-22 10:29:38.241504", "warnings": []}

2016-03-22 18:31:20,877 p=366 u=jarvis |  stderr: Traceback (most recent call last):

  File "/usr/bin/neutron-db-manage", line 10, in <module>

    sys.exit(main())

  File "/usr/lib/python2.7/dist-packages/neutron/db/migration/cli.py", line 238, in main

    CONF.command.func(config, CONF.command.name)

  File "/usr/lib/python2.7/dist-packages/neutron/db/migration/cli.py", line 105, in do_upgrade

    run_sanity_checks(config, revision)

  File "/usr/lib/python2.7/dist-packages/neutron/db/migration/cli.py", line 229, in run_sanity_checks

    script_dir.run_env()

  File "/usr/lib/python2.7/dist-packages/alembic/script.py", line 382, in run_env

    util.load_python_file(self.dir, 'env.py')

  File "/usr/lib/python2.7/dist-packages/alembic/util.py", line 241, in load_python_file

    module = load_module_py(module_id, path)

  File "/usr/lib/python2.7/dist-packages/alembic/compat.py", line 79, in load_module_py

    mod = imp.load_source(module_id, path, fp)

  File "/usr/lib/python2.7/dist-packages/neutron/db/migration/alembic_migrations/env.py", line 109, in <module>

    run_migrations_online()

  File "/usr/lib/python2.7/dist-packages/neutron/db/migration/alembic_migrations/env.py", line 90, in run_migrations_online

    engine = session.create_engine(neutron_config.database.connection)

  File "/usr/lib/python2.7/dist-packages/oslo_db/sqlalchemy/session.py", line 413, in create_engine

    _test_connection(engine, max_retries, retry_interval)

  File "/usr/lib/python2.7/dist-packages/oslo_db/sqlalchemy/session.py", line 591, in _test_connection

    six.reraise(type(de_ref), de_ref)

  File "<string>", line 2, in reraise

oslo_db.exception.DBConnectionError: (OperationalError) (2003, 'Can\'t connect to MySQL server on \'172.31.17.151\' (111 "Connection refused")') None None

2016-03-22 18:31:20,878 p=366 u=jarvis |  FATAL: all hosts have already failed -- aborting

I have checked the IP showed in the log "172.31.17.151" is one of the load balancer and I can connect to the IP through SSH without issue from the management node.

Would anyone could help on this issue? Thanks a lot.

Lewis

Reply
0 Kudos
6 Replies
admin
Immortal
Immortal

‌what are .157 and .158 VMs?

make sure those ip are not already under use.

in the same log file do you see any other errors related to those IPs?

Reply
0 Kudos
yangyuopenstack
VMware Employee
VMware Employee

Please try to command 'telnet 172.31.17.151 3306' to check if the mysql port is monitoring and IP address is not dup.

Reply
0 Kudos
lch12mc
Contributor
Contributor

.157 and .158 VM are the VIO Controller VM

Checked the two IPs are not in use by others.

And seems there is no other error related to these IPs

I have uploaded the full log files to below link

http://gofile.me/1UvHi/fCPog82O

Thanks

Reply
0 Kudos
lch12mc
Contributor
Contributor

I found 17.151 is quite strange that I have defined 17.151-17.168 as  management Network and 18.151-18.152 as API Network and 18.153 as Public Virtual IP.

But 17.151 is assigned to VIO-LoadBalancer-0 (from VM Summary) and can be PING but not SSH or telnet.

On VIO-LoadBalancer-0 NIC adapter, I can only see 2 interface which carry 17.163 and 18.151 but not 17.151.

Reply
0 Kudos
ZhangAdam
VMware Employee
VMware Employee

Did you deployed on this environment before, or have previous unsuccessful provision.

I think the /etc/hosts file get corrupted, it has invalid records.

You can delete the failed deployment and clean the /etc/hosts on management server,  try to deploy again.

We have fixed this by force refresh /etc/hosts right after the VM is created in VIO 2.5 release.

Reply
0 Kudos
lch12mc
Contributor
Contributor

Yes, I tried to deploy before and was unsuccessful.

I tried to clean up the /etc/hosts and deploy again though same error found ..

Reply
0 Kudos