VMware Cloud Community
yedongcan
Contributor
Contributor

NSX Manager return 400 “Invalid SecurityCertificate”

I had deployed a environment with NSX + vSphere + OpenStack. I'm not use VOVA, OpenStack is deployed by RDO.

I have modify neutron configuration and nsx.ini in controller node,  nova.conf in nova-compute node.

When I use OpenStack dashboard to create a vm, in neutron-server.log:

NSX Manager return 400, I checked code, this means  Invalid SecurityCertificate.

What's reason for this?

what does this error info mean, Is it caused by security group of port?

Server Error Message: LogicalSwitchPortConfig.security_profiles.$item.0: LogicalSwitchPortConfig.security_profiles.$arrayitems: must be an RFC 4122 UUID

Errror log paste here:

2015-01-13 08:49:46.712 15498 ERROR neutron.plugins.vmware.api_client.client [req-12735c8d-74fa-46ba-8f48-a692cd5596de None] Received error code: 400

2015-01-13 08:49:46.713 15498 ERROR neutron.plugins.vmware.api_client.client [req-12735c8d-74fa-46ba-8f48-a692cd5596de None] Server Error Message: LogicalSwitchPortConfig.security_profiles.$item.0: LogicalSwitchPortConfig.security_profiles.$arrayitems: must be an RFC 4122 UUID

2015-01-13 08:49:46.716 15498 ERROR NeutronPlugin [-] An exception occurred while creating the neutron port 52321fbe-227d-4bea-885c-12a140151785 on the NSX plaform

2015-01-13 08:49:46.716 15498 TRACE NeutronPlugin Traceback (most recent call last):

2015-01-13 08:49:46.716 15498 TRACE NeutronPlugin   File "/usr/lib/python2.6/site-packages/neutron/plugins/vmware/plugins/base.py", line 453, in _nsx_create_port

2015-01-13 08:49:46.716 15498 TRACE NeutronPlugin     True)

2015-01-13 08:49:46.716 15498 TRACE NeutronPlugin   File "/usr/lib/python2.6/site-packages/neutron/plugins/vmware/plugins/base.py", line 414, in _nsx_create_port_helper

2015-01-13 08:49:46.716 15498 TRACE NeutronPlugin     port_data.get(addr_pair.ADDRESS_PAIRS))

2015-01-13 08:49:46.716 15498 TRACE NeutronPlugin   File "/usr/lib/python2.6/site-packages/neutron/plugins/vmware/nsxlib/switch.py", line 351, in create_lport

2015-01-13 08:49:46.716 15498 TRACE NeutronPlugin     cluster=cluster)

2015-01-13 08:49:46.716 15498 TRACE NeutronPlugin   File "/usr/lib/python2.6/site-packages/neutron/plugins/vmware/nsxlib/__init__.py", line 96, in do_request

2015-01-13 08:49:46.716 15498 TRACE NeutronPlugin     res = cluster.api_client.request(*args)

2015-01-13 08:49:46.716 15498 TRACE NeutronPlugin   File "/usr/lib/python2.6/site-packages/neutron/plugins/vmware/api_client/client.py", line 119, in request

2015-01-13 08:49:46.716 15498 TRACE NeutronPlugin     exception.ERROR_MAPPINGS[status](response)

2015-01-13 08:49:46.716 15498 TRACE NeutronPlugin   File "/usr/lib/python2.6/site-packages/neutron/plugins/vmware/api_client/exception.py", line 83, in fourZeroZero

2015-01-13 08:49:46.716 15498 TRACE NeutronPlugin     raise BadRequest()

2015-01-13 08:49:46.716 15498 TRACE NeutronPlugin BadRequest: The server is unable to fulfill the request due to a bad syntax

2015-01-13 08:49:46.716 15498 TRACE NeutronPlugin

2015-01-13 08:49:46.717 15498 ERROR NeutronPlugin [-] Unable to create port or set port attachment in NSX.

2015-01-13 08:49:46.727 15498 ERROR neutron.api.v2.resource [req-12735c8d-74fa-46ba-8f48-a692cd5596de None] create failed

2015-01-13 08:49:46.727 15498 TRACE neutron.api.v2.resource Traceback (most recent call last):

2015-01-13 08:49:46.727 15498 TRACE neutron.api.v2.resource   File "/usr/lib/python2.6/site-packages/neutron/api/v2/resource.py", line 87, in resource

2015-01-13 08:49:46.727 15498 TRACE neutron.api.v2.resource     result = method(request=request, **args)

2015-01-13 08:49:46.727 15498 TRACE neutron.api.v2.resource   File "/usr/lib/python2.6/site-packages/neutron/api/v2/base.py", line 448, in create

2015-01-13 08:49:46.727 15498 TRACE neutron.api.v2.resource     obj = obj_creator(request.context, **kwargs)

2015-01-13 08:49:46.727 15498 TRACE neutron.api.v2.resource   File "/usr/lib/python2.6/site-packages/neutron/plugins/vmware/plugins/base.py", line 1206, in create_port

2015-01-13 08:49:46.727 15498 TRACE neutron.api.v2.resource     self._delete_port(context, neutron_port_id)

2015-01-13 08:49:46.727 15498 TRACE neutron.api.v2.resource   File "/usr/lib/python2.6/site-packages/neutron/openstack/common/excutils.py", line 82, in __exit__

2015-01-13 08:49:46.727 15498 TRACE neutron.api.v2.resource     six.reraise(self.type_, self.value, self.tb)

2015-01-13 08:49:46.727 15498 TRACE neutron.api.v2.resource   File "/usr/lib/python2.6/site-packages/neutron/plugins/vmware/plugins/base.py", line 1190, in create_port

2015-01-13 08:49:46.727 15498 TRACE neutron.api.v2.resource     port_create_func(context, port_data)

2015-01-13 08:49:46.727 15498 TRACE neutron.api.v2.resource   File "/usr/lib/python2.6/site-packages/neutron/plugins/vmware/plugins/base.py", line 468, in _nsx_create_port

2015-01-13 08:49:46.727 15498 TRACE neutron.api.v2.resource     lport and lport['uuid'])

2015-01-13 08:49:46.727 15498 TRACE neutron.api.v2.resource   File "/usr/lib/python2.6/site-packages/neutron/plugins/vmware/plugins/base.py", line 430, in _handle_create_port_exception

2015-01-13 08:49:46.727 15498 TRACE neutron.api.v2.resource     LOG.exception(msg)

2015-01-13 08:49:46.727 15498 TRACE neutron.api.v2.resource   File "/usr/lib/python2.6/site-packages/neutron/openstack/common/excutils.py", line 82, in __exit__

2015-01-13 08:49:46.727 15498 TRACE neutron.api.v2.resource     six.reraise(self.type_, self.value, self.tb)

2015-01-13 08:49:46.727 15498 TRACE neutron.api.v2.resource   File "/usr/lib/python2.6/site-packages/neutron/plugins/vmware/plugins/base.py", line 453, in _nsx_create_port

2015-01-13 08:49:46.727 15498 TRACE neutron.api.v2.resource     True)

2015-01-13 08:49:46.727 15498 TRACE neutron.api.v2.resource   File "/usr/lib/python2.6/site-packages/neutron/plugins/vmware/plugins/base.py", line 414, in _nsx_create_port_helper

2015-01-13 08:49:46.727 15498 TRACE neutron.api.v2.resource     port_data.get(addr_pair.ADDRESS_PAIRS))

2015-01-13 08:49:46.727 15498 TRACE neutron.api.v2.resource   File "/usr/lib/python2.6/site-packages/neutron/plugins/vmware/nsxlib/switch.py", line 351, in create_lport

2015-01-13 08:49:46.727 15498 TRACE neutron.api.v2.resource     cluster=cluster)

2015-01-13 08:49:46.727 15498 TRACE neutron.api.v2.resource   File "/usr/lib/python2.6/site-packages/neutron/plugins/vmware/nsxlib/__init__.py", line 96, in do_request

2015-01-13 08:49:46.727 15498 TRACE neutron.api.v2.resource     res = cluster.api_client.request(*args)

2015-01-13 08:49:46.727 15498 TRACE neutron.api.v2.resource   File "/usr/lib/python2.6/site-packages/neutron/plugins/vmware/api_client/client.py", line 119, in request

2015-01-13 08:49:46.727 15498 TRACE neutron.api.v2.resource     exception.ERROR_MAPPINGS[status](response)

2015-01-13 08:49:46.727 15498 TRACE neutron.api.v2.resource   File "/usr/lib/python2.6/site-packages/neutron/plugins/vmware/api_client/exception.py", line 83, in fourZeroZero

2015-01-13 08:49:46.727 15498 TRACE neutron.api.v2.resource     raise BadRequest()

2015-01-13 08:49:46.727 15498 TRACE neutron.api.v2.resource BadRequest: The server is unable to fulfill the request due to a bad syntax

2015-01-13 08:49:46.727 15498 TRACE neutron.api.v2.resource

Tags (3)
Reply
0 Kudos
8 Replies
yedongcan
Contributor
Contributor

From danwendlandt:

Note, this type of question should really be a 'Discussion' thread (OpenStack  ), rather than a 'Document', but here are a few things to look at:

I'm assuming you are using NSX-Multihypervisor, rather than NSX-vSphere in this setup.  RDO's NSX plugin will only work with NSX-multihypervisor.

- 400 is more general than invalid security certificate.  It generally just means "bad request".

- Above you said that you pointed Neutron to the "NSX Manager".  If you are using NSX-multihypervisor, you should be pointing the plugin to one of the NSX controllers, as the NSX-MH "manager" is just a web gui.  If you are using NSX-vSphere, as I mentioned, RDO does not have a plugin for NSX-vSphere.

Reply
0 Kudos
yedongcan
Contributor
Contributor

Thank you danwendlandt,

I'm only using ESXi as hypervisor.

I had installed openstack-neutron-vmware from RDO.

And in Neutron nsx.ini, I had configured the connection with the NSX controller.

Reply
0 Kudos
admin
Immortal
Immortal

Even if you are just using ESXi, there are actually two versions of NSX that work with ESXi:

- NSX-vSphere (built on top of the former vCNS platform)

- NSX-multihypervisor (built on top of the former Nicira platform)

Which of these two are you using?  Only the latter will work with the Neutron plugin contained in RDO.

Dan

Reply
0 Kudos
yedongcan
Contributor
Contributor

I'm using NSX-multihypervisor.

Reply
0 Kudos
admin
Immortal
Immortal

Ah, Ok.  It looks like the detailed message from NSX is actually buried higher in the stack trace:

ERROR neutron.plugins.vmware.api_client.client [req-12735c8d-74fa-46ba-8f48-a692cd5596de None] Server Error Message: LogicalSwitchPortConfig.security_profiles.$item.0: LogicalSwitchPortConfig.security_profiles.$arrayitems: must be an RFC 4122 UUID 


I'll ping someone on the dev team to see if they have a suggestion.  Btw, can you send me a private message indicating which VMware account team you are working with so that we can pull in the right folks?

Reply
0 Kudos
salvorlando
Contributor
Contributor

I think what Dan found is correct - that's the reason of your failure.

Needless to say, this is an error which "should not happen". As the error message says, Neutron is sending to NSX a request with a reference to a security profile (NSX terms for security groups), which is not a UUID. The problem here appears to be DB corruption, and there might be several reasons for that.

The most likely has to do with some mapping tables introduced in the DB schema in icehouse. Before the NSX plugin assigned NSX-mh UUIDs to Neutron resources. This forced us to process requests in a given order. Starting with Icehouse this constraint has been removed and a mapping table has been introduced. However, the mapping table needs to be populated, and this happens with DB migrations. If something went wrong during the migration, then there might be wrong values there. If the migration has not been executed the plugin should be able to fetch the right identifier directly from the NSX backend.

My gut feeling here is that we have some sort of invalid data in the mapping table (neutron_nsx_security_group_mappings)

It would be good to check if the values for the nsx-mh backend there are UUIDs and map to actual NSX security profiles.

I can help you with instrumenting code to find out data passed around between neutron and NSX.

Alternatively you can try and wipe out the above mentioned table. This will allow Neutron to fetch correct values from the backend (http://git.openstack.org/cgit/openstack/neutron/tree/neutron/plugins/vmware/common/nsx_utils.py#n135)

Reply
0 Kudos
yedongcan
Contributor
Contributor

Dan, I'm not use commercial version.

Reply
0 Kudos
yedongcan
Contributor
Contributor

Thank you Orlando, I will check the DB.

Reply
0 Kudos