Error when issuing command "vimsh -n -e /hostsvc/maintenance_mode_enter"

[WhyNot201110141]

Hello everyone,

During the upgrade of a few hosts from ESX 4.0 to ESX 4.1 I've run into an issue with one of the hosts that none of the others seem to have encountered. When running the command: vimsh -n -e /hostsvc/maintenance_mode_enter - I get the following error output:

vimsh -n -e /hostsvc/maintenance_mode_enter
Using system libcrypto, version 90810F

[2011-08-29 08:19:10.890 F66B36D0 info 'App'] Current working directory: /home/vmadmin
[2011-08-29 08:19:10.891 F66B36D0 info 'App'] Vmacore::InitSSL: doVersionCheck = true, handshakeTimeoutUs = 20000000
[2011-08-29 08:19:10.913 F6530B90 warning 'Libs'] [Vmdb_Unset] Unsetting unknown path: /vmomi/
[2011-08-29 08:19:11.111 F66B36D0 warning 'App'] ============BEGIN FAILED METHOD CALL DUMP============
[2011-08-29 08:19:11.111 F66B36D0 warning 'App'] Invoking [login] on [vim.SessionManager:ha-sessionmgr]
[2011-08-29 08:19:11.111 F66B36D0 warning 'App'] Arg userName:
"local-root"
[2011-08-29 08:19:11.111 F66B36D0 warning 'App'] Arg password:
(not shown)

[2011-08-29 08:19:11.111 F66B36D0 warning 'App'] Arg locale:
"en"
[2011-08-29 08:19:11.111 F66B36D0 warning 'App'] Fault Msg: "Permission to perform this operation was denied."
[2011-08-29 08:19:11.111 F66B36D0 warning 'App'] ============END FAILED METHOD CALL DUMP============
Failed to login: vim.fault.NoPermission
[2011-08-29 08:19:11.171 F66B36D0 warning 'App'] ============BEGIN FAILED METHOD CALL DUMP============
[2011-08-29 08:19:11.171 F66B36D0 warning 'App'] Invoking [enterMaintenanceMode] on [vim.HostSystem:ha-host]
[2011-08-29 08:19:11.171 F66B36D0 warning 'App'] Arg timeout:
30
[2011-08-29 08:19:11.171 F66B36D0 warning 'App'] Arg evacuatePoweredOffVms:
false
[2011-08-29 08:19:11.171 F66B36D0 warning 'App'] Fault Msg: "The session is not authenticated."
[2011-08-29 08:19:11.171 F66B36D0 warning 'App'] ============END FAILED METHOD CALL DUMP============
(vim.fault.NotAuthenticated) {
   dynamicType = <unset>,
   faultCause = (vmodl.MethodFault) null,
   object = 'vim.HostSystem:ha-host',
   privilegeId = "Host.Config.Maintenance",
   msg = "The session is not authenticated.",
}

-------------------------------------------------------------------------------------------------------------------------------------

When running as 'SU' I pretty much get the same result:

vimsh -n -e /hostsvc/maintenance_mode_enter
Using system libcrypto, version 90810F

[2011-08-29 08:31:20.196 F668D6D0 info 'App'] Current working directory: /home/vmadmin
[2011-08-29 08:31:20.197 F668D6D0 info 'App'] Vmacore::InitSSL: doVersionCheck = true, handshakeTimeoutUs = 20000000
[2011-08-29 08:31:20.219 F650AB90 warning 'Libs'] [Vmdb_Unset] Unsetting unknown path: /vmomi/
[2011-08-29 08:31:20.419 F668D6D0 warning 'App'] ============BEGIN FAILED METHOD CALL DUMP============
[2011-08-29 08:31:20.419 F668D6D0 warning 'App'] Invoking [login] on [vim.SessionManager:ha-sessionmgr]
[2011-08-29 08:31:20.419 F668D6D0 warning 'App'] Arg userName:
"local-root"
[2011-08-29 08:31:20.419 F668D6D0 warning 'App'] Arg password:
(not shown)

[2011-08-29 08:31:20.419 F668D6D0 warning 'App'] Arg locale:
"en"
[2011-08-29 08:31:20.419 F668D6D0 warning 'App'] Fault Msg: "Permission to perform this operation was denied."
[2011-08-29 08:31:20.419 F668D6D0 warning 'App'] ============END FAILED METHOD CALL DUMP============
Failed to login: vim.fault.NoPermission
[2011-08-29 08:31:20.479 F668D6D0 warning 'App'] ============BEGIN FAILED METHOD CALL DUMP============
[2011-08-29 08:31:20.479 F668D6D0 warning 'App'] Invoking [enterMaintenanceMode] on [vim.HostSystem:ha-host]
[2011-08-29 08:31:20.479 F668D6D0 warning 'App'] Arg timeout:
30
[2011-08-29 08:31:20.479 F668D6D0 warning 'App'] Arg evacuatePoweredOffVms:
false
[2011-08-29 08:31:20.479 F668D6D0 warning 'App'] Fault Msg: "The session is not authenticated."
[2011-08-29 08:31:20.479 F668D6D0 warning 'App'] ============END FAILED METHOD CALL DUMP============
(vim.fault.NotAuthenticated) {
   dynamicType = <unset>,
   faultCause = (vmodl.MethodFault) null,
   object = 'vim.HostSystem:ha-host',
   privilegeId = "Host.Config.Maintenance",
   msg = "The session is not authenticated.",
}

-------------------------------------------------------------------------------------------------------------------------------------------

Within the vCenter client I have no issue placing the host into maintenance mode, however, I get the following error when attempting to review the maintenance mode status when running: vimsh -n -e "/hostsvc/hostsummary" | grep inMaintenanceMode

vimsh -n -e "/hostsvc/hostsummary" | grep inMaintenanceMode
Using system libcrypto, version 90810F

Failed to login: vim.fault.NoPermission
(vim.fault.NotAuthenticated) {
   dynamicType = <unset>,
   faultCause = (vmodl.MethodFault) null,
   object = 'vim.HostSystem:ha-host',
   privilegeId = "System.Read",
   msg = "The session is not authenticated.",
}

When running the esxupdate obviously it as well fails with the following error:

esxupdate --bundle="/vmfs/volumes/MHT-VM2 Local Volume/upgrade-from-esx4.0-to-4.1-update01-348481.zip" update
Encountered error MaintenanceModeError:
The error data is:
   Message     - Unable to determine if the system is in maintenance mode: vim-
                 cmd returned error (255).  Please see esxupdate.log for more
                 details.  To be safe, installation will not continue.
   Errno       - 18
   Description - Maintenance mode is not enabled or could not be determined.

Everything I have encountered seems to relate to a permissions issue. Running as "root" I thought for sure would allow me to run these commands unfortunately this is not the case. Any thoughts?

[WhyNot201110141]

I have to stop answering my own questions... lol.

I went into the vCenter client under permissions and noticed that 'root' had "no access", not to sure how that happened - most like the ID10T error. Anyway, after giving the root user "Administrator" privileges and running the esxupdate I was successful.

Thanks for reviewing the issue.