vCenter 6.0 Upgrade to 6.5

[MattKimler]

I'm receiving the following error when I attempt to upgrade from vCenter 6.0 Update 2 to the vCenter 6.5. This is an appliance to appliance upgrade. I receive an error in Step 2 of the migration ("Set up target vCenter Server and start services". It get's to 62% with the error "A problem occurred while - Starting VMware vCenter Server...".

Anyone know how to resolve this one?

The message is as follows:

VMware VirtualCenter failed firstboot.

Database in-place upgrade failed. Please see vcdb_inplace.err and vcdb_inplace.out for details.

Resolution

Please refer to vSphere documentation to troubleshoot or Please contact VMware Support.

Here's the copy of the vcdb_inplace.err log:

Error while executing ./Upgrade-v2015-to-v2016/postgresql/upgrade_PostgreSQL.sql:2374, reason: Cannot execute statement(rc=-1).

DO $$

DECLARE r RECORD;

DECLARE sql_stmt VARCHAR;

BEGIN

    ALTER TABLE VPX_USAGE_STAT SET WITHOUT CLUSTER;

EXCEPTION WHEN OTHERS THEN

NULL;

END$$;

DO $$

DECLARE l_sn_vdevice_backing_seq int;

DECLARE l_vdevice_backing_seq int;

DECLARE sql_stmt VARCHAR;

BEGIN

    SELECT COALESCE(MAX(backing_id), 0)

       INTO l_sn_vdevice_backing_seq

    FROM VPX_SN_VDEVICE_BACKING_REL;

    SELECT COALESCE(MAX(backing_id), 0)

       INTO l_vdevice_backing_seq

    FROM VPX_VDEVICE_BACKING_REL;

    IF l_sn_vdevice_backing_seq >= l_vdevice_backing_seq THEN

       l_sn_vdevice_backing_seq = l_sn_vdevice_backing_seq + 200;

       sql_stmt = 'ALTER SEQUENCE VPX_VDEVICE_BACKING_REL_SEQ RESTART WITH '

          || l_sn_vdevice_backing_seq || ';';

       EXECUTE sql_stmt;

    END IF;

END$$;

^^^^^^^^^^

1 [42501](7) ERROR: must be owner of relation vpx_vdevice_backing_rel_seq;

Error while executing the query

ERROR: Cannot execute statement(rc=-1).

DO $$

DECLARE r RECORD;

DECLARE sql_stmt VARCHAR;

BEGIN

    ALTER TABLE VPX_USAGE_STAT SET WITHOUT CLUSTER;

EXCEPTION WHEN OTHERS THEN

NULL;

END$$;

DO $$

DECLARE l_sn_vdevice_backing_seq int;

DECLARE l_vdevice_backing_seq int;

DECLARE sql_stmt VARCHAR;

BEGIN

    SELECT COALESCE(MAX(backing_id), 0)

       INTO l_sn_vdevice_backing_seq

    FROM VPX_SN_VDEVICE_BACKING_REL;

    SELECT COALESCE(MAX(backing_id), 0)

       INTO l_vdevice_backing_seq

    FROM VPX_VDEVICE_BACKING_REL;

    IF l_sn_vdevice_backing_seq >= l_vdevice_backing_seq THEN

       l_sn_vdevice_backing_seq = l_sn_vdevice_backing_seq + 200;

       sql_stmt = 'ALTER SEQUENCE VPX_VDEVICE_BACKING_REL_SEQ RESTART WITH '

          || l_sn_vdevice_backing_seq || ';';

       EXECUTE sql_stmt;

    END IF;

END$$;

^^^^^^^^^^

1 [42501](7) ERROR: must be owner of relation vpx_vdevice_backing_rel_seq;

Error while executing the query

The log for the output is much larger, but not sure if that's necessary to post.

[admin]

> ERROR: must be owner of relation vpx_vdevice_backing_rel_seq;

As the error message suggests there is a conflict of sequence ownership.

Run the following to get more details on your environment:

1.

/opt/vmware/vpostgres/current/bin/psql -U postgres -l

2.

/opt/vmware/vpostgres/current/bin/psql -U postgres -d VCDB

\ds vpx_vdevice_backing_rel_seq

Please provide the output resultset from both.

[black88mx6]

I noticed that there was not an answer to this issue.   I have successfully migrated one my VCSA appliances from 6.0.0.20000 Build to 6.5.0.5178943 Build after encountering the same issue.

The following command shows that the sequence below has the wrong owner, along with several others.

/opt/vmware/vpostgres/current/bin/psql -U postgres -d VCDB

VCDB=# \ds vpx_vdevice_backing_rel_seq

                     List of relations

Schema |            Name             |   Type   |  Owner

--------+-----------------------------+----------+----------

vc     | vpx_vdevice_backing_rel_seq | sequence | postgres

(1 row)

The following command will list all sequence's and their owner.

VCDB=# \ds

From what was found, I build the fix below.

The following is run from from the VCSA shell prior to the migration of the VC, it will change the owner of all the sequences from postgres to vc and allow the migration to finish.

/opt/vmware/vpostgres/current/bin/psql -U postgres -d VCDB -c "ALTER SEQUENCE vpx_vdevice_backing_rel_seq OWNER TO vc"

/opt/vmware/vpostgres/current/bin/psql -U postgres -d VCDB -c "ALTER SEQUENCE vpx_access_seq OWNER TO vc"

/opt/vmware/vpostgres/current/bin/psql -U postgres -d VCDB -c "ALTER SEQUENCE vpx_alarm_seq OWNER TO vc"

/opt/vmware/vpostgres/current/bin/psql -U postgres -d VCDB -c "ALTER SEQUENCE vpx_ansfile_seq OWNER TO vc"

/opt/vmware/vpostgres/current/bin/psql -U postgres -d VCDB -c "ALTER SEQUENCE vpx_array_seq OWNER TO vc"

/opt/vmware/vpostgres/current/bin/psql -U postgres -d VCDB -c "ALTER SEQUENCE vpx_binary_data_seq OWNER TO vc"

/opt/vmware/vpostgres/current/bin/psql -U postgres -d VCDB -c "ALTER SEQUENCE vpx_change_tag_seq OWNER TO vc"

/opt/vmware/vpostgres/current/bin/psql -U postgres -d VCDB -c "ALTER SEQUENCE vpx_compliance_status_seq OWNER TO vc"

/opt/vmware/vpostgres/current/bin/psql -U postgres -d VCDB -c "ALTER SEQUENCE vpx_drsrules_seq OWNER TO vc"

/opt/vmware/vpostgres/current/bin/psql -U postgres -d VCDB -c "ALTER SEQUENCE vpx_entity_seq OWNER TO vc"

/opt/vmware/vpostgres/current/bin/psql -U postgres -d VCDB -c "ALTER SEQUENCE vpx_event_seq OWNER TO vc"

/opt/vmware/vpostgres/current/bin/psql -U postgres -d VCDB -c "ALTER SEQUENCE vpx_ext_client_seq OWNER TO vc"

/opt/vmware/vpostgres/current/bin/psql -U postgres -d VCDB -c "ALTER SEQUENCE vpx_ext_server_email_seq OWNER TO vc"

/opt/vmware/vpostgres/current/bin/psql -U postgres -d VCDB -c "ALTER SEQUENCE vpx_ext_server_seq OWNER TO vc"

/opt/vmware/vpostgres/current/bin/psql -U postgres -d VCDB -c "ALTER SEQUENCE vpx_field_def_seq OWNER TO vc"

/opt/vmware/vpostgres/current/bin/psql -U postgres -d VCDB -c "ALTER SEQUENCE vpx_file_system_mount_info_seq OWNER TO vc"

/opt/vmware/vpostgres/current/bin/psql -U postgres -d VCDB -c "ALTER SEQUENCE vpx_host_bus_adapter_seq OWNER TO vc"

/opt/vmware/vpostgres/current/bin/psql -U postgres -d VCDB -c "ALTER SEQUENCE vpx_image_lib_seq OWNER TO vc"

/opt/vmware/vpostgres/current/bin/psql -U postgres -d VCDB -c "ALTER SEQUENCE vpx_ip_pool_seq OWNER TO vc"

/opt/vmware/vpostgres/current/bin/psql -U postgres -d VCDB -c "ALTER SEQUENCE vpx_license_seq OWNER TO vc"

/opt/vmware/vpostgres/current/bin/psql -U postgres -d VCDB -c "ALTER SEQUENCE vpx_profile_seq OWNER TO vc"

/opt/vmware/vpostgres/current/bin/psql -U postgres -d VCDB -c "ALTER SEQUENCE vpx_psa_adapter_seq OWNER TO vc"

/opt/vmware/vpostgres/current/bin/psql -U postgres -d VCDB -c "ALTER SEQUENCE vpx_psa_device_seq OWNER TO vc"

/opt/vmware/vpostgres/current/bin/psql -U postgres -d VCDB -c "ALTER SEQUENCE vpx_psa_path_seq OWNER TO vc"

/opt/vmware/vpostgres/current/bin/psql -U postgres -d VCDB -c "ALTER SEQUENCE vpx_psa_target_seq OWNER TO vc"

/opt/vmware/vpostgres/current/bin/psql -U postgres -d VCDB -c "ALTER SEQUENCE vpx_role_seq OWNER TO vc"

/opt/vmware/vpostgres/current/bin/psql -U postgres -d VCDB -c "ALTER SEQUENCE vpx_sample_seq OWNER TO vc"

/opt/vmware/vpostgres/current/bin/psql -U postgres -d VCDB -c "ALTER SEQUENCE vpx_sample_time1_seq OWNER TO vc"

/opt/vmware/vpostgres/current/bin/psql -U postgres -d VCDB -c "ALTER SEQUENCE vpx_sample_time2_seq OWNER TO vc"

/opt/vmware/vpostgres/current/bin/psql -U postgres -d VCDB -c "ALTER SEQUENCE vpx_sample_time3_seq OWNER TO vc"

/opt/vmware/vpostgres/current/bin/psql -U postgres -d VCDB -c "ALTER SEQUENCE vpx_sample_time4_seq OWNER TO vc"

/opt/vmware/vpostgres/current/bin/psql -U postgres -d VCDB -c "ALTER SEQUENCE vpx_scheduledtask_seq OWNER TO vc"

/opt/vmware/vpostgres/current/bin/psql -U postgres -d VCDB -c "ALTER SEQUENCE vpx_scsi_lun_seq OWNER TO vc"

/opt/vmware/vpostgres/current/bin/psql -U postgres -d VCDB -c "ALTER SEQUENCE vpx_scsilun_descriptor_seq OWNER TO vc"

/opt/vmware/vpostgres/current/bin/psql -U postgres -d VCDB -c "ALTER SEQUENCE vpx_sn_vdevice_backing_rel_seq OWNER TO vc"

/opt/vmware/vpostgres/current/bin/psql -U postgres -d VCDB -c "ALTER SEQUENCE vpx_sn_virtual_device_seq OWNER TO vc"

/opt/vmware/vpostgres/current/bin/psql -U postgres -d VCDB -c "ALTER SEQUENCE vpx_stat_def_seq OWNER TO vc"

/opt/vmware/vpostgres/current/bin/psql -U postgres -d VCDB -c "ALTER SEQUENCE vpx_stat_id_seq OWNER TO vc"

/opt/vmware/vpostgres/current/bin/psql -U postgres -d VCDB -c "ALTER SEQUENCE vpx_tag_seq OWNER TO vc"

/opt/vmware/vpostgres/current/bin/psql -U postgres -d VCDB -c "ALTER SEQUENCE vpx_task_seq OWNER TO vc"

/opt/vmware/vpostgres/current/bin/psql -U postgres -d VCDB -c "ALTER SEQUENCE vpx_template_seq OWNER TO vc"

/opt/vmware/vpostgres/current/bin/psql -U postgres -d VCDB -c "ALTER SEQUENCE vpx_usage_sample_time_seq OWNER TO vc"

/opt/vmware/vpostgres/current/bin/psql -U postgres -d VCDB -c "ALTER SEQUENCE vpx_virtual_device_seq OWNER TO vc"

/opt/vmware/vpostgres/current/bin/psql -U postgres -d VCDB -c "ALTER SEQUENCE vpx_vm_fle_disk_layout_seq OWNER TO vc"

/opt/vmware/vpostgres/current/bin/psql -U postgres -d VCDB -c "ALTER SEQUENCE vpx_vm_fle_snapshot_disk_seq OWNER TO vc"

/opt/vmware/vpostgres/current/bin/psql -U postgres -d VCDB -c "ALTER SEQUENCE vpx_vm_fle_snapshot_layout_seq OWNER TO vc"

[Mnemonic]

Thank you for this VERY useful information.

I wrote a short blog article referring to this, and also to other problems that you might run into when upgrading.

The article can be found here: Things to know about upgrading vCSA 6.0 to vCSA 6.5 – The vGoodie-bag