VMware Cloud Community
TSSpain
Contributor
Contributor
Jump to solution

Configuring Cassandra for vCloud Director 9.1: Empty graphs

Hello.

Im using vCD 9.1.0 and few days ago configured Cassandra 3 in our LAB.

I used for installation and integration:

vCloud Availability for vCloud Director: Part 5: Deploy and Configure Cassandra Cluster

We had problems with ssl, so we disabled encrypted connection: vCloud Director 9 can't connect to cassandra DB

It seems installation worked fine but graphs are empty:

Capture001.jpg

Any help will be apreciated.

0 Kudos
1 Solution

Accepted Solutions
sk84
Expert
Expert
Jump to solution

I can't see in this guide that vCD has been configured for Cassandra. Did you use the "cell-management-tool" to tell vCD to use the Cassandra database for performance metrics?

I also wrote a blog article a few weeks ago on how to use these performance metrics in vCD. But without vCloud Availability.

Maybe it will help you:

Configuring Performance Metrics for vCloud Director 9.5 • Just virtual things

--- Regards, Sebastian VCP6.5-DCV // VCP7-CMA // vSAN 2017 Specialist Please mark this answer as 'helpful' or 'correct' if you think your question has been answered correctly.

View solution in original post

0 Kudos
11 Replies
sk84
Expert
Expert
Jump to solution

I can't see in this guide that vCD has been configured for Cassandra. Did you use the "cell-management-tool" to tell vCD to use the Cassandra database for performance metrics?

I also wrote a blog article a few weeks ago on how to use these performance metrics in vCD. But without vCloud Availability.

Maybe it will help you:

Configuring Performance Metrics for vCloud Director 9.5 • Just virtual things

--- Regards, Sebastian VCP6.5-DCV // VCP7-CMA // vSAN 2017 Specialist Please mark this answer as 'helpful' or 'correct' if you think your question has been answered correctly.
0 Kudos
TSSpain
Contributor
Contributor
Jump to solution

Hi, you're right.

I forgot to mention we followed another guide: https://anthonyspiteri.net/enabling-configuring-and-viewing-metrics-in-vcloud-director-9-0/

We checked both Cassandra Keyspace and table were created but contained empty rows.

I think we had some issue installing Cassandra database. We'll create a new Cassandra environment following your install guides:

Installation of Cassandra Database for vCloud Director 9.5 • Just virtual things

Configuring Performance Metrics for vCloud Director 9.5 • Just virtual things

I'll keep you informed

0 Kudos
TSSpain
Contributor
Contributor
Jump to solution

Hi again,

I've just followed your articles (congrats!!!) but problem persists.

It seems tables were created but vCloud Director cells do not send any information regarding vm metrics...

ScreenShot002.jpg

0 Kudos
sk84
Expert
Expert
Jump to solution

It may take some time until the metrics are populated in the database.

But can you please provide the output of the following commands, just to check if vcd is configured correctly:

/opt/vmware/vcloud-director/bin/cell-management-tool manage-config -n cassandra.keyspace.name -l
/opt/vmware/vcloud-director/bin/cell-management-tool manage-config -n cassandra.vmm.clusters -l

You can also look in the debug log file "/opt/vmware/vocoud-director/logs/vcloud-container-debug.log" and search for the string "cassandra".

--- Regards, Sebastian VCP6.5-DCV // VCP7-CMA // vSAN 2017 Specialist Please mark this answer as 'helpful' or 'correct' if you think your question has been answered correctly.
0 Kudos
TSSpain
Contributor
Contributor
Jump to solution

Hi Sebastien,

I'll wait a couple of days...

The output for the requested commands:

[root@cell01 logs]# /opt/vmware/vcloud-director/bin/cell-management-tool manage-config -n cassandra.keyspace.name -l

Property "cassandra.keyspace.name" has value "vcloud_metrics"

[root@cell01 logs]# /opt/vmware/vcloud-director/bin/cell-management-tool manage-config -n cassandra.vmm.clusters -l

Property "cassandra.vmm.clusters" has value "192.168.0.111,192.168.0.112,192.168.0.113,192.168.0.114"

It seems everithing well configured.

I also look in the debug log file:

[root@cell01 logs]# cat /opt/vmware/vcloud-director/logs/vcloud-container-debug.log | grep cassandra

        at com.vmware.vcloud.cassandra.client.impl.CassandraClientImpl$1.doWithRetry(CassandraClientImpl.java:193)

        at com.vmware.vcloud.cassandra.client.impl.CassandraClientImpl$1.doWithRetry(CassandraClientImpl.java:187)

        at com.vmware.vcloud.cassandra.client.impl.CassandraClientImpl.initialize(CassandraClientImpl.java:187)

        at com.vmware.vcloud.cassandra.client.impl.CassandraClientImpl$1.doWithRetry(CassandraClientImpl.java:193)

        at com.vmware.vcloud.cassandra.client.impl.CassandraClientImpl$1.doWithRetry(CassandraClientImpl.java:187)

        at com.vmware.vcloud.cassandra.client.impl.CassandraClientImpl.initialize(CassandraClientImpl.java:187)

        at com.vmware.vcloud.cassandra.client.impl.CassandraClientImpl$1.doWithRetry(CassandraClientImpl.java:193)

        at com.vmware.vcloud.cassandra.client.impl.CassandraClientImpl$1.doWithRetry(CassandraClientImpl.java:187)

        at com.vmware.vcloud.cassandra.client.impl.CassandraClientImpl.initialize(CassandraClientImpl.java:187)

2019-02-08 16:39:47,638 | DEBUG    | Spring Context: com.vmware.vcloud.statsfeeder-application | EventManagerImpl               | Registered listener com.vmware.vcloud.cassandra.cache.VirtualMachineParentChangedListenerImpl@24a8934a for com.vmware.vcloud.inventory.events.impl.VirtualMachineParentChangedEvent |

        at com.vmware.vcloud.cassandra.client.impl.CassandraClientImpl$1.doWithRetry(CassandraClientImpl.java:193)

        at com.vmware.vcloud.cassandra.client.impl.CassandraClientImpl$1.doWithRetry(CassandraClientImpl.java:187)

        at com.vmware.vcloud.cassandra.client.impl.CassandraClientImpl.initialize(CassandraClientImpl.java:187)

        at com.vmware.vcloud.cassandra.client.impl.CassandraClientImpl.getSession(CassandraClientImpl.java:230)

        at com.vmware.vcloud.cassandra.client.impl.CassandraClientImpl.buildQueryString(CassandraClientImpl.java:414)

        at com.vmware.vcloud.cassandra.client.impl.CassandraClientImpl.readMetrics(CassandraClientImpl.java:313)

        at com.vmware.vcloud.cassandra.client.impl.CassandraClientImpl$1.doWithRetry(CassandraClientImpl.java:193)

        at com.vmware.vcloud.cassandra.client.impl.CassandraClientImpl$1.doWithRetry(CassandraClientImpl.java:187)

        at com.vmware.vcloud.cassandra.client.impl.CassandraClientImpl.initialize(CassandraClientImpl.java:187)

        at com.vmware.vcloud.cassandra.client.impl.CassandraClientImpl.getSession(CassandraClientImpl.java:230)

        at com.vmware.vcloud.cassandra.client.impl.CassandraClientImpl.buildQueryString(CassandraClientImpl.java:414)

        at com.vmware.vcloud.cassandra.client.impl.CassandraClientImpl.readMetrics(CassandraClientImpl.java:313)

        at com.vmware.vcloud.cassandra.client.impl.CassandraClientImpl$1.doWithRetry(CassandraClientImpl.java:193)

        at com.vmware.vcloud.cassandra.client.impl.CassandraClientImpl$1.doWithRetry(CassandraClientImpl.java:187)

        at com.vmware.vcloud.cassandra.client.impl.CassandraClientImpl.initialize(CassandraClientImpl.java:187)

        at com.vmware.vcloud.cassandra.client.impl.CassandraClientImpl.getSession(CassandraClientImpl.java:230)

        at com.vmware.vcloud.cassandra.client.impl.CassandraClientImpl.buildQueryString(CassandraClientImpl.java:414)

        at com.vmware.vcloud.cassandra.client.impl.CassandraClientImpl.readMetrics(CassandraClientImpl.java:313)

        at com.vmware.vcloud.cassandra.client.impl.CassandraClientImpl.buildQueryString(CassandraClientImpl.java:414)

        at com.vmware.vcloud.cassandra.client.impl.CassandraClientImpl.readMetrics(CassandraClientImpl.java:313)

        at com.vmware.vcloud.cassandra.client.impl.CassandraClientImpl$1.doWithRetry(CassandraClientImpl.java:193)

        at com.vmware.vcloud.cassandra.client.impl.CassandraClientImpl$1.doWithRetry(CassandraClientImpl.java:187)

        at com.vmware.vcloud.cassandra.client.impl.CassandraClientImpl.initialize(CassandraClientImpl.java:187)

        at com.vmware.vcloud.cassandra.client.impl.CassandraClientImpl.getSession(CassandraClientImpl.java:230)

        at com.vmware.vcloud.cassandra.client.impl.CassandraClientImpl.buildQueryString(CassandraClientImpl.java:414)

        at com.vmware.vcloud.cassandra.client.impl.CassandraClientImpl.readMetrics(CassandraClientImpl.java:313)

        at com.vmware.vcloud.cassandra.client.impl.CassandraClientImpl$1.doWithRetry(CassandraClientImpl.java:193)

        at com.vmware.vcloud.cassandra.client.impl.CassandraClientImpl$1.doWithRetry(CassandraClientImpl.java:187)

        at com.vmware.vcloud.cassandra.client.impl.CassandraClientImpl.initialize(CassandraClientImpl.java:187)

        at com.vmware.vcloud.cassandra.client.impl.CassandraClientImpl.getSession(CassandraClientImpl.java:230)

        at com.vmware.vcloud.cassandra.client.impl.CassandraClientImpl.buildQueryString(CassandraClientImpl.java:414)

        at com.vmware.vcloud.cassandra.client.impl.CassandraClientImpl.readMetrics(CassandraClientImpl.java:313)

        at com.vmware.vcloud.cassandra.client.impl.CassandraClientImpl$1.doWithRetry(CassandraClientImpl.java:193)

        at com.vmware.vcloud.cassandra.client.impl.CassandraClientImpl$1.doWithRetry(CassandraClientImpl.java:187)

        at com.vmware.vcloud.cassandra.client.impl.CassandraClientImpl.initialize(CassandraClientImpl.java:187)

        at com.vmware.vcloud.cassandra.client.impl.CassandraClientImpl.getSession(CassandraClientImpl.java:230)

        at com.vmware.vcloud.cassandra.client.impl.CassandraClientImpl.buildQueryString(CassandraClientImpl.java:414)

        at com.vmware.vcloud.cassandra.client.impl.CassandraClientImpl.readMetrics(CassandraClientImpl.java:313)

        at com.vmware.vcloud.cassandra.client.impl.CassandraClientImpl.buildQueryString(CassandraClientImpl.java:414)

        at com.vmware.vcloud.cassandra.client.impl.CassandraClientImpl.readMetrics(CassandraClientImpl.java:313)

2019-02-08 16:52:47,650 | DEBUG    | Spring Context: com.vmware.vcloud.statsfeeder-application | EventManagerImpl               | Registered listener com.vmware.vcloud.cassandra.cache.VirtualMachineParentChangedListenerImpl@3483da30 for com.vmware.vcloud.inventory.events.impl.VirtualMachineParentChangedEvent |

2019-02-08 17:10:19,665 | DEBUG    | auto-pool-StatsFeederCellApplicationManager-28-thread-2 | CassandraClient                | Shutting down cassandra session |

2019-02-08 17:10:19,677 | DEBUG    | auto-pool-StatsFeederCellApplicationManager-28-thread-2 | CassandraClient                | Shutting down cassandra cluster |

Have a nice weekend!

0 Kudos
sk84
Expert
Expert
Jump to solution

So it looks like this is part of a Java stack trace. Open the log file and search for the complete error message. 😉

--- Regards, Sebastian VCP6.5-DCV // VCP7-CMA // vSAN 2017 Specialist Please mark this answer as 'helpful' or 'correct' if you think your question has been answered correctly.
0 Kudos
TSSpain
Contributor
Contributor
Jump to solution

Hi,

We created a VMware SR. The problem was the "vCenter Statistics settings". The "1 day statistitcs" value was defined 2 minutes.

The min_reporting interval is 300 on the VCD side.

If you run the following the query " select * from metric_configuration;" against the vCloud Director DB you will obtain the metrics that vCD is using for historic counters. In my case:

id category name    entity instance current_interval historic_interval min_reporting_interval aggregator

0x00EC654741B74F04AC576468AB975740 VMM cpu.usage.average VM 20 20 300 AVERAGE

0x1726C1F7C9544848BC80EBDD33A5F32C VMM    cpu.usage.maximum VM 20 20 300 AVERAGE

0x24B4374849FD4976A63A7BA78EBA6E3E VMM disk.write.average VM 20 20 300 AVERAGE

0x8548A983BB7C41E1A2C33250A51B6912 VMM    mem.usage.average VM 20 20 300 AVERAGE

0xA86FEBEC0F604D2CA7A5B8A041A8F933 VMM cpu.usagemhz.average VM 20 20 300         AVERAGE

0xE1442C7E6ABC420A90B42E063C8F2842 VMM disk.used.latest VM 300 300 1800 AVERAGE

0xE53D24E12E7848568DD6A6ABDDDF02C3 VMM    disk.provisioned.latest VM 300 300 1800 AVERAGE

0xE67BEEF388824413A518EABBDD5AFBA5 VMM disk.read.average VM 20 20           300 AVERAGE


So, I replaced  the "1 day" statistics to 5 minutes and graphs are working fine.

Thank you for your suport.

0 Kudos
SebastianGrugel
Hot Shot
Hot Shot
Jump to solution

This query can be performed where ? on vCD Postgress DB ? or on CASSANDRA ?

vExpert VSAN/NSX/CLOUD | VCAP5-DCA | VCP6-DCV/CMA/NV ==> akademiadatacenter.pl
0 Kudos
sk84
Expert
Expert
Jump to solution

Hi,

SebastianGrugel

This query can be performed where ? on vCD Postgress DB ? or on CASSANDRA ?

This query can be used for the vCloud Director database (for example, MS SQL or Postgres) to retrieve the metric configuration settings.

TSSpain

Thanks a lot for sharing this information.

--- Regards, Sebastian VCP6.5-DCV // VCP7-CMA // vSAN 2017 Specialist Please mark this answer as 'helpful' or 'correct' if you think your question has been answered correctly.
0 Kudos
TSSpain
Contributor
Contributor
Jump to solution

Hi,

on vCD database. In my case SQL Database (we are still using vCloud Director 9.1).

Best regards

Eduard

0 Kudos
ManivelR
Hot Shot
Hot Shot
Jump to solution

Hi Team,

Im also facing the same isse.I troubleshooted almost all the above given steps.

VCD is 10.1 V.Earlier it was 10.0 and upgraded to 10.1

VCDDB is postgress.

Cassandra is 3.11.4 running on Centos.It has 2 cassandra nodes.

VC is 6.7 and the statistics is 5 mins & VCDDB is also 300 secs.

 

Performed all the below steps,

VCD restart

VCDDB restart

Cassandra DB removed and reconfigured.

VC performance graph service restart

Still the graph is coming empty.Any advise?

0 Kudos