vCloud Director 9.7 JMS Certificate Issue

Are you still on vCloud Director 9.7 (VCD) in multi-cell configuration? Then you are susceptible to Java Message Service (JMS) certificate expiration issue. Read on.


In multi-cell set up VCD cells need to communicate between themselves. They use shared database but for much faster and efficient communication they also use internal ActiveMQ message bus. It is used for activity sharing and vCenter Server events notifications. If the message bus is dysfunctional it slows any operations almost to halt. For this particular certificate issue you will see in the logs similar message:

Could not accept connection from tcp://<primary-cell-IP:port> : Received fatal alert: certificate_unknown

In vCloud Director 9.7 the bus communication become encrypted in preparation for other use cases (read here). On upgrade or new deployment of each cell new certificate was issued by internal VCD_CA with 365 day duration. In vCloud Director 10.0 or VMware Cloud Director 10.1 the certificate is regenerated upon upgrade and its duration is extended to 3 years.

To find out the certificates expiry date, run the following command from any cell:

/opt/vmware/vcloud-director/bin/cell-management-tool jms-certificates -status

It will for every cell print out its JMS certificate details:

Cell with UUID fd0d2ca0-e357-4aae-9a3b-1c1c5d143d05 and IP has jms certificate: [
Version: V3
Subject: CN=vcd-node2.vmware.local
Signature Algorithm: SHA256withRSA, OID = 1.2.840.113549.1.1.11

Key: Sun RSA public key, 2048 bits
modulus: 25783371233977292378120920630797680107189459997753528550984552091043966882929535251723319624716621953887829800012972122297123129787471789561707788386606748136996502359020718390547612728910865287660771295203595369999641232667311679842803649012857218342116800466886995068595408721695568494955018577009303299305701023577567636203030094089380853265887935886100721105614590849657516345704143859471289058449674598267919118170969835480316562172830266483561584406559147911791471716017535443255297063175552471807941727578887748405832530327303427058308095740913857590061680873666005329704917078933424596015255346720758701070463
public exponent: 65537
Validity: [From: Wed Jun 12 15:38:11 UTC 2019,
To: Thu Jun 11 15:38:11 UTC 2020]


Yes, this particular cell’s certificate will expire Jun 12 2020 – in less than two months!

The Fix

Set a calendar reminder and when the certificate expiration day is approaching run the following command.

/opt/vmware/vcloud-director/bin/cell-management-tool jms-certificates --certgen

Or upgrade to vCloud Director 10.0 or newer.

Update 21/05/2020: KB 78964 has been published on this topic. Also if the CA signing certificate is expired you will need to disable SSL altogether, restart the cell, regenerate the cert and re-enable SSL.


16 thoughts on “vCloud Director 9.7 JMS Certificate Issue

  1. Hello Fojta, to run this renew command I have to stop the cells ? I have to do this command in each cell or just one ? I have to restart the cells ?

  2. Fojta just one more question I promess, If I update vCloud from to, it will correct this issue or not ?

  3. Hi Tomas

    Just FYI there seems to be an issue with vCloud 9.7 and 10 where this process creates certificates with an incorrect IP address in the certificate subject alt name (adds the IP address for the cell that you ran the certgen on).

    From the support call I have with VMware this is a known issue, apparently fixed in an upcoming patch. The workaround is to either disable JMS SSL (run /opt/vmware/vcloud-director/bin/cell-management-tool jms-certificates –disablessl on one cell and restart all cells) or delete the certificate from the vCloud database then start each cell one at a time.

    I don’t see a KB that I can cite as yet.

    1. Jono, thanks for your reply
      your solution is correct:
      I`ll share required postgress commands to delete JMS certs from DB

      select * from truststore where tag = ‘JMS’;
      delete from truststore where tag = ‘JMS’;

      You should execute this commands with stopped services.
      And than bring your cells up one by one.

  4. Hi Tomas, quick question. I want to renew http and consoleproxy certs. The common names for the existing certs are for example and When running the csr generate command as;

    keytool -keystore /opt/vmware/vcloud-director/certificates.ks -storetype JCEKS -storepass root_passwd -certreq -alias http -file http.csr -ext “,dns:http”

    the csr is generated, however when submitting to public CA for cert creation, the common name is still set to the internal FQDN and private internal domain of the vcd cell. Would you know how i can generate the csr to use the public common name instead by any chance?

    1. Update; unsure if this is the correct method, however i resorted to deleting both untrusted http and consoleproxy certs from certificates.ks –

      keytool -delete -alias http -keystore /opt/vmware/vcloud-director/certificates.ks -storepass root_passwd
      -storetype JCEKS -v

      I than used the old linux command (as per vCD 9 KB) to re-create untrusted certs for http and consoleproxy, which let me specify the CN i wanted to use (my public DNS) –

      -keystore certificates.ks
      -alias http
      -storepass passwd
      -keypass passwd
      -storetype JCEKS
      -keyalg RSA
      -keysize 2048
      -validity 365
      -dname “, OU=Engineering, O=Example Corp, L=Palo Alto, S=California, C=US”
      -ext “,dns:vcd1,ip:”

      Only after this, did i create the CSR’s again and submit to online public CA, and i no longer got the error ‘Common name invalid’ when renewing the certificates.

      1. You can use “keytool -delete” subcommand to delete a specific certificate from certificate.ks, however you can also recreate a new certificate.ks from scratch. The keystore file is used only during the import of the certs to VCD (via CMT or configure commands) and not used after that.

  5. Hi Tomas
    I have a 1 cell vCD (v10.1) for lab on which I had a public certificate that has expired few days ago.
    I just realized so generated now self signed certificate with :
    ./cell-management-tool generate-certs -j -p -o xxx.ks -w yyy
    Then imported cert :
    /opt/vmware/vcloud-director/bin/cell-management-tool certificates -j -p –keystore xxx.ks –keystore-password yyy
    and restarted vCD : service vmware-vcd restart
    When connecting to the vCD UI, I can see the new certificate is used. However when accessing the console, it is disconnecting after a few sec, and I have the following message in console-proxy.log
    2021-04-20 14:25:49,257 | DEBUG | consoleproxy | ReadOperation | IOException while reading data. Closing transfer. | java.nio.channels.SocketChannel[connected local=/:444 remote=/:39246] Received fatal alert: bad_certificate
    Any idea where the issue is ?

    PS: I have also tried to run the commands you recommend in this post with jms:/opt/vmware/vcloud-director/bin/cell-management-tool jms-certificates –certgen and restart, but did not help


    1. jms certs have nothing to do with console certs. As you generated self signed certs for https and console you need to trust them in your browser. See if https:/ gives you cert error in the browser.

      1. Hi Tomas
        I know it is not related, however as this post was related to certificates and you are very experienced with all vcd versions … i thought you may help.
        When i open with the browser the url, i have a warning because it is self signed … but that should be ok, no ?

        1. No. Your client (browser/OS) must trust the certificate explicitly. So it has to be green without any error. The easiest is to add the self signed cert to your trused certificate store

  6. Not sure if this helps; I created a untrusted cert for HTTP and one for consoleproxy, generated a CSR for both using my CA and imported both into java keystore.

    I guess as your cert is self signed, you would verify all certs are imported in java keystore;
    keytool -storetype JCEKS -storepass passwd -keystore certificates.ks -list

    if so, run configure script for network & DB connections

    Finally paste the certificate chain in vCD public addresses in PEM format;

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.