vCloud Director Federation with VMware Identity Manager

Although in the past I have already blogged about vCloud Director federation with VMware Workspace and Microsoft AD FS I still have not wrote a guide how to achieve organization federation with VMware Identity Manager.

VMware Identity Manager (VIDM) is Single Sign-On solution that integrates with multiple identity providers (such as Active Directory) and offers App Store like access to multiple subscribed services with adaptive access (including multi factor authentication such as VMware Verify, DUO or RSA SecurID).

Users can easily log in into multiple different cloud services from a single page thanks to SSO. One such service can be vCloud Director and this article describes how to achieve this.

VIDM is provided as virtual appliance that can be deployed in  a load balanced distributed or simple configuration with external Microsoft SQL database or for evaluation purposes with embedded PostgreSQL. It is also provided as a cloud service.

I have deployed in my lab VIDM 3.2 in a single embedded database configuration and connected it to my lab Active Directory. VIDM also provides its own internal IdP (System Domain) so can be used also without an external AD/LDAP.

  1. In vCloud Director Organization enable Federation by setting Entity ID to Org Name (or any other unique string), generate fresh certificate and download Metadata from the link provided (file spring_saml_metadata.xml). This can be done as system or Organization Administrator.
  2. In VIDM go to Catalog and create new web application. You have to be logged in as VIDM Administrator. Write application name, description and upload nice icon and choose category. 
  3. In the next screen keep Authentication Type SAML 2.0 and paste the xml metadata from step #1 into the URL/XML window. Scroll down to Advanced Properties. 
  4. In Advanced Properties we will keep the defaults but add Custom Attribute Mappings which describe how VIDM user attributes will translate to VCD user attributes. Here is the list:
Name                                                               Value
-------------------------------------------------------------------------------------
UserName                                                           ${user.userName}
EmailAddress                                                       ${user.email}
http://schemas.xmlsoap.org/ws/2005/05/identity/claims/surname      ${user.lastName}
http://schemas.xmlsoap.org/ws/2005/05/identity/claims/givenname    ${user.firstName}
Groups                                                             CokeAdmins
Roles                                                              ${user.employeeID}

Format is always basic and namespace is blank. Let me explain what is going on here. The first two mappings should be clear. UserName specify how will the user be recognized by vCloud Director and EmailAddress will be his/her email address for notifications. The next two parameters contain given name and surname which VCD will combine into full name. Not sure why the assertion name needs to be specified in such long format but only this way it worked for me. In theory you could also pass ‘fullname’ if you have field in your directory that contains full name (my Active Director does not).

Next we have Groups. I do not know how to pass AD groups in a dynamic way, so this is a hardcoded example – all users who use this SaaS definition will belong to CokeAdmins group. You can obviously omit this if you will only import users by name.

The last property is Role – again this is optional and should be used only if we want to manage roles in IdPs and not in VCD, where we would import the user with Defer to Identity Provider role – see here for more details. I am using here unused AD Employee ID field. 

  1. Now we can finish the wizard by clicking next, select access policy (keep default) and reviewing the Summary on the next screen. As last step we can click Save & Assign, where we are presented to select users that should have this newly created App in their catalog.
  2. Next we need to retrieve metadata configuration of VIDM – this is by going back to Catalog (all the way up) and clicking Settings. From SAML Metadata download Identity Provider (IdP) metadata. 
  3. Now we can finalize SAML configuration in vCloud Director. Still on Federation page click Use SAML Identity Provider checkbox and import the downloaded metadata (idp.xml) with Browse and Upload buttons and click Apply. 
  4. From now on, login screen will default to SAML authentication, but you can always revert to local authentication with: https://<vcloud_fqdn>/cloud/org/<org-name>/login.jsp
    However, we first need to import some users/groups to be able to use SAML. You might need to re-login to see the option to add SAML users and groups. You can import VIDM users by their user name or group (the hardcoded name CokeAdmins). We can assign role or optionally leverage Defer to IdP (if you have role field populated in AD). 
  5. Login to VIDM and click on the application tile to be logged straight into VCD Organization (the Flex legacy UI). 

You can also directly enter the Flex or HTML 5 URL and you will be redirected to VIDM login screen.

As mentioned in step 4, I am managing user roles from Active Directory so I just need to import one group into VCD with Defer to IdP role and I am done with user management in VCD. The role field that VIDM uses is EmployeeID, however that field in AD can have only numerical value and VCD expects role as text string. The workaround I am using is that in AD I actually use User Description field and then in VIDM I have changed the mapping. So lets first have a look how it looks in AD:

Here is my VIDM custom AD IdP mapping:

 

Federation of Multiple vCloud Director Instances


While vCloud Director offers impressive scalability with possibility to manage up to 20 vCenter Servers there are valid reasons why service providers deploy multiple vCloud Director instances due to fault domains (availability zones) or latency (multiple geographical regions) requirements.Federation

In such case a single tenant (end-customer) has multiple organization accounts in multiple vCloud Director instance each with access to subset of VDCs, catalogs, VMs, etc. The service provider would usually hide this under one custom overarching portal however when the tenant wants to discover and manage his resources in programmable fashion he has to access multiple vCloud API endpoints of the vCloud Director instances.

vCloud Director version 8.0 offers a new feature: Organization Associations. If you search through vCloud API v 9.0 you will see new Admin Types:  OrgAssociationType  and OrgAssociationsType and related new Admin Element OrgAssociationMember. So what it is and how does it work?

The service provider can associate multiple Organizations (belonging to the same customer) together and simplify discovery of all cloud resources through single API endpoint.

Example

In my lab I have two different vCloud Director instances with URLs vcloud.fojta.com and vcloud2.fojta.com. In the first instance I have created two organizations belonging to the same customer: ACME and ACME2. In the second instance I have created organization ACME. All three organizations use the same identity source (in my case LDAP) and the same user exists in all of them with Organization Administrator credentials.

  1. The system administrator creates the organization associations. In this example I have associated org: ACME in the second instance with ACME and ACME2 in the first instance. This is done with the following vCloud API call:
    PUT https://vcloud2.fojta.com/api/admin/org/ca5295f0-a521-4d4c-8b2e-322f154fbbea/associations
    Content-Type: application/vnd.vmware.admin.organizationAssociations+xml
    
    Body:
    <?xml version="1.0" encoding="UTF-8"?>
    <OrgAssociations xmlns="http://www.vmware.com/vcloud/v1.5">
        <OrgAssociationMember href="https://vcloud2.fojta.com/api/admin/org/ca5295f0-a521-4d4c-8b2e-322f154fbbea/associations/02b433db-0b37-4304-b07b-0717255ec297" type="application/vnd.vmware.admin.organizationAssociation+xml">
            <MemberUrl>https://vcloud.fojta.com/api/org/02b433db-0b37-4304-b07b-0717255ec297</MemberUrl>
            <MemberName>ACME</MemberName>
            <MemberEndpointCertificate>-----BEGIN CERTIFICATE-----
    MIIFfzCCBGegAwIBAgITTgAAARuwZOW3iRv9KQABAAABGzANBgkqhkiG9w0BAQsF
    ADBCMRMwEQYKCZImiZPyLGQBGRYDY29tMRUwEwYKCZImiZPyLGQBGRYFZm9qdGEx
    FDASBgNVBAMTC2ZvanRhLURDLUNBMB4XDTE1MDExNTE2NTAwNFoXDTE3MDExNDE2
    NTAwNFowZzELMAkGA1UEBhMCQ1oxDzANBgNVBAcTBlByYWd1ZTESMBAGA1UEChMJ
    Zm9qdGEuY29tMRgwFgYDVQQLEw92Q2xvdWQgRGlyZWN0b3IxGTAXBgNVBAMTEHZj
    bG91ZC5mb2p0YS5jb20wggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQC1
    UeaPIsHjRQb8PkybTv7tfCe6oyq8UUxc7tiwX+nWvHWKJ9X6ASis1v/gT0CCa4cG
    fP+tezXayXMrwFSRu6OvanBSTVYvaAbUQl5CsVnQaaeCC5bTAMfGlDsl/q+LnqzW
    i0eG4hTpWG78v88AZkaHjIZdr5CQuDaPGJUqzOgrHjpYTLDJs+oK+S7ScpMyKhia
    hgeRKDfrEeQGGvSEMdHhg3Bg8RK8eyLQLjwCSCVkhYTrM5wyt4fow65beoMnOBbx
    LO+QpB6/amy5mJuVLVx+WJivVvuId2hmELorm2iJMjUAabybLmbMPmqHTTGyZaYw
    vxaDRDr0DbTMUYFyOh6LAgMBAAGjggJHMIICQzAdBgNVHQ4EFgQUi1Dhxpbkz9Dh
    tYOljP+MW/9GF+AwHwYDVR0jBBgwFoAUs0GCJG1KfknG9couJQXq4CZq4SQwgfoG
    A1UdHwSB8jCB7zCB7KCB6aCB5oaBr2xkYXA6Ly8vQ049Zm9qdGEtREMtQ0EoMSks
    Q049REMyLENOPUNEUCxDTj1QdWJsaWMlMjBLZXklMjBTZXJ2aWNlcyxDTj1TZXJ2
    aWNlcyxDTj1Db25maWd1cmF0aW9uLERDPWZvanRhLERDPWNvbT9jZXJ0aWZpY2F0
    ZVJldm9jYXRpb25MaXN0P2Jhc2U/b2JqZWN0Q2xhc3M9Y1JMRGlzdHJpYnV0aW9u
    UG9pbnSGMmh0dHA6Ly9EQzIuZm9qdGEuY29tL0NlcnRFbnJvbGwvZm9qdGEtREMt
    Q0EoMSkuY3JsMIG7BggrBgEFBQcBAQSBrjCBqzCBqAYIKwYBBQUHMAKGgZtsZGFw
    Oi8vL0NOPWZvanRhLURDLUNBLENOPUFJQSxDTj1QdWJsaWMlMjBLZXklMjBTZXJ2
    aWNlcyxDTj1TZXJ2aWNlcyxDTj1Db25maWd1cmF0aW9uLERDPWZvanRhLERDPWNv
    bT9jQUNlcnRpZmljYXRlP2Jhc2U/b2JqZWN0Q2xhc3M9Y2VydGlmaWNhdGlvbkF1
    dGhvcml0eTAhBgkrBgEEAYI3FAIEFB4SAFcAZQBiAFMAZQByAHYAZQByMA4GA1Ud
    DwEB/wQEAwIFoDATBgNVHSUEDDAKBggrBgEFBQcDATANBgkqhkiG9w0BAQsFAAOC
    AQEArNmceh3GkHx5Qf9s/NUlG7lh+i5nPUzlbnlhTJQeOJXSKG3DzU8ocb1xWguT
    x1ICyLZTQq11q3D3/3xgi9KJJaWzo8X5O/Mj81x4V8Jp9d8OgERc7lrVyrAJPbJA
    k7q/4tY41VOu8P5i+A21Pxuo3xELkOt5rcb2qt6QH1QizSA8Dzjq8uwCpdDo8eCP
    ZWUwc2lNUOyCmhFD7boNecHRJZN92i3W0YKfV+BC0cIXnqU2Y+4YEKAHWwH/gRm0
    ZI41oyatyoHpTjCGFtKzrSo/mxitIoj5ZTY9wwSNPlcmziw29hOTM1fOx//rqgrW
    17CREB/BoFj12Hd9bVXFgMGUSg==
    -----END CERTIFICATE-----</MemberEndpointCertificate>
        </OrgAssociationMember>
        <OrgAssociationMember href="https://vcloud2.fojta.com/api/admin/org/ca5295f0-a521-4d4c-8b2e-322f154fbbea/associations/13e52807-3d0a-4c0f-abdb-62d8fccb36ea" type="application/vnd.vmware.admin.organizationAssociation+xml">
            <MemberUrl>https://vcloud.fojta.com/api/org/13e52807-3d0a-4c0f-abdb-62d8fccb36ea</MemberUrl>
            <MemberName>ACME2</MemberName>
            <MemberEndpointCertificate>-----BEGIN CERTIFICATE-----
    MIIFfzCCBGegAwIBAgITTgAAARuwZOW3iRv9KQABAAABGzANBgkqhkiG9w0BAQsF
    ADBCMRMwEQYKCZImiZPyLGQBGRYDY29tMRUwEwYKCZImiZPyLGQBGRYFZm9qdGEx
    FDASBgNVBAMTC2ZvanRhLURDLUNBMB4XDTE1MDExNTE2NTAwNFoXDTE3MDExNDE2
    NTAwNFowZzELMAkGA1UEBhMCQ1oxDzANBgNVBAcTBlByYWd1ZTESMBAGA1UEChMJ
    Zm9qdGEuY29tMRgwFgYDVQQLEw92Q2xvdWQgRGlyZWN0b3IxGTAXBgNVBAMTEHZj
    bG91ZC5mb2p0YS5jb20wggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQC1
    UeaPIsHjRQb8PkybTv7tfCe6oyq8UUxc7tiwX+nWvHWKJ9X6ASis1v/gT0CCa4cG
    fP+tezXayXMrwFSRu6OvanBSTVYvaAbUQl5CsVnQaaeCC5bTAMfGlDsl/q+LnqzW
    i0eG4hTpWG78v88AZkaHjIZdr5CQuDaPGJUqzOgrHjpYTLDJs+oK+S7ScpMyKhia
    hgeRKDfrEeQGGvSEMdHhg3Bg8RK8eyLQLjwCSCVkhYTrM5wyt4fow65beoMnOBbx
    LO+QpB6/amy5mJuVLVx+WJivVvuId2hmELorm2iJMjUAabybLmbMPmqHTTGyZaYw
    vxaDRDr0DbTMUYFyOh6LAgMBAAGjggJHMIICQzAdBgNVHQ4EFgQUi1Dhxpbkz9Dh
    tYOljP+MW/9GF+AwHwYDVR0jBBgwFoAUs0GCJG1KfknG9couJQXq4CZq4SQwgfoG
    A1UdHwSB8jCB7zCB7KCB6aCB5oaBr2xkYXA6Ly8vQ049Zm9qdGEtREMtQ0EoMSks
    Q049REMyLENOPUNEUCxDTj1QdWJsaWMlMjBLZXklMjBTZXJ2aWNlcyxDTj1TZXJ2
    aWNlcyxDTj1Db25maWd1cmF0aW9uLERDPWZvanRhLERDPWNvbT9jZXJ0aWZpY2F0
    ZVJldm9jYXRpb25MaXN0P2Jhc2U/b2JqZWN0Q2xhc3M9Y1JMRGlzdHJpYnV0aW9u
    UG9pbnSGMmh0dHA6Ly9EQzIuZm9qdGEuY29tL0NlcnRFbnJvbGwvZm9qdGEtREMt
    Q0EoMSkuY3JsMIG7BggrBgEFBQcBAQSBrjCBqzCBqAYIKwYBBQUHMAKGgZtsZGFw
    Oi8vL0NOPWZvanRhLURDLUNBLENOPUFJQSxDTj1QdWJsaWMlMjBLZXklMjBTZXJ2
    aWNlcyxDTj1TZXJ2aWNlcyxDTj1Db25maWd1cmF0aW9uLERDPWZvanRhLERDPWNv
    bT9jQUNlcnRpZmljYXRlP2Jhc2U/b2JqZWN0Q2xhc3M9Y2VydGlmaWNhdGlvbkF1
    dGhvcml0eTAhBgkrBgEEAYI3FAIEFB4SAFcAZQBiAFMAZQByAHYAZQByMA4GA1Ud
    DwEB/wQEAwIFoDATBgNVHSUEDDAKBggrBgEFBQcDATANBgkqhkiG9w0BAQsFAAOC
    AQEArNmceh3GkHx5Qf9s/NUlG7lh+i5nPUzlbnlhTJQeOJXSKG3DzU8ocb1xWguT
    x1ICyLZTQq11q3D3/3xgi9KJJaWzo8X5O/Mj81x4V8Jp9d8OgERc7lrVyrAJPbJA
    k7q/4tY41VOu8P5i+A21Pxuo3xELkOt5rcb2qt6QH1QizSA8Dzjq8uwCpdDo8eCP
    ZWUwc2lNUOyCmhFD7boNecHRJZN92i3W0YKfV+BC0cIXnqU2Y+4YEKAHWwH/gRm0
    ZI41oyatyoHpTjCGFtKzrSo/mxitIoj5ZTY9wwSNPlcmziw29hOTM1fOx//rqgrW
    17CREB/BoFj12Hd9bVXFgMGUSg==
    -----END CERTIFICATE-----</MemberEndpointCertificate>
        </OrgAssociationMember>
    </OrgAssociations>
    

    As can be seen I am supplying URI for each Organization, its name and endpoint certificate.

  2. I can review organization associations with GET /admin/org/{id}/associations call or I can add or remove single association with similar POST / DELETE calls.
  3. Now when the end-user authenticates against the organization that has these associations he can run query calls that will run against all associated organizations. To make the query federated he must add federated=global string to the Accept header.
    GET https://vcloud2.fojta.com/api/query?type=organization
    Accept: application/*;version=9.0;federated=global
    x-vcloud-authorization: 4a1f241b371b46f5a36abac85f5962c7
    

    The reply lists all three organizations:

    <?xml version="1.0" encoding="UTF-8"?>
    <QueryResultRecords xmlns="http://www.vmware.com/vcloud/v1.5" name="organization" page="1" pageSize="128" total="3" href="https://vcloud2.fojta.com/api/query?type=organization&amp;page=1&amp;pageSize=128&amp;format=records" type="application/vnd.vmware.vcloud.query.records+xml" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.vmware.com/vcloud/v1.5 http://vcloud2.fojta.com/api/v1.5/schema/master.xsd">
        <Link rel="alternate" href="https://vcloud2.fojta.com/api/query?type=organization&amp;page=1&amp;pageSize=128&amp;format=references" type="application/vnd.vmware.vcloud.query.references+xml"/>
        <Link rel="alternate" href="https://vcloud2.fojta.com/api/query?type=organization&amp;page=1&amp;pageSize=128&amp;format=idrecords" type="application/vnd.vmware.vcloud.query.idrecords+xml"/>
        <OrgRecord canPublishCatalogs="false" deployedVMQuota="0" displayName="ACME Corporation" isEnabled="true" isReadOnly="false" name="ACME" numberOfCatalogs="1" numberOfDisks="0" numberOfGroups="7" numberOfVApps="1" numberOfVdcs="2" storedVMQuota="0" href="https://vcloud.fojta.com/api/org/02b433db-0b37-4304-b07b-0717255ec297" numberOfRunningVMs="1"/>
        <OrgRecord canPublishCatalogs="false" deployedVMQuota="0" displayName="ACME Inc." isEnabled="true" isReadOnly="false" name="ACME2" numberOfCatalogs="0" numberOfDisks="0" numberOfGroups="1" numberOfVApps="1" numberOfVdcs="1" storedVMQuota="0" href="https://vcloud.fojta.com/api/org/13e52807-3d0a-4c0f-abdb-62d8fccb36ea" numberOfRunningVMs="0"/>
        <OrgRecord canPublishCatalogs="false" deployedVMQuota="0" displayName="Acme Inc." isEnabled="true" isReadOnly="false" name="ACME" numberOfCatalogs="0" numberOfDisks="0" numberOfGroups="1" numberOfVApps="0" numberOfVdcs="0" storedVMQuota="0" href="https://vcloud2.fojta.com/api/org/ca5295f0-a521-4d4c-8b2e-322f154fbbea" numberOfRunningVMs="0"/>
    </QueryResultRecords>
    

Design Considerations

  • Only GET requests for the top level Org Level data (org, sessions, tasks, templates/catalogs) and all Query Service requests by Org users can be federated.
  • All System Admin query request span single vCloud Director instance and cannot be federated.
  • All non-GET requests cannot be federated.
  • All associated organizations must use the same identity source as the user is authenticated against all organizations and vCloud Director stores the session tokens for subsequent requests. Local accounts are not supported.
  • OAuth identity source offers the lowest administration overhead  when managing multiple organizations belonging to the same end-user as the user/role management can be performed centrally (see vCAT-SP for more information).
  • Association works only in one direction: when organization A is associated with organization B it does not mean that organization B is associated with A. Bidirectional association must be explicitly configured for both organization.