Load Balancing vCloud Director Cells with NSX Edge Gateway

About two years ago I have written article how to use vShield Edge to load balance vCloud Director cells. I am revisiting the subject; however this time with NSX Edge load balancer. One of the main improvements of the NSX Edge load balancer is SSL termination.

Why SSL Termination?

All vCloud GUI or API requests are made over HTTPS. When vShield (vCNS) Edge was used for load balancing it was just passing through the traffic untouched. There was no chance to inspect the request – the load balancer saw only source and destination IP and encrypted data. If we want to inspect the HTTP request we need to terminate the SSL session on the load balancer and then create a new SSL session towards the cell pool.

SSL Termination

 

This way we can filter URLs, modify the header or do even more advanced inspection. I will demonstrate how we can easily block portal access for a given organization and how to add X-Forwarded-For header so vCloud Director can log the actual end-user’s and not only load balancer’s IP addresses.

Basic Configuration

I am going to use exactly the same setup as in my vShield article. Two vCloud Director cells (IP addresses 10.0.1.60-61 and 10.0.1.62-63) behind Virtual IPs – 10.0.2.80 (portal/API) and 10.0.2.81 (VMRC).

vCloud Director Design

While NSX Edge load balancer is very similar to vShield load balancer the UI and the configuration workflow has changed quite a bit. I will only briefly describe the steps to set up basic load balancing:

  1. Create Application Profiles for VCD HTTP (port 80), VCD HTTPS (port 443) and VCD VMRC (port 443). We will use HTTP, HTTPS and TCP types respectively. For HTTPS we will for now enable SSL passthrough.
  2. Create new Service Monitoring (type HTTPS, method GET, URL /cloud/server_status)
    Service Monitor
  3.  Create server Pools (VCD_HTTP with members 10.0.1.60 and 62, port 80, monitor port 443; VCD_HTTPS with members 10.0.1.60 and 62, port 443, monitor port 443 and VCD_VMRC with members 10.0.1.61 and 63, port 443, monitor port 443). Always use monitor created in previous step. I used Round Robin algorithm.
    Pools
  4. Create Virtual Servers for respective pools, application profiles and external IP/port (10.0.2.80:80 for VCD_HTTP, 10.0.2.80:443 for VCD_HTTPS and 10.0.2.81:443 for VCD_VMRC).
  5. Enable load balancer in its Global Configuration.
    Global LB Config

Now we should have load balanced access to vCloud Director with identical functionality as in vShield Edge case.

Advanced Configuration

Now comes the fun part. To terminate SSL session at the Edge we need to create and upload to the load balancer vCloud http SSL certificate. Note that it is not possible to terminate VMRC proxy as it is a poor socket SSL connection. As I have vCloud Director 5.5 I had to use identical certificate as the one on the cells otherwise catalog OVF/ISO upload would fail with SSL thumbprint mismatch (see KB 2070908 for more details). The actual private key, certificate signing request and certificate creation and import was not straightforward so I am listing exact commands I used (do not create CSR on the load balancer as you will not be able to export the key to later import it to the cells):

  1. Create private key with pass phrase encryption with openssl:
    openssl genrsa -aes128 -passout pass:passwd -out http.key 2048
  2. Create certificate signing request with openssl:
    openssl.exe req -new -in http.key -out http.csr
  3. Sign CSR (http.csr) with your or public Certificate Authority to get http.crt.
  4. Upload the certificate and key to both cells. See how to import private key in my older article.
  5. Import your root CA and http certificate to the NSX Edge (Manager > Settings > Certificates).
    Certificates

Now we will create a simple Application Rule that will block vCloud portal access to organization ACME (filter /cloud/org/acme URL).

acl block_ACME path_beg – i /cloud/org/acme
block if block_ACME

Application Rule

Now we will change previously created VCD_HTTPS Application Profile. We will disable SSL Passthrough, check Insert X-Forwarded-For HTTP header (which will pass to vCloud Director the original client IP address) and Enable Pool Side SSL. Select previously imported Service Certificate.

Application Profiles

And finally we will assign the Application Rule to the VCD_HTTPS Virtual Server.

Virtual Servers

Now we can test if we can access vCloud Director portal and see the new certificate, we should not be able to access vCloud Director portal for ACME organization and we should see in the logs the client and proxy IP addresses.

ACME Forbidden

Client IP

For more advanced application rules check HAProxy documentation.

Advertisements

2 thoughts on “Load Balancing vCloud Director Cells with NSX Edge Gateway

  1. Great article Tom, we’re looking into using NSX edge for our vCD cell load balancing. One question – would it be possible to redirect (presumably using application rules) individual source ip/source ranges to specific client organization URLs?

    I’ve had a look through the HAProxy documentation and examples but can’t figure out if this is either possible or straightforward.

    e.g.
    Client 1 has external IP 4.4.4.4 so gets redirected to https://cloud.company.com/cloud/org/client1
    Client 2 has external IP range 5.5.5.0/24 so gets redirected to https://cloud.company.com/cloud/org/client2
    etc.

    Since we’re limiting access to our vCD portal based on source IP, this would allow all tenants to use the same base URL (cloud.company.com) and not have to remember/include the correct /cloud/org// path.

    1. I think it should be possible. You would create acl based on source IP address and reqrep to replace string in the HTTP request. Something like:
      acl ACL_ACME src 4.4.4.4
      reqrep search-expression "/cloud/org/ACME" if ACL_ACME

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s