vCloud Director not reporting CPU Utilization

I came across a customer site recently where their vCloud Director (vCD) 5.1 implementation was not reporting CPU utilization for Organisation Virtual Datacenters (Org VDCs). RAM and storage allocation was fine, just CPU was not showing a usage bar, and the mouse over tip reported 0% allocated. Turns out this is a known issue with vCloud Director 5.1, and VMware have released KB 2054043 relating to this issue. Their advice (via the KB) is to: Upgrade vCloud Director to version 5.1.2 or later. Enable Elastic Allocation Pool mode for vCloud Director. To do this: Login to vCD as a System Administrator user. Navigate to Home > Administration > General. Unter the Miscellaneous section, check the box next to 'Make … [Read more...]

vCloud Director 5.1 to 5.5 Cell Upgrade ‘cpio: chown failed’


Upgrading my lab environment from vCloud Director v5.1 to v5.5, I came across an interesting error whilst upgrading the cells. My lab has the following vCD configuration: 2 x RHEL 6.2 Cells 1 x RHEL 6.2 NFS Server 2 x vShield load balancer instances 1 x Windows 2008 R2 DB server running SQL Server 2005 The upgrade process was: Quiesce the cell using the Cell Management Tool commands. (Upgrade Guide) Upload the vCD .BIN file to the /install directory of the cell (using WinSCP or similar). Change the execution parameters for the vCD .BIN file. (Upgrade Guide) Running the installation .BIN file. (Upgrade Guide) Confirm the existing v5.1 cell instance can be upgraded. This is where the interesting error came in. The … [Read more...]

Troubleshooting vCloud Automation Center Endpoints and vCloud Director 5.1

BACKGROUND Note: For this troubleshooting article, I'm using vCloud Director 5.1 and vCloud Automation Center 5.2. Having fun with my lab. Lots of fun - and as readers may have noticed, my current focus is around deploying vCloud Director (vCD) and integrating it with vCloud Automation Center (vCAC). However, all is not plain sailing. Firstly, the installation of vCAC v5.2 is much better than installing v5.1. By better I mean more streamlined, the deployment packages retain more explanation, and lots of the installation 'gotchas' have been either removed, fixed or simply hidden from the installation process. I'll not detail the installation process here as there are lots of other guides out there. This break-fix article is about a … [Read more...]

Fixing vCloud Director ‘HTTP NFC Error’ & Sysprep


Recently I've been doing some work on vCloud Director deployments, typically new VMs within Organisations. On powering-on VMs or deploying vApps, I got the error 'Unable to start vApp '<vApp Name>'. The more specific error related to one or more of the following errors: 'HTTP NFC Error'. Cannot deploy package from /opt/vmware/vcloud-director/.... etc etc. Unable to start vApp. Unable to start virtual machines in resource pool. Broken pipe. Broken pipe. Solution: A little bit of digging on the vCloud cell (RHEL in this case) showed the permissions were incorrect on the 'guestcustomization' folder, and that changing these fixed the problem. A little more investigation showed-up the … [Read more...]

Site-2-Site VPN Issue between vShield 5.1 and Checkpoint

I came across an interesting issue whilst configuring a site-2-site VPN connection between a vCloud Director 5.1 customer vDC and a remote customer site. The requirement was for the VPN to be configured to pass through the perimeter hardware firewalls and terminate on the external interface of the vShield Edge device connected to the vCloud external network. Firewall configuration was done on the vShield device connecting the vDC routed organisation network and the vCloud external network - defining the peer and local network configurations, plus vShield firewall rules to allow traffic through the Edge device to reach the VMs inside the organisation. The problems we had in this instance were multiple - the customer only had a change … [Read more...]

Manually Removing vCloud Director Agents

There are a few scenarios where you need to manually remove a vCloud Director agent from an ESXi host, for example troubleshooting or re-preparing a host following a failed vCloud Director installation. It's quite simple to achieve in both ESXi 4.1 and 5.x host versions. (You need to enable SSH for the host in the security profile for this method, if it isn't usually running). 1) In both cases, SSH to the ESXi host, authenticating as root user. 2a) For ESXi 4.1 hosts, run the following command: /opt/vmware/uninstallers/ 2b) For ESXi 5.x hosts, run the following command: esxcli software vib remove -n vcloud-agent 3) DIsconnect your SSH session. Re-secure the host in the security profile if SSH is not usually … [Read more...]

Isilon, UCS, Nexus get the nod for UK Community Cloud Infrastructure


(Point of note: I work for Eduserv, this is my technology take on my current project deployment). Eduserv are building a new cloud service called 'The Community Cloud Infrastructure', with the first service to be deployed being the 'University Modernisation Fund Cloud Pilot'. (Subsequent public and hybrid clouds are planned for deployment on the platform in the future, to support Education, Health, Public Sector and other public bodies). As part of this project, there are some exciting choices and this post is to provide some background information about the technology stack to be used in the infrastructure, and here is the technology stack outline. The purpose of the platform is to be both scalable and resilient, whilst providing … [Read more...]

Troubleshooting vCloud Director Cells

So, you have a single or multi-cell vCloud Director installation, but some of the cells are misbehaving or are having issues? Here are some tips, tricks and gems of information I have collected over time..... NTP. Time is critically important to vCD cells. If there is a difference between the cells local time is greater than 2 seconds, there will be connectivity issues. DNS. This is critical - make sure the cells have forward and reverse DNS entries in contactable DNS server. Check the 'Transfer' folder is read/write. If the NFS share is mounted as read-only - the cells will complain and not function. If a single cell is working, and another cell is added to the configuration, when presenting the transfer NFS mount to the second … [Read more...]

Ephemeral Ports with VMware vCloud Director 1.5 and vSphere 5

This is something that I have been thinking about in a recent build project, where we are considering using vCDNI-Backed networks in vCloud Director. Dave Hill writes a great article on using (or not) Ephemeral Ports in vCD here: ...where the focus was on 4.0 vs. 4.1 deployments. With the recent release of vSphere 5, the arguments for and against using Ephemeral ports in a capacity context come into sharper focus: vSphere 4.1: 1,016 PortGroups per vDS/vCenter vSphere 5: 256 PortGroups per vDS/vCenter Source: vSphere 5 Maximums Guide: vSphere 4.1 Maximums … [Read more...]

Building vCloud Director Cells using CentOS6

In my lab infrastructure (build information soon to be published for information), I have been playing with vCloud Director 1.5. As part of my work day, I am developing a cloud platform as part of a team at Eduserv (for more information see, and part of that is to design availability zones so customer information when populated will be resilient across geographical locations. Now, not being part of the RedHat brigade (and a novice getting up to speed with CentOS), I have been playing with getting vCloud Director cells to work ok on CentOS (an 'unsupported' activity, according to the documentation - RHEL being a pre-req for vCD). In order to get this to work, I followed the following process. NOTE: … [Read more...]