Manual vCNS / vShield Edge HA Little Gem!

vCNS-HARecently, I have been doing lots with vCNS and manual creation / manipulation of vShield Edge devices (posts coming soon). One thing that drive me crazy is a tiny little thing that prompted me to write this quick Little Gem – ‘Edge HA’ sat on my to do list, and gloated at me…..

When creating a manual vShield Edge device in vCNS, there is the usual opportunity to create an pair of appliances for running the pair in High Availability mode. Trouble is, the options for deployment are limited and not very clear. (This might be clear / obvious to some, but weren’t to me!)

When creating an HA pair, in the vShield Manager console editing the Edge device in question under Settings – the HA Configuration gives few options. Essentially, ‘Enabled’ or ‘Disabled’, vNIC, Declared Dead Time and Management IPs. Here’s where my confusion was based. Management IPs. So many questions……!

The option for Management IPs is even outlined. 2 IP entry boxes, and note text: ‘You can specify pair of IPs (in DIDR format with /30 subnet. Management IPs must not overlap with any vnic subnets’.

OK, so I need Management IPs to manually create a HA pair. What /30 address range do I need to specify? Can the IP range share an existing vNIC, or does the Edge device need another interface or uplink. Where do I define the /30 addresses. Do they need their own vLANs? Must I create a whole new private address range specifically for HA heartbeat? Like I said – so many questions. Scour the documentation, Google ‘vShield Edge Management IPs’ produces no helpful results. So – to the LAB!

Turns out, you don’t need Management IPs at all. Simply change the HA Status to ‘Enable’, select a vNIC to support HA heartbeat, and add a second Edge appliance via the green plus symbol (it will prompt for the parameters) to deploy the HA pair! When both report as ‘Deployed’, HA is configured and your Edge device is protected.

Sigh. Like I said. This might seem obvious to some, but it wasn’t to me. ‘Edge HA’ is no longer on my to-do list!

Windows 2008 R2 Domain Trust – Fixing The Security DB Trust Relationship

I came across an interesting problem recently. A customer is trialling child DNS domains for their server in their lab set-up, and we came across an interesting problem when changing the FQDN of the server.

In this instance, we were playing with using dcpromo to promote and demote the domain controller (DC) into a single domain. At one point, we got out of step with the reboots and got the following message when logging-in to the domain:

The Security database on the server does not have a computer account for this workstation trust relationship.

Interesting. Somehow, we managed to get the domain out of sync with the FQDN of the server. We found 3 ways to resolve this issue, 2 simple (which felt a little like a hack) and 1 that permanently fixed the issue:

  1. Login to the DC with local admin credentials, and in Computer / Properties: change the FQDN.local name to just the machine name, then reboot the server.
  2. Update the DC FQDN in the domain DNS system, so the current DC FQDN and that in the DNS server match.
  3. Update the computer account in the domain to reflect the FQDN of the DC. To do this:
    1. Open your domain Users & Computers console.
    2. Under View, select Advanced Features.
    3. Find the computer account of your DC. (Depending on what your policy is, this might be under <AD Host> / <Domain> / Domain Controllers, or elsewhere in another OU. If in doubt, use the domain search for the machine name, or ask the domain admin!
    4. Open Properties of the computer account, then select “Attributes Editor”. (If you can’t see it, re-check step 2 above).
    5. Look for attributes “dNSHostName” and “servicePrincipleName” (and anywhere else the FQDN of the DC is found).
    6. If incorrect (they should be – to produce the error on the DC), update them to the correct values.
    7. Reboot the DC and try connecting again. If the error persists, re-check the attributes in step 5 again for any missed values that need resetting. Also bear in mind that the DCs may need some time to replicate if your domain has more than one DC.

Also worth pointing out that these errors don’t always require a reboot, so if you are doing this under a change window or are limited for time then it’s possible the fix will be applied fine without a reboot. But then hey – your DCs are all virtual, so will reboot in about 10 secs anyway. Right?

Windows DNS Queries Failing

I was putting together a PoC late last year and encountered an issue that I’ve not seen before that was caused by some functionality within Windows 2008 that I did not know existed at the time. In fact, no one who I’ve mentioned it to since knew about it either. It seemed sufficiently obscure that I thought I should write about it quickly.

In the PoC, I had created a Windows domain based on Server 2008 R2. Setting that up was simple enough and I’d done it many times before. What I began to notice though was that DNS queries forwarded outside of my PoC infrastructure were failing more often than not. This made Microsoft Updates impossible to install amongst other issues.

After fiddling with the DNS timeouts and talking with the hosting provider at the remote datacenter to no avail, I discovered this Microsoft KB article:

Some DNS name queries are unsuccessful after you deploy a Windows Server 2003 or Windows Server 2008 R2-based DNS server

After following the workaround instructions and issuing the following command on the Windows DNS server…

dnscmd /config /enableednsprobes 0

…external DNS queries were successfully resolved 100% of the time. Following up with the hosting provider, they confirmed that the larger UDP packets would have been dropped by their firewalls.

Home Lab Build: vSpecialistLabs v2

So, time to update the home labs information, and yes, this time I may have overdone it a little in one or two areas.

I spend most of Sunday rebuilding my home lab (christened some time ago as vSpecialistlabs v2), adding some elements, changing and tweeking some hardware, and removing other hardware I wasn’t using at the time.

Essentially, I’ve ended-up with a home lab that comprises the following aspects:

  • Multi-site VM configuration, with multi-host clusters at both sites.
  • iSCSI shared storage for main ‘production’ site.
  • vSphere Replication to backup ‘DR’ site.
  • Managed networking.

Below is a picture of my home lab set-up, and you can immediately see where I may have gone OTT – screens! For some reason, I love to have screen real estate.

overview

The components of the lab / set-up are as follows:

  1. Servers:
    1. Server 1: IBM S5520HC chassis with 2 x E5520 2.26GHz, 24GB RAM, 1TB SATA, H/W iSCSI & Dual 1GB NICs.
    2. Server 2: As server 1 above.
    3. Server 3: HP NL35l MicroServer with 8GB RAM.
    4. Server 4: As server 3 above.
    5. Main PC. Desktop PC from Servers Plus. (Updated range can be found here). Intel i7-2700 Quad core @ 3.50GHz, 32GB RAM, 2 x OCZ Vertex 4 SSDs and 2TB SATA, X64 Windows 7 Pro. Eizo CE210W (main monitor) plus Dell E177FP (second monitor).
  2. Storage:
    1. 8 TB QNAP 459 Pro II NAS. (4 x 2TB drives in 2 RAIDs).
    2. Iomega external 1TB USB/FW disk.
  3. Networking:
    1. HP 1910-16G Managed gigabit switch.
    2. HP 1410-8G Un-managed gigabit switch.
  4. Accessories:
    1. Belkin Soho 4-port VGA KVM, with bluetooth USB keyboard – for all servers.

I will add more information about how the lab grows and is configured – especially in the light of required revision for updating my VCAP certification to v5. Things to note:

  • The cabling is far from finished! I’m still on connectivity at the moment – looking pretty is next phase.
  • Power configuration is top of the list. Running this from multi-plugs is not ideal (at least they aren’t daisy-chained!) The servers and PCs are all connected to surge protector PDUs.
  • For my PC, iMac and laptops I use Synergy across all clients for a single KVM view. For the servers, I use the Belkin KVM and separate keyboard.
  • I’m not a specific network focused bod, but I am looking at expanding the lab in the near future into the Cisco arena, for CCENT certification and beyond.

In the meantime, please feel free to ask questions or comment on my set-up, I’m always looking for ways to improve!

Do my ESXi hosts have the same VLANs?

PowerCLIIn a small vSphere environment that I’ve recently been working on, I started to notice that some of my VMs were disappearing off the network from time to time. Reboots of the VM didn’t seem to fix the issue but a quick vMotion of the VM to another host did.

If you haven’t figured it out yet, one of my hosts was missing a VLAN and VMs connected to a certain portgroup were affected whenever they ran on the host.

vSphere will warn you if a host that you’re trying to migrate a VM to doesn’t have the right portgroup and host profiles (if you’re using Enterprise Plus licensing) will alert you to the fact that a portgroup isn’t configured with the right VLAN ID but nowhere in vSphere will you get an alert if a required VLAN is not being presented to a host. So you have to use other means to check this information.

You could manually examine the properties of each physical NIC in turn but that could take some time. The method that I used on this occasion was a PowerCLI script. I could have written one myself but a quick google lead me to a script written by Luc Dekens that did what I wanted already (and a little more besides). I modified it to suit my needs (demonstrating to the person in the remote datacenter that there was a network misconfiguration) and ran it. The output is below:

Host:  esx1.mydomain.com

vmnic0  VLAN224 VLAN227

vmnic1  VLAN224 VLAN227

vmnic2  VLAN250 VLAN252 VLAN251

vmnic3  VLAN250 VLAN252 VLAN251

Host:  esx2.mydomain.com

vmnic0  VLAN227 VLAN226 VLAN224

vmnic1  VLAN227 VLAN226 VLAN224

vmnic2  VLAN251 VLAN252 VLAN250

vmnic3  VLAN251 VLAN252 VLAN250

Host:  esx3.mydomain.com

vmnic0  VLAN224 VLAN227 VLAN226

vmnic1  VLAN224 VLAN227 VLAN226

vmnic2  VLAN250 VLAN252 VLAN251

vmnic3  VLAN250 VLAN252 VLAN251

Host:  esx4.mydomain.com

vmnic0  VLAN224 VLAN226

vmnic1  VLAN224 VLAN226

vmnic2  VLAN250 VLAN251

vmnic3  VLAN250 VLAN251 VLAN252

As you can see, there are some discrepancies in which VLANs are presented to the four hosts that I ran it against and vmnic2 on Host4 was the one causing my problems. The hosts are supposed to have the vmnics paired (vmnic0/vmnic1 in one pair and vmnic2/vmnic3 in another) with identical configuration between the hosts.

The modified script that I used is attached below. Many thanks, as always, LucD.

Show-PNICVLANs.ps1