vCAC 5.2/6.0 Certificate checking - Error "The underlying connection was closed: Could not establish trust relationship for the SSL/TLS secure channel"

During a recent challenging installation I had a situation where I needed to front both the vCAC web servers with a customer generated verified certificate, and keep the rest of the certificates a Self sighed.

This was simple in theory, just replace the certificates on the web servers.

We generated certificate requests from both web servers and submitted them to the CA.  Once I had the certificates they were installed on the vCAC Web servers and I changed the bindings in IIS for 443.  I then placed both the new certs on the MGMT servers and restarted the services.  The MGMT service failed to start and then gave the following error

"The underlying connection was closed: Could not establish trust relationship for the SSL/TLS secure channel"  

I then looked in some of the vCAC documentation and found that I needed the following.

1. The intermediate CA

2. Domain certificate

Placed these on all the machines via group policy and then tried to start the service again.  Still the MGMT service failed.

I then investigated some of the vCAC code as I thought there must be something going on that was not apparent.  I found that all the components try to check sighed certs against the issuing CA authority root.  This is not documented clearly (if at all) so it was a bit of a find.  I then looked at how to over come this.  Some of you who have installed vCAC many times will recall there is an option to "Suppress certificate mismatch" when you install some of the site components.  

This setting stops the full chain check of each service.  But there is no option for this when installing many of the components.  So this is how you add it to the code.

Find the  "ManagerService.exe.config" file located in "/Program Files (x86)/VMware/vCAC/Server" right click and edit the file.  Go to the bottom of the file and just before "</Configuration>" tag insert the following code.

"<system.net>
            <settings>
            <servicePointManager checkCertificateName="false" checkCertificateRevocationList="false"/>
            </settings>

 </system.net>"

This will tell the service not to check the cert chain and will allow it to start.  You will also need to add this to the CONFIG files for all the other services such as the DEM's and the designer.  Same file and insert the same code in the same place. 

Placing all vCAC 5.2 web components and REPO404 error

Hi all

I know vCAC 5.2 is the previous version now with vCAC 6.0 being released.  However there are still a large number of vCAC 5.2 projects running.  Including the one I am working on at the moment.

I had a problem where the initial install was done by some colleagues  and I continued with the installation building out the vCAC web layer to a 2 node load balanced construct.  The install completed by my colleagues  was following my design.  I had placed the vCAC Manager Server role and the vCAC Model Manager Web services role on separate machines.  My reasoning around this was based on the security requirements of the customer.

The installation was functioning well until we came to build the second web server.  We had some major problems with all the portals.  We lost both the vCAC admin portal and the vCAC Self Service Portal and got REPO404 errors when trying to access them.

I looked over the logs of the vCAC web servers and could see loads of SSL errors.  So I check the certificates on all the servers and all of them were correct.

After some discussions internally I decided to place the Model manager web service on the web servers with the other website components.  We installed vCAC from the start and found this removed the error.

I think the problem was a combination of the SSL and communication URL that the model manager was using was being replaced with a new one when the second Admin portal service was installed.  This then conflicted with URLs and certs in the Manager service and maybe in the repository.  We could have over come this with a load balancer, but at the time we didn't have this available. 

Hope this helps if you see the REPO 404 error as there isn't much documented about it. 

Using Network Profiles in vCAC 5.2

Hi All

Another quick thing I thought I would blog during an install I am doing.  The customer is building from my design and came over to me and asked "Where is the Network Profiles option"  After going over and seeing what they were asking it became apparent that they were unaware of the obscure place this is enabled. 

Go to vCAC Administrator > Customization and then tick the box for "Enable Static IP service" Click OK and this will enable the Network Profiles option along the side bar.

Thanks
Phil

Installing ESXi via USB drive

Being mainly an architect or a design lead on many projects I am finding I don't tend  to get too involved with installs as much as I would like.  But when rebuilding my lab I found a good USB boot device tool.

Universal USB installer

Select the Other Linux Distro.  This will then let you select the ESXi install and then your USB. 

Very good tool and works a treat.

vSphere 5.5 - SSO Setup and additional configuration

Hi all,
So when I was rebuilding my lab I took note from internal communications that we had made some changes to SSO, both under the hood and for the install.

So I started the install and selected the "Simple Install" as I was only building a lab.

SSO Admin Account

Over the installation steps you have to configure the SSO admin password.  Now in 5.1 the default admin user was admin@system-domain First thing I noted was this had changed.  The default administrator account for SSO is now Administrator@vsphere.local

 

Attach Identity Source

So after the installation I logged in with the new Administrator@vsphere.local account.  I then logged out to see if the AD that the vCenter was a member of, had been added to the SSO identity sources.  With vSphere 5.1 the AD was added automatically.  I received an error message stating that it could not log in.

So I went to add the identity source to the SSO configuration using the Web Client. 

 

Select the configuration option down the side menu and then select the identity sources tab, and click the green plus.  You will get the below popup window.

 Because this was my Lab I used the domain Administrator account and I used port 389 for LDAP.  In a production environment you would need to configure a Read Only AD account specificity for vSphere to allow you to track AD access.  You would also want to use SSL for LDAP to make the network communication secure.

Test the connection and then click ok.  Give the Web Client a second to perform the action of adding the source.

Once the ID source is added you don't get access straight away, you need to add a group from the ID source to SSO and then to vCenter to allow it access.

Add ID source group to SSO 

Next you need to add a group from the ID source to SSO to allow access from the ID source to vCenter and SSO.

If I just login with an account from the ID source I get presented with an empty vCenter or an access denied as the account has no permissions.

I now need to add the accounts I am going to administer vSphere with, to the SSO group that is assigned Administrator permissions in vCenter.

By default the SSO group called Administrators is added to the vCenter group called Administrators. This is why the account Administrator@vsphere.local has access to the vCenter and all the objects it controls.

The account I want to administer the vCenter with is called vsphere-admin and the domain admin account called Administrator this is more a backup account.

This time load the Web Client select the configuration tab, and under SSO select Users and groups.  This will allow you to see the groups that are configured in SSO.

 

Select the Administrators group and the accounts included in this group are listed below.  If you select the green cross in the window below you can add an account to the group.

Select the correct ID source from the drop down, and then find the account you want to add.  Click add and then select the Ok button.

 

Web Client in my lab took a few mins to add this to the group but after the account has been added you should see a screen like the one below, showing the accounts you have added to the group.

 

 You can add accounts directly to the vCenter  Administrator group using the traditional methods.  So we can add the domain accounts to vCenter and bypass SSO altogether, I would not recommend this as SSO is used in other products and adoption of it now will also make it less painful when it is an essential component in future releases.

 

 

vSphere 5.5 - Whats changed

Hi All

I have tried not to call this a "Whats New" blog post as I plan on doing one of those in the future.  However I did want to post up some info about things I have noticed when building my vSphere 5.5 lab.

Coincidentally I have been asked by one of two customers about things they have noticed and so I thought it was worth publishing something I could reference to future questions. So a few posts will follow about things I have noted in my Lab setup.

My Lab is running the following

1 vCenter - Windows 2K8 R2 SQL Express
1 DC - Windows 2K8 AD, DNS.  AD running 2008 R2 domain level.
2 ESXi hosts - Resource
1 ESXi host - MGMT Runs vCNS
1 vC AC 5.2 Server

I may add more machines to the Lab later on but it is all running on a MacBook with Fusion Pro so I am limited to how many machines I can run.

Keep looking and Ill post updates soon.

vCD without vCNS

One of the most interesting solutions I have been working on recently is a vCD project where the customer has decided to use an alternative to vCNS (vCloud Networking and security).  I can not name the product they want to use as an alternative but for the discussion inside this blog I think the principle applies to most third party products.  

The first thing I asked the customer was why do you not want to use vCNS? The answer was quite simply that there security department doesn't see the vCNS firewall as secure. When asking for a bit more detail we were not provided with anything credible in my view.  One of the main problems they communicated as an issue is that the appliances are deployed on demand and are all applied with the same default password. (this can be changed after the appliance is deployed)  Anyhow this is not the aim of my blog, to detail my customer problems, and so we were asked to use an virtual firewall that they already use. 

Now the first thing I needed to comunicate to the customer we that we HAVE to install vCNS manager, this is a requirement of vCD and all network related instructions are sent to vCNS. This includes port group creations for vCD networks.  I also needed to point out to the customer we will be using VXLAN and vCNS is used in securing multitenancy within the VXLAN.

So once we had cleared all the above and it was agreed that there was no way we could push the customer back to using vCNS we looked at the use cases and the limitations that would be imposed. 

First lets break down what not using vCNS removed from vCD.

1. Limits the use of OrgvDC networks, we can't create a routed or an isolated OrgvDC network with IP addressing ranges specified. - When creating a network with an IP range  a vCNS appliance is applied to the networks corresponding dVportGroup.  This is to assign DHCP. 
2. We can't create any vApp networks with routed connections.
3. Networks and network services configuration is no longer made visible to vCD. This restricts what we can do for network service creation when using vCD. 
4. No vCO plugin for any third party software defined networking service. 
5. No Charge Back monitoring for any Network elements, usage of bandwidth, IP addresses and network services such as firewall rules and NAT rules.

So how did I design around this?

1. The customer was given two options of how to manage IP addressing inside the vCD networks. a) Was to use there third party firewall appliance, b) Was to use a vCO workflow to populate the IP address of the VM's in the vApp from there IPAM system.
2. We had to place an extra VM in the vApp.  This extra VM is the firewall VM and because vCD just thinks this is another VM it creates it in the users OrgvDC and uses the resource provisioned to the user/Organization.  If this was vCNS the appliances are provisioned in the SYSTEM resource pool and do not impact the users resource allocation.
3. The IPAM system and the firewall VM has no integration with vCD and this particular firewall vendors API is not very good. So using vCO to talk to it is a challange.  This gives the customer another system to monitor and manage as we cant pull any information from either of these systems.
4. We can try and develop a vCO plugin, most of the plugin's are developed by VMware but the limitation of this firewall vendors API presents a big challenge and my customer is not going to pay for a plugin development.
5. We can add a fixed cost for the VM in CBM and charge a fixed amount each month.  We can monitor bandwidth but not in the same way we can with a vCNS appliance. 

This is becoming more and more common and the above are some small use cases for my customer.  Other customers may have more risks and may also have more constraints and requirements of vCNS and/or a third party firewall appliance. 

With the release of NSX in the not so distant future I hope people start to see VMware network virtualization as a more credible technology.

So if you get asked this question I hope this post can give you some food for thought on the impact and the design considerations/constraints it will impose.