Wednesday, 12 June 2013

vCloud Director - Set Administrator Password and then you cant login?

Hi All

Had a very strange problem today, I was delivering a vCD POC.  I left the database creation to the user and we just connected to it.  I provided them the scripts that located in the Installation Guide. We were provided a SQL user account as normal and ran the ./configure script and connected to the database with no problems.

Once we had completed the configuration we connected the web address for the cell and went over the initial configuration.  We set the password for the administrator and completed the wizard.

After this we then tried to login to the cell

User: administrator
Password: ******** (same as was set in the wizard)

Result = Access denied?

Very strange, as i was 100% certain the password was correct.  So I asked the SQL admin to run the following to rest the cell to a un-configured state. update config set value='false' where name = 'vcloud.system.initialized';   Once this was done we went over the initialization of the cell again and went set the same password, just incase I and typed the password incorrectly.  Still had the same Access Denied message.

So I ran the above SQL query again to reset the cell.  I then asked to have a look at the SQL management studio.  The DBA let me have a look and I could see he had made the account being used by vCD a sysadmin! this was not in the scripts in the install guide so it must have been done after.  Now I have seen strange things when sysadmin accounts have been used in previous VMware products for DB access.  Traditionally VMware products have suggested giving the accounts sysadmin access at install and then lock down after, but most the newer releases give more info on user rights needed so we no longer need to give sysadmin rights at all.

I removed the sysadmin privilege and reconfigured the cell with the same password as the last two times. Amazingly at the login screen I was then able to login.

Now I have no idea why this resolved the login problems and I have asked if I can log a bug internally to see if this is as designed or if it is a small bug.  I was using 5.1.2 vCD binaries on SQL 2008 Express, again I dont know if this had an affect on what was happening.  I just wanted to post this incase anyone has a similar problem, hopefully this will help.

Tuesday, 11 June 2013

vBrownbag's and Professionalvmware.com

Hi again

I wanted to let everyone know I have been hosting some sessions on the popular community site. professionalvmware.com I have covered 2 objectives from the VCAP-CID blueprint and I am planning on hosting a networking one soon to go a little more in-depth into the networking virtualization we introduce in vCNS and vCD.

Here are the links to my two vBrownbags.  Be sure to add me and the vBrownbag crew on twitter.

Phil Monk  vBrownbags

#vBrownbag Follow-Up Phil Monk Covering VCAP5-CID Objective 4 – vCloud Security

#vBrownbag with Philip Monk covering VCAP5-CID Objective 3

Thanks
Phil

Please Leave Feedback..... I Would love to hear what you think, good or bad

Hi Everyone

I have had some impressive views on some of the stuff I have published, especially the SRM RecoverPoint article I posted.  But next to no one leaves any feedback.  :-(  Good or bad, or even a follow up question.  Please feel free to ask me anything.

Many thanks
Phil

CBT - Change Block Tracking and its common miss interpretations.

I have been working on a vCloud project for the last few months and it has now come to looking at the backup of the solution.  Commvault is the existing backup vendor and is going to continue to provide backup services in this Private cloud.

Now the customer came to me with an interesting question about how CBT worked, they wanted to understand how the changed blocks in the vmdk were tracked.  This was partly down to there thirst for information and partly because I think they were concerned at contraints that might be seen with IO and CPU.

The perception of most people is that CBT creates snapshots to record all the information in the snapshot and then it is consolidated, much like the usage of a normal snapshot in VMware. This is incorrect and although I am no storage expert I will attempt to explain how it works from my VMware background.

How it actually works is by keeping track of the blocks that have changed in a vmdk based on significant disk events recorded for a specific vmdk.

Simple right?  Well the next question from my customer was "Well how does it do that? And is it going to cause me pain on my storage IO and/or my hosts"  - This is a valid question as recording all this information has to have some overhead.

So how does it do it? Lets use the most common usage of CBT, Backup of a Virtual Machine.
  1. We take a full Backup of a VM with CBT enabled on all disks.  This takes a snapshot to record all IO, while the backup is being processed on the original vmdk.  The backup application records the Change Clock timestamp (T1) at the time the snapshot is created to facilitate this backup. After the backup is completed normal snapshot consolidation is conducted and  the changes in the delta are merged into the original vmdk.
  2. When the next backup is started 24 Hours later, another snapshot is taken of the VM's vmdk's.  The backup application records the Change Clock timestamp (T2) at the time of creating this snapshot. All changes are written to the delta vmdk while we backup the changed blocks in the origonal vmdk.  We still create a snapshot as we need to have access to the original vmdk.
  3. The backup application then backups up only the blocks of the original vmdk that have changed between the two time stamps (C).  
Now the next part of the proccess I wanted to examine was "How does it know what blocks are changed"

When we enable change block tracking, detailed here in KB1020128 and the VM is powered on for the first time. Have a look in the Datastore Browser and the VMs home directory.  You will see a file named "vm_name-ctk.vmdk" this file is used to track the blocks that are changed on a given vmdk.  This auxiliary file has a pointer configured in the vmdk to point to this file to track changes made to blocks in the correcponding vmdk.


So when step three, listed above, is performed the auxiliary file is used to identify blocks based on the two time stamps, and the backup application then performs a backup of the changed blocks.

Now the change clock is not based on a normal clock and utilizes a Unix Epoch based clock,  Now I dont work in engineering and I am under NDA from VMware (being an employee) so I cant share any more information around the nitty gritty details.

But stepping back, my customer wanted to know if this was going to cause them any pain with IO and/or CPU on the hosts.  The simple answer is that the impact of using CBT is minimal.  It will maybe cause the kernal to in use 1 or 2 % more CPU per 20 or 30 VMs. However it is important to remember that the small increase the kernal sees now will help reduce the IO and/or CPU increase if the VM was being fully backed up every night because CBT was not being used.  backups will also be much shorter as well due to smaller amounts of data being backed up.

Hope this has been helpful, many thanks.

Phil