XenDesktop and Provisioning Services on vSphere 5

I want to preface this post by saying if you are not interested in any backstory just skip down to the summary section for a wrap up…

Coming from a consulting background I learned the hard way that you always have to check the compatibility of all components in a project.  Since VMware released vSphere 5 many people found out the hard way that Citrix Provisioning Services (PVS) was not supported on VMware’s latest hypervisor platform.  People without in-depth knowledge of the Citrix solution stack may have read XenDesktop 5 was supported on vSphere 5, but in reality that only meant when using Machine Creation Services (MCS).  This forced companies to have to make some hard decisions.  You could either go down the route of using MCS with vSphere 5, or go with Provisioning Services on vSphere 4.x.  Hopefully most customers consulted with their trusted Citrix advisor prior to making this decision.  Even as recently as a few weeks ago the lines were blurred when it came to using PVS with vSphere 5.  One of my clients has been having issues with PVS on vSphere 5 for close to two months.  Two months ago Citrix flat out did not support this configuration.  Citrix technical support will typically give a best effort to help in these situations, but in the end the recommendation is made to revert to vSphere 4.x.  Unfortunately, reverting to vSphere 4.x was not an option for my particular client because there were many man hours put in to building their vSphere 5 infrastructure and they had a deadline closing in fast to launch their desktop virtualization pilot.  I just began working with the client a few weeks ago and was unaware of all the pain they had been having with Provisioning Services on vSphere 5.  In a last ditch effort the PM for the project reached out to my ERM (Enterprise Resource Manager) and asked if there was anything they could do to get PVS to work before the end of the week, and if not they would have to scrap the pilot.  In theory they could have used MCS for XenDesktop and manually provisioned the XenApp servers for the pilot.  The problem is this is a government client, and the approved architecture and all pre-implementation documentation was built on the premise of PVS.  In essence they would have to start over, and that was not an option.  When my ERM sent me the client’s request for help I was concerned the pilot would fail because of PVS not being supported on vSphere 5, but wanted to do everything I could to help.  The issue the client saw was when booting multiple target devices to a vDisk in Standard Mode they would just hang at a blank black screen.  I had seen this issue at a previous project I worked on prior to coming to work for Citrix, and in the end we had to resort to using MCS.  I started digging around to try and find any updates on the issues related to PVS on vSphere 5, and I quickly learned we may have hit the jack pot!

About a week prior to jumping in to help this client I remembered Provisioning Services 6.1 came out, so the first thing I checked was if Citrix eDocs had added ESXi 5 to the supported hypervisors list for PVS 6.1.  I was disappointed to see that only ESX 4.1 was listed as supported, and even as I am writing this post ESXi 5.x is still not listed (See here).  Next I started searching the Citrix forums, which pointed me in the direction of this Citrix KB article that was just updated on March 26th, 2012 stating how to get PVS 5.6/6.0 working on vSphere 5!  While on a GoToMeeting with the client I had them pull up the article and they quickly said they had tried that resolution a few days prior with no luck.  Before moving on I walked through the article with them and we double checked everything was set correctly.  The PVS server bootstrap file did have “Interrupt safe mode (check this if the device hangs during boot)” like the article’s resolution stated but the issue was still occurring.

Figure 1

Many people, including myself at times, will skip to the “Resolution” section of a KB without reading all of the “Issue” section.  In this case the kb article above states that ONLY the VMXNET3 virtual NIC is supported, not the more commonly used E1000 virtual NIC. When checking the target devices and PVS virtual machine settings I noticed that they were still using E1000 virtual NIC’s.  The client mentioned Citrix technical supported told them in a previous call not to use VMXNET3 because it was not stable with PVS.  In fact, it was fairly well known in the consulting world that VMXNET3 caused issues with PVS so we typically used E1000 NIC’s.  Here are a couple older KB’s discussing the issue with VMXNET3, CTX128160 and CTX125361.  At this point I had the client change the virtual NIC settings on both the target devices and the virtual PVS servers to use VMXNET3.  Next we set the vDisks to Standard Mode, and booted a few target devices.  Voila!  The vDisk was being streamed without issue on vSphere 5!!

Summary:

Many customers running Provisioning Services on vSphere 5 were seeing target devices hang at a black screen while booting to a vDisk in Standard Mode.  On top of that Citrix did not support running Provisioning Services on vSphere 5 so people had to change their architecture to either use MCS on vSphere 5, or Provisioning Services on vSphere 4.x.  The good news is Citrix now officially supports running Provisioning Services on vSphere 5.x (ESXi 5.x)!  The caveat to this is you must use vSphere’s VMXNET3 virtual NIC on your target devices.  If you are running virtual Provisioning Servers you should also use VMXNET3 virtual NICs on those VM’s.  The last thing that must be done is to configure the Provisioning Server’s bootstrap file to “Interrupt safe mode” as seen above in Figure 1.  I would also recommend applying ESXi 5 Update 1 per the article referenced below in the third bullet.  ESXi 5 Update 1 resolved a very similar issue with Provisioning Services on vSphere 5.

References:

  • Another important article was published by VMware regarding a separate issue with vSphere 5 and Citrix Provisioning Services. Essentially the article states that ESXi 5 Update 1 resolves the issue mentioned in the article.
  1. Gerald ElizaldeGerald Elizalde08-12-2012

    I was working on a POC of XD 5.6 wth PVS 6.0 sitting on a vSphere 5. This was last March and I experienced the same thing. The NICs were on E1000, but I only got to find out when I was doing an isolation process wherein I created another VM with VMXNET3 NICs. From there on, everything worked.

    My take on this though is that when you search on the web, the results barely points you to the right articles, in my case.. never.

    Anyway, this was a good article to have confirmed that the troubleshooting I did was correct. And this site is full of useful articles.

    • Dane YoungDane Young08-31-2012

      Thanks so much for your response. Please follow any of our authors on Twitter and keep up the great work!

Leave a Reply