Virtualise GPU with XenApp and VDI
I was at BriForum Europe and one of the more interesting presentations was actually a sponsored session from NVidia showing us their Virtualized Graphic card options.
I don’t really believe in VDI (apart from specialized use cases), but this was interesting! Because the technology can also be used for XenApp/RDS services. They offered “Grid” cards “K1” and “K2” hardware GPU’s to place in servers (Dell, IBM and HP are mentioned), and on some servers you could fit 2 cards (like the Dell R720). The K1 has 4x GPU’s on board and the K2 has 2x GPU’s. Another difference is that the GPU’s on the K2 are more heavier than on the K1. This is due to how the hardware could be used on different platforms, but more on that later.
The whole session is available on Brian Madden, but i thought the overview/understanding could be better so that’s why i wrote this blog. Btw, if you are going to see the presentation now, let me warn you that the first 25 minutes is “sales talk”.
I made a screenshot of the slide with the scenario’s;
This is a good slide to start explaining i think, let’s elaborate a bit;
GPU pass-through, where the VM “owns” the GPU;
- This can be done with all the major hypervisors like VMware, XenServer and Hyper-v.
- As you assign a GPU per VM, you would get 2x VM’s with a K2 or 4x VM’s with a K1 (with the K1 being less powerful). If you use 2 cards (which is the max currently) this will double.
- You could just run XenApp in those VM’s (6.5 with OpenGL add-on), XenDesktop or VMware View.
- This means you can use the NVidia driver inside the VM, which has benefits for those applications who look at the driver (like Autocad) to decide which GPU acceleration to use. If those applications wouldn’t see that NVidia driver you would probably get the fallback methods based on DirectX and OpenGL.
- In the presentation there is also a different chapter on XenApp on bare metal (38:44). This is a supported scenario but leaves GPU’s unused as this uses GPU pass-through and there is only one OS instance. NVidia advises to use XenServer so you can create more VM’s with XenApp inside to use all of the GPU’s. So this falls in the GPU pass-through solution.
GPU sharing (in the datasheet it’s called API, more on that later).
- The NVidia driver lives in the hypervisor but is not exposed to the VM. The VM’s will use generic drivers and use DirectX and OpenGL (for specifics see the datasheet). They call it “API intercept”.
- This is possible today on Hyper-v, VMware and XenServer, but.. I quote Milan: “it has limitations”. If you want to see those limitations, skip to the demo part (41:50) of the presentation.
- You can use XenApp and XenDesktop and View with this.
Hardware vGPU (grid with K1/K2) is more like the virtualization as we want to see it. There are quite a few terms for it as you can see;
- This works by placing the grid software in the hypervisor and installing the driver in the VM.
- This gives us the possibility to create as many VM’s as the hardware/apps can handle.
- Today, this is supported on XenServer only, and available after Citrix Synergy.
- VMware has got it working in a lab version (given to NVidia to show at demo ‘s), but this hasn’t been released to the public. They told us in the presentation this is a decision of VMware to whether to actually use the technology or not, so i guess we have to wait. There were some hints that they would announce support on Citrix Synergy, but i haven’t seen them yet (please correct me if i’m wrong).
- Hyper-V doesn’t have support yet as well, although i expected a session on TechEd to give more clues on this subject. I haven’t seen it (yet), but if you have, let me know!
- In my opinion, i want Hyper-V support (or at least VMware) to use this so we’ll have to wait a bit more.
The 3 scenario’s and details are all in a datasheet from NVidia, but it confused me. NVidia calls XenApp “Virtualized Applications” and XenDesktop is called “Virtual Remote Workstations”. They are correct, but these terms apply for many other solutions (didn’t they pay attention to BriForum? :-P) so it might be clear to them but certainly not for me. Creating this post made it more clearer for me, and i hope to saved you some time. All the above is explained in the presentation from 26:14, with demo time starting from 41:50 with some figures on scaling. You can find the presentation here.
There were some other things that i would like to share with you;
- There was talk about licensing the grid cards (so you would need to get software licenses to use the hardware), NVidia told us this is off the table and no separate licenses from NVidia are required.
- Milan (the presenter) told me: If you are going to use XenServer: Call Citrix and get a Citrix engineer assigned. Apparently the process is not yet in the next-next-finish stage yet.
- Dell offers list prices of 2700 euro for the K1 and 5200 euro for the K2 if you configure the Dell R720. We use mostly Dell servers, so if you are going to buy Dell servers, let us know, we can probably give you a good deal (20% off the list price if you are in Europe).
- If you add 2x K2 cards in a Dell R720 on the Dell website, it requires you to add a 1100 Watt power supply.. 😉
We will be using the grid cards in combination with VMware and XenApp (in the GPU pass through mode) for some applications, so expect more blog posts about this subject.
And if you want to see more good content about this subject, look at these
- http://www.brianmadden.com/blogs/brianmadden/archive/2013/05/24/NVidia-wows-the-citrix-synergy-audience-with-xendesktop-and-xenapp-graphics-demos-gabe-knuth-digs-in.aspx
- http://www.brianmadden.com/blogs/morevideos/archive/2013/05/24/gabe-knuth-interviews-NVidia-s-will-wade-at-synergy-2013-about-gpu-virtualization-in-xendesktop.aspx
- http://www.NVidia.co.uk/object/NVidia-grid-desktop-virtualization-may22-2013-uk.html
update 12 june 2013: i talked to a NVidia support tech and it seems the above story still applies, only Xenserver allows only the virtual gpu scenario. No ETA on VMware of Hyper-v exists.
update 22 juli 2013: some added details on XenApp and XenDesktop versions, read it here.
update april 2014: vGPU for VMware not until 2015, read it here.