Configuring Tesla M60 cards for NVIDIA GRID vGPU

Configuring Tesla M60 cards for NVIDIA GRID vGPU

There are a couple of steps which need to be taken to configure the Tesla M60 cards with NVIDIA GRID VGPU in a vSphere / Horizon environment. I have listed them here quick and dirty. They are an extract of the NVIDIA Virtual GPU Software User Guide.

  • On the host(s):
    • Install the vib
      • esxcli software vib install -v directory/NVIDIA-vGPUVMware_ESXi_6.0_Host_Driver_390.72-1OEM.600.0.0.2159203.vib
    • Reboot the host(s)
    • Check if the module is loaded
      • vmkload_mod -l | grep nvidia
    • Run the nvidia-smi command to verify the correct communictation with the device
    • Configuring Suspend and Resume for VMware vSphere
      • esxcli system module parameters set -m nvidia -p “NVreg_RegistryDwords=RMEnableVgpuMigration=1”
    • Reboot the host
    • Confirm that suspend and resume is configured
      • dmesg | grep NVRM
    • Check that the default graphics type is set to shared direct
    • If the graphics type were not set to shared direct, execute the following commands to stop and start the xorg and nv-hostengine services
      • /etc/init.d/xorg stop
      • nv-hostengine -t
      • nv-hostengine -d
      • /etc/init.d/xorg start
  • On the VM / Parent VM:
    • Configure the VM, beware that once the vGPU is configured that the console of the VM will not be visible/accessible through the vSphere Client. An alternate access method should already be foreseen
    • Edit the VM configuration to add a shared pci device, verify that NVIDIA GRID vGPU is selected
    • Choose the vGPU profile
      more info on the profiles can be found here under section ‘1.4.1 Virtual GPU Types’:
    • Reserve all guest memory
  • On the Horizon pool
    • Configure the pool to use the NVIDIA GRID vGPU as 3D Renderer

Unsupported upgrade of VCSA 6.5 U2 to 6.7

Unsupported upgrade of VCSA 6.5 U2 to 6.7

We will upgrade the vCenter Server Appliance from 6.5 U2 to 6.7 though it is not supported. As this is not supported you will NOT want go ahead with this in a production environment. Maybe I will have regrets later on too … but this is my lab environment so the alternative is to redeploy a new VCSA.

I have applied the following knowledge base articles on the source VCSA

The first KB was applied because the installer is failing due to a lack of disk space on the source appliance. The installer gives the opportunity to supply a location on the source VCSA to export the necessary files that facilitate the upgrade.

The second KB was applied because the VMware Directory failed during the firstboot phase after the upgrade succeeded.

I downloaded the sources for VCSA 6.7.0 but had to go and download the sources for VCSA 6.7.0a. The VCSA 6.7.0 sources stalled at 5% on VMware Identity Management Service.

I also went to change the root password expiration to no and set the administrator@vsphere.local account password to only include alphabet characters.

The installer will also fail after the first phase if the VAMI port is not reachable, the first phase will finish succesfully though. I forgot to add an exception to my firewall. You can then continue the installer by going to the VAMI interface on port 5480.

 Setting up the lab in Ravello – Part 1 : the jumphost

This entry is part 1 of 1 in the series Ravello Cloud Lab 1.0

 Setting up the lab in Ravello – Part 1 : the jumphost

In these series we will create a lab with multiple components, a jumphost, vcsa, esxi, a vsan enabled cluster, nsx and maybe more. The aim of the series is to learn about deploying all components onto the Ravello cloud.

Part 1: Creating the Jumphost

Part one of the series will be about creating the jumphost. I’m looking at a linux system as we do not need any license to run it and it is already available in Ravello

Creating the Ravello Application

The first step is to create an application. We will create a 0.1 version of the LAB:

Creating the Jumphost VM in the Application

Drag a ‘Xubuntu Desktop 14.04.1 with qemu-kvm pre-installed’ onto the Canvas. Once the VM has been dragged onto the Canvas, there will be an error: ‘Key pair must be supplied’

You can see that the error has its source on the General tab. To correct this a Key Pair must be created.

On the General tab – Cloud Init Configuration – Key Pair

Select the Option: Create a Key Pair

In the following screenshot you can see that I already created a Key Pair

Once created the private key will be available for download. To be able to use the private key with a ssh session from putty, you will need to convert the key.pem to key.ppk. Open puttygen and load the key.pem file and save the file as key.ppk.

Now that we have created our key pair we can save the VM and the error should disappear.

On the System tab, change the # CPU to 2 and the memory to 3 GB.

On the Disks and NICs tab we leave everything as is.

On the Services tab, Add Supplied Service. We will use this Service to connect to the VM via RDP.

A second service will be added. I changed the name to RDP and chose protocol RDP which sets the Port to 3389.

We are ready to publish the application:

Change the ‘Schedule application to stop in:’ countdown timer to ‘04:00hr’. This will give us the time to update and change the VM to our needs.

Publish will power on the VM. When Powered on we will have access to the Console. Powering on the VM takes a couple of minutes.

Customizing the Jumphost VM


The Console will open in a new tab. The initial password for this VM is ‘ravelloCloud’.

The first thing we will do is upgrade the VM to the latest release available. Open the ‘Byobu Terminal’.

Run the command ‘sudo apt-get update && sudo apt-get upgrade’ and confirm you want to upgrade all proposed packages. I tried do-release-upgrade first, which failed because of an apt dependency.

sudo apt-get update && sudo apt-get upgrade

Now we are ready to upgrade to the lastest release. Confirm to all new version configuration files from the package maintainer. In the end all obsolete packages can be removed and reboot when finished.

Run the command ‘sudo apt-get dist-upgrade’ and confirm you want to upgrade all proposed packages. Now your system will be fully up-to-date.

XRDP 0.9.x

Install xrdp 0.9.x so that we can connect via RDP. This will be a more pleasant way of working.

We will add a PPA (Personal Package Archive) to add the package source location to the /etc/apt/sources.list file. This will enable updates through the apt update process. We will install the latest version of xrpd from this location. At the time of writing the version integrated is in the ubuntu sources is 0.6.x. The latest stable version has quite some enhancements like shared clipboard support.

sudo add-apt-repository ppa:hermlnx/xrdp
sudo apt-get update
sudo apt-get install
xrdp xrdp -v

The version installed at the time of writing is 0.9.4

Create xsession file with contents xfce4-session. The latest xrdp version should be detecting the desktop environment by default but in my case it did't and wouldn't work without the following xsession file.

cd $HOME
echo xfce4-session > ~/.xsession

Generate new certificate and key

openssl req -x509 -newkey rsa:2048 -nodes -keyout key.pem -out cert.pem -days 365

Update XRDP to use the new certificates

cd /etc/xrdp sudo vi xrdp.ini

Change the following lines to use the certificate and key generated

cd /etc/X11/
sudo vi wrapper.config

Change the following line


Reboot the VM Now you can access the VM through RDP. You will need to confirm the self-signed cert as it has not been signed by a trusted root CA.

Powershell Core

Import the public repository GPG keys

curl | sudo apt-key add -

Register the Microsoft Ubuntu repository

curl | sudo tee /etc/apt/sources.list.d/microsoft.list

Update the list of products

sudo apt-get update

Install PowerShell

sudo apt-get install -y powershell

Start PowerShell


PowerCLI 10

Install the PowerCLI module from the PowerShell Gallery

Install-Module -Name VMware.PowerCLI -scope CurrentUser

Verify PowerCLI version


OPTIONAL: Opt-out from the Customer Experience Improvement Program (CEIP)

Set-PowerCLIConfiguration -scope user -ParticipateCeip $false

OPTIONAL: Do not display the warning about using self-signed certificates

Set-PowerCLIConfiguration -InvalidCertificateAction Ignore

OPTIONAL: Visual Studio Code

Installing Microsoft Visual Studio Code can be usefull for creating scripts that will/could be used within the environment.

curl | gpg --dearmor > microsoft.gpg
sudo mv microsoft.gpg /etc/apt/trusted.gpg.d/microsoft.gpg sudo sh -c 'echo "deb [arch=amd64] stable main" > /etc/apt/sources.list.d/vscode.list'
sudo apt-get update
sudo apt-get install code # or code-insiders

The next part will be setting up the ESXi machines and VCSA.

Many thanks to:

boot failure: systemctl status system-fsck-root.service


boot failure: systemctl status system-fsck-root.service

I had downtime in my lab due to an power failure which resulted in a boot failure of my VCSA 6.5 appliance. Looking on the console showed me a “[FAILED] Failed to start File System Check on /dev/dis…uuid/uuid. See ‘systemctl status system-fsck-root.service’ for details.” message. Therefor it booted into ‘Emergency Shell’ or ‘Emergency mode’.

boot failure: systemctl status system-fsck-root.service

I ran the command ‘systemctl status systemd-fsck-root’ manually. This showed me that the ‘/dev/sda3’ partition was having issues.

UPDATE: It also states “RUN fsck MANUALLY”. I did not notice this the first time

systemctl status system-fsck-root

I tried to run fsck with no options to see if the command was known to the CLI. I then ran the command with the partition as a parameter ‘fsck /dev/sda3’. I answered ‘y(es)’ to all ‘Fix<y>?’ questions.

fsck /dev/sda3

In the end I received the message ‘FILE SYSTEM WAS MODIFIED’ and tried to reboot. The reboot command gave me an error so I went through the ESXi to reset the virtual machine. Afterwards I was able to login again.