Automation of the ESX(i) hypervisor application in the continuous integration process using AutoIT language

This article is devoted to the theme of how to deploy the continuous integration using ESX(i) hypervisor as a test environment storage. There are some easy-to-use mechanisms to manage this environment by means of vmrun.exe console utility included into the VMWare VIX installation package.

This article describes how to use ESX(i) hypervisor in the automated testing process for the scheduled build installation by means of the particular example. It is meant for the beginners in the testing automation, who have some skills of writing autotests in a scripting language. Here we will use AutoIT language – it is intended for the Desktop application automation and is freeware, but reader can use any scripting language.

Contents:

Introduction
What is Continuous Integration
What is ESX(i) hypervisor
Examples of the scripts to automate work with ESX in Continuous Integration context
Problems with AutoIt concerning Continuous Integration testing automation
TFS options to use AutoIt scripts created for Continuous Integration
Conclusion
Useful links

Introduction

The first section provides some information about Continuous Integration and ESX(i) hypervisor. This is a theoretical minimum that an automation specialist should know before starting to create continuous integration process by the example described below.

The second section of this article describes the work of the AutoIT scripts to perform automated testing of the product installation on various configurations using ESX(i). Please note: there won’t be any autotest implementation examples as each product has its own specific installation steps. The example describes development of the scripts to organize the continuous integration process.

In the third section, I mention a possibility to use AutoIT scripts in the Microsoft Team Foundation Server (TFS) system.

What is Continuous Integration

Continuous Integration is a practice of software development that consists in the frequent automated project builds to discover and resolve possible integration problems on the early stages. Usually, in the projects where developers work on the different system parts independently, integration is the final stage. It can unpredictably delay the development cycle to be completed. Continuous integration introduction enables to reduce integration complexity and make it more predictable due to the early bugs and contradictions detection and fix.

It is implemented by means of the service hosted on the dedicated server whose tasks are:

Get the source code from the repository;
Build the project;
Perform tests;
Deploy the built project;
Send reports.

Scheduled builds are usually performed automatically each night in the specified time – so-called night builds (so that the testing results are ready by the morning of the next working day).

What is ESX(i) hypervisor

Hypervisor is a software program providing the possibility to run simultaneously several or many OS on the same physical host. Hypervisor also provides the isolation of the OS, security and protection, resource distribution among different running OS and resource management. Hypervisor also can (but it’s not obligatory) provide OS running on it with some means to communicate and interact one with another (e.g. via file exchange or network connections) just like these OS were running on the different physical hosts.

Hypervisor per se is in some way a small OS. It provides OS running on it with the virtual machine service by virtualizing or emulating real hardware of the machine, and manages these virtual machines, allocate and release resources for them. Hypervisor allows to independently start, restart, “turn-off” each virtual machine. An OS running on the virtual machine managed by a hypervisor can “know” or not “know” about it is running on the virtual machine and not on the real hardware.

VMWare ESX(i) Server is a virtualization OS, which allows to divide physical machine into the logical partitions named virtual machines. It includes means to manage virtual resources. We can say that it is a hypervisor.

VMware vCenter Server is an expandable and scalable platform to manage virtual infrastructure and provide the possibility to get the comprehensive information about it.

Examples of the scripts to automate work with ESX in Continuous Integration context

To work with ESX(i) concerning Continuous Integration, we will use the vmrun.exe console application as virtual machine management mechanism – to start and revert to the snapshot. This application is provided with the VMWare VIX product installation. VIX API is intended to the development of the programs or scripts to automate some routine actions with virtual machines. The rest of the scripts are written in AutoIT.

First, let’s imagine the architecture of the process to implement:

To implement the described example you’ll need: ESX(i) with virtual machines, vmrun.exe utility from VMWare VIX package, AutoIt code editor SciTE. Continuous Integration process looks as follows:

Scheduled build system places the new build to the FTP server and starts Manager.exe (AutoIt script to manage testing process).
Manager.exe manages the ESX(i) hypervisor by means of the vmrun.exe utility (start virtual machines, checks logs, revert virtual machines to the snapshot, sets testing parameters).
The DownLoader.exe (DL) AutoIt script, placed on the each virtual machine, downloads the new build, scripts and configuration files for the autotests.
When all autotests are finished, generated logs are placed to the FTP server.
Manager.exe checks logs to discover bugs and execution errors and compose the testing report.

Let’s consider the DownLoader.exe implementation. This script must start on the virtual machine after its start. To implement this, we create a virtual machine snapshot: when it starts the user autologon is performed and our script is starting. To perform system autologon we must change some registry keys:

Bash

HKLM\SOFTWARE\Microsoft\Windows NT\CurrentVersion\Winlogon", "AutoAdminLogon", "REG_SZ", "1") 
HKLM\SOFTWARE\Microsoft\Windows NT\CurrentVersion\Winlogon", "ForceAutoLogon", "REG_SZ", "1") 
HKLM\SOFTWARE\Microsoft\Windows NT\CurrentVersion\Winlogon", "DefaultUserName", "REG_SZ", [User Name]) 
HKLM\SOFTWARE\Microsoft\Windows NT\CurrentVersion\Winlogon", "DefaultPassword", "REG_SZ", [User Password]).

To autorun the DownLoader.exe script we change the registry key:

Bash

HKLM\SOFTWARE\Microsoft\Windows\CurrentVersion\Run", "Downloader", "REG_SZ", [Full local file path])

Now, when virtual machines starts, the DownLoader.exe file is executed. It performs such actions:

Download the new build, configuration file (/Autotests/Constants.ini) and autotest files (/Autotests/Installation.exe) to the virtual machine.
Starts autotests specifying the name of the installation file to be executed.

Here is an example of the DownLoader.exe script code written in AutoIt:

;~ Handle to be used in _FTP_FindFileNext() or _FTP_FindFileClose().
$h_Handle
$sLogFile=@ScriptDir & "logsLog.log"
$open = _FTP_Open ('ftpserver')
$connection = _FTP_Connect ($open, 'ftpserver', "", "")
	If $connection <> 0  Then
		FileWrite ($sLogFile, "Connected to FTP://vcenter. Handle of connection is: " & $connection & @CRLF)
	Else
		FileWrite ($sLogFile, "Unable to connect to FTP://vcenter. _FTP_Connect returned: " & $connection & @CRLF)
	EndIf
_FTP_FileGet($connection, "/Autotests/Installation.exe ", @ScriptDir & "Installation.exe")
_FTP_FileGet($connection, "/Autotests/Constants.ini ", @ScriptDir & "Constants.ini")
;~ As we don’t know the name of build file we need to find it
$aFile = _FTP_FindFileFirst($connection, "/Autotests/Builds/", $h_Handle)
_FTP_FileGet($connection, "/Autotests/Builds/"&$aFile[10] , @ScriptDir & ""& $aFile[10])
;~ Start of Autotesting
ShellExecute (@ScriptDir& "Installation.exe", $aFile[10])
_FTP_Close($connection)

Now let’s proceed to the Manager.exe implementation. We need to send commands to the ESX hypervisor by means of the AutoIt script. The vmrun.exe utility enables to send commands to start virtual machine and revert it to the snapshot in such formats:

Bash

[Path to VMware VIX]\vmrun.exe -T [host system type] -H [host network address] -u [name of the user who has permissions to manage hypervisor] -p [password] [the action to be performed by the virtual machine] [The path to the.vmx file of the virtual machine on the hypervisor]

Here are some examples of the vmrun.exe commands to start virtual machine:

Bash

C:\Program Files\VMware\VMware VIX\vmrun.exe -T ESX -H https://10.1.0.15/sdk/ -u Admin -p Test start "[DATA-I] VirtualMachine1.vmx"

So, to start virtual machine we need to execute AutoIt script containing just one line:

Run(@ProgramFilesDir & "\VMware\VMware VIX\vmrun.exe -T ESX -H https://10.1.0.15/sdk/ -u Admin -p Test start "[DATA-I] VirtualMachine1.vmx")

To revert to the snapshot:

Run(@ProgramFilesDir & "\VMware\VMware VIX\vmrun.exe -T ESX -H https://10.1.0.15/sdk/ -u Admin -p Test revertToSnapshot "[DATA-I] VirtualMachine1.vmx" Clear)

Clear is the name of the snapshot.

By means of these commands Manager.exe performs such actions:

Check the availability of the new build (/Autotests/Builds/).
Edit the autotest configuration file (/Autotests/Constants.ini ).
Start one of the virtual machines and wait for the logs on the FTP server.
When the logs have appeared, revert the virtual machine to the snapshot.
Change the autotest configuration file and starts the next virtual machine in the list.
After the testing is over, change the installation type and repeat for all configurations.

Here is an example of the Manager.exe script code written in AutoIt (file is placed to the Autotests directory on the FTP server):

1. Check the availability of the new build, old logs, configuration file.

#include <Array.au3>
#include <file.au3>
$sLogFile=@scriptdir & “logs.log”
$PathToConstants= "Constants.ini" ;Type path to configuration file ;here
$aListOfLogs=_FileListToArray(@ScriptDir & "logs")
	if @error<>4 Then
		_FileWriteLog($sLogFile,"Testing is stoped. Please clear logs directory from any files before testing")
		Exit
	EndIf
	$aListOfInstallFiles=_FileListToArray(@ScriptDir & ”install ”)
	if @error=4 Then
		_FileWriteLog($sLogFile,"Testing is stoped. There are no files in install directory")
		Exit
	EndIf
	If _ArraySearch($aListOfInstallFiles,IniRead ($PathToConstants, "Installation File Name", "1", Default))<0 Then
		_FileWriteLog($sLogFile,"Testing is stoped. There is no install file that specified in Configuration.ini. Please specified valid file or add file to install directory")
		Exit
	EndIf

2. Edit the autotest configuration file and start the first virtual machine in the configuration list.

Local $aOSLocations[IniRead ($PathToConstants,"OS locations of tested systems", "Number of systems", Default)+1]
Local $aInstallTypes[IniRead ($PathToConstants,"Installation types for testing", "Number of types", Default)+1]
$aOSLocations[0]=IniRead ($PathToConstants,"OS locations of tested systems", "Number of systems", Default)
$aInstallTypes[0]=IniRead ($PathToConstants,"Installation types for testing", "Number of types", Default)
IniWrite (@scriptdir & "AutotetsConstants.ini","Installation File Name","1", IniRead($PathToConstants,"Installation File Name","1",""))
;~ installs different types of installation
For $j=1 to $aInstallTypes[0] step 1
	$aInstallTypes[$j]=IniRead ($PathToConstants, "Installation types for testing", "Type" & $j, Default)
	IniWrite (@scriptdir & "ConstantsConstants.ini", "Installation Type", "1",$aInstallTypes[$j])
	For $i=1 to $aOSLocations[0] step 1
		$aOSLocations[$i]=IniRead ($PathToConstants, "OS locations of tested systems", "System" & $i, Default)
		IniWrite (@scriptdir & "ConstantsConstants.ini", "OS location on the ESX", "OSLocation",$aOSLocations[$i])
		Run(@ProgramFilesDir & "VMwareVMware VIXvmrun.exe -T ESX -H https://10.1.0.15/sdk/ -u Admin -p Test start """ & $aOSLocations[$i] & ".vmx""")
	$NameOfLogFile=StringStripWS($aInstallTypes[$j]&StringReplace($aOSLocations[$i],"",""),8)&"logs.log"
			$ListOfLogFiles=_FileListToArray(@ScriptDir & "logs")
			$k=0

3. Wait when the log file appears on the FTP server. When it has appeared or time out is over, revert virtual machine to the snapshot and repeat the process:

While _ArraySearch($ListOfLogFiles, $NameOfLogFile)<=0 and $k<600
				Sleep (2000)
				$k=$k+1
				$ListOfLogFiles=_FileListToArray(@ScriptDir & "logs")
			WEnd		
Run(@ProgramFilesDir & "VMwareVMware VIXvmrun.exe -T ESX -H https://10.1.0.15/sdk/ -u Admin -p Test revertToSnapshot""" & $aOSLocations[$i] & ".vmx"" Clear")
Next	
Next

Here is the scheme of the script interaction:

The code described above can be used to perform automated testing of some product installation (if there are corresponding autotests) or for other types of testing.

Problems with AutoIt concerning Continuous Integration testing automation

This article contains only AutoIt scripts implementation. This examples can be applied to the installation testing automation, AutoIt functionality is quite enough for that. The implementation won’t take more than 2 weeks. But AutoIt os not enough to automate other types of testing by many reasons:

Limited possibilities of work with interface elements,
Graphical representation verification can’t be performed,
Sound verification can’t be performed,
There is no central test management.

It should be mentioned that there are special systems enabling to deploy continuous integration process in one environment, e.g. TFS included to Microsoft Team System, or Test Complete.

TFS includes: version control system, testing result compilation, report generation, central tests and test sets management, and many other options. Surely, this system considerably excels the script described above in possibilities. But TFS requires significant time to configure it, and also works with the ESX(i) hypervisor only by means of vCenter, included into SCVMM (System Center Virtual Machine Manager), that is rather expensive. All these consideration make it very dependent on the hardware, financial, and human resources as well as configuration time.

Test Complete enables to perform functional, unit, regression, distributed testing, and also the testing of the HTTP availability on the project level. Its language tools allow to create scripts based on the main programming languages (i.e. used languages are similar to the usual programming languages). For example, if I use to develop in C++, then I can write scripts in this language, so I won’t waste time on learning language syntax, but spend it on the functionality research.

But let’s consider the case when we have already deployed installation automation by means of the AutoIt scripts and now we need to migrate to the more serious system to perform more complex automated testing types. Below, I will describe the way to use AutoIt automation scripts in the TFS system.

TFS options to use AutoIt scripts created for Continuous Integration

As we’ve already mentioned, one of the TFS shortcomings is a big amount of hardware, human and time resources required to configure and maintain the solution. On the other hand, it provides much more possibilities than the AutoIt scripting language.

Naturelly, we would like to use ready AutoIt autotest scenarios while deploying TFS.

Generic Test is one of the TFS test types. It starts an existent executable file (Manager.exe, in our case) and reports about this application execution results (autotest logs). So, we add our already implemented installation automation to TFS using this test type. After the scheduled build is created we upload it to the specified directory and start this Generic Test.

Here is the illustration for the modified scheme of Continuous Integration process architecture with TFS.

Generic Test starts the Manager.exe script and after the latter finishes its work, Generic Test checks autotests results. TFS can also work with virtual machines and perform some other tests for them.

Conclusion

Continuous Integration is a useful and sometimes necessary process of software development.
To automate the ESX(i) hypervisor usage for continuous integration process, we can use the AutoIt scripting language and implement the autotesting of the scheduled build. It should be mentioned that this approach is the most painless, fast and economical one. But still AutoIt has a number of shortcomings, so it is worth considering when the project resources are considerably limited.
The option of Generic Test in TFS gives possibility to use autotests developed in AutoIt for continuous integration process with ESX(i) hypervisor and also develop new tests in TFS for more complex application testing.

Learn more tips for software QA with our next article about what is impact analysis.