Virtualization in Software Testing: Advantages and Disadvantages

As it is generally known, the software testing is a comprehensive and important part of the software development process. Therefore, costs and time spent on testing have an effect on the final result. When the project comes to testing, a tester has the following tasks: preparation of the test environment, testing itself, and reporting on results.

Preparation of the environment is often related with large time and resources expenses. The largest part of time must be taken by testing process but not by the creation of the system backup copies and their configurations or the system recovery after the failure in a result of the unstable work of the software product built.

Such technologies as virtualization enable the optimization of these processes. Several simultaneously started virtual test environments on a single physical machine or group of machines considerably increase the IT infrastructure flexibility and the efficiency of hardware usage.

At Apriorit, we widly use virtualization testing all kinds of applications on extensive range of OS types, versions, and environment configurations. It also helps a lot when performing load testing. Learn more about the Apriorit QA and testing tasks and processes in this section.

Tester Tasks

Some software products require their work to be checked under definite conditions (various OS versions, various RAM memory size, channel capacity, CPU clock, etc.). In this case, the testing team can not always have the required hardware configuration within their testing lab.

Of course, one of the most serious problems of the software testing is the fact that on early implementation stages, software product builds can cause damage to the system during their work (e.g., different device drivers). That is why we have to plan the recovery of operating systems and their settings after failures from backup copies and to waste a significant amount of time on their recovery. If we count the amount of time and resources spent on solving these tasks, there will be rather an impressive number. This number, put into the money equivalent, forms rather a big part of the project budget.

Advantages of the Virtualization in Software Testing

Virtualization technologies can considerably reduce the man-hours and increase the process efficiency if they are applied in the development and testing process correctly. In particular, the virtualization in software testing gives such benefits:

• Testers, while working with virtual testing environment, can create the unlimited number of user configurations on their physical machines and choose the most suitable configuration at each stage. This gives the possibility to do the following:
• to experiment with potentially incompatible applications;
• to perform testing with different user profiles.

Fig. 1 VMware: User configurations tabs

• Easy portability to other hardware without the necessity to reset and the independence from the hardware platform. If there is a serious failure on the virtual machine, this will not affect the real tester’s machine in any way.
• A backup of the virtual machine can be created by copying a folder or creating a snapshot. If testers use virtual machines on their work stations, they can create their backups by copying a folder with files of a virtual machine. If the system fails, there is no need to recover the saved copy as it is quite ready for work. Besides, many virtualization platforms make it possible to create several snapshots of the virtual machine state. The rollback to each of these snapshots can be performed in a few minutes.

Fig. 2 VMware: Snapshots tree

Also, the virtualization technologies make it possible to clone virtual machines (their current state) or some definite snapshot. Usually, there are two types of cloning:

• linked cloning;
• full cloning.

The specifics of these two types are that the linked cloning is performed instantly and the created virtual machine refers to its “parent”. It means that all changes made in the clone are displayed in the clone “parent” and vice versa. The full cloning creates a new, independent virtual machine and this process takes much more time than the linked cloning does.

Fig. 3 VMware: Cloning of virtual machines

• When finding a bug, a tester can simply save a system state, where the bug appears, to a snapshot and continue testing of the system. If it is necessary to reproduce the bug, the virtual machine can be given to a developer who can work with it without fear to damage tester’s environment. Besides, resources become available for further testing: the tester can continue working and not wait till the developer finishes with the test environment.
• Required conditions for testing can be quickly created due to flexible settings of virtual machine hardware (RAM memory space, number of virtual processors, limitation of resources). Often, great flexibility of hardware components configuration is required when testing a software product. For example, checking of software product work in extreme or limiting conditions is required while stress testing (low disk space, network connection break). In this case, we can add new virtual devices to a virtual machine or limit resources assigned to it with a help of a virtualization platform.

Also, if we add a virtual disk in a guest system, we can create it as a dynamically expandable disk. This helps to save free space on the disk and also to create so-called undo disks, whose changes are applied only while working with the virtual machine and can be cancelled at the end of the session that is very convenient for testing.

Fig. 4 VMware: Creation of virtual networks

Fig. 5 VMware: Hardware settings of the virtual machine

• The possibility of a quick rollback to the saved state of a virtual machine with required configuration or switching between several simultaneously working guest systems shortens time for testing;

Disadvantages of the Virtualization in Software Testing

Despite all advantages of the virtual testing environments, sometimes we face some disadvantages:

• Impossibility of emulation of all devices. At the current moment, all main devices of hardware platforms are supported by vendors of virtualization systems. But if some controllers or devices, which are not supported, are used, we have to refuse the virtualization of such environment.
• Virtual test environment requires additional hardware resources. Existing virtualization technologies make it possible to bring performance of virtual machines to real. But to start some definite configuration with a number of machines, we need a physical host to allow this.
• Equipment conflict. Drivers used by a virtualization system can conflict with drivers of tested equipment.
• In case of low disk space on the hard disk of the virtual machine, we cannot increase it if the virtual machine contains snapshots.

Fig. 6 VMware: Error when trying to change the disk of the virtual machine containing snapshots.

• We should perform final testing of the developed software on real machines because no virtual environment can completely replace the real one.
• There are training costs on work with virtual machines.
• When a virus is detected on one of snapshots, it is hard to define on which stage it appeared. We have to check all snapshots because the antivirus that can detect and delete viruses on all snapshots of the virtual machine isn’t developed yet.
• Single destruction point. If a virtualization system fails, it is impossible to start virtual machines till the reason of failure is found and system is recovered.

Conclusion

From everything mentioned above, we can conclude that using virtualization in software testing helps in the following aspects:

• It considerably reduces the time costs on preparing test user environments and configuration of test environments.
• Bug detection and bug fix become more effective when development team members and product users have general access to found bugs.

But testing process on virtual test environments has some limitations:

• It is not recommended to apply virtualization in perfromance testing and also while testing applications with high requirements to the physical resources of the computer.
• The final testing of the developed software must be performed on real machines.

For other types of testing, the virtual test infrastructure is one of the best solutions to  increase effectiveness of development process and also to simplify the interaction between the team members.