Installation Testing Tips

Working with the software starts with its installation on the computer. During the installation, user gets the first impression of a product. That’s why this process is very important and claims certain attention.

This article includes practical recommendations on testing of software installation process and also includes a bulk of ready tests that are a good basis for a valid test plan preparation.

Installation and uninstallation process notions

Installer is a program that performs certain actions for software installation on a user computer such as:

• file copying to a destination folder on a hard disk;
• shortcuts creation;
• changes in the Registry;
• system files changing.

Widespread types of installers:

• Windows: Windows Installer, InstallShield, Macrovision InstallAnywhere, Wise, SetupBuilder, Actual Installer, Smart Install Maker
• Linux: apt, rpm, dpkg, portage.

Software uninstallation means deletion of all program parts from the system (records in the Registry and in other system files, DLL libraries in the WINDOWS/SYSTEM folder, etc.)

Types of installation

Manual installation is a case when the software installation is performed without the installer help or requires considerable user participation.

Automatic installation is an installation that does not require any user actions except its starting (password entering, license agreement acceptance, etc.).

Self-installation is a fully automatic installation that does not require an initial start of the installation process.

Silent installation. Messages and windows are not displayed during the installation of this type.

Remote installation is a type of installation when the control is performed from the remote computer that is connected via LAN or serial cable.

Clear installation is an installation that is performed in the absence of interfering factors (files of the previous installation of this program, unstable OS mode, etc.).

Testing of the software installation

During the first launch of the installer, it proposes you to choose the installation mode.

The principal moment, on which you should concentrate during the testing, is the control of correspondence of installed components to those chosen by the user:

• Typical: For less experienced users, components are installed by default.
• Custom: For advanced users, only the components chosen by the user are installed.
• Full: Full installation of all components. All options are included.
• After the installation, it is necessary to check recording of the new data to the system Registry for the correctness, and successful registration of a new application.
• Test it on different OS because the same installation package can install different sets of files for different OS.

Peculiarities of installation testing for various applications

While testing, you should take into account specific of installations of various applications.

Program packages

Pay close attention to the work of the license agreement, i.e. check the program behavior at the illegal license key entering. If the program validates the key via the security server using the internet connection then the following tests can be also performed:

• Try to install the program package with the disabled internet connection.
• Check if it is possible to reuse the same license key on several machines.

Antivirus software

Antivirus software in most cases conflicts with other installed antiviruses. That’s why starting the antivirus installation testing you can single out two main test cases:

• The system already has another antivirus application installed. In this case, the launched installation must warn the user about the possible conflict with the installed antivirus and propose to remove it or to cancel the installation.
• If the firewall is the only PC security tool, check that the antivirus can redefine the protection responsibilities by default to itself during the installation. This can be checked in the Security Center (Control Panel -> Security Center).

Drivers

Driver installation testing should be performed carefully and attentively.

• Make sure in the operability of the device after installation, in case the OS installs standard drivers for it automatically. Usually there are no problems in such cases.
• Test the incompatibility of drivers and the third-party driver installation in case you use “Hardware Installation wizard” when a user is proposed to choose driver and device it will be installed for.
• During the installation of drivers, which are supplied with the software and hardware, check only the correctness of installation and the operability of the device in the end of installation process.
• Check the possibility and correctness of rollback after the canceling of unsigned driver installation by the system after warning about the possible incompatibility.
• Test the installation of unsigned drivers in case such installation is disabled (Control Panel -> System -> Hardware -> Driver Signing).
• Make sure those temporary files, which are not necessary for the work of application (*.tmp…), are deleted in case the program contains files signed with MS certificate.

To learn more about driver testing, explore our article about Windows driver testing basics.

Games

Full-screen 3D games are the most requiring for the PC hardware. Many games collect the information about the system at the preliminary stage of installation.

• Check the installation of a game on PC with low technical characteristics (e.g. install it on the corresponding virtual machine). As a result a user should be informed about the mismatch of system requirements.

Mail clients

The installation of mail clients implies the choice of basic settings of work with a client.

• Check various configurations: type of the server access protocol (POP, IMAP4, MAPI), type of authentication and the connection port.

How OS security tools affect software installation

Installation under different users

As is known, for every Windows user there is a folder with his own profile where different system settings are stored. It refers also to the programs installed. That’s why installation testing and starting a program for different users are the separate interesting classes of testing. In this case, the following situations may occur:

• The program is installed for all users by default regardless of who installs it.
• The program is installed by default only for the user who launches the installation.
• There is an option in the installation: “To install the program for all users”/ “To install the program only for the current user”.

It should be mentioned in the specification which of the installation variants your program uses. If it is missed it is necessary to focus on common sense and the program functionality.

Thus, the following tests should be executed:

• Install the program under one user and then log in as another user. Check the program behavior: either the program launches or it is unavailable and its shortcuts must not appear in Start menu, Desktop, etc.
• Test the program launch for users with limited rights in case it was installed for all users. In this case, the program can simply fall at the attempt of performing some actions (e.g., at the first attempt to write data to logs).

User permissions

• Also it is necessary to test the installation and the first launch of a program for users with different rights. If the user rights are not sufficient for starting of installation, the installation should not run. Special case: a user has rights for program installation but installation ends unsuccessfully. Reason: during the installation program executes different actions with objects for which additional rights are needed, e.g., the right to write in certain folders or to connect to the data base.

UAC impact

User Account Control is a Microsoft Windows component that appeared first in Windows Vista OS. This component inquires the confirmation of actions that require the administrator’s rights to protect the computer from unauthorized use.

• Check the success of software installation with enabled UAC.

There is a certain list of actions that call the UAC message, e.g., writing to the system folders or Registry branches, adding accounts, etc. If the program performs some actions from this list, it must response correctly to the UAC reaction; especially it concerns the silent installation.

It can be enabled in such a way:

Vista:

• Select Control Panel -> User Accounts.
• Turn User Account Control on or off.
• Select the options and click on the OK button.
• Restart your computer to apply the UAC settings.

Win7 has 4 UAC gradations (which differ in the list of events that call UAC message), it is necessary to run testing on the strictest gradation (Always notify). It can be enabled in such way:

• Select Control Panel -> User Accounts.
• Change User Account Control settings.
• Move the slider to the very top.
• Click OK.
• Restart the computer.

DEP impact

Data Execution Prevention is a set of software and hardware techniques that allow you to execute additional testing of RAM contents and to prevent launching of a harmful code.

DEP allows blocking viruses and other malicious software that attempt executing their code out of system memory space that is reserved for Windows and other authorized programs. After discovering that a program uses system memory incorrectly, the DEP closes the program forcibly and displays the appropriate message. Unlike the firewall or antivirus program the DEP feature does not prevent the installation of potentially unsafe programs. Instead of it monitoring is performed to make the programs use system memory in the proper way. By default the DEP feature is enabled only for basic programs and services of Windows OS.

• Test the installation and the first launch of the program with enabled DEP function to make sure that your product uses the memory correctly and the end user will not face problems if DEP is enabled.

DEP can be enabled in such way:

1. Right click on My Computer.

2. Select Properties.

3. In the Advanced tab, select Performance and click Settings.

4. On the Data Execution Prevention tab, select the Turn on DEP for all programs and services except those I select option.

Remote installation and Firewall

Your program must be ready to be installed not from the local computer.

• Try to install the program from the network drive or the share folder. In this case, installation must run successfully, just like from the local disk.

Special case: programs that allow the installation from a single computer to all computers in the network, domain, certain range of ip addresses, etc. Such installation is not rather an object of installation testing, but a separate functionality and is tested accordingly.

Anyway, the remote installation first of all faces the use of Firewall.

• Test the possibility of remote installation with enabled Windows Firewall. Usually, having discovered, that the application connects via the network, firewall proposes to enable or to reject or to add an exception. You can try all 3 cases to look how the user will work with this application.

To enable Firewall go to Control Panel -> Windows Firewall -> Change settings. Then launch the remote installation.

During the configuration of environment for testing of remote installations pat attention to Exceptions in the Firewall settings: Control Panel -> Windows Firewall -> Change Settings -> Exceptions. Pay attention to all options that begin with the word Remote. To perform the remote installation Firewall is often disabled but if these options are included to Exceptions they will function and will not allow to execute the installation.

Testing of repeated installation

During the repeated installation it is often proposed to delete, update, recover or modify the installed software.

Uninstall (Remove) – deletion of a program.

• Check if all the components, which the program entered to the system (files, registry records, library links, etc.), were deleted after its uninstallation. Here special utilities for system status monitoring can help.
• Check if the data, created during the work with application, were saved.
• Check if the warning message appears when trying to uninstall running application.

Repair – an application recovery if it was damaged.

• Delete some data from the installation folder and the Registry. After using Repair, all the data must be recovered.

Modify (change) – change of installation options after its termination.

• Install only the part of available options during the first installation and while using the Modify function add the rest (or cancel the installation of chosen options).

Update – update the program version.

• Check for the existent version update after using Update.
• Make sure that after the installation of updates all created earlier objects open and function correctly.
• Check the possibility of saving the document in the old format in the updated version.
• The program must be tested for correctness after the canceling the update process.

Installation wizard testing peculiarities

It is very important to test the installation wizard as well.

• Test the navigation on wizard pages. Expected results: entered data must be saved; having changed the data on the previous page check for the appropriate change (or the answers reset) on the next pages.
• Make sure that wizard blocks the Next moves if the data were not entered properly or the answer was chosen incorrectly.
• Check the operability of the Cancel (Close) button on any wizard page.
• Perform validation testing for entry field: Unicode and special symbols, empty field; if it is the field to enter the path: inexistent path, path with incorrect structure, etc.

Conclusion

We made sure that software installation is really a very important moment that has its own nuances and bottlenecks. That’s why the installation testing must be performed with account taken of all peculiarities of installation process. The basic moments were covered in this article. Let’s summarize.

Performing the installation testing it is necessary to take into account the following things:

• type of the installation being tested (e.g., how it is executed);
• peculiarities of the application which installation must be tested
• features of the OS where the software is being installed
• interaction of the software being installed with the OS security tools
• peculiarities of the installation wizard testing
• also the testing nuances of the repeated installation (Uninstall, Repair, Modify, Update)

Wish your product be of high quality!