GUI Testing: Basic tips

This article will be useful for those specialists who are not indifferent to the GUI problems and are interested in getting good GUI. It’s hard to give clear and full definition for the good GUI. So let’s at the moment agree that we’ll understand good GUI as the GUI which helps user to perform his tasks effectively and provide software product with commercial success. Further in this article we’ll extend this definition.

 

Introduction

Most likely reader has already guessed that we will speak about the HCI (Human Computer Interaction) – interaction between a user (human) and computer software. We’ll discuss the aspects of this interaction by the example of Windows-applications. Also we’ll define the common principles independent from operating system and features of application, those principles that are true for the interaction between human and every information system as soon as they are based on the use psychology.

So let’s ask a question about getting the good user interface…

Only testing can prove the interface ability to work.

Vlad Golovach

User interface testing: what, why and how?

The answer to the question of expediency of user interface testing is quite obvious.

Let’s imagine ourselves to be the user who decided to try our product. We start the application with a sinking heart… and… O, God! It’s absolutely impossible to understand what’s what here. All important elements are small (it’s like the developer paid crazy taxes for each superfluous square pixel used), and moreover you should work hard to find them at all. There is almost no working space left because of 5 rows of toolbar. And don’t say a word about the style unity… It’s natural that we uninstall this application, forget it like a nightmare and go to find analogues with more pleasant and clear interface.

It’s sad, isn’t it? And the only method to avoid these sad situations is interface testing. I hope that I convinced you of the importance of this type of testing.

The other good question is about what is testing of interface? From the global point of view interface testing includes a lot of things. They are usability testing with the exterior users and experts, complicated measurement system and rates; testing of all graphic elements – if they are working in the proper way, optimally situated, logically functioning, good looking; and much more other activities. We will concentrate on the graphical user interface testing – GUI testing – we’ll consider the “bricks” that it consists of and the rules and principles that help us to build beautiful palace from those “bricks”.

So when it is effective to test interface? As my own experience tells interface testing usually gets the second priority or even lower as far as the resources and time are always limited and the main attention is paid on the functional testing. Some bugs in interface can be fixed only if they are very striking.

I think that interface testing should be started when the system prototype is ready and also it should be performed every time when the new features appeared or some functionality was changed. And one should test not only that local part with changes but also the whole product to be sure of the integrity of style and to improve the interface taking into account new functions.

Vlad Golovach in his book “User interface design 2. The art of washing of elefant” (russ. Влад Головач, “Дизайн пользовательского интерфейса 2. Искусство мыть слона” [2]) recommends to use the understanding of the “good” interface each time with the corresponding context of the current application. He also recommends to ask yourself some questions, based on the quality conceptions that form the definition of the “good” interface (see Appendix 1).

Here is this useful list of questions:

• Is it possible to accelerate the interaction of user with this interface?
• Where are the places which can produce the human errors? Is it possible to change these fragments?
• What in this interface doesn’t assist in teaching? What does user need to know to work successfully with this interface? Is there something in this list that interface doesn’t report itself?

If all three questions have a negative answer we pass to the following portion of questions from other conceptions of quality:

• Do I know something about the users that makes this interface bad?
• Does this interface satisfy all of the user’s motives that are known for me?
• Is this interface compatible with the environment users work in?
• Are there tasks which are processed ineffectively in interface?
• Do I want to have this interface?

And a few words about automation. In our time automation got to many areas. It’s also used in software testing, in particular testing of interface. I can recommend the following test automation tools for interface testing: Test Complete, AUTOIT, IBM Rational Functional Tester, AutoHotKey, SilkTest, HP Mercury WinRunner.

In this article we will not deepen in automation, but consider the manual testing in detail. You can find a number of so-called checklists in the internet for verification of interface. Majority of them are either supersaturated with information (as for example Microsoft UI Guideline [3]) or contain little of useful information. I tried to choose the most basic and most important information from all of these sources and to represent it in easy-to-read form.

But before testing let’s consider some patterns of user-system interaction that will be useful to build the good interface.

Basic principles

A graphic interface deals with the questions of motivation or collecting necessary reactions from the people. All of people are different and that’s why there is no universal solutions workings always and for everybody. But there are commonly used principles – something that like most of users.

Let’s consider two basis principles of interaction between user and system.

One of such principles is the so-called Fitts’ law ( [9], [10], [11] ) formulated as early as 1954 by Paul Fitts:

The time of achieving a goal is in direct proportion to distances to the goal and inversely to the size of goal.

As applied to GUI it states that the farther object is the more time user will spend moving a pointer of mouse to it. Therefore it makes sense to shorten the distance to the most high-usage objects and make them bigger, more accessible to the user. It is possible, certainly, to draw a reverse conclusion: if we want to keep user from hasty decisions (for example, not to click some button quickly) then we make it small. As example we can mention a context menu (pop-up menu) that possesses greater availability for a user then pull-down menu as he is not required to move a mouse to call it.

Other psychological principle applied in planning of interfaces carries the name of Hick’s law ([12], [13]) and sounds as follows:

The time of reaction when choosing from a number of alternative signals depends on their number – when one should choose from n possible variants the time of choosing one of them will be in proportion to the logarithm to the base 2 of variant number plus 1, in condition that all variants are equally probable:

Time (ms) = a + b log_2(n+1)

If variants are not understood clearly then values a and b increase. If there are some skills and habits in the system usage then b is reduced.

It means that it takes time to make one or another decision; difficult decisions require more time than simple ones; and that interdependence is logarithmic. And thus, for example, the fewer items menu has the less time it’s occupies to choose one of them. One menu is better than two.

So being armed with knowledge of these principles and penetrated by the sympathy to the user we pass to the testing of interface.

Verification Checklist

We will strip GUI into bricks and define the checklist for verification of these bricks. We’ll form the list of rules based on the above mentioned principles and experience of specialists Of course it’s not a complete list – any interface needs its own thoughtfully created checklist for GUI verification, but if GUI you test comes through this list we can say that it has reliable «foundation.

Elements

Buttons

• It should be hard for the user to click wrong button. Thus either the state of the button changes when there is a cursors on it or there are enough empty spaces between the buttons.
• The clickable size of the buttons coincides with their visible or logical size.
• Elements which can not be clicked or chosen should not be removed but disabled.
• The names of the command buttons are expressed by verbs in infinitive (for example «Search», «Delete»). One can give “ОК” text to the button only if there is not enough space for the verb.
• It is good to use icons for the often used buttons because it makes user work more comfortable and it is easier to memorize them.
• Unity and consistency of interface require that if there are icons then they should be everywhere; if it is impossible then it’s better not to use them at all.
• It shouldn’t be different states of the button that look identically.

Checkboxes and radiobuttons

• Radiobuttons should be always embedded in the scope of grouping; it is optional for checkboxes.
• Checkboxes and radiobuttons are the buttons of postponed action – their activation shouldn’t initiate any immediate action.
• One of the radiobuttons in the group should be checked by default
• The group of two radiobuttons can be often replaced by one checkbox, but unfortunately it not always works. Checkbox text represents only description of what will happen after its checking but it is not described what will happen if not to check it. Such construction does not work in situations when the functionality can be unclear for the users because of one or another reason.
• Both checkboxes and radiobuttions should be placed on a vertical line, as it accelerates the search of necessary element considerably.
• Every caption should clearly describe an effect from the choice of the proper element.
• Captions of the parallel buttons should have approximately equal length.
• All of captions should be positive (not to contain a negative forms)
• It is better to place a repetitive word to the title of grouping box.
• If a caption doesn’t go in one line then the indicator of the button (circle or square) is aligned on the first line of caption.
• If there are more than 10 checkboxes in a group the additional one should be created to check/uncheck all.
• If there is only a group of radiobutton in a window besides the terminal buttons (i.e. buttons starting an action) a double click on radiobutton checks it and closes a window.

Listboxes and comboboxes

Lists can be scrolling and opening, and scrolling ones can provide both unique (like the group of radiobuttons) and plural choice (checkboxes); opening ones work only as radiobuttons.

Combobox is a hybrid of list and the edit field: user can choose an existent element or enter some data himself.

Tips:

• The list should be wide enough to show difference between elements. When it’s impossible to make the width of all elements smaller then the list width a horizontal scrollbar is not added and the text of elements is shorten (the most important fragments are determined – for example for URL it’s beginning and end of the link) and the rest is replaced with triple-dot (…) ).
• The most probable values should be already chosen in the lists by default.
• If a list contains more than 50 elements a filter or search mode should be proposed.
• There are no often used short lists (less than five elements); such lists are presented as groups of radiobuttons or checkboxes.
• The elements of list are assorted either structurally (i.e. on general attributes) or in alphabetical order or by frequency (for only lists containing less then 7 elements).
• If there are more than 50 elements assorted in alphabetical order in the list than the first three elements are the most used elements. They are also repeated on their alphabetical places.
• The multiline lists of plural choice are supplied with checkboxes near the each element.
• Multiline list’s height is no less than 4 lines.
• If there is available space then extended comboboxes are used and not single-line ones.

Editboxes

• Edit box should contain the most probable value by default.
• If a numeral value is entered in the edit box the value range is shown in the hint.
• If a numeral value is entered in the edit box and it has limited range then the box is supplied with the spinner.
• Box width is not less and, if possible, not more than the lengths of the information entered.
• If the box is intended to input a noticeable amount of information it is multiline.
• The multiline boxes have a maximally possible height; there is no space to make them bigger.
• On the often used screens captions should be above the box (in order to make it easier not to read them); on the rarely used screens a caption must be on the left (in order to be noticed every time and reduce the number of errors).
• When the screen space is worth its weight in gold a caption to an edit box can be placed into the field itself (it is necessary only to watch focus of input and clear caption when cursor is in the field).

Menu

A menu is a method of interaction between user and the application when user chooses some command from the proposed list (not giving it his own command).

• Each menu item name starts with the capital letter.
• All items of the first level menu activate drop down menus.
• No more than two sublevels of menu are used.
• If there are icons in a menu, they are used only for the most used items.
• Items that open submenus look different from the terminal items.
• Each object visible in an interface has specific context menu.
• Context menu has no more than 10 items.
• Items in context menu are assorted in the decrease of frequency of their use.
• All items of context menus are present in other fragments of interface; there are no commands activated from context menus only.

Windows

• There is an indicator of stretching on the stretching windows.
• The titles of windows correspond to the names of elements that called these windows. If a window is called by an element that doesn’t have explicit name then there is the title of the screen form in the window title.
• The type of a window (modal, modeless, with possibility of minimization/maximization) corresponds to the users’ tasks.
• There are no menus or toolbars in dialogs.
• The «Apply» button is used in windows-palettes only (in place of the “ОК” button).
• There is no «Apply» button in the modal dialogs.

Icons

• There are no icons with the similar colors and form in the groups of icons.
• There are no icons with standard appearance but non-standard scenarios.
• There is no text in icons.
• In the groups icons of one value but different sizes are used for the same features and/or scenarios.

Status line

• There are only information about the current system status and buttons (not looking like the buttons) for functions intended to the experienced users only.
• Progress bar controls are placed in the status line. Exception: wizards, where progress bar controls can be shown inside the window.

Input forms

• Input form has a title.
• All input forms that are used for information collection include items «Other» and “Not applicable” or similar.
• All required fields are marked and there is the corresponding explanation.
• The multipage input forms have the buttons «Back» and «Next».
• All input forms that are used for information collection contain the explanation why these data are collected and what will be and will not be performed with them.

The Structure of interface forms

• Groups of interactive elements (form fields, menu items etc.) contain no more than seven elements.
• The “Cancel” button is always the rightmost.
• Multipage forms have pointing on that they are multipage; user always sees the amount of remaining screens (example: «Screen x of y»).
• If there are a few buttons in a form, one of them is the default button. If there is only one button in the form it cannot be the default button. Buttons that are dangerous for user are not the default buttons.
• If there is available space in a window the most frequency terminal button (i.e. button that manages a window) is bigger than others.
• Buttons are placed in that section which they directly effects on.
• Terminal buttons are located either below in a row or on the right in a column.
• The buttons related to the whole block of tabs are located outside the block.
• If a window or tab has automatically replenished content, for example, there are incoming messages in it, then the title of interface element that opens this window or tab contains the number of objects in this window and separately the number of new objects. Example: Documents (8/3).
• Menu items and button that initiate some complex actions with additional parameters is marked with suspension points (…). Examples: element «Save As…» requires suspension points as user must choose the name of a file; element «About» does not require suspension points as there are no independent interface elements on the opened window.
• Captions to the interface elements are placed uniformly.

Text

• All of interface elements have hint with the text describing the result of the use of these elements.
• There are no slang words in an interface.
• Not a single element is named variously in different places (an interface glossary is documented and reviewed).
• All confirmation text includes the name of an object on which the confirmable action is performed.
• To make text easy-to-read all long numbers are separated into the groups of 3 digits with blanks: 1 234 567.
• Each items of the list begins with a capital letter. All items end with the final letter of the word without any punctuation mark, except for the last item that ends with a dot. Exception: if at least one item of the list contains more than one sentence then all items start with a capital letter and end with a dot.
• There is at least one paragraph of text before an every list.
• In tables all of columns with numbers are right justified.
• There are no dots at the end of headings (if it is separated from the text), images captions and table names.
• Interface element captions begin with a capital letter and end with a colon.

Interaction

• The system makes a sound signal after it has completed some long operation (more than one minute of work).
• If the direct manipulation is not used in the system then its interface does not have its own cursors. If direct manipulation is used, the own cursors are used only if there are no analogues in the OS.

System messages and error processing

• The input data verification is performed in the input forms just after user has entered them; if improper information is entered the system notifies user about it at once, not waiting for a moment, when user completes the input of data in the whole form.
• Invalid data notifications are shown near those control elements where these data were entered.
• The text of invalid data notification does not say that error was made but informs what type and format valid data should have.
• If the window with invalid data notification is pop-up it should not cover the input field with the error.
• The text of reports about problems consists of three parts: a problem is briefly described in the first, the second part tells how to resolve it, and the third describes how to avoid such problems in future.
• Status reports («Synchronization is completed successfully») are shown only in the status line.

Keyboard

• If user presses tab key he moves between elements in the correct order.
• Form processing starts both after clicking terminal button and pressing Enter on the last field of this form.
• There are hotkeys combinations for the most frequently used control elements (including menus).
• ALT-combinations are assigned to the each menu item (ALT+ underlined letters of item name).
• ALT-combinations and hotkeys are standard.
• If there are more than 40 hotkeys, then there is a method to change them in an interface.
• Pressing Tab key user moves from an element to the element from the top to the bottom from the left to the right.

Visual representation

• The direction of shadows of all controls must be identical: from below on the right.

Indication

• Color indication is not the only one; if it is used the system is supplied with other indication too.

Layout

Talking about layout we will understand sizes, indentions, location of content inside a window or page. This aspect of interface is important not only because it makes a window or page good looking but also because it helps user to find what he needs quickly and effectively. The elements should be arranged in such way that visual hierarchy is followed.

Visual hierarchy

Visual hierarchy of window shows the interconnections and priorities of its elements. It’s hard for user to orient in interface without visual hierarchy.

Visual hierarchy is achieved by the followings aspects:

• Focus. Layout shows user where to look in the first turn.
• Order. The order of elements should be organized in such way that user «slides» with eyes on the surface of interface finding the elements in an order, corresponding to their usage.
• Grouping. The logically connected elements should have clear visual relations. Elements that are not interconnected should be separated.
• Accent. The elements of GUI should be marked out taking into account their relative importance.
• Alignment. Layout of interface elements should be coordinated – to make it easy to look over.

It is important to remember that users don’t read – they look over only, scan content of a window or page with their eyes. And this scanning is not performed from the left to the right as reading. At first the most «attractive» elements are scanned. And, probably, an order of scan-out will be the following:

• Interactive elements situated in the center
• Buttons starting an action
• Interactive elements situated in other parts of a window
• Basic guidances
• Additional explanations
• Text with a warning icon
• Title bar
• Other static text is in the body of a window
• Foot-notes

Accordingly to the above mentioned we have the following list of layout verification:

1. Elements that start some operating are disposed in the left upper corner or above on a center
2. Elements that finish some action are disposed in the right lower corner.
3. Important information is placed on interactive elements, but not as static text in the body of a window.
4. Important information is placed neither in the left lower corner nor at the bottom of a page or long element of interface with a scrollbar.
5. There are no large blocks of text.

Effective usage of the screen space

The effective usage of the screen space requires the reasonable distributing of space: if there is too much available space a window looks poor and due to the above-described Fitts’ law it is hard to use it.

Screen space is used effectively if:

• User doesn’t have to change the size of windows, panels and other elements of interface. If user first of all changes the size of the element that he are going to work with it means that the size of element is wrong.
• Information which must be seen for implementation of some action should not be cut.
• Windows and other elements of interface have a size allowing to avoid the unnecessary page changing.
• There should not be large empty areas.

The combination of cut information or scrollbars with a generous amount of blank space is the main indicator of inefficient elements layout.

Element sizes

Element sizes and distance between them are also important. There is a list of recommendations for sizes and distances between elements carefully given by Microsoft [14]

Conclusion

Let’s give some resume for this article. So, in this work we

• learned more about «good» interface
• discussed questions that should be asked each time during the testing of interface
• learned some common principles of human perception coming from psychology
• talked about conceptions of interface quality
• considered an interface by elements and defined what and why is good and correct for each element.
• considered the correct layout of interface elements.

Certainly one should remember that the fact that an interface satisfies the mentioned checklists is not sufficient but only necessary condition to get “good” interface. There is always something more than the standard, some salt of the current application. And of the tasks of QA specialist is to find and understand this specialty and bring it to the users.

Appendix. Conceptions of quality for user interface

It’s very hard to give the general definition of «good» interface, but it is possible to compose it from some attributes from different conceptions of interface quality, taking into account the specific of testable application.

Here is the approximate list of things making an interface good (it is taken from [2]).

So, an interface can be considered as good one if it:

• is «comfortable», «simple» and «usable»

In this case it is necessary to define clearly what these concepts mean for the future users of this software product.

• possesses the high ergonometric indices

For example we can take so-called Shneiderman’s indexes: speed of user work, number of human errors, subjective satisfaction, speed of learning to operate with an interface, extent of these skills remaining when user doesn’t use the product.

• is optimized for its users

Here we mean User Сentered Design, UCD, the main idea of it is the following: if we study our audience and optimize an interface for it then we can create really good interface.

• Is optimized for the user tasks

Now it’s about Task Centered Design, TCD. In obedience to this conception interface is good if it user tasks are executed effectively in it. This method is broader than UCD and includes it in a non-obvious form: really, how will we choose the best solution for the users if we don’t know everything about these users?

• Is optimized for the user motives

Goal Centered Design, GCD states that users do something for satisfaction of the personal necessities, otherwise — motives; when we identify these necessities and compare them to the user tasks we gain the better understanding of what should be done.

• possesses the high usability indices

Usability concept is more integral than above-mentioned. The most commonly used definition of usability (from the standard ISO 9241-11) states:

Usability is theextenttowhich a productcanbeusedbyspecifieduserstoachievespecifiedgoalswithefficiency, effectivenessandsatisfactionin and specifiedcontextofuse.

Here I also want to mention the definition produced by Vlad V. Golovach [1]:

Usability is an index of amount of errors, speed of interaction with a product, speed of gaining skills of interaction and subjective satisfaction of certain users of product achieving certain goals/reasons in the certain context of use.

• Is adequate to the user activity

There is conception of Activity Theory originated from the works of Russian psychologists Vygotskiy, Rubinstein and Leont’ev.

• Is commercially successful.

Other measure of interface quality is commercial success. Usually product should be sold and its interface should help sales. If it does than it is a good interface; if it doesn’t — it’s bad. This means that users like this interface.