Techniques for Estimating the Time Required for Software Testing

Finding the right balance between software quality and the release date is one of the most significant challenges in software development. While software testing ensures your product meets all customer requirements, addressing each product risk with a test case and running those test cases may significantly postpone the product’s launch.

This article contains methods and recommendations that can help you obtain realistic and functional QA time estimates for your project.

The importance of estimating testing time in software engineering

Ensuring accurate time estimation in software engineering is essential, and this includes both development and testing processes. A properly planned testing process is a must for ensuring the required level of software quality without exceeding a project’s time and budget. Misestimation can cause a delay in product delivery or decrease the product’s quality and competitiveness. Estimating software testing is a rather complicated and volumetric process, but it’s critical for creating a successful project.

There are two things you should always keep in mind when estimating the time required for testing your product. The first is that people often struggle to precisely estimate the time required for a task. The second is that testing is no longer a one-time task but a continuous process. Let’s take a look at testing estimation techniques in agile. Many companies work with agile methodologies, where testing doesn’t stop even when software reaches the production stage.

Let’s consider what methods will help you precisely estimate your testing time, taking into account these two factors. But first, let’s see what stages of the on-premise and cloud testing process you must include in your estimates.

Testing stages you should estimate

Software testing is a process that consists of multiple stages, and thoroughly estimating them is your key to success. According to the International Software Testing Qualifications Board (ISTQB), there are five fundamental testing stages:

1. Test planning and control
2. Test analysis and test design
3. Test implementation and execution
4. Evaluating exit criteria and reporting
5. Test closure activities

Let’s take a close look at each testing stage and discuss the peculiarities of estimating the time required.

Test planning and control

The test planning and control stage consists of two sub-stages:

• Researching project
• Developing a testing strategy

Project research is important for gathering all the necessary information and getting the whole picture of the project. There are many ways to research a new project: by analyzing project documentation, discussing the project with the customer, working out project details with managers, etc.

On average, we recommend spending one to two days on this sub-stage. This doesn’t necessarily need to take the whole working day, but a one- or two-day estimation process will allow you enough time in case new ideas and questions appear.

However, additional time may be required in the following cases:

• The project is rather large and has a lot of documentation.
• The project requires additional research due to a high number of unfamiliar tasks.
• Specialists who take part in the project discussions have different schedules and are geographically dispersed.

Designing a testing strategy will let you prioritize your actions to conduct tests in the most effective and efficient way. To create a well-planned testing strategy, you need to answer a number of questions:

• What types of testing will be performed?
• What kinds of specialists are required for developing test plans and test cases?
• How many specialists will perform project testing and what skills should they have?
• What testing tools should be available and how should they be configured for project testing?

Our company has a set of empirically tested best practices and approaches that help us answer these questions. The final choice of practices to be used depends on the particular project.

Additionally, to ensure precise time estimates for the whole test planning and control stage, it’s important to decide whether your project requires the following activities:

• Risk management — This often includes creating a risk matrix, assessing risks, and eliminating them.
• Additional testing — Some projects might require running extra tests like non-functional tests, localization tests, or compatibility tests.
• Testing environments — You need to plan in advance whether your project will have both alpha and beta testing.
• Test automation — Some manual testing activities can and should be automated. Plan this ahead as well.

Note: Defining what specialists will work on the project will help you more precisely estimate the time required, as you’ll be able to take into account their skills and experience. Don’t forget to include their vacations and days off in the testing schedule.

Test analysis and test design

Before analyzing and designing your tests, make sure you clearly understand the product requirements. ISTQB defines the following tasks for this stage of testing:

• Reviewing project documentation
• Identifying test conditions based on analysis of test items
• Defining test objectives and writing test cases
• Preparing test data according to test conditions and test cases
• Designing the test environment

Below, we discuss the peculiarities of estimating the time required for some of these tasks.

Defining test objectives and writing test cases

Defining test objectives and writing test cases is a labor-intensive process that requires considerable time expenditures. To take everything into account, we recommend you start with creating a test plan. Your test plan should be based on a testing strategy that shows what kinds of test cases should be created and helps you set their priority.

In our practice, we follow the rule that each product requirement must be covered by at least five test cases. This rule helps us calculate approximately how many test cases we’ll need and how long will they take.

For example, it takes about two to three days to build and review a simple test plan without test cases. Creating one test case usually takes about 10 minutes. The more test cases your product needs, the more time you should estimate for their development.

However, not all projects require full test case coverage. Instead, you can create your tests based on ready-to-use checklists, saving some time on your testing activities.

There are a few other aspects you should keep in mind when estimating the time for writing test cases:

• QA experience — The time a QA specialist needs for preparing a test plan and test cases directly depends on their experience. The more experienced the specialist, the less time they’ll need for reviewing the test plan and test cases and making changes.
• Competencies — When faced with a new technology, your team might need anywhere from a day to several weeks for additional research. The precise estimate depends on your team’s qualifications and the complexity of the new technology.
• Test data — Depending on the types of testing and the project, you may need some time for creating test data.
• Time for reviewing test plans and cases — Allot time both for reviewing newly created test cases and altered test cases. When planning this time, you should take into account the workload and schedules of the specialists who will review the tests.

Preparing test data

When writing test cases, you need to have test data that will be used as an input for detecting defects in certain product features or scenarios. This information should be precise and complete, and it takes time to achieve this. The time needed for preparing test data depends on many factors, such as:

• the size and scope of the project
• the amount of data needed for project testing
• the variety and types of test data
• the need for generating new test data.

Designing a test environment 

The test environment consists of the software and hardware necessary for executing test cases. For example, for mobile app testing environment can consist of real or virtual mobile devices. Its configuration must mimic a user-like environment to identify any possible issues that end users may face on their devices. When designing the test environment, a QA specialist should properly configure hardware and software as well as the operating system and configure test terminal settings.

The time required for this stage of work depends on the following factors:

• Test environment complexity. The time for installing and configuring the test environment directly depends on its complexity. Usually, it takes from one hour to one day to configure a test environment for a mid-sized project that works with popular operating systems and doesn’t require complicated solutions.
• Additional equipment. When you need to expand the testing laboratory, take into account the time necessary for getting additional equipment. If the required equipment is on sale and the company can buy it at once, the time will be short. However, it may take longer if specific equipment needs to be delivered from abroad or if the company can’t afford it at the moment.

Test implementation and execution

Test implementation and execution is a fundamental process in which the actual work is done. During this process, test cases with test data are executed.

This stage includes the following tasks:

• Conducting full testing based on a checklist or test cases
• Performing regression testing
• Executing other types of testing like smoke testing and impact testing

One of the main difficulties of planning the test implementation and execution stage is that you can’t precisely predict the number of bugs you’ll find and the complexity of localizing them. On average, writing a report on one bug takes 10 to 15 minutes. The more bugs are found, the more time will be required for each report. Let’s explore examples of how to estimate a software project in man hours.

If a bug is too complicated, it may even take several hours to find its exact location. There are methods to estimate the number of possible bugs in every subsequent product version, but it’s difficult to exactly estimate the time for detecting and fixing them.

It’s also important to include time for other activities in the work schedule: reporting on found bugs, defining the location of the most complicated bugs, etc. According to our testing estimation experience, we recommend you add about 20% to 25% of the total testing time for such activities to your final estimate.

At any rate, you should follow your common sense and not let other activities go beyond the allotted time. If product testing is estimated for 10 hours but you need to spend more than 2 to 3 hours localizing one bug, it’s logical to postpone this activity unless the bug is too critical or blocks product performance. Return to this issue if time remains at the end of product testing or request additional time.

Evaluating exit criteria and reporting

The main goal of this stage is to validate the received test results with the exit criteria or the conditions for completing tests at each test level that were defined and agreed with stakeholders at the planning stage. According to ISTQB, this test stage includes such activities as:

• checking test logs against the exit criteria
• determining if further testing or exit criteria revision is required
• writing a test summary report for stakeholders.

The time for these activities is included in the time necessary for performing each level of testing.

Test closure activities

The main task of this stage is to collect and archive all test artifacts (scripts, environments, etc.) for further use and conduct a retrospective to define lessons learned.

The time needed for this type of activity doesn’t always need to be estimated.

Time estimation factors

All the testing stages we’ve mentioned depend on a range of factors that you should keep in mind when estimating working hours. Some of these factors we’ve already mentioned, but let’s summarize them:

Human factors	Technical factors
Knowledge and testing experience with the project technology Domain area knowledge Buffer time for accidents and emergencies Time for team communication	Availability of ready-to-use tools Readiness of test environment and test data Availability of open-source or third-party tools and libraries Internal product quality (code, instruments, specifications) Use of automation tools

Figure 1. Time estimation factors

Time estimation techniques

So, how to do estimation in software testing? Below, we describe effective software testing estimation methodologies for achieving accurate estimates.

Let’s start with the types of estimation techniques in testing that require detailed mathematical calculations:

• The Delphi method (Estimate-Talk-Estimate) is a communication technique that relies on a panel of experts for providing more accurate estimates.
• Three-point estimation is a technique based on statistical data and includes three values for every task: the best case, the most likely case, and the worst case estimates.
• The function point method measures the project size and weights each function point based on past projects.

On the bright side, all these methods are great for gaining precise estimates. On the downside, they are:

• time-consuming
• labor-intensive
• inflexible

However, given a fast development speed and agile methodologies, the process of time estimation should also be flexible. Otherwise, your most thorough plans will be easily disrupted if unforeseen events suddenly appear. Thus, let’s discuss some more flexible methods that you can use intuitively.

Trial-and-error approach

The trial-and-error approach is used every time you do something new. Put simply, you try, fail, then try again. This pattern is repeated until you finally succeed.

Analogy-based and expert estimation methods

If your team already has experience in projects similar to the current one, you can use an analogy-based method to plan the time based on a known time for a previous project.

In addition, you can use recommendations from your team members who have experience performing similar tasks or external experts with domain-specific knowledge. It’s better to consider estimates made by several experts. However, keep in mind that the estimate of an external expert may be incorrect, as those performing the task may lack the necessary qualifications and require time for learning.

Test task breakdown method

To make testing time estimates more accurate and realistic, break down test tasks, i.e. divide the testing process into several parts and estimate the time for each.

This is a formalized method, but it requires the least effort for assessment. You just need to divide your scope of work into the smallest tasks you can easily estimate.

However, during task decomposition, consider the following questions:

• What test task changes will impact product testing?
• What test environment configurations should we use for verifying test results?
• What test data should be prepared?
• How will feature changes impact product performance?
• What test preparations are required?
• What processes can be automated?

Percentage of development approach

This approach is based on what percentage of time the testing process usually takes from the overall development estimate. Say your expect project development will take about 1.000 hours and you know that your company usually spends 40% of your programming time on testing for similar products. In this case, you can estimate that for this project, testing will take about 400 hours.

However, keep in mind that this approach should take into account project risks, staff expertise, the type of application under development, and so on.

Top-down and bottom-up estimates

Time estimation for software testing can be carried out by managers based on previous projects. In this case, managers tell the executors how much time a particular testing stage should take. However, this method doesn’t actually take into account human factors. To ensure accuracy, it’s often helpful to compare top-down estimates against your bottom-up estimates or against estimates made by task executors.

The Cone of Uncertainty

The Cone of Uncertainty, described by Steve McConnell, shows that it’s hard to estimate the time for performing any task at an early stage. The accuracy of an estimate depends on the project stage when it’s made. For instance, at the early stage, the execution of a 12-month project may easily be estimated at either three or 48 months.

The level of uncertainty is higher at early stages, as many project variables are still unclear: software-specific details, requirement details, staffing, project plan, etc. It’s impossible to take into account all risks that may occur during the work process.

You can narrow the Cone of Uncertainty by making time re-estimates after implementing any changes to the project.

Software development time estimation template

Let’s see how we can apply the described time estimation methods in practice.

Say we need to test a new product that consists of three main features. Preliminarily, product testing is divided into two iterations.

• Iteration #1: feature #1 + feature #2
• Iteration #2: feature #3

These product features have different levels of complexity, which will impact the time required for their testing:

• feature #1 has a middle level of implementation complexity
• feature #2 has a low level of implementation complexity
• feature #3 has a high level of implementation complexity

We know that product testing will be performed by one QA specialist. So we need to estimate the time for product testing and submit our estimates to the client.

Let’s suggest our time estimation task has the following variables:

• Our QA specialist may be either junior or senior
• Our QA specialist may or may not be familiar with the specific domain

To estimate the time necessary for testing our product, we’ll use a combination of the following methods:

• Trial-and-error approach
• Analogy-based method
• Expert estimation method

In the tables below, you can see the results of our estimation:

Iteration #1	Junior	Senior	Comments
	new	experienced	new	experienced
1. Specification revision
1.1. Reading and analyzing specifications for all three features	10	10	10	10	The time required for reading and analyzing the documentation depends on the volume and complexity of the specifications.
1.2. Clarifying specification details	4	3	3	2	Estimate 30% to 40% of the time for 1.1 for preparing qualifying questions if these features are new for the tester (for conducting additional research and following best practices) and 20% for cases when a tester has experience testing similar features (knowledge about how a feature should react in abnormal situations).
1.3. Researching domain area	16		12		Estimate the time for this activity in case the features are new to the tester.
2. Preparing test documentation
2.1. Writing test cases/ checklists for full/acceptance/smoke testing for feature #1 and feature #2	17.5	17.5	12.5	12.5	When estimating this task, take into account the QA specialist’s expertise as well as the function complexity and specification requirements for which we will write test cases. Estimation can be performed with the help of expert advice or previous experience in similar projects.
2.2. Writing test cases/ checklists for full/acceptance/smoke testing for feature #3	17.5	17.5	12.5	12.5
2.3. Reviewing test documentation (for a reviewer)	12	12	9	9	Estimate the time required for reviewing test documentation. No matter who performs testing, the review will be done by a senior tester or another tester. Let’s take 30% to 35% of the time calculated for writing test documentation for the whole product (2.1 and 2.2).
2.4. Making corrections to test documentation after review	12	12	5	5	For a junior, it will take more time to make corrections to the test documentation after its review because we assume a lower quality of tests. Let’s estimate 35% of all time calculated for writing test documentation (tasks 2.1 and 2.2) for a junior and 20% for a senior.
3. Designing the test environment and preparing test data	16	10	16	10	This activity depends on the project and doesn’t need to be allotted time if your company already has the necessary data and equipment. Otherwise, you’ll need time to design an environment and prepare test data.
4. Testing
4.1. Smoke testing (first check after product implementation)	4	4	4	4	The main goal of this task is to find bugs that block product performance and report on them within four hours of the build implementation.
4.2. Full testing	24	20	17	16	Full testing is performed in accordance with test cases and checklists. Feature testing may take less time if a tester has previous experience dealing with similar test cases.
4.3. Modifying tests for feature #1 and feature #2	3.5	3.5	2.5	2.5	New code often appears that hasn’t been covered with tests. Let’s estimate 10% of all time calculated on writing test documentation (tasks 2.1 and 2.2) for this activity.
4.5. Confirmation testing	4.8	4	3.4	3.2	Confirmation testing usually takes 20% of the time calculated for full testing (task 4.2).
4.6. Regression testing based on feature #1 and feature #2 impact on the product	12	10	8.5	8	Let’s estimate 50% of the time calculated in 4.2 for full testing for regression testing.
Total working hours for testing activities	153.3	122.5	115.4	94.7
6. Risks	30.66	24.5	23.08	18.94	Let’s estimate 20% of the total testing time planned for iteration #1.
Total working hours for iteration #1	183.96	147	138.48	113.64

Figure 2. Iteration #1

Iteration #2	Junior	Senior	Comments
	new	experienced	new	experienced
1. Testing
1.1. Smoke testing (first check) for feature #3	4	4	4	4	The main goal of this task is to find bugs that block product performance and report on them within 4 hours of the build implementation. During smoke testing, we perform only the most important test cases.
1.2. Acceptance testing for feature #1 and feature #2	6	5	4	4	Estimate the time for verifying that after implementing feature #3, everything works well in feature #1 and feature #2. Acceptance testing usually takes longer than smoke testing, especially for a junior tester, as it includes both positive and negative test cases.
1.3. Full testing for feature #3	24	20	17	16	Feature#3 has a high level of implementation complexity, so testing it will take the same time we spent on full testing of both feature #1 and feature #2 in iteration #1.
1.5. Modifying tests for feature #3	2	2	1.5	1.5	Let’s estimate 10% of all time calculated for writing test documentation for feature #3 (see iteration #1, task 2.2).
1.6. Confirmation testing	3.5	3.5	3	3	Note that non-critical bugs in feature #1 and feature #2 may be fixed during task 1.2 of iteration #2. Estimate some time for checking them.
1.7. Regression testing based on impact for feature #1, feature #2, and feature #3	24	20	17	16	Let’s estimate 50% of the full product testing time (the full testing of feature #1 and feature #2 plus the full testing of feature #3).
1.8. Acceptance testing of all product features before sending the product to the customer	9.6	8	6.8	6.4	Let’s estimate 20% of the full product testing time (time for 4.2 from iteration #1 and 1.3 from iteration #2).
Total working hours for testing activities	73.1	62.5	53.3	50.9
6. Risks	14.62	12.5	10.66	10.18	Let’s estimate 20% of the total testing time planned for iteration #2.
Total working hours for iteration #2	87.72	75	63.96	61.08
Total working hours for product testing	271.68	222	202.44	174.72

Figure 3. Iteration #2

Conclusion

Quality assurance is a complicated process with a high risk of uncertainty, and there’s always some deviation from all software test time estimates. Thus, it’s challenging to indicate exact test estimation techniques for software testing, as well as define product metrics in software engineering.

That’s why it’s efficient to combine different software testing estimation techniques and methods, taking into account the specifics of the project at hand and the testing team. In this way, you’ll have a clear understanding of the factors that influence the cost, time, and factors, like the team’s knowledge or the specific project development model. To learn how to improve the testing process, check out our article on 6 ways to improve software testing.

Apriorit has a team of ISTQB qualified QA specialists who have mastered accurate testing estimates. We’ll be glad to assist you in testing your product and provide reliable estimates for doing so. Feel free to contact us using the form below.