Quick and Thorough SaaS Product Testing: Is It Possible?

The SaaS model is one of the most successful on the cloud computing market according to the 2018 SaaS Industry Market Report. This cost-saving technology is forcing many businesses to move to the cloud. However, any issues with cloud products can make your clients confused or angry or even cause them to panic.

The key to smooth performance of your SaaS product is conducting thorough testing before its release. But there’s often not enough time to perform testing by the book. There are a few tricks to ensure you don’t miss anything important in this process. In order to speed up testing, your QA team can use data from client analytics and prioritize testing of user-favorite functionality or platforms.

The Apriorit testing team examines SaaS products in several directions. To do that, our testers become experts in the client’s business and study the product’s logic and user stories. In this article, we talk about SaaS testing best practices and tools, and how to work around tricky parts of the SaaS testing process.

Pros and cons of the SaaS development model

There’s a high level of competition on the SaaS market, leading to high standards for service providers. SaaS providers that don’t work on the quality of their services risks losing their clients.

The SaaS development model has several disadvantages. Most relate to service–level agreements. Service-level agreements often lack specifics because SaaS providers write them in their own best interests. The essential prerequisite for an SLA should be ensuring 99.7% to 99.9% availability in most cases. However, the failure recovery process is often not written down at all.

 It’s important to understand the SaaS development process in order to make your testing process faster. An Agile methodology is typically used, but it depends on the specific project requirements. However, there are six key development phases for any SaaS project.

SaaS testing types

Apps, infrastructure, and the network are considered the main components of the SaaS testing methodology. There are several key types of testing for SaaS products:

 1.  Functional testing

Functional testing verifies that an application works how it’s supposed to. It’s implemented in a target environment (your data processing center or your cloud) by conducting manual tests according to specific test plans, considering the needs and requirements of the end user. Functional testing includes the following tests:

• Browser compatibility test to check app performance in various browsers.
• Regression test for every release, minor update, integration, or data migration.
• Automated functional and regression tests.
• Reliability test to find app weaknesses and reduce the number of failures during deployment.

2.  Performance testing

Performance is one of the most important attributes for a SaaS application. Therefore, it’s essential to carefully choose relevant software performance testing parameters. Performance of each app module should be tested alongside the workflow. In addition, testers can determine bandwidth performance that’s expected in the work process.

The team may estimate the application’s ability to handle loads and find the maximum load by putting high loads on the system. SaaS product testing with an emphasis on performance is essential for the success of a SaaS provider.

Performance testing also includes load, stress, and scalability tests. In order to evaluate an application’s response, you need to test it with various loads, including ones that exceed the load expected under normal operating conditions.

Failure and recovery tests are also important for SaaS testing. These tests check the system for functional disaster recovery after simulation of various crashes both internal (for example, failures when writing data to a database) and external (for example, internet connection issues or power cuts).

3.  Compatibility testing

One of the most important advantages of cloud services is the ability to access them regardless of:

• Browsers (Internet Explorer, Safari, Chrome)
• Platforms
• Operating systems (Windows, iOS, Linux)
• Mobile devices (tablets, smartphones)
• Hardware versions

Testing the compatibility of a SaaS app with different hardware is the most difficult part. There’s no way to perform SaaS app testing for all possible hardware configurations. You need a client’s help in order to define the testing methodology.

Here are several tips to help you choose which hardware to test on:

• Gather user statistics to find out the most popular platforms and app versions among end users.
• New platforms. Your client might want their profile to be connected with new platforms that haven’t been found in user statistics yet.
• When several platforms use the same core, testing the core of one of these platforms might be enough to cover the most important issues.
• Focus on the most commonly used routes in service.
• Test one type of technical services (client scripts, forms, data storage, management types).
• Test various technologies used in the application (GPS/no GPS, camera/no camera, Wi-Fi/no Wi-Fi, and so on).
• Request all screens and menus once only.

4.  Infrastructure and safety testing

This type of testing checks SaaS application safety as well as typical web application security failures (HTTP headline insert, cross-site scripting (XSS), SQL injection, and so on).

Here’s a SaaS testing checklist to ensure safety:

• Test the security of the network where the SaaS application is deployed.
• Review possible attacks and security threat scenarios.
• Test access privileges for various roles (especially in an environment with several tenants).
• Audit the security, integrity, and availability of test data.
• Define situations in which your SaaS application could be vulnerable.
• Confirm compliance with Payment Card Industry Data Security Standards (PCI DSS).
• Log security warnings, errors, and requests from unreliable sources.

5.  API integration and testing

The success of SaaS applications is based on working out scenarios, when a third-party developer creates their own applications using your API and thereby add value to your product. Therefore, it’s vital to test all API interface functionality, safety, usability, and performance as well as the accuracy of documentation.

6.  Maintenance testing

Maintenance testing is performed by an operational group. This team is responsible for ensuring the correct performance of applications, customer service, and billing. Usually, there are built-in instruments to help the operational team track and analyze problems in the following search fields:

• Applications
• Services
• Application servers
• Platforms (operating systems)
• Databases
• Data-level logs
• Alerts
• Warnings
• Functionality and performance errors

SaaS product testing features

Though there are not so many aspects of SaaS application testing, demand for testing in SaaS is actually high. There are two reasons for this: frequent releases and customer expectations for fast fixes. If some functionality doesn’t work, customers will expect it to be fixed quickly.

Speed is the centerpiece in the development and testing of your SaaS product. The best way to achieve this speed is using Agile methods. They allow you to start testing as fast as possible during the development cycle. Test cycles should be short, fast, and partly automated.

A short cycle allows you to speed up user access to application. This means that a QA team has to implement innovative approaches to catch up with product development. With unit testing, simulating incomplete components with visualization of services, and automating regression testing, the QA team can perform simultaneous tests instead of waiting for the cycle to end.

Some processes aren’t tested in SaaS applications, including support of various app versions and backends. These aspects should be taken into account when you’re planning the SaaS testing processes.

The user experience is the basis for prioritizing tests

The client experience is critical to the SaaS testing process. Different audiences will use different software and hardware configurations to access the product. Client analytics can tell the QA team which browser versions should be compatible with your app. Detecting the configurations most used by your target audience is important for prioritizing tests. This data helps you to improve the user experience.

Another step to quick testing is detecting users’ favorite features. This can help you to point out areas that deserve the most attention when writing automated and manual tests.

For quick SaaS testing, it’s important to understand the business objectives of the platform. It’s particularly important that any suggestions or improvements be made in full compliance with the goals of the entire organization.

Due to strong competition, the success of a SaaS platform depends not only on the user experience but also on the customer support experience. As for the user experience, an application has to be user-friendly. SaaS testers should become experts in user experience and understand the business logic and user scenarios.

There’s also another level of performance testing for SaaS applications. You have to consider not only the functionality of the platform in small environments but its general functionality as well. Intense competition is a direct reflection of the model itself, so testing and quality control are very important for the stability and success of your SaaS product on the market.

What makes SaaS testing tricky?

There are a lot of difficulties in testing any SaaS product. Here are the most common challenges a testing team may face:

• Lack of time. To compete in the market, applications are updated frequently. It’s necessary to check quality and safety of the product before each release.
• Verifying all licensing factors, including UX, number of users, and application features whenever a new version is released.
• User data confidentiality. It’s hard to assure confidentiality of data during integration, data migration, and simultaneous use of different user modes.
• Paying attention to performance testing. You have to identify the most accessible areas and test them with a large number of users from different places.

There are several general tips to overcome these issues, though solutions may be different from project to project.

• Create automated tests for SaaS products. This will help you to concentrate on complex bugs instead of wasting time on the most common issues connected with updates.
• Based on monitoring, define the most used features of your app. This will help you to conduct more effective performance testing if there’s a time limit.
• Use strong encryption during integration to ensure the security of the SaaS application.

Best SaaS testing tools and solutions

Testing speed is one of the most critical parts of the SaaS product lifecycle. Therefore, test automation is crucial. But besides automated testing, there are a number of tools that allow SaaS QA teams to optimize, simplify, and speed up the testing process.

Functional testing tools

Functional testing is one of the key types of testing, as it confirms the compliance of the developed product with the original functional requirements. The choice of tools for functional tests depends primarily on the specifics of the product.

Tools for changing IP addresses

In some scenarios, testers need to use VPN services because, for example, a SaaS product is developed for a market in a different country and has some inbuilt IP-based functionality limitations.

To avoid these SaaS limitations, we recommend using the following VPN services:

• TunnelBear is a cross-platform, multi-country VPN client with great features. It requires installation and registration with an email address. With TunnelBear, users can connect to local VPNs in 20 countries and simultaneously connect up to five devices. Good encryption, reasonable speeds, and minimal connection logs are the main benefits of Tunnelbear. But its free version provides only 500 MB of traffic per month.
• Hotspot Shield VPN Free Proxy. The free version of this service contains some restrictions on the choice of countries. Hotspot Shield VPN guarantees that all online transactions will pass through the secure HTTPS protocol.

If you need to use VPN services, you can also use Touch VPN, ZenMate VPN, Browsec, Hoxx VPN Proxy, friGate, or Hola VPN.

• React developer tools — a special tool to work in Chrome. When working with React in the Chrome browser, you may need the React Developer Tools extension, which can be found here. Usually, we use this extension when we need to work with React, namely when we need to enter data in the UI that isn’t supported by the field format.

To inspect sites that use React and view their components, do the following:

1. Open the site in Google Chrome.
2. Go to the Additional Tools tab and click on Developer Tools.
3. Find the React tab, which contains all information about the components used.

 API integration testing

The success of your SaaS solution depends on your script design allowing third-party developers to create their own applications using your APIs and thus add value to your product. Therefore, testing all APIs for functionality, security, usability, performance, and technical requirements is crucial to making your SaaS product successful.

The following tools will help you test API response codes for system requests and compare the received data from the API with your data in the UI:

• Advanced Rest API — This construction tool helps web developers test Rest APIs and services by sending custom HTTP/HTTPS requests.
• Postman — This is a convenient HTTP client for testing websites that allows you to quickly create requests with required HTTP methods and parameters, send the requests, and check the results.
• Requestly — This Chrome client plugin modifies HTTP requests.

Using these extensions, you can compose and edit simple or complex HTTP requests. Compiled requests are saved automatically for future use. Responses from the server can also be saved locally on the hard disk. In addition, the history of past requests can be saved by category (project). 

Generating random test data

If you need to get random addresses, first and last names, or ZIP codes for testing, you can use the US Address Generator. This website provides fake random addresses (with street, city, state, etc.), basic information (name, gender, birthday), phone numbers, credit card numbers, social security numbers, and even fake online profiles.

Non-functional testing tools

Unlike functional testing, which aims to verify that the actual product meets functional requirements, non-functional testing checks for compliance with non-functional requirements:

• User convenience (how easy it is for users to work with your application)
• Security (protection of user data, protection of application data, resistance to hacking), system performance at different loads)

Load testing and performance testing

There are several well-known testing tools such as JMeter, Gatling, and Tsung. Although they’re quite easy to use, it may be difficult for testers to analyze the results and come up with conclusions.

Our team has experience using JMeter. This application is designed to perform load testing of functionality and measure performance.

One of the convenient features of JMeter is that it works in proxy mode. For example, when we specify 127.0.0.1:8080 as a proxy in the browser settings and visit various website pages, JMeter saves all our actions and all related requests in the form of a script that you can edit later if necessary. This function makes it much easier to create HTTP tests.

Using JMeter, you can determine the maximum number of simultaneous users that a website can support.

JMeter simulates a high system load. Using JMeter, you can create and duplicate samples of user requests and launch them in separate threads.

JMeter provides a wide variety of options for visualizing performance reports.

Since SaaS apps are products that require authorization, you need to add an HTTP cookie manager (to store an authorized session cookie) when conducting load, performance, or stress testing of your solution.

UI testing

At the UI testing stage, it’s necessary to check whether it‘s easy to use the interface and whether it corresponds to the specified requirements and approved prototype. For UI testing, we generally use these tools:

Chrome DevTools is automatically embedded in the Google Chrome browser. It can be launched by going to Settings (icon in the upper right corner) –> More tools –> Developer tools. The Chrome DevTools are very useful for debugging websites. They can help you edit pages on the fly and diagnose problems quickly, which ultimately helps you build better websites faster. DevTools is effective when you need to introduce changes in the design mode.

Here are some examples of when DevTools may come handy:

• You face some challenges in the UI with menu names.
• You need to see how a menu item would look with a certain number of characters to determine the expected result in the bug report.
• You need to make a clear screenshot of the data you want to see after fixing.
• You want to see the hyphenation of words.

The WhatFont tool is an extension that helps you to identify all fonts on a page. By hovering over any text on the page, you can see the name of the font. In order to find out complete information about the font, name, size, and even color, just click on the text.

Cross-browser and cross-platform testing

As a rule, a website or web app doesn’t work equally well in all browsers. This may be due to the version of the browser under test, different behavior of the app or site on mobile and desktop devices, as well as variations in environments.

Cross-browser testing is a method of quality assurance for web applications across multiple browsers. To ensure cross-browser capabilities, we often use BrowserStack for app testing.

To assist with cross-browser testing, the Chrome extension Cross Browser Testing Local Connection allows you to test in 1500+ real desktop and mobile browsers.  

Sometimes, you may need to conduct cross-browser testing or check your product on a mobile browser without having an appropriate device. In these cases, you can use User-Agent Switcher for Chrome. This extension allows you to change the user-agent string so you can easily check how websites are displayed in different browsers and on different devices.

The User-Agent Switcher settings allow you to emulate browsers including Chrome, Internet Explorer, Firefox, Opera, and Safari, and platforms including iOS, Android, and Windows Phone.

Penetration testing tools

Penetration testing is a family of testing practices that simulate various actions to penetrate a system and disrupt its functioning. A pen test can detect vulnerabilities in the protection of SaaS solutions.

Pen testing is usually performed either by an in-house team or by an independent penetration testing team. As a rule, we use the following web scanners:

• OWASP Zap — a fairly easy-to-use tool for pen testing as well as for finding vulnerabilities in apps
• Burp Suite — one of the most popular tools for penetration testers around the world due to its flexibility and ways to combine manual and automated analysis methods when conducting web application security testing

Tools for test automation

The sooner the testing begins, the faster the product gets to actual operation. Therefore, we recommend starting automated tests at the very earliest stages of development.

A common practice with SaaS solutions is to build a framework based on Selenium WebDriver for test automation. Here’s an example of a framework that uses Java.

Tool type	Tool
IDE (integrated development environment)	IntelliJ IDEA Community Edition
Tool for integration with the browser	Selenium Webdriver
Plugin for test run automation	TestNG
Test build tool	Apache Maven
Performance testing tool	Apache JMeter
Continuous integration and reporting tool	Jenkins
Repository	Bitbucket

It’s worth noting that the topic of automation and selection of tools is quite extensive.

Conclusion

The quality of SaaS testing is one of the key factors in the success of your product. High-quality testing will help you stand up to the competition in the market. Before releasing your SaaS application, you need to launch an MVP for your SaaS solution, test the product functionality, performance, infrastructure, security, API integration, and compatibility with different operating systems and platforms. You also need to conduct maintenance testing.

SaaS software development is one of our specialties at Apriorit. We pay much attention to all types of product testing, ensuring the smooth performance of SaaS products before their release. To do this, we study not only a product itself but also its tasks, the role of customers in the business, and the functions it will perform. Contact us if your project requires specialized QA expertise.