Traffic Spikes in Cloud-Based Applications: How to Mitigate Them on a Budget

Trying to predict traffic load is one of the trickiest tasks in cloud application development. If you migrate an already functioning application to the cloud, you can estimate the load based on existing usage data. But if you launch a startup, the only available load prediction tool is market research, which can’t be 100% accurate.

At Apriorit, we often work with projects where our clients expect their application to have one thousand users a day but it actually gets one million users a day after the release. Such traffic spikes in cloud-based applications can easily crash them. From our experience, we’ve learned that the most reliable way to mitigate this risk is to implement flexible scaling mechanisms in the product before it attracts lots of users.

In this article, we discuss how to prepare cloud-based application for traffic spikes, plan and budget software development with no traffic load predictions, and what pitfalls to avoid.

This article will be useful for both startups and established businesses that want to launch a new cloud application and want to learn all the necessary variables to consider.

Preparing for software development: what information do you need to gather?

Stories of many successful IT projects start with In the beginning, there was just an idea. But in fact, you need much more than just an idea to build functioning and competitive software. Before a single line of code is written, the outsourcing project team works closely with their client to gather all kinds of data and form a clear understanding of the nuances of the future product.

These activities happen during the project discovery phase, which usually involves project managers (PMs), business analysts (BAs), software developers, quality assurance (QA) specialists, and stakeholders. 

The key aspects of project discovery for a cloud-based solution are:

• Business logic. The project team needs to understand how the product should work, who its end users are, and what value it provides. It’s essential to form a single understanding of the product’s business logic, as various stakeholders within the client’s organization may see it differently. 
• Budget. This includes the planned cost of both the development process and further product maintenance. Based on these estimates, developers choose the tools they can use, a platform for product deployment, infrastructure elements, and features they can implement.
• Total number of users. Knowing the maximum number of users, developers can estimate how much data these users will generate and interact with, what size of storage the project needs, and how to prepare for traffic spikes.
• Requests per minute. The application has to work flawlessly and process user requests as fast as possible. To achieve this, the development team must know the estimated number of requests users will make per minute. Thus, they can choose suitable hardware and a scaling system for the product.

Ideally, the client can figure out all four parameters by themselves. However, many clients start discussing their projects with us when they only know the business logic.

In such cases, we help our clients figure out their potential number of users and requests and suggest various budget options. Let’s take a look at the budget proposal for our recent cloud application development case.

How to decide on a development budget when the traffic load is unclear

Software development and maintenance budgets are the key criteria for choosing deployment and scaling options for a cloud-based product. When estimating the development budget, make sure to include costs for maintenance, which are usually discussed at the project discovery or software development stages.

It also can be challenging to understand how the maintenance budget correlates with your application’s scaling capabilities. Especially if you have yet to gain experience with cloud scaling technologies.

That’s why at Apriorit, we prepare several budget proposals that differ in the number of processed requests per second, recommended technologies, monthly maintenance cost, and DevOps integration time.

Let’s take a look at a real proposal from one of our recent AWS-based projects:

Such a proposal helps our clients better understand their options if they aren’t sure about the load on their application.

Even when our client knows how many users they expect to interact with their system, we recommend implementing one of the AWS auto scaling strategies. Doing so will increase the development time by 30-40%, but it will barely influence the maintenance budget for cloud software with a consistent traffic load. In case of an unexpected traffic spike, such a solution will allow your software to remain stable and usable. 

Let’s take a look at the example of one of our projects that relied on auto scaling. We used Firebase to deploy our client’s software. With a consistently small load of about 300 users, using cloud infrastructure was quite affordable:

The next month, our client’s product attracted 35,000 users. The software had to process ten times more requests without losing performance. The system started using autoscaling mechanisms, but the maintenance costs remained relatively low:

With peak traffic load, cloud maintenance cost our client twice as much. But their product performed correctly, quickly responding to user requests and helping our client make more money.

Using Firebase isn’t the only way to ensure reliable and affordable autoscaling. Let’s take a look at some other options we often use for Apriorit projects.

4 tools for efficient cloud autoscaling

Each cloud computing provider has built-in autoscaling services that you can implement to prepare your app for traffic spikes. And there are dozens of third-party solutions for implementing or improving autoscaling. Analyzing all of them and choosing the best option for each project can be time-consuming, so we have figured out the tools we can safely use for most of our cloud-based projects.

Let’s take a quick look at them.

Firebase

Firebase is a cloud application development platform created by Google. It allows you to build cross-platform mobile and web applications while using Firebase hosting, data storage, databases, authentication mechanisms, performance monitoring services, and more.

Firebase provides developers with enough tools to design an architecture for a simple application:

You can also combine Firebase elements with services from other cloud providers like AWS or Google Cloud.

Firebase has free and paid versions. The free version has significant load limitations, including a cap on the maximum number of users per month, available storage, and access to the test lab. So if you plan on using Firebase for a commercial product that may experience a heavy traffic load, it’s best to use the paid version.

In our experience, Firebase is one of the best autoscaling solutions on the market for startups and small organizations, as it allows you to quickly deploy a simple cloud-based application and prepare your web app for a traffic spike on a budget.

AWS Elastic Load Balancing

Elastic Load Balancing is an AWS service that monitors the condition of instances used in your application and evenly distributes traffic load between them. It allows you to use the computing power of virtual hardware most efficiently. There are two types of Elastic Load Balancing:

• The Application Load Balancer works on the application layer and provides you with lots of features to configure custom traffic routing: routing requests to several ports, dynamic host port mapping, path-based routing, and more.
• The Network Load Balancer works at the transport layer and selects a target port for traffic automatically, making it able to process millions of requests per second.

Here’s an example of scalable architecture you can build using a load balancer:

Such a scheme is easy to configure for a small project. When your product starts getting more user requests, you can add more EC2 instances to process them instantly. You can also limit the number of instances your application uses to keep cloud maintenance costs under control.

AWS Lambda

AWS Lambda is a serverless computing service that helps AWS-based applications scale without any hardware configurations. This service automatically adds more computing power when your application experiences a traffic spike and reaches its performance limit.

Here’s an example of an application architecture that scales using Lambda:

The key benefit of using Lambda is the ease of configuration: you can simply add this service to your application instead of configuring the type and hardware for EC2 instances. It’s also more flexible in terms of payment. With EC2 instances, you pay for the whole instance you rent, even if you use only a fraction of its computing power. With Lambda, you can pay for the exact computing resources you use.

However, if your application is constantly working under a heavy load, scaling it with Lambda will get pricey compared to other autoscaling tools.

AWS CloudWatch

If you aren’t ready to adopt autoscaling, you can set up notifications on incoming traffic spikes and scale your cloud infrastructure manually. AWS provides CloudWatch alarms — a service designed for monitoring infrastructure and collecting metrics on your application’s performance.

With CloudWatch, you can create simple metric alarms that will notify you about spikes in some of the metrics related to your application’s performance. CloudWatch also allows for setting up composite alarms that include several conditions and will notify you only when all of the conditions are met.

Here’s how you can configure a similar architecture to the one we showed above, only using alarms instead of a load balancer:

In this scheme, you can only use static instances, and the system can’t add new ones if it registers a traffic spike. When the system reaches a certain level of traffic load, it notifies the development team and they can add new instances or rebalance the load manually.

Using CloudWatch alarms instead of autoscaling is beneficial when you have a strictly limited maintenance budget and know what signs of traffic spikes to look for.
Autoscaling seems like the most convenient option to prepare your software for unexpected traffic spikes. But such services have disadvantages that may come at a cost during the later stages of the application lifecycle. Let’s look at the key downsides and ways to mitigate them.

Autoscaling challenges you should consider

When misused, autoscaling mechanisms can increase maintenance costs, force you to make unplanned changes to the product, and make you dependent on the platform. Let’s take a look at the three biggest drawbacks of autoscaling that we have experienced in our projects.

1. Dependency on the cloud platform. When you choose cloud scaling, you have to use the services of the cloud provider to build all of the product infrastructure. This means you’ll have to play by the rules of the provider and will have little influence on them. For example, if your provider changes privacy or pricing policies, you can accept and implement them into your product or take your product off the platform.

For one of our clients, this dependency became a major challenge when Google Cloud changed the way they calculated costs for their cloud services and the price of hosting the client’s product multiplied several times over. Due to that, our client had to perform an unplanned migration to another cloud service.

It’s impossible to avoid or mitigate this risk of using cloud services. Before deploying your project in the cloud, you need to assess the possible implications of depending on a cloud provider and decide whether it’s worth using the cloud for your project.

2. Challenges in code migration. Each cloud platform has its nuances that you have to adapt your product to. Depending on the size and complexity of your product, migrating from one platform to another can take from a couple of months to a year.

However, you can simplify possible code migration by:

• Using libraries that support most cloud-based platforms. For example, you can use Node.js for the REST API and “Express” library for AWS Lambda, AWS EC2, and Google Cloud Compute Engines.
• Separating business logic from code that uses databases, notifications, and other services. This way, you can quickly migrate the core of your project to another cloud platform and add new infrastructure elements to it. This also helps you quickly change third-party services if you need to.

3. Dependency on your development team’s skills. The autoscaling mechanism is only as efficient as the team that configures and deploys it. Even a small error in application code can significantly increase the cost of using cloud services. Here are two examples of code in React.js:

// costs $30 per day
useEffect(() => {
    requestToServer();
});

// costs nothing
useEffect(() => {
    requestToServer();
}, []);

Both code examples work, but the first sends a request to the server each time when the UI component rerender the content. Constant requests to the back end force our system to scale. In the second example, we send requests when the component renders first time and that means that request is sent only once.

You need at least one person in the development team who deeply understands scaling systems, can build one, and can fix development errors like the one shown above during code review.

The way to mitigate such a dependency on certain skills is to work with teams that have successfully delivered cloud-based solutions and have experience with configuring scaling for the cloud platform of your choice.

Conclusion

Unpredicted traffic spikes can crash your application and make it unusable for your customers. You need to take this risk into account when planning software development and drafting the budget.

The most reliable way to prevent a web application from crashing from traffic spike is to use autoscaling services. Such services allow your application to scale almost infinitely using the resources of a cloud service provider. This way, you’ll make your product usable under heavy loads and keep maintenance costs under control.

The efficiency of autoscaling depends on the scaling tools you choose and the skills of your development team. At Apriorit, we have experience building and managing cloud infrastructure for projects of various scales and with various technology stacks.

Reach out to leverage our experience and secure your software from traffic spikes!