Choosing an Effective Python Dependency Management Tool for Flask Microservices: Poetry vs Pip

The microservices software architecture has gained popularity over traditional monolithic architectures due to its scalability and flexibility. But the benefits come with a dark side.

A microservices architecture consists of small, loosely coupled services, and each service may have its own set of dependencies. Effectively managing and coordinating these dependencies is challenging, especially as the system grows. To deal with this, you need dependency management tools.

In this article, Apriorit’s Python development experts discuss Pip and Poetry — two popular tools for managing dependencies and packages in Python. 

What is dependency management in microservices, and why is it important?

Managing dependencies is a crucial part of developing microservices. It helps developers handle external libraries and packages that the application relies on. Python developers at Apriorit often rely on external libraries to enhance application functionality, boost performance, and allow microservices to interact with other systems. 

Dependency management can become complex when dealing with distributed applications, as you need to track and update your packages and libraries across multiple services and environments. To tackle this challenge, you need to make your dependency management process efficient and preferably automated. 

Effective dependency management in microservices involves:

• Tracking required packages and their versions
• Installing and updating packages
• Ensuring that the application functions as expected with different package versions
• Creating isolated development environments to encapsulate dependencies and configurations specific to each microservice
• Managing different environments to reduce conflicts 
• And more

In this article, we focus on dependency management in Python microservices. At Apriorit, we often use Python to build microservice-based software due to its versatility, cross-platform nature and features like asynchronous development. Python allows us to build scalable, reliable microservices quickly and efficiently. 

Python has multiple frameworks for building microservices. In this article, we use Flask to demonstrate how to build microservices in Python, as it’s highly customizable and flexible. Flask integrates smoothly with other Python libraries and tools, which allows for easy dependency and package management.

Let’s first discuss the technologies we’ll work with and why they’re great for building microservices.

Overview of top Python dependency managers

Pip and Poetry are two popular Python dependency management tools. They allow developers to automate the process of managing dependencies and simplify such operations as package installation, dependency resolution, and versioning. Both these tools allow developers to focus on building their applications while maintaining a Python application’s stability, security, and smooth operation in any environment.

Pip and Poetry use virtual environments as their core functionality to manage Python dependencies in microservices. Virtual environments allow developers to create and manage isolated environments with independent sets of dependencies, packages, and Python interpreters. These environments provide a clean space for developers to experiment with various packages and configurations without affecting the global Python environment or other projects. 

By using virtual environments, Pip and Poetry can help to prevent version conflicts between packages and ensure that each microservice has access to the correct set of dependencies. 

Let’s now discuss Pip and Poetry in detail and explore their advantages and disadvantages for dependency management.

Pip

Pip is a command-line tool for installing packages and managing dependencies that’s pre-installed in most Python distributions. It can install packages from the Python Package Index (PyPI) or other sources, resolving package dependencies and installing packages globally or in virtual environments.

Using Pip is straightforward. Developers can use the Pip install command followed by the name of the package they want to install. This command will automatically download and install the package and its dependencies.

Pip also supports various other useful commands, including:

• pip list to display installed packages
• pip freeze to create a requirements file containing a list of installed packages
• pip uninstall to remove installed packages

Pip is an essential tool for Python developers, providing a convenient and efficient way to install and manage Python packages and libraries. Let’s look at its implementation.

Below are the main advantages of Pip for dependency management.

1 . Built-in. Pip comes with Python by default, making it easily accessible.

2 . Ability to install individual packages. Pip supports the installation of individual packages and their dependencies, making it ideal for microservices with specific package requirements.

3. Flexible dependency management. Packages can be installed globally or in a virtual environment, providing flexibility in managing dependencies.

4. Large community. A large number of contributors and users ensures that many packages are available for your project.

Pip also has its shortcomings. 

1. Centralized dependency management. This type of management can lead to issues with conflicting package versions.

2. No project metadata management. Dependencies can be a form of metadata. The absence of metadata management in Pip makes it challenging to ensure a project’s consistency across environments.

3. No lockfile management by default. The lack of this feature makes it hard to recreate environments across multiple machines.

Next, let’s explore the key specifics of Poetry.

Read also

How to Accelerate Microservices Development: a Practical Guide to Applying Code Generation

Get our free eBook to explore tried ways to speed up software development and gRPC-based microservices to a GraphQL client.

Learn more

Poetry

Poetry is a recent addition to Python dependency management tools. It offers a simple and intuitive method for managing project dependencies, managing virtual environments, and building systems. 

Poetry allows developers to declare project dependencies in a standardized and easy-to-manage way. It generates a lock file to ensure that the same versions of all dependencies are installed across different environments.

Using Poetry, developers can also easily create distributable packages for their Python projects, which can be shared with other users or deployed to production environments. 

Advantages of Poetry

1. Simplicity. Poetry makes project dependency management clear and intuitive.

2. Metadata management support. Developers can easily manage metadata like package dependencies, virtual environments, and build systems.

3. Default lockfile creation. Developers can recreate the same environment across multiple machines without breaking a sweat.

4. Easy version management. Developers can manage Python versions and create project-specific virtual environments, ensuring compatibility across environments.

Disadvantages of Poetry

1 . Small community. Poetry has a smaller user community and fewer packages compared to Pip.

2. Transition challenges. The workflow may be unfamiliar to developers who are used to Pip.

Let’s compare Pip vs Poetry based on factors that influence your dependency management process in Python microservices.

All in all, Pip is a simpler and more established Python dependency manager, while Poetry is a newer tool that offers more advanced features and better integration with virtual environments.

Developers may prefer Pip for its simplicity and compatibility with older Python versions, but if you prioritize advanced dependency management and reproducibility, Poetry may be a better option.

Now, let’s see how to use these tools to simplify dependency management, enhance the reliability and performance of your microservices, and avoid common issues such as dependency conflicts. 

We’ll use a small microservices-based app as an example so you can see how these tools work in practice and follow the described steps of implementing Python Flask dependency management in real-world conditions.

Read also

Microservices and Container Security: 11 Best Practices

Know what security challenges of working with microservice solutions to expect. Explore the details of Apriorit’s experience in protecting such apps.

Learn more

Building Python microservices using Flask: A practical example

We’ll demonstrate dependency management based on the example of a simple ordering app. It consists of two microservices — an order manager and a user manager — and one shared internal module: a discount calculator. Here’s the general architecture we’ll use in our app. 

Graphics: Flask-based microservices app architecture

Let’s explore each of these microservices in detail and see how to implement them using Flask.

1. Order manager microservice in the main.py file

import os
from discount_calculator.calculator import random_discount
import flask

app = flask.Flask(__name__)


@app.route('/api/order')
def create_order():
   discount = random_discount()
   return f'Order created, discount is {discount}.'


if __name__ == '__main__':
   app.run(host='0.0.0.0', threaded=True, port=os.getenv('ORDER_MANAGER_PORT'))

This code defines a microservice component responsible for managing orders. The component is implemented in the main.py file, which uses the Flask web framework to define an API endpoint at /api/order. 

This service has an endpoint for creating orders. When accessing this endpoint, the app generates a string indicating an order created with the discount — we’ll talk about the discount module below. The application listens on a port specified by the environment variable and is threaded to handle multiple simultaneous requests. 

This microservice component provides an API endpoint for creating orders with randomly generated discounts using Flask and a discount calculator module.

2. User manager microservice in the main.py file

import os
import flask

app = flask.Flask(__name__)


@app.route('/api/user')
def create_user():
   return 'User created'


if __name__ == '__main__':
   app.run(host='0.0.0.0', threaded=True, port=os.getenv('USER_MANAGER_PORT'))

This microservice component provides an API endpoint for creating users using Flask.

It’s implemented in the main.py file. 

When the endpoint is accessed, the function create_user() is executed. It simply returns the string “User created.” This is just a placeholder we use to indicate that a user has been created, and it can be modified to perform actual user creation operations.

Like the order service, the app listens on a port specified by the USER_MANAGER_PORT environment variable or on the default Flask port if the environment variable is not set.

3. Shared / Discount calculator module in the calculator.py file

import random


def random_discount():
   return random.randint(0, 20)

This function generates a random integer between 0 and 20 using the random module. The function returns the generated integer as its output. This code will be used in another part of our app to randomly generate order discounts.

To make sure our app is scalable, we need to use containerization. We’ll do this with Docker — a platform that allows you to package and distribute applications as containers that can be run on any machine with Docker installed. Thanks to this, you don’t need to worry about installing and configuring the application and its dependencies manually.

You can learn more about using Docker in the comprehensive official documentation.

To manage the deployment and scaling of these containers, we can leverage Kubernetes, an open-source container orchestration platform. Kubernetes simplifies deploying, scaling, and managing containerized applications by providing a unified API and extensive features such as load balancing, automatic scaling, and rolling updates. 

Docker and Kubernetes provide a robust and scalable infrastructure for deploying Python microservice applications.

You can find the full code of our project on Apriorit’s GitHub.

Now that we have our application ready, let’s discuss implementing dependency management with Pip and Poetry.

Related project

Building a Microservices SaaS Solution for Property Management

Discover how Apriorit’s experts replaced client’s monolithic SaaS architecture with microservices and made the solution more flexible, scalable, and easier to maintain.

Project details

Implementing dependency management with Poetry

We’ll implement Poetry in two steps: 

• Managing dependencies
• Deploying microservices

1. Managing dependencies

In Poetry projects, dependencies are managed by pyproject.toml files. These files define the metadata and dependencies. They use the Tom’s Obvious, Minimal Language (TOML) format, a popular configuration file format for Python projects.

The pyproject.toml file contains various sections that define the project’s metadata, such as name, version, description, author, and license. It also includes a section for defining project dependencies, which Poetry uses to manage and install the required packages for the project. Let’s look at an example of such a file in our project.

User manager component in the pyproject.toml file

[tool.poetry]
name = "user_manager"
version = "0.1.0"
description = ""
authors = ["admin"]

[tool.poetry.dependencies]
python = "^3.10"
Flask = "2.2.3"

[tool.poetry.dev-dependencies]

[build-system]
requires = ["poetry-core>=1.0.0"]
build-backend = "poetry.core.masonry.api"

2. Deploying microservices

Now, let’s implement the deployment stage using Docker files.

User manager component in the Dockerfile

# First build stage: build the application
FROM python:3.10-buster as py-build

# Install Poetry for package management
RUN curl -sSL https://install.python-poetry.org | POETRY_HOME=/opt/poetry python3 -

# Copy the entire application to the container and set the working directory
WORKDIR /app
COPY . .

# Second build stage: run the application
FROM python:3.10-slim-buster as py-run

# Copy the poetry directory from the first build stage to the second build stage
COPY --from=py-build /opt/poetry /opt/poetry

# Add the poetry bin directory to the PATH
ENV PATH=/opt/poetry/bin:$PATH

# Copy the application code from the first build stage to the second build stage
COPY --from=py-build /app /app

# Create a new user for running the application and set ownership of the app directory
RUN useradd -ms /bin/bash myuser
RUN chown -R myuser:myuser /app
USER myuser

#Install dependencies using Poetry
WORKDIR /app/Microservices/OrderManager
RUN poetry install

# Start the application using poetry run
CMD ["poetry", "run", "python", "order_manager/main.py"]

These instructions are designed to generate an image for building and running a Python application using the Poetry package manager, and they contain two build stages. 

1. The first build stage, named py-build, installs the Poetry package manager and copies application code to the container. This stage is responsible for building the application.
2. The second build stage, named py-run, copies the Poetry directory and application code from the first build stage, sets up a new user for running the application, installs dependencies using Poetry, and finally starts the application using Poetry run. This stage is responsible for running the application.

Read also

2 Ways to Implement Atomic Database Transactions in a Microservice-based Solution

Explore a practical example of establishing atomic communications between microservices in an e-commerce solution.

Learn more

Implementing dependency management with Pip

Just like with Poetry, let’s implement Pip in two steps, starting with managing dependencies. 

1. Managing dependencies

In Pip, the requirements file is a text file that lists the Python packages and their versions required by a project. Pip can read this file and automatically install all listed packages and their dependencies. 

A requirements file typically has the extension .txt and is usually named requirements.txt. The format of a requirements file is simple, with each line containing the name of a package and an optional version number, separated by ==.

Pip allows you to use requirements files to create virtual environments for Python projects, which helps to isolate the project’s dependencies and avoid version conflicts with other projects. Pip can also use requirements files to upgrade or downgrade packages to specific versions when necessary.

Overall, the requirements file is a powerful and popular feature in Pip, which provides a simple and efficient way to manage dependencies in Python projects.

Here is an example of the Pip requirements.txt file in our project.

Order manager component in the requirements.txt file

setuptools==63.2.0
Flask==2.2.2
discount_calculator==0.1.0

This is a list of Python packages the Order manager component requires in order to run:

• setuptools is a package that provides infrastructure for building and distributing Python packages.
• Flask is a Python web framework we use for building our app.
• discount_calculator is a custom package required by the Order manager component for discount calculations.

2. Deploying microservices

Pip can’t install packages from source code, which can be time-consuming and may require the compilation of native extensions. This can be a problem when deploying packages to remote servers or systems with different architectures or operating systems.

We recommend building and distributing the pre-built binary distributions in the form of wheels. 

A wheel is a distribution format that contains all files needed for installing a package, including the compiled code and any other necessary files, such as configuration files, templates, or static assets.

Wheels simplify the installation of Python packages and ensure that packages are installed consistently across environments. Since wheels contain pre-built binaries, there is no need to compile the code during installation, which can save a lot of time and reduce potential compatibility issues.

When you use Pip to install a package, Pip will first look for a pre-built wheel that matches your system configuration and Python version. If there’s a wheel available, Pip won’t need to build a package from source code. This reduces the risk of errors when installing packages on different systems.

Additionally, a build script can customize the build process by including additional build steps or optimizing the package for a particular platform.

You can check out the script we created for our app on GitHub.

Also, we need to create Docker files that use created wheels. We provide an example of such a file below.

Order manager component in the Dockerfile

FROM python:3.10

RUN apt-get update && apt-get install -y vim nano lsb-release apt-utils
RUN python3 -m Pip install -U Pip==22.2.1

COPY . /app

RUN Pip3 install /app/wheels/* && Pip3 install -r /app/requirements.txt

CMD ["python3.10", "/usr/local/lib/python3.10/site-packages/order_manager/main.py"]

This script uses Python 3.10 to:

• build a Docker image
• install additional packages and dependencies
• copy application code to the image
• install project dependencies
• run the application using the specified command

Conclusion

Dependency management is an integral part of microservice app development in Python. It allows developers to handle libraries and packages that an app can’t operate without. 

When building a Python application using Flask, you have two convenient tools for managing packages and dependencies — Pip and Poetry. With these tools, you can streamline your development process and make your software more scalable, secure, and maintainable. Using our side-by-side comparison, you can easily choose the tool that will work best for your project.

Apriorit’s Python development team will gladly assist you with building a cost-effective, scalable, and maintainable microservices-based solution that stays performant and secure from version to version.