PART IV Cloud Programming Paradigms

Chapter 12 Microservices

Introduction
• The previous chapter describes the MapReduce paradigm that employs parallel computation to solve a single
problem.

• This chapter introduces an alternative way to structure software that takes advantage of multiple computers in a
data center to scale computation. Instead of focusing on one problem, however, the approach divides each
application into pieces, allowing the pieces to scale independently, as needed.

Traditional Monolithic Applications

• Two approaches:

o Build each application as a single, self-contained program (monolithic).

o Divide applications into multiple pieces that work together (microservices).

• A monolithic application is constructed as a single, self-contained piece of software where everything is bundled
into a single executable so a user only has one piece of software to install, configure, and use.

• Bundling eliminates failures that arise if a user fails to install all the needed pieces as well as incompatibility
problems that can arise if a user upgrades or reconfigures one piece without upgrading or reconfiguring the
others.

Figure 12.1 Illustration of a monolithic application with all the functions needed to support online
shopping built into a single program.

Monolithic Applications In A Data Center

• Traditional monolithic applications can run in a data center exactly the way they run on a traditional server by
simply launching and use a VM to run monolithic applications.

• This is not a good approach because monolithic applications cannot be replicated as quickly as cloud-native
applications as starting a VM has higher overhead than starting a container and a monolithic design means all
code must be downloaded when the application starts, even if pieces are not used.

The Microservices Approach

• A microservices architecture divides functionality into multiple, independent applications. Each of the
independent applications is much smaller than a monolithic program, and only handles one function. To
perform a task, the independent applications communicate over a network.
 • The microservices approach can be used to implement a new application or to divide an existing monolithic
application.

• Disaggregation refers to the division of a monolithic application into microservices.

Figure 12.2 Illustration of one possible way the shopping application c a n b e disaggregated into a set of
microservices that communicate with one another.

The Advantages Of Microservices

• The microservices approach does introduce the extra overhead of running multiple, small applications and using
network communication among the pieces instead of internal function invocation.

• In a cloud environment, however, the microservices approach has advantages that can outweigh the overhead.
The advantages can be divided into two broad categories: advantages for software development and advantages
for operations and maintenance.

Advantages For Software Developmen

• Smaller scope and better modularity - Software engineers focus on one small piece of the problem at a time and
define clean interfaces. The limited scope encourages better decomposition and allows engineers to understand
each piece completely.

• Smaller teams - A microservice can be designed and implemented independent of other microservices, each
microservice only requires a small development team, meaning that the resulting code will be more uniform and
less prone to errors.

• Less complexity - Complexity leads to errors, and the monolithic approach creates complexity.

• Choice of programming language - When using the monolithic approach, all code must be written in a single
programming language. With the microservices approach, software engineers can choose the best language for
each service.

• More extensive testing - With the microservices approach, each service can be tested independently, allowing
more extensive and thorough assessment.

Advantages For Operations And Maintenance

• Rapid deployment - Because microservices are small, they can be created, tested, and deployed rapidly as well
as be changed easily and quickly.

• Improved fault isolation – Having multiple microservices makes fault isolation easier because a problematic
issue with a microservice can be tested and resolved while allowing applications and other microservices to
continue normal operations.
 • Better control of scaling – Microservices can be scaled independently.

• Compatibility with containers and orchestration systems – Because it is small and only performs one task, a
microservice fits best into the container paradigm. Furthermore, using containers means that microservices can be
monitored, scaled, and load balanced by a conventional container orchestration system, such as Kubernetes

• Independent upgrade of each service - Once an improved version of a microservice has been created, the new
version can be introduced without stopping existing applications and without disturbing other microservices.

The Potential Disadvantages Of Microservices

• Cascading errors - One microservice can invoke another, which can invoke another, and so on, so if one of the
microservices fails, the failure may affect many others as well as the applications that use them.

• Duplication of functionality and overlap - Due to the ease with which microservices can be created, when
functionality is needed that differs slightly from an existing microservice, it is often easier to create a completely
new one than to modify the existing microservice resulting in many similar microservies.

• Management complexity - Each microservice must be monitored and when hundreds of microservices are
running simultaneously, it can be difficult to understand their behaviors, interdependencies, and the interactions
among them.

• Replication of data and transmission overhead - Each microservice must obtain a copy of the needed data,
either from a storage server or by being passed a copy when the microservice is invoked.

• Increased security attack surface - A monolithic application represents a single attack point. The microservices
approach has a much larger security attack surface having multiple points that an attacker can try to exploit.

• Workforce training - The microservices approach require software engineers to consider the cost of running
each microservice as well as data communication costs between microservices as well as the need to develop
new skills to create software for microservices.

Microservices Granularity
• The question of microservice size forms one of the key decisions a software engineer faces when following the
microservices approach.

Figure 12.3 Two alternative designs for the payment microservice from Figure 12.2. (a) divides the functionality into four
separate microservices, and (b) places the four under an intermediate microservice.

• The figure only shows two ways to structure the payment microservices and does not show several of the other
microservices from Figure 12.2 that are needed for the application.

• The following are three heuristics that can help developers choose a granularity:
 o Business process modeling - Each microservice should be based on a business process. Instead of merely
disaggregating existing applications, development teams identify how the applications are being used and the
steps along the workflow can be transformed into microservices.

o Identification of common functionality - Instead of building a microservice for exactly one application,
consider how related applications might use the service and plan accordingly.

o Adaptive resizing and restructuring - The small size of a microservice means it can be redesigned quickly
and accommodates new applications and new underlying functionality.

Communication Protocols Used For Microservices

• Communication protocols specify the message details that enable microservices to have meaningful and
unambiguous communication.

• Like other applications in a data center, microservices communicate using Internet protocols. Doing so means a
microservice can be reached from inside or outside the data center, subject to security restrictions.

• For the transport layer protocol, most microservices use the Transmission Control Protocol (TCP), with
TCP being sent in Internet Protocol (IP) packets. TCP merely delivers streams of bytes between a pair of
communicating entities.

• When communicating over TCP, the pair must also employ a transfer protocol that defines how bytes are
organized into messages. In essence, the set of transfer protocol messages defines the service being offered.

• Using an existing protocol makes it easier to write code as several transfer protocols exist and satisfy most needs.
Thus, a software engineer merely needs to choose one when designing a microservice. Consider two examples:
o HTTP – The HyperText Transfer Protocol used in the Web

o gRPC – An open source high-performance, universal RPC framework

• HTTP
o When an entity uses HTTP to communicate with a microservice the entity can send data to the
microservice or request that the microservice send data.

o In addition to specifying an operation to be performed, each request message specifies a data item by
giving the item’s name in the form of a Uniform Resource Identifier (URI).

o For some operations, the sender must also supply data to be used for the request. Below are six basic
operations that HTTP supports:
▪ GET - retrieve a copy of the data item specified in the request
▪ HEAD - retrieve metadata for the data item specified in the request (ie. last
modified time)
▪ PUT - replace the specified data item with the data sent with the request
▪ POST - append the data sent with the request onto the specified data item
▪ PATCH - use data sent with request to modify part of the data item specified in
the request
▪ DELETE - remove the data item specified in the request
 • gRPC
o Unlike most transfer protocols, gRPC does not define a specific set of operations that can be performed.
Instead, it provides a general framework for communication and allows a specific set of operations to be
defined for each instance (microservice).

o gRPC incorporates the Remote Procedure Call (RPC) approach that has been used to build distributed
systems for decades. The general idea is straightforward: create a program that runs on multiple
computers by placing one or more of the procedures from the program on remote computers that are
invoked with messages containing the arguments for the procedure being called and return reply
messages containing the value returned by the procedure.

o To make RPC easy to use, technologies exist that generate message passing code automatically. RPC
technologies generate code known as stubs. In the program, a stub replaces each procedure that has
been moved to a remote computer; on the remote computer, a stub calls the procedure with a local
procedure call, exactly like a program does.

Figure 12.6 (a) An application program that calls two procedures, and (b) the same application using RPC technology that
allows the procedures to run remotely.

o gRPC extends traditional RPC technology by supporting many programming languages, allowing
the user to define data serialization, and supporting streaming of multiple data items using a
technology known as protocol buffers.

Communication Among Microservices

• A variety of interactions have been used with microservices primarily separated into two broad types:

o Request-response (REST/RESTful interface) - A request-response style of interaction used on the

Web: a web browser sends a request to which a web server sends a response. Typically requires multiple
requests and responses. A REST API implies that the microservice uses HTTP as its transfer protocol.

o Data streaming (continuous interface) - Data streaming interaction avoids repeated requests by allowing
a microservice to combine a small number of items into a single response by providing a stream of data
items as a response. However, all items must be available, and the processing required to create the
combined response must be reasonable. When using a streaming interface, an entity establishes a network
connection with the microservice and sends a single request. The microservice sends a sequence of
one or more data items in response to the request (i.e., the microservice streams a sequence of data
items).

• An important distinction between traditional RPC and gRPC are the interactions they support.
 o A traditional RPC follows a request-response interaction where every is accomplished by sending single
messages back and forth over the network to the computer containing the remote procedure and a single
message to travel back.

o gRPC extends remote procedure call to allow a remote procedure to stream multiple data items in
response to a request.

• Microservices have also used variations of basic communication interactions. For example, some microservices
follow the publish-subscribe variant of data streaming.

o subscribe – An entity contacts a microservice and specifies a topic. The network connection remains
open perpetually, and the microservice sends any data items that arrive for the specified topic.

o publish - An entity contacts the microservice and sends data items labeled with a topic.

Using A Service Mesh Proxy

• A service mesh is a software system that handles such tasks as:
o Creating instances of a microservice
o Forwarding requests to a given instance
o Translating requests to an internal form
o Discovering microservices discover another?

• Instead of allowing entities to contact an instance of a microservice directly, a service mesh requires that the
entities contact a proxy that manages a set of instances and forwards each request to one instance.

• The use of a proxy allows a microservice to be scaled and isolates the communication used internally from
the communication used to access the microservice.

Figure 12.7 Illustration of a proxy for a service mesh. To use the service, an external entity contacts the proxy.

The Potential For Deadlock

• Although the microservice approach has many advantages, a distributed system composed of many
microservices can fail in unexpected ways and is to circular dependencies where a set of microservices all depend
on one another. If each microservice in the cycle is waiting for another microservice, a deadlock can result.

• Consider a trivial example of four microservices: a time service, a location service, a file storage service, and an
authentication service. Although the services access one another, the system runs with no problems. The file
storage service uses the time service to obtain the time of day that it uses for timestamps on files. The location
service uses the file storage service to obtain a list of locations. The authentication service uses the location
service to find the location of the file storage service (which it uses to obtain stored encryption keys). If any
service is completely terminated and restarted, the system continues to work correctly as soon as the restart
occurs. However, a deadlock could occur if all four microservices attempt to start at the same time.
 Figure 12.8 A dependency cycle among four microservices.

Microservices Technologies
• Many technologies have been created to aid software engineers in the design and operation of microservices:

o Commercial and open source service mesh technologies exist, including Linkerd, a project of the Cloud
Native Computing Foundation (CNCF).

o Istio, a joint project among Google, IBM, and Lyft.

o Many frameworks exist that help developers create and manage microservices, including Spring Boot.

Summary
• The microservices approach disaggregates a monolithic application into multiple services, allowing them to scale
independently.

• The approach has advantages for software development, including smaller scope, smaller teams, less complexity,
a choice of programming language, and more extensive testing.

• The approach has advantages for operations and maintenance, including rapid deployment, improved fault
isolation, better control of scaling, compatibility with containers and orchestration systems, and independent
upgrade of each microservice.

• The approach also has potential disadvantages, including cascading errors, duplication of functionality and
overlap, management complexity, replication of data and transmission overhead, an increased security attack
surface, and the need for workforce training.

• Microservices communicate over a network using transport protocols, such as HTTP and gRPC. HTTP supports
a request-response (REST) interaction. gRPC generalizes conventional RPC to provide a streaming interface in
addition to remote procedure invocation and return.

• Service mesh software automates various aspects of running a microservice. The use of a proxy for the service
hides both the internal structure and internal communication protocols, which allows the protocols used for
external access and internal microservice invocation to differ.

• An important weakness of the microservices approach arises because each microservice is created and
maintained independently. Subtle problems, such as circular dependencies, can arise and remain hidden until an
unusual event occurs, such as a power failure that causes all microservices to restart simultaneously.