Microservice Architectures: API Gateway, Data Management, and Event-Driven

Architecture

Let's first discuss the problem of connecting a microservice architecture to clients. Here we see an example of a
cinema booking application. In it, a cinema catalog service is connected to a movie service and a booking service.
Booking service is connected to payment service and both payment service and booking service use the notification
service. The cinema catalog service and the booking service are exposed as RESTful API resources to the client.

There are many possible problems in exposing services directly to clients. If clients interact directly with many
services in a system, the client code can get more complex than needed. Clients should not be directly coupled to
the back end services. Usually you want APIs exposed to the clients to stay relatively stable, whereas the system in
the back end should be able to cope with changing requirements if the clients access services directly, single
operations might require multiple service calls. This leads to unnecessary complexity and coupling too.

Central tasks of an API must be handled by each service, such as authentication, SSL or rate limiting. API services
need to offer client friendly protocols. So the choices for protocols are limited if every service is an API service. For
instance, often you might want to use RESTful HTTP on the client API, but use other protocols in the back end, such
as messaging protocols. Finally, there is a security consideration. Each API service is a potential attack surface and
overall you want to minimize the attack surface of the application.

One solution to this problem is an API gateway. An API gateway sits between clients and service. It acts as a reverse
proxy, routing requests from clients to services. It may also perform various cross cutting tasks such as
authentication, SSL termination and rate limiting.

Here is the example from before with an API gateway. You can see that the client is now only connected to the API
gateway interface which abstracts the joint services of the cinema catalog service and the booking service. This way
clients can rely only on the cinema booking API interface provided by the API Gateway and don't need to rely on the
details of the service implementations in the back end.

Next, let us consider data management in microservice architectures. In a monolithic application, you usually have
a shared data store. Here we have a similar microservice based architecture with the shared database, accessed
by the different services of the example system. In the microservice world, this can lead to all kinds of problems.

That's wherever possible you usually want to avoid shared data between microservices. The main reason for this is
that you want to avoid unintentional coupling between services through the data, read from and written to the
database. If services share the underlying data schemas coupling emerges. If there is a change to the data schema,
the change must be coordinated across every service that relies on that chat database. Data independence of
services means we can limit the scope of changes. This in turn enables independent deployments. Using a shared
data store limits each team's ability to optimize data storage for their particular service.

This problem is addressed by the database per service pattern. A basic principle of microservices is that each
service manages its own data. Two services should not share a data store. Instead, each service is responsible for
its own private data store, which other services cannot access directly.

Here you can see a modified design of our example with databases per service. All services that require a data store
have their own database. As can be seen, three different technologies are chosen for the different databases.

As a consequence, this approach naturally leads to polyglot persistence. That is, the use of multiple data storage
technologies within a single application.

1
As mentioned before, many microservice systems leverage polyglot programming languages, communication
technologies and many other technologies as well.

This then leads us to the question which approach to choose for microservice data management. The usual general
advice on this is embrace eventual consistency where possible. Understand the places in the system where you
need strong consistency or ACID transactions and the places where eventual consistency is acceptable.

When you really need strong consistency guarantees, one service may represent the source of truth for a given
entity, which is exposed through an API. Other services might hold their own copy of the data, or a subset of it, that
is eventually consistent with the master data.

What transactions use patterns such as Scheduler Agent Supervisor, SAGA and Compensating Transaction to keep
data consistent across several services.

Store only the data that the service really needs. A service might only need a subset of information about a domain
entity.

Consider whether your services are coherent and loosely coupled. If two services are continually exchanging
information with each other, resulting in chatty APIs you may need to redraw your service boundaries.

Finally, consider where possible to use an event driven architecture style. In this architecture style a service
publishes an event when there are changes to its public models or entities. Interested services can subscribe to
these events. The event driven architecture style feels very natural where eventual consistency is needed.

Here is our example redesigned with the publish subscribe component that serves as an event store. All services
that exchange messages are connected to it. The published topics and the subscribed topics are modeled for the
connectors. For instance, the booking service publishes events about bookings and notifications and subscribes to
catalog updates. This enables looser coupling of services, but requires service developers to deal with eventual
consistency.

A related technique that is enabled by event driven architectures is event sourcing. When using this technique,
instead of storing just the current state of the data in the domain, you use an append only store to record the full
series of actions taken on that data. The store acts as a system of record and can be used to materialize the domain
objects

This can potentially simplify tasks in complex domains by avoiding the need to synchronize the data model and the
business domain while improving performance, scalability and responsiveness.

It can also provide consistency for transactional data and maintain full audit trails and history that can enable
compensating actions. Event Sourcing provides a fully reliable audit log of all changes made to a domain object.
With it, it is possible to determine the state of a domain object at any point in time.