AGENDA

• Netflix – background and evolution

• Monolithic Apps
• Characteristics

• What are Microservices?

• Microservices
• Why?
• Challenges
• Best practices
• Tools of the trade

• InterProcess Communication
• Takeaways
Netflix - Evolution
Netflix - Evolution
• Old DataCenter (2008) • AWS Cloud (~2010)
• Everything in one WebApp (.war) • 100s of Fine Grained Services
Netflix Scale
• ~ 1/3 of the peak Internet traffic a day
• ~50M subscribers
• ~2 Billion Edge API Requests/Day
• >500 MicroServices
• ~30 Engineering Teams (owning many microservices)
Monolithic Apps
MONOLITHIC APP
 Monolithic Architecture

Load Balancer

Monolithic App
Account Catalog
Component Component

Recommendation Customer Service Database

Component Component
Characteristics
• Large Codebase
• Many Components, no clear ownership
• Long deployment cycles
Pros
• Single codebase
• Easy to develop/debug/deploy
• Good IDE support
• Easy to scale horizontally (but can only scale in an “un-differentiated”
manner)
• A Central Ops team can efficiently handle
Monolithic App – Evolution
• As codebase increases …
• Tends to increase “tight coupling” between components
• Just like the cars of a train
• All components have to be coded in the same language
 Shopping Cart

User Accounts
Product Catalog

Customer Service

Evolution of a Monolithic App

 Monolithic App - Scaling
• Scaling is “undifferentiated”
• Cant scale “Product Catalog” differently from “Customer Service”
AVAILABILITY
 Availability
• A single missing “;” brought down the Netflix website for many hours (~2008)
MONOLITHIC APPS – FAILURE & AVAILABILITY
MicroServices

You Think??
TIPPING POINT

& &
Organizational Growth Disverse Functionality Bottleneck in
Monolithic stack
What are
MicroServices?
NOT ABOUT …
• Team size
• Lines of code
• Number of API/EndPoints
MicroService

MegaService
CHARACTERISTICS
• Many smaller (fine grained), clearly scoped services
• Single Responsibility Principle
• Domain Driven Development
• Bounded Context
• Independently Managed
• Clear ownership for each service
• Typically need/adopt the “DevOps” model

Attribution: Adrian Cockroft, Martin Fowler …

 Composability– unix philosophy

• Write programs that do one thing and do it well.

• Write programs to work together.

tr 'A-Z' 'a-z' < doc.txt | tr -cs 'a-z' '\n' | sort | uniq | comm -23 - /usr/share/dict/words

Program to print misspelt words in doc.txt

 Comparing Monolithic to MicroServices

MONOLITHIC APP (VARIOUS COMPONENTS LINKED TOGETHER)

MICROSERVICES – SEPARATE SINGLE PURPOSE SERVICES
Monolithic Architecture (Revisiting)

Load Balancer

Monolithic App
Account Catalog
Component Component

Recommendation Customer Service Database

Component Component
 Microservices Architecture

Load Balancer

API Gateway

Account Service Catalog Catalog

Service DB

Recommendation Customer Service Customer

Service Service DB
 Concept -> Service Dependency Graph
Service X
Service L
Your App/Service Service Y
Service M

Service Z
MicroServices
- Why?
WHY?
• Faster and simpler deployments and rollbacks
• Independent Speed of Delivery (by different teams)
• Right framework/tool/language for each domain
• Recommendation component using Python?, Catalog Service in Java ..
• Greater Resiliency
• Fault Isolation
• Better Availability
• If architected right 
MicroServices
- Challenges
 CHALLENGES

Can lead to chaos if not designed right …

OVERALL COMPLEXITY
• Distributed Systems are inherently Complex
• N/W Latency, Fault Tolerance, Retry storms ..

• Operational Overhead
• TIP: Embrace DevOps Model
SERVICE DISCOVERY
• 100s of MicroServices
• Need a Service Metadata Registry (Discovery Service)

Account Service Catalog

Service Registry
Service
(e.g. Netflix Eureka)
Recommendation Customer Service
Service Service

X
Service Y Z
Service Service
CHATTINESS (AND FAN OUT)
~2 Billion Requests per day on Edge Service
Results in ~20 Billion Fan out requests in ~100 MicroServices

1 Request 1 Request

Monolithic App MicroServices

DATA SERIALIZATION OVERHEAD
Data transformation
getMovies() getMovie() getMovieMetadata()
C C C
l l l
i i i
Service A e Service B e Service C e Service D
n n n
t t t
B C D

X XAvro
JSON Xml
 CHALLENGES - SUMMARY
• Service Discovery
• Operational Overhead (100s of services; DevOps model absolutely required)
• Distributed Systems are inherently Complex
• N/W Latency, Fault Tolerance, Serialization overhead ..
• Service Interface Versioning, Mismatches?
• Testing (Need the entire ecosystem to test)
• Fan out of Requests -> Increases n/w traffic
Best Practices/Tips
 Best Practice -> Isolation/Access
• TIP: In AWS, use Security Groups to isolate/restrict access to your MicroServices

http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/using-network-security.html
 Best Practice -> Loadbalancers
Choice
1. Central Loadbalancer? (H/W or S/W)

OR

2. Client based S/W Loadbalancer?

 Central (Proxy) Loadbalancer
API Gateway

Customer Service Load

Balancer
Account Service Load
Balancer
Customer Service
Service 1
Account Service 1 Account Service N
Reco Service
Load Balancer Customer Service
Service N

Recommendation Recommendation
Service 1 Service N
 Client Loadbalancer
API Gateway

Recommendation Customer Service

Account Service LB Service LB
Service LB

Customer Service
Service 1
Account Service 1 Account Service N

Customer Service
Service N

Recommendation Recommendation
Service 1 Service N
Client based Smart Loadbalancer

Use Ribbon (http://github.com/netflix/ribbon)

 Best Practice -> LoadBalancers

• TIP: Use Client Side Smart LoadBalancers

BEST PRACTICES CONTD..

• Dependency Calls
• Guard your dependency calls
• Cache your dependency call results
• Consider Batching your dependency calls
• Increase throughput via Async/ReactiveX patterns
Dependency Resiliency
 Service Hosed!!

A single “bad” service can still bring your service down

 AVAILABILITY

MicroServices does not automatically mean better Availability

- Unless you have Fault Tolerant Architecture
Resiliency/Availability
HANDLING FAN OUTS
SERVER CACHING
Service X

Your App/Service Service Y

Service Z
Cache
Cache
Cluster
Cluster

Tip: Config your TTL based on flexibility with data staleness!

 Composite (Materialized View) Caching
Service X

Your App/Service
Service Y

Service Z
Fn {A, B, C}
Cache
Cache
Cluster
Cluster
 BottleNecks/HotSpots
A/B Test Service

App Service X Service Y Service Z

User Account
Service
 Tip: Pass data via Headers
A/B Test Service

App Service X Service Y Service Z

User Account
Service

reduces dependency
load
TEST RESILIENCY (of Overall MicroServices)
* Benjamin Franklin
 three

* Inspired by Benjamin Franklin

 Best Practices contd..

• Test Services for Resiliency

• Latency/Error tests (via Simian Army)
• Dependency Service Unavailability
• Network Errors
Test Resiliency – to dependencies
 TEST RESILIENCY
Use Simian Army https://github.com/Netflix/SimianArmy
BEST PRACTICES - SUMMARY
• Isolate your services (Loosely Coupled)
• Use Client Side Smart LoadBalancers
• Dependency Calls
• Guard your dependency calls
• Cache your dependency call results
• Consider Batching your dependency calls
• Increase throughput via Async/ReactiveX patterns

• Test Services for Resiliency

• Latency/Error tests (via Simian Army)
• Dependency Service Unavailability
• Network Errors
Tools of the Trade
AUTO SCALING
• Use AWS Auto Scaling Groups to automatically scale your microservices
• RPS or CPU/LoadAverage via CloudWatch are typical metrics used to scale

http://docs.aws.amazon.com/AutoScaling/latest/DeveloperGuide/WhatIsAutoScaling.html
 USE CANARY, RED/BLACK PUSHES
• NetflixOSS Asgard helps manage deployments
Service Dependency Visualization
MicroServices at Netflix
 SERVICE DEPENDENCY GRAPH
How many dependencies does my service have?

What is the Call Volume on my Service?

Are any Dependency Services running Hot?

What are the Top N Slowest “Business Transactions”?

What are the sample HTTP Requests/Responses that had a 500 Error Code in
the last 30 minutes?
 SERVICE DEPENDENCY VISUALIZATION

You

Your Service Dependency Graph

Service Dependency Visualization
Dependency Visualization
Polyglot Ecosystem
Homogeneity in A Polyglot Ecosystem
TIP: USE A SIDECAR
• Provides a common
homogenous
Operational/Infrastructural
component for all your non-
JVM
based MicroServices
 Prana Open Sourced!
• Just this morning!
• http://github.com/netflix/Prana
Inter Process
Communication
 Netflix IPC Stack (1.0)
Client Server (Karyon)
A
Ribbon p Apache
Bootstrapping (Governator)

a HTTP Admin Console

c
Load
h
Balancing Metrics (Servo)
H E e
Tomcat
y V Eureka Integration

s C
t a Metrics
(Servo)
H
r c T
Registration

i h T
P
x e C
l
Eureka (Service Registry)
Eureka
Integration
i
e Fetch Registry
n

A Blocking Architecture
t
 Netflix IPC Stack (2.0)
Client (Ribbon 2.0) Server (Karyon)
HTTP
Ribbon Transport Bootstrapping (Governator)
Ribbon UDP
TCP Admin Console
Load
Balancing WebSockets
RxNetty
SSE
Hystrix R Metrics (Servo)

x
N Eureka Integration
Metrics
(Servo)
e
t
Registration
EVCache t
y

Eureka (Service Registry)

Eureka
Integration
Fetch Registry

A Completely Reactive Architecture

 Performance – Throughput

Details: http://www.meetup.com/Netflix-Open-Source-Platform/events/184153592/
NetflixOSS
LEVERAGE NETFLIXOSS

http://netflix.github.co
• Eureka – for Service Registry/Discovery
• Karyon – for Server (Reactive or threaded/servlet container based)
• Ribbon – for IPC Client
• And Fault Tolerant Smart LoadBalancer

• Hystrix – for Fault Tolerance and Resiliency

• Archaius – for distributed/dynamic Properties
• Servo – unified Feature rich Metrics/Insight
• EVCache – for distributed cache
• Curator/Exhibitor – for zookeeper based operations
• …
Takeaways
Takeaways
• Monolithic apps – good for small organizations
• MicroServices – have its challenges, but the benefits are many
• Consider adopting when your organization scales
• Leverage Best Practices
• An Elastic Cloud provides the ideal environment (Auto Scaling etc.)
• NetflixOSS has many libraries/samples to aid you