Load Balancing 101 Nuts and Bolts
Load Balancing 101 Nuts and Bolts
Load Balancing 101 Nuts and Bolts
Bolts
White Paper
byF5
WHITE PAPER
Introduction
In today's dynamic, application-centric marketplace, organizations are under more
and more pressure to deliver the information, services, and experiences their
customers expectand to do it quickly, reliably, and securely. Key network and
application functions, such as load balancing, encryption, acceleration, and security,
can be provided via Application Delivery Controllers (ADC), which are physical or
virtual appliances functioning as proxies for physical servers. With the explosion of
applications, as well as the demands placed on organizations by the rigorous cycle
of Continuous Integration and Continuous Deployment (CI/CD), it's no wonder that
the market for ADCs is projected to reach $2.9 billion a year by 2020.1
But before heading into the future, let's look at how we got here. Network-based
load balancing is the essential foundation upon which ADCs operate. In the mid-
1990s, the rst load balancing hardware appliances began helping organizations
scale their applications by distributing workloads across servers and networks.
These rst devices were application-neutral and resided outside of the application
servers themselves, which means they could load balance using straightforward
network techniques. In essence, these devices would present a "virtual IP address"
to the outside world, and when users attempted to connect, they would forward the
connection to the most appropriate real server doing bi-directional network address
translation (NAT).
However, with the advent of virtualization and cloud computing, a new iteration of
load balancing ADCs arrived as software-delivered virtual editions intended to run on
hypervisors. Today, virtual appliances can deliver application services with the same
breadth of features as those that run on purpose-built hardware. In addition, these
virtual editions eliminate much of the complexity involved in moving application
services between virtual, cloud, and hybrid environments, allowing organizations to
quickly and easily spin up application services in private or public cloud
environments.
The newest trend to hit the data center is containerization, which is a method of
application virtualization that helps in deploying and running distributed applications.
The process isolates applications and contains them in clear delineated memory
spaces on a shared OS, which not only makes developing and deploying an
application easier than building a virtual appliance, but also make it quicker. Due to
the dramatic improvements in portability and performance, containerization could
provide businesses with greater scalability and agility. In the future, container
architectures could also help organizations take better advantage of different cloud
environments.
Today's ADCs evolved from the rst load balancers through the service virtualization
process. And now, with software-only virtual editions, ADCs can not only improve 1
availability, but also help organizations deliver the scalable, high-performance, and
application virtualization that helps in deploying and running distributed applications.
The process isolates applications and contains them in clear delineated memory
spaces on a shared OS, which not only makes developing and deploying an
application easier than building a virtual appliance, but also make it quicker. Due to
the dramatic improvements in portability and performance, containerization could
WHITE PAPER with greater scalability and agility. In the future, container
provide businesses
Load Balancing
architectures 101:also
could Nuts and
help Bolts
organizations take better advantage of different cloud
environments.
Today's ADCs evolved from the rst load balancers through the service virtualization
process. And now, with software-only virtual editions, ADCs can not only improve
availability, but also help organizations deliver the scalable, high-performance, and
secure applications that their business requires. In the end, though, all these
virtualized application services, shared infrastructure deployments, and intelligent
routing capabilities wouldn't be possible without the solid foundation of load
balancing technology.
To understand how enterprises can better address the complex challenges of the
dynamically evolving marketplace, let's explore the foundation of application delivery:
Load Balancing 101.
2
The second concept is expressed by the term "member" (unfortunately also called a
WHITE PAPER
The second concept is expressed by the term "member" (unfortunately also called a
node by some manufacturers). A member is usually a little more dened than a host
in that it includes the TCP port of the actual application that will be receiving trafc.
For instance, a host named www.example.com may resolve to an address of
172.16.1.10, which represents the host, and may have an application (a web server)
running on TCP port 80, making the member address 172.16.1.10:80. Simply put,
the member includes the denition of the application port as well as the IP address
of the physical/virtual server. For the remainder of this paper, we will refer to this as
the service.
Why all the complexity? Because the distinction between a server and the
application services running on it allows the load balancer to individually interact
with the applications rather than the underlying hardware or hypervisor, in a
datacenter or in the cloud. A host (172.16.1.10) may have more than one service
available (HTTP, FTP, DNS, etc.). By dening each application uniquely
(172.16.1.10:80, 172.16.1.10:21, and 172.16.1.10:53, for example), the ADC can
apply unique load balancing and health monitoring (a concept we'll discuss later)
based on the services instead of the host.
Remember, most load balancingbased technology uses one term to represent the
host, or physical server, and another to represent the services available on itin this
case, simply host and services.
WHITE
Remember,PAPER
most load balancingbased technology uses one term to represent the
Load
host, or physical101:
Balancing Nuts
server, and
and Bolts to represent the services available on itin this
another
case, simply host and services.
Virtual server
A virtual server is a proxy of the actual server (physical, virtual, or container).
Combined with a virtual IP address, this is the application endpoint that is presented
to the outside world.
WHITE PAPER
While Figure 2 may not be representative of any real-world deployment, it does
This very simple example is relatively straightforward, but there are a couple of key
elements to note. First, as far as the client knows, it sends packets to the virtual
server and the virtual server respondssimple. Second, the NAT takes place. This is
where the ADC replaces the destination IP sent by the client (of the virtual server) 5
with the destination IP of the host to which it has chosen to load balance the
not touched).
The host accepts the connection and responds back to the original source,
the client, via its default route, the ADC.
The ADC intercepts the return packet from the host and now changes the
source IP (and possible port) to match the virtual server IP and port, and
WHITE PAPER
forwards the packet back to the client.
Load
Balancing 101: Nuts
The client receives theand Bolts
return packet, believing that it came from the virtual
server, and continues the process.
This very simple example is relatively straightforward, but there are a couple of key
elements to note. First, as far as the client knows, it sends packets to the virtual
server and the virtual server respondssimple. Second, the NAT takes place. This is
where the ADC replaces the destination IP sent by the client (of the virtual server)
with the destination IP of the host to which it has chosen to load balance the
request. Third is the part of this process that makes the NAT "bi-directional". The
source IP of the return packet from the host will be the IP of the host; if this address
were not changed and the packet was simply forwarded to the client, the client
would be receiving a packet from someone it didn't request one from, and would
simply drop it. Instead, the load balancer, remembering the connection, rewrites the
packet so that the source IP is that of the virtual server, thus solving this problem.
Let's discuss the second question rst. What happens if the selected host isn't
working? The simple answer is that it doesn't respond to the client request and the
connection attempt eventually times out and fails. This is obviously not a preferred
circumstance, as it doesn't ensure high availability. That's why most load balancing
technology includes some level of health monitoring to determine whether a host is
actually available before attempting to send connections to it.
There are multiple levels of health monitoring, each with increasing granularity and
focus. A basic monitor would simply ping the host itself. If the host does not
respond to the ping, it is a good assumption that any services dened on the host
are probably down and should be removed from the cluster of available services.
Unfortunately, even if the host responds to the ping, it doesn't necessarily mean the 6
service itself is working. Therefore, most devices can do "service pings" of some
working?
Let's discuss the second question rst. What happens if the selected host isn't
working? The simple answer is that it doesn't respond to the client request and the
connection attempt eventually times out and fails. This is obviously not a preferred
WHITE PAPER
circumstance, as it doesn't ensure high availability. That's why most load balancing
Load Balancing
technology 101:some
includes Nuts and
level Bolts
of health monitoring to determine whether a host is
actually available before attempting to send connections to it.
There are multiple levels of health monitoring, each with increasing granularity and
focus. A basic monitor would simply ping the host itself. If the host does not
respond to the ping, it is a good assumption that any services dened on the host
are probably down and should be removed from the cluster of available services.
Unfortunately, even if the host responds to the ping, it doesn't necessarily mean the
service itself is working. Therefore, most devices can do "service pings" of some
kind, ranging from simple TCP connections all the way to interacting with the
application via a scripted or intelligent interaction. These higher-level health monitors
not only provide greater condence in the availability of the actual services (as
opposed to the host), but they also allow the load balancer to differentiate between
multiple services on a single host. The load balancer understands that while one
service might be unavailable, other services on the same host might be working just
ne and should still be considered as valid destinations for user trafc.
This brings us back to the rst question: How does the ADC decide which host to
send a connection request to? Each virtual server has a specic dedicated cluster of
services (listing the hosts that offer that service) that makes up the list of
possibilities. Additionally, the health monitoring modies that list to make a list of
"currently available" hosts that provide the indicated service. It is this modied list
from which the ADC chooses the host that will receive a new connection. Deciding
on the exact host depends on the load balancing algorithm associated with that
particular cluster. Some of these algorithms include least connections, dynamic ratio
and a simple round robin where the load balancer simply goes down the list starting
at the top and allocates each new connection to the next host; when it reaches the
bottom of the list, it simply starts again at the top. While this is simple and very
predictable, it assumes that all connections will have a similar load and duration on
the back-end host, which is not always true. More advanced algorithms use things
like current-connection counts, host utilization, and even real-world response times
for existing trafc to the host in order to pick the most appropriate host from the
available cluster services.
Load Balancing
services. 101: Nuts is
This functionality and Bolts
increasingly important as services become more
differentiated with HTML and scripting.
Connection maintenance
If the user is trying to utilize a long-lived TCP connection (Port 21: FTP, Port 23:
Telnet, or other) that doesn't immediately close, the load balancer must ensure that
multiple data packets carried across that connection do not get load balanced to
other available service hosts. This is connection maintenance and requires two key
capabilities. The rst is the ability to keep track of open connections and the host
service they belong to. Second, the load balancer must be able to continue to
monitor that connection so the connection table can be updated when the
connection closes. This is rather standard fare for most ADCs.
Persistence
Increasingly more common, however, is when the client uses multiple short-lived
TCP connections (for example, Port 80: HTTP) to accomplish a single task. In some
cases, like standard web browsing, it doesn't matter and each new request can go
to any of the back-end service hosts; however, there are many instances (XML, e-
commerce, and so on) where it is extremely important that multiple connections
from the same user go to the same back-end service host and not be load
balanced. This concept is called persistence, or server afnity.
There are multiple ways to address this, depending on the protocol and the desired
results. For example, in modern HTTP transactions, the server can specify a "keep-
alive" connection, which turns those multiple short-lived connections into a single
long-lived connection, which can be handled just like the other long-lived
connections. However, this provides only a little relief, mainly because, as the use of
web and mobile services increases, keeping all the connections open longer than
necessary strains the resources of the entire system. That's why todayfor the
sake of scalability and portabilitymany organizations are moving toward building
stateless applications that rely on APIs. This basically means that the server will
forget all session information to reduce the load on the resources and in these
cases, the state is maintained by passing session IDs as well as through the
concept of persistence.
Load
cases,Balancing
the state 101: Nuts andby
is maintained Bolts
passing session IDs as well as through the
concept of persistence.
Today, the intelligence of ADCs allows organizations to open the data packets and
create persistence tables for virtually anything within them. This enables them to use
unique information, such as user name, to maintain persistence. However,
organizations must ensure that this identiable client information will be present in
every request made, as any packets without it will not be persisted and will be load
balanced again, most likely breaking the application.
Conclusion
In the beginning, load balancing focused on distributing workloads throughout the
network and ensuring the availability of applications and services. As the technology
evolved, however, load balancers became platforms for application delivery, ensuring
that an organization's critical applications were highly available and secure. While
basic load balancing remains the foundation of application delivery, modern ADCs
offer much more enhanced functionality.
Enterprises realize that simply being able to reach an application doesn't make it
usableand unusable applications mean wasted time and money for the
organization deploying them. That's where the modern ADC comes in, allowing
organizations to consolidate network-based services like SSL/TLS ofoad, caching,
compression, rate-shaping, intrusion detection, application rewalls, and even
remote access into a single strategic point that can be shared and reused across all
application services and all hosts to create a virtualized Application Delivery Network.
This allows network, application, and operations teams better respond to business
demands for shorter delivery timelines and greater scalabilitywhile never sacricing
the need for security.
If you would like to learn more about how advanced application delivery works and
the future of ADCs, read The Evolution of Application Delivery Controllers and Go
Beyond Plain Old Load Balancing.
http://www.strategyr.com/MarketResearch/Application_Delivery_Controllers_ADC_Market_Trends.asp
9
application services and all hosts to create a virtualized Application Delivery Network.
This allows network, application, and operations teams better respond to business
demands for shorter delivery timelines and greater scalabilitywhile never sacricing
the need for security.
WHITE PAPER
If you would like to learn more about how advanced application delivery works and
Load Balancing
the future of ADCs,101: Nuts
read and
The Bolts of Application Delivery Controllers and Go
Evolution
Beyond Plain Old Load Balancing.
http://www.strategyr.com/MarketResearch/Application_Delivery_Controllers_ADC_Market_Trends.asp
F5 Networks, Inc.
401 Elliott Avenue West, Seattle, WA 98119 Americas Asia-Pacific Europe/Middle-East/Africa Japan
888-882-4447 f5.com info@f5.com apacinfo@f5.com emeainfo@f5.com f5j-info@f5.com
2017 F5 Networks, Inc. All rights reserved. F5, F5 Networks, and the F5 logo are trademarks of F5 Networks, Inc. in the U.S. and in certain other countries. Other F5
trademarks are identified at f5.com. Any other products, services, or company names referenced herein may be trademarks of their respective owners with no
endorsement or affiliation, express or implied, claimed by F5. CS01-00094 0113
10