• In this course, basic DC concepts, including the topology and architecture, are based on

Layer 2 or above.
• DCN: is a network that provides interconnection between computing units within a DC
and interconnection between computing units in a DC and external egresses of the DC.

• SAN: consists of storage arrays and fibre channel (FC) switches and provides block
storage. A SAN that uses the fibre channel protocol (FCP) is called an FC SAN, and a
SAN that uses the IP protocol is called an IP SAN.
• Distributed storage: is different from centralized storage in terms of the deployment
mode. In distributed storage, data is stored on independent servers (storage nodes) in
a distributed manner. Distributed storage can also be deployed in cloud storage mode.

• Server (compute node): provides compute services.

• DCNs of different enterprises or industries are divided into different zones, without
fixed zone division rules. For example, a financial DC is divided into different zones
based on functions, including production zone 1, production zone 2, test zone 1, test
zone 2, big data zone, and operations management zone.

• In this example,
▫ Internet access zone: transmits traffic generated when users access the Internet,
for example, traffic of online banking services.

▫ Office network access zone: transmits traffic generated when office users access a
campus network.

▫ WAN access zone: connects to the WAN built by an enterprise, as well as the
remote DCN and remote campus network.

▫ Production environment zone: connects to the production environment network.

▫ Test environment zone: connects to the test environment network.
• This solution ensures the reliability of Layer 2 and Layer 3 switching. However, with the
continuous increase in the DC scale and service bandwidth, the following problems
become serious:

▫ Low network throughput cannot meet increasing traffic requirements:

▪ The STP blocking mechanism results in poor link efficiency.

▪ The single-active gateway mechanism of VRRP cannot fully exploit the
device forwarding performance.

▪ Layer 2 traffic is not forwarded along the shortest path. The root bridge has
a bandwidth bottleneck, causing a long forwarding delay.

▫ STP network scale is limited, resulting in poor convergence performance.

▫ Maintenance is complex due to a large number of management nodes and
complicated logical topologies.
• Either with stacking or M-LAG, DCN 2.0 has the following characteristics:
▫ Layer 2 and Layer 3 traffic is fully load balanced, making full use of all links.
▫ Inter-device link aggregation is deployed to support protection against physical
node faults.

▫ Compared with STP, DCN 2.0 provides a shorter Layer 2 forwarding path and
delay.
▫ Compared with traditional STP, DCN 2.0 can build a larger Layer 2 network.
• Compared with DCN 2.0, DCN 3.0 greatly improves the scalability and flexibility of
service networks. DCN 3.0:

▫ Uses the scalable spine-leaf architecture where more than two spine nodes can
be used.

▫ Supports equal-cost multi-path (ECMP) at Layer 3.

▫ Uses M-LAG to support load balancing of Layer 3 gateways.
▫ Implements the distributed gateway function provided by EVPN to transmit Layer
3 traffic along the optimal path.

▫ Uses common IP encapsulation at the outer layer, ensuring good network

expandability and reuse of assets, experience, and tools.
• A standard DC cabinet is 42 U (4.445 cm per unit).
• TOR: TOR switches are deployed on the top of a cabinet. Servers in the cabinet are
connected to the TOR switches through optical fibers or network cables and the TOR
switches are connected to upper-layer aggregation switches. This solution is applicable
to the scenario where there are a large number of access devices or the density of
devices in a single cabinet is high. The distributed access mode reduces the number of
connections between server cabinets and a network cabinet, ensuring simple
connection management. However, in cloud DCs, it is quite common to find that access
switches are distributed in multiple cabinets, which is inconvenient for centralized
maintenance and management of switches.
• EOR: In traditional DCs, EOR switches are usually deployed in one or two cabinets at
the end of each cabinet row. All servers in the cabinet row are connected to EOR
switches through horizontal cables. If the EOR mode is used, a large number of cables
are aggregated from multiple server cabinets to a network cabinet. This complicates
cable connection management, but facilitates centralized maintenance and
management of switches.
• MOR: MOR switches and EOR switches have similar cable connection modes where
access switches are deployed in one or two cabinets of a cabinet group in a centralized
manner. The only difference is that the network cabinet of MOR switches is located in
the middle of the cabinet group. As such, compared with the EOR mode, the MOR
mode simplifies the cable connections from server cabinets to a network cabinet and
manages switches in a centralized manner. In this sense, the MOR mode is a
compromise between the TOR mode and EOR mode.
• The definitions of PoDs, indicating physical partitions, vary in different enterprises. For
example, in some large-sized enterprise DCs, a PoD is smaller than an equipment room
module, while in others, a PoD consists of multiple TOR switches and spine switches.

• Many enterprises plan PoDs to normalize hardware specifications and facilitate the
modular and standardized deployment of IT infrastructure.
• OpenvSwitch is an open-source Apache2.0 project. It is a vSwitch running on a
virtualization platform (such as KVM and Xen) and is the most mainstream vSwitch in
the industry. The OVS is a distributed virtual switch.

• The OVS provides Layer 2 switching for dynamically changed endpoints to control
access policies, network isolation, and traffic monitoring on virtual networks.
• Characteristics of the spine-leaf architecture:
▫ Each lower-level node (leaf node) connects to all higher-level nodes (spine
nodes) to form a full-mesh topology.

▫ There is no east-west link between nodes at the same level.

▫ In the standard spine-leaf architecture, leaf nodes are similar to LPUs of modular
switches and are responsible for transmitting external traffic. Spine nodes are
similar to switch fabric units (SFUs) on modular switches and are responsible for
traffic forwarding between leaf nodes.

• The spine-leaf architecture provides high scale-out capabilities.

▫ The number of spine nodes can be expanded to four or more. The maximum
number of spine nodes depends on the number of uplink interfaces on leaf
nodes.
▫ A two-level spine-leaf architecture can be extended to a three-level one to
implement high-speed data exchange between more leaf nodes.
• Xen: is a virtualization technology developed by Ian Pratt of Cambridge University and
is included in the Linux kernel (VMware and OpenVZ are also based on Linux). In the
simplified Xen virtualization mode, device drivers are not required and each virtual user
system is independent of each other, with some functions implemented by service
domains.
• Computing virtualization, storage virtualization, and network virtualization are all
required to implement the complete functions of virtualization.

▫ Computing virtualization is implemented through CPU virtualization (such as

Intel VT-x and AMD-V), memory virtualization (such as the shadow page table
and extended page table), and I/O virtualization (such as QEMU, Virtio, and SR-
IOV).

▫ Storage virtualization: Huawei FusionCompute is used as an example. It is

implemented using the virtual cluster file system (VIMS), a high-performance
cluster file system.
▫ Network virtualization is implemented through vSwitches. For example, Huawei
FusionCompute is implemented using distributed virtual switches (DVSs).
• Mainstream virtualization platforms in the industry include
Huawei FusionCompute VRM, VMware vCenter, and Microsoft System Center.
• OVS is a vSwitch under the open source Apache 2.0 license and runs on a virtualization
platform such as KVM and Xen. OVS is the most popular virtual switch among all other
counterparts in the industry. Strictly speaking, an OVS is a distributed vSwitch.

• vSwitches have various functions. As for the OVS, Layer 2 switching can be provided for
dynamically changing endpoints to control access policies, network isolation, and
traffic monitoring on virtual networks.
• DVSs are also used in Huawei FusionCompute VRM and connect to the controller in
the network virtualization solution.

• vSphere distributed switches (VDSs) are used in VMware vCenter.

• In the network virtualization – computing scenario, iMaster NCE-Fabric pushes
configurations of port groups to DVSs on the virtualization platform.
• FusionSphere OpenStack is Huawei's commercial release with a built-in Huawei KVM
virtualization engine.

• Based on open source OpenStack, FusionSphere OpenStack incorporates various

enterprise-level enhancements to its computing management, storage management,
network management, installation and O&M, security, and reliability. This product is
the optimal commercial OpenStack choice for enterprise private cloud, carrier NFV, and
public cloud service providers.
• Layer 2 network functions are the prerequisite and must-have for other network
functions. In this sense, core Neutron plug-ins must exist as they implement Layer 2
network functions, with Modular Layer 2 (ML2) (as mentioned in the course) as the
mainstream implementation mode.

• Neutron APIs can be invoked by external software and the database is used to store
Neutron data.
• ManageOne is the smallest set of compute, storage, and network resources. For
example, if a user is a VDC and has multiple project teams or service systems,
ManageOne can allocate a project to an independent project team or service system.
This project corresponds to a tenant on iMaster NCE-Fabric. VPC is a concept for
ManageOne and iMaster NCE-Fabric, while VDC is a concept specifically for
ManageOne. Neither of two concepts applies to OpenStack.
• A region is an OpenStack system that has multiple availability zones (AZs).

• An AZ has same compute and storage resources, providing high availability. Different
resources can be allocated to different AZs, for example, common servers and high-
performance servers are allocated to a unique AZ respectively.
• Does an AZ consist of multiple DCs or just some regions in a DC? This depends on the
application scenario of the AZ: an AZ in the public cloud scenario contains multiple
DCs, while an AZ in the private cloud scenario (such as the telco cloud scenario or
financial cloud scenario) contains only some regions of a DC. In this course, the cloud-
network integration solution mainly applies to the private cloud scenario.
• VPCs provide isolated VMs and network environments to meet the network isolation
requirement of different departments.

• VPCs use resources in VDCs. Each VPC belongs to one VDC, and each VDC can have
multiple VPCs.

• Each VPC can provide independent services, such as virtual firewalls, elastic IP
addresses, security groups, firewalls, and NAT gateways.
• The Huawei cloud-network integration solution uses ManageOne as the cloud
management platform and FusionSphere OpenStack as the cloud platform.
• In Linux, the kernel space and user space are separated. The Linux OS and
drivers run in kernel space, whereas applications run in user space.

• More specifically, a container is a running instance of a container image.

• Container image:

▫ Packages an application and its dependencies (including all files and directories
of the complete OS).
▫ Contains all dependencies required for application running that is achieved
through image running in the isolated sandbox without any modification or
configuration.

▫ Focuses on packaging applications and their running environments in a unified

format. This ensures high consistency between the local environment and the
cloud environment.
• In terms of container technology development, leading container orchestration
platforms include Docker Swarm, Apache's Mesos (initially developed by AMPLab of
the University of California, Berkeley), and Google's Kubernetes. After years' of
development, Kubernetes has become the de facto standard of container management
platforms.
• Functions of components on the Kubernetes master node:
▫ Kubernetes API server: exposes all the APIs of the Kubernetes master node and is
the only entry for adding, deleting, modifying, and querying all resources in
Kubernetes.
▫ ETCD: is used as Kubernetes' backing store for all cluster data. It is used to store
metadata of the Kubernetes cluster, such as the IP address and Layer 4 port
number of a PoD.
▫ Kubernetes controller manager: is the Kubernetes cluster control center. It
contains many complex components and is used to monitor and manage the
PoD, service, and namespace information.
▫ Kubernetes scheduler: schedules a PoD to be run to a specified Kubernetes
worker node.
• Functions of components on the Kubernetes worker node:
▫ Kube proxy: is the access entry for managing services, including the access from
PoDs to services inside or outside clusters.
▫ kubelet: dynamically creates and deletes Pods on a Kubernetes worker node, and
periodically reports the resource usage of the worker node to the API server by
eavesdropping on the Kubernetes API server.
▫ Container runtime: creates and runs containers.
• The vSwitch integrates the distributed firewall function to implement Kubernetes
network policies and integrates the network address translation (NAT) and load
balancing functions to provide east-west load balancing for PoDs to access services.

• The API server watcher eavesdrops on Kubernetes objects, including the PoD, service
and ingress, and associates with iMaster NCE-Fabric to configure the physical network.
• The API server watcher integrates the IP address management (IPAM) function to
manage the IP resource pool of the container network.
• FusionStorage Manager (FSM): a management module of FusionStorage, providing
O&M functions including alarm management, service monitoring, operation logging,
and data configuration. In most cases, the FSM is deployed in active/standby mode.

• FusionStorage Agent (FSA): an agent module of FusionStorage, which is deployed on

each node to enable the node to communicate with the FSM nodes.
• Metadata Controller (MDC): a metadata control component of FusionStorage for
controlling distributed cluster node status, data distribution rules, and data rebuilding
rules. MDC components are deployed on the ZooKeeper disks of three nodes to form
an MDC cluster.
• Virtual Block System (VBS): a virtual block storage management component for
managing volume metadata and providing the access point (AP) service for distributed
clusters, enabling computing resources to access distributed storage resources. Each
node has one VBS process running by default, forming a VBS cluster. Multiple VBS
processes can be deployed on one node to improve the I/O performance.
• Object Storage Device (OSD): performs specific I/O operations. One server can have
multiple OSD processes deployed and one OSD process is deployed for each disk by
default. When an SSD card is used as the main storage disk, multiple OSD processes
can be deployed on the SSD card to maximize the SSD card performance. For example,
one 2.4 TB SSD card supports a maximum of six OSD processes, each of which
manages the I/O operations for 400 GB of space.
• Tenant services are carried on the overlay network and decoupled from the physical
network.

• VXLAN network virtualization:

▫ VXLAN uses a 24-bit segment ID known as the VXLAN network identifier (VNID),
which supports up to 16 million multi-tenant networks.
▫ Different terminals (servers/VMs) communicate with each other at Layer 2 across
the IP network.

▫ Smooth VM migration brings network change requirements, facilitating VXLAN

virtualization.

▫ Virtual networks can be flexibly deployed in an on-demand manner.

▫ Elastic network expansion can be implemented as the network border can be
extended to NVE nodes of servers.
• The access IP address provided by a load balancer is called the virtual IP address in this
example and is called the floating IP address (FIP) in OpenStack.

• The load balancing algorithm determines the server to which an external request is
distributed. Typical load balancing algorithms include:

▫ Round robin: selects the first server in the list for the first request, and traverses
the list one by one in a cyclic way.
▫ Least connections: selects the server with the least number of active connections.

▫ Hash: selects the server to forward packets based on the hash value of source IP
addresses. This mode ensures that requests of a specific user are distributed to
the same server.
▫ Random weight: randomly distributes requests to nodes based on their weights.
For example, if the weight of Node1 is two times that of Node2 and 30 requests
are to be sent to the two nodes, Node1 will receive about 20 requests and Node2
about 10.
• For enterprises with multiple DCs in different regions, GSLB ensures that users access
the nearest DCs based on their locations. GSLB has multiple solutions and this example
describes the DNS-based GSLB solution commonly used in DCs.

• In the GSLB solution, the domain name service provider sets the name server (NS) to
the GSLB device that provides the intelligent DNS resolution function. In this way, the
GSLB device is responsible for resolving the domain name. If GSLB devices are deployed
in multiple places, all of they should be added to the NS to ensure high availability.
GSLB devices can perform health check on backend servers and public IP addresses of
other DCs. Health check results are synchronized between GSLB devices in different
IDCs using proprietary protocols. Then the GSLB devices select the optimal addresses
for the DNS servers based on global load balancing policies and send the addresses to
the user.
• SLB in a DC intelligently forwards service data requests to several, or hundreds or
thousands of backend application servers based on the information contained in the
requests. With load balancing algorithms, the optimal servers are selected based on
predefined policies. This solves the availability and scalability problems of the
application to some extent.
1. A DC comprises a set of complex facilities, including the equipment room, the computer
system and the devices related to the computer system (for example, communications
and storage systems), as well as redundant devices, say, data communication devices,
environmental control devices, monitoring devices, and various security devices.
2. The main difference between a common DC and a cloud DC is that a cloud DC
implements large-scale cloud computing deployment. Cloud DCs are low-carbon and
energy-saving where compute, storage, and network resources are loosely coupled.
3. Key IT services of DCs include but are not limited to cloud computing, virtualization,
container, HPC, and AI.
4. Load balancing falls into three types. They are: 1. GSLB: is based on the domain name
resolution mechanism; 2. HTTP load balancing: involves HTTP redirection and reverse
proxy; 3. Network layer load balancing: works at the network layer and implements
service load balancing by modifying network layer information such as the IP address,
MAC address, and Layer 4 port.
5. DCN 1.0: uses the VRRP + STP mechanism and achieves basic reliability. A VRRP group
is configured to work in master/backup mode. STP is configured to eliminate loops,
leading to low utilization.
▫ DCN 2.0: uses stack/M-LAG and supports inter-device link binding for server
access and full load balancing among links. The network has a limited scale,
supporting only two aggregation devices. East-west Layer 3 traffic is transmitted
over a non-optimal path.
▫ DCN 3.0: uses a spine-leaf architecture and VXLAN EVPN. It supports full load
balancing among links. Four or more spine nodes are supported and east-west
Layer 3 traffic is transmitted over an optimal path.