Paper 1 - O-RAN-Minimum-Delay
Paper 1 - O-RAN-Minimum-Delay
Paper 1 - O-RAN-Minimum-Delay
Computer Networks
journal homepage: www.elsevier.com/locate/comnet
Minimum delay function placement and resource allocation for Open RAN
(O-RAN) 5G networks
Nasim Kazemifard, Vahid Shah-Mansouri ∗
School of Electrical and Computer Engineering, College of Engineering, University of Tehran, Tehran 14395-515, Iran
Keywords: Digitalization is a journey that has been started and put ICT industry in a crucial situation to provide required
Open radio access network (O-RAN) infrastructure for diverse range of data hungry, short tempered applications and services. One of the main
Hierarchical data center technologies that will pave the way towards new digital ecosystem is fifth Generation of mobile technology.
Containerized network function (CNF)
To meet 5G network service requirements, innovative architectures, technologies and standards focusing on
Resource allocation
cloudification are employed. Cloudification of network functions along with the use of virtualized network
Minimum delay function placement
functions (VNFs) and containerized network functions (CNFs) allows agile and scalable service provisioning.
The use of VNFs and CNFs has been started from core and networking middleboxes but then extended to
RAN functions. Open radio access network (O-RAN) proposes an interoperable and standard architecture for
cloudified RAN. The main idea behind this architecture is to make RAN more flexible. O-RAN allows different
layers of RAN to be split and deployed as virtual function and openly communicate with each other for service
provisioning. In this paper, we model an End-to-End mobile network operator (MNO) employing O-RAN. We
consider a mobile network architecture, with three layer hierarchical data centers (Local, Regional, and Core)
to add flexibility in resource allocation, and increase reliability, taking the advantages of O-RAN. MNO receives
various service function requests (SFRs) requiring accommodation on the network. We assume RAN and core
functions are deployed as CNFs on the data centers. Users of SFRs connects to remote radio heads (RRH) to
receive the service. In this paper, we mathematically model the CNF placement and resource allocation of an
O-RAN enabled LTE/5G network while trying to minimize the End-to-End delay of the data plane. We study
the problem in two different cases First, we assume that the SFR traffic traverse through a single path across
the RAN functions and model this problem. This is a mixed integer non-linear programming problem. With
some change of variables, we make it a linear mixed integer programming problem but it is still non-trivial to
solve. Then, we model the problem for the case where traffic of an SFR can be split and be served via multiple
CNFs. We proposed a gradient based scheme to solve the minimum delay problem in this case.
Experimental results indicate that by increasing the number of service requests in a network, the proposed
GBMD(Gradient-Based Minimum Delay) algorithm serves up to 90% End-to-End Delay decrease. Another
improvement on the performance of a network will occur by levering GBMD algorithm for around 72%
End-to-End delay reduction in case of limited resources.
1. Introduction strategic and business changes, they have to think about technical
conversions in their networks.
Day to day life is being affected by digital ecosystem. Increasing Taking a look over standardization trends and technology roadmap
in variety of services and applications brings more demand from cus-
indicates an inevitable migration for telecom operators towards virtual-
tomers in any aspect of life, such as shopping, learning, health care,
traveling, sport, etc. Besides application developers and OTT players, ization. There is a history behind current mobile network technologies.
Telco Operators play a key role on realization of this life style. Opera- Starting from first generation, to 2G with poor data throughput in
tors, mainly MNOs, in the role of infrastructure and service providers form of GPRS, continued with third generation of mobile network
should cope with such increasing demand from customers and appli- (3G) that supports better speed of data up to 8 Mbps. More data
cation developers, in terms of huge investment, agile strategic plans, speed requirements lead standardization bodies to changing priority in
architectural review, business review, etc. [1]. In other words, besides
∗ Corresponding author.
E-mail addresses: na.kazemifard@gmail.com (N. Kazemifard), vmansouri@ut.ac.ir (V. Shah-Mansouri).
https://doi.org/10.1016/j.comnet.2021.107809
Received 13 May 2020; Received in revised form 16 October 2020; Accepted 31 December 2020
Available online 4 January 2021
1389-1286/© 2021 Elsevier B.V. All rights reserved.
N. Kazemifard and V. Shah-Mansouri Computer Networks 188 (2021) 107809
architecture based on data which happened in 4G or LTE solution with network tuning with scale-up/scale-down designs rather than expensive
150 Mbps. In this generation, many advanced features are added to hardware expansion in vendor locked situation. Admitting new services
increase data throughput such as carrier aggregations, massive MIMO, and applications, realization of network slicing, and DevOps concept
and etc. to reach up to 1 Gbps data rate per customer [2]. is a result of implementing O-RAN in network. In comparison to tradi-
In legacy mobile network architecture, shown in Fig. 1, there are tional RAN which had integrated RRU and BBU with high cost last mile
some physical functions that serve customers with voice and data transport network, deployment and management in O-RAN is flexible
services. The network consists of radio access network, which connects according to its agnostics front-haul. Programmable application and
customer to Antennas, and Backhaul part plus IP backbone (IPBB) network adaptability in O-RAN can be compared to pre-programmed
network that includes high speed switches and routers to deliver user and fixed control logic and fixed network resources in legacy network
data to core part of the network. The functionality of serving customer as well.
data will happen in several Physical Network Functions called PNF, However, O-RAN splits the control-plane (CP) from the user-plane
such as SGW, MME, HSS, etc. (UP) through E1 interface (3GPP standard) inherited from SDN concept,
This fixed and physical oriented architecture cannot support all and will handle Radio Resource Management functions in terms of NG-
ambitious targets for future solutions. Customers need more data, in RRM via hierarchical (Non-RT and Near-RT) RAN Intelligent Controller
very dense areas, with a minimum End-to-End delay and reliability. (RIC) with A1 and E2 interfaces as shown in Fig. 2.
Fifth generation of network is planned to meet these targets. 5G will In this paper, our focus is to propose a mathematical model for the
bring more flexibility and reliability to network providers via NR (New problem of CNF placement and resource allocation in O-RAN networks
Radio) [3], and 5GC (5G Core). The minimum required bandwidth and where different layers of RAN are placed on geographically distributed
new products are not the concerns in this paper. But, there are some places. We provide a model for the end to end delay of the data plane
enablers that will use for migration towards 5G. One of these enablers traffic and minimizes this delay subject to resource constraint of the
is Virtualization of network functions which is the fundamental of Telco data centers and communication links. In the next section of the paper,
data centers and orchestration in network that we will use for our work. we consider a system model by combining NFV and O-RAN reference
In order to reduce CAPEX and OPEX, minimize time-to market of models. A three layer hierarchical cloud based network architecture is
new services, and maintain profit by creating new revenue streams, used for NFV deployment. Mobile network operator receives service
operators are adopting network function virtualization (NFV) technolo- function requests (SFRs) which have different resource requirements
gies and are shifting from physical hardware towards NFV platforms and place them on the virtual infrastructure. We model the problem of
with help of softwarization and cloudification [4]. Through NFV, net- CNF placement and resource allocation of various SFRs for RAN and
work functions become virtualized and are called Virtualized Network core functions while minimizing the End-to-End delay for data plane.
Functions (VNFs) in standards. VNFs are deployed as VMs (virtual The system model is shown in Fig. 2. The contributions of the paper
machines) with hypervisors such as Linux KVM or VMware vSphere are listed as follows:
on Commercially Off-The-Shelf hardware (COTS) and bring vendor
independency for operators [5]. For lightweight implementation of • We mathematically formulate the problem of resource alloca-
functions, in some cases, Containerized Network Functions (CNFs) are tion and CNF placement on a virtualized 5G network employing
employed instead of VNFs. CNFs are lighter and more flexible compared O-RAN.
to VNFs. CNFs can run in microservices architecture which provides • First, we consider the case where the traffic of an SFR traverses
a dynamic, flexible, and scalable architecture towards 5G [6]. Tran- via a single path through CNFs of its chain. It is a nonlinear
sitioning from a hardware-centric to a software-centric paradigm is mixed integer programming problem. Via some re-formulations,
challenging. Different targets for 5th generation of mobile technology we convert it to a linear programming problem. However, it is
defines tough targets such as five nine reliability, one millisecond still non-trivial.
latency, and up to 10 Gbps throughput [7]. 5G networks simultane- • We then formulate the case where SFR traffic can traverse multi-
ously support several services with different requirements in categories ple paths. We propose a gradient based solution which achieves
including Enhanced Mobile Broadband (eMBB), Ultra Reliable Low the optimal solution efficiently.
Latency Communications (URLLC), and Massive Machine Type Commu- • We evaluate and compare the performance of the proposed
nications (mMTC) [8]. These technologies lead to re-design the service schemes via simulations.
provisioning architecture from dedicated hardware appliances or mid-
dleboxes to white-boxes. By moving packet-processing activities from The rest of the paper is organized as below. After presenting the
proprietary hardware middleboxes to virtualized entities, dependency state of the art, the system model is introduced in Section 3. The
on underlying hardware and vendors, which generally push operators single path placement and resource allocation problem are modeled
into Vendor Lock-In situation, will be reduced. To handle VNF/CNF in Section 4. In Section 5, we propose the alternative multiple path
functions, end-to-end cloud based architecture should be employed [9]. problem and the gradient based efficient solution. Finally, investigation
Hierarchical data centers are used in operator networks to distribute of the performance of optimal solution with exhaustive approach via
the deployment of the functions. VNFs or CNFs can be deployed in some simulations is shown in Section 6.
local, regional, and core data centers. Each service contains a ser-
vice chain composing of several VNFs/CNFs connected back to back. 2. State of the art
Various VNFs/CNFs of a service can be placed in different data centers.
The ORAN Alliance, co-founded by several telecom operators in- The European Telecommunications Standards Institute (ETSI) [10]
cluding AT&T, China Mobile, Deutsche Telekom, NTT DOCOMO, and proposed an architectural framework for NFV. The resources of NFV
Orange, combines and extends the efforts of the C-RAN Alliance and Infrastructure (NFVI) are compute (i.e., processing), storage, and net-
xRAN Forum into a single, global, operator lead effort. They pro- work capacity. A Virtualized Infrastructure Manager (VIM) (e.g., Open-
pose Open RAN (O-RAN) which is a vendor-neutral disaggregation Stack [11] or Kubernetes) manages NFVI and utilizes existing virtual-
of RAN at both the hardware and software levels on general pur- ization technologies to provide virtual resources for VNFs and CNFs.
pose processor-based platforms which implements an open interface The CNFs represent virtualized instances of different network func-
between components RU/CU/DU using hardware- and software-defined tions [12]. In this reference model, there is an Orchestrator
functions, and brings cloud-scale economics and agility to radio access (i.e., MANO) that manages the life-cycle of CNFs of the service. It
part of the network by means of modular softwarization for capacity utilizes resource allocation and placement algorithms to ensure optimal
management, increasing reliability and availability, easily and quickly usage of both physical and software resources [12].
2
N. Kazemifard and V. Shah-Mansouri Computer Networks 188 (2021) 107809
Fig. 2. End-to-End LTE/5G network-considering O-RAN architecture, with 3 layer data centers.
Cloudification in core network is employed and implemented in instead of TAS. In the proposed architecture, controllers and MANO
many operators’ networks until now. Nevertheless, in radio access net- roles are not considered. In [22], authors first study the deployment of
work (RAN), recently, standardization is completed and new reference CRAN in cellular networks. Then, virtualized RAN (VRAN) is studied as
models are proposed. Previously, C-RAN architecture was proposed and an evolution of CRAN. It is mentioned that unlike C-RAN, the V-RAN
its standardization is introduced in [13–15]. facilitates flexible control, low cost, efficient resource usage, and diver-
In [16], authors modeled the minimum power consumption problem sified applications. It can also resolve many underlying issues of cloud
while considering resources of RRH, BBU pool and backhaul links. based RANs. ORAN is evolved from VRAN by adding openness and
Authors proposed a study in which C-RAN is deployed in the network intelligence to that. Open interfaces allows multi-vendor deployment
while minimum changes are applied to the network and available of RAN. It is mentioning that by developing embedded intelligence,
infrastructure such as site locations and transmission links are re- the O-RAN architecture aims to not only extend the SDN concept of
used with the aim of cost reduction. In [17], authors considered a decoupling the control-plane (CP) from the user-plane (UP), but also to
cellular network with mobile cloud computing. They model the C- enhance the traditional RRM functions with embedded intelligence via
RAN problem with the objective of minimum energy consumption with RAN Intelligent Controller (RIC) near-RT.
constraint on task execution time. Afterwards, O-RAN alliance defines a
cloud based architecture evolving towards an open, agile, flexible, and
3. System model
programmable RAN paradigm in [18] and [19].
In 2013, M. Yang, et al. proposed an architecture for software-
We consider a virtualized multi-layer cellular network with several
defined RAN based on virtualization [20], in which three main parts,
hierarchical data centers including local (i.e., edge), regional (i.e., fog),
wireless spectrum resource pool (WSRP) that enables virtual RRUs
(vRRUs) to support different wireless protocols in one shared pRRU, and core data centers. Data center infrastructure allows support of net-
Cloud Computing Resource Pool (CCRP) that includes a large amount work function virtualization (NFV) which not only gives more flexibil-
of physical processors to support a high speed cloud computing net- ity to network designer and orchestrator, but also increases reliability
work via virtualized functions such as vBBUs and vBSCs, and an SDN and availability in terms of resource distribution and utilization [23].
controller to play a role of control plane to execute the strategies Local data centers are those close to radio sites with limited amount of
of each vBBU and vBSC that contains a SDN agent to communicate resources. Each local data center can connect to several regional data
with controller through SDN protocol were introduced. Although the centers physically via high capacity fiber optical links. The regional
proposed architecture is considerable in access part of the network, data centers have higher capacity and amount of resources; these nodes
it did not take care of Core functions, and there is no End-to-End receive data from local data centers and send to core data center. Core
view of O-RAN implementation. O-RAN alliance has proposed O-RAN data centers are limited in terms of number but they are significantly
Controller instead of SDN controller in this architecture which has more rich in terms of resources. Core data centers supports running virtual-
complicated functionality and we use it in this paper. ized EPC and 5GC cores [7]. Multiple instances of core can be employed
Another Cloud-RAN architecture is presented in [21], that supports to increase reliability and availability.
indoor front-ends and the BBU server can be installed inside the build- We consider a cellular network employing O-RAN architecture.
ing that is covered with multiple RRH considering DAS algorithms Fig. 2 shows the system model. In O-RAN architecture, low physical
3
N. Kazemifard and V. Shah-Mansouri Computer Networks 188 (2021) 107809
layer part (i.e., open remote unit (O-RU)) is running on RRH. It includes The same constraint can be used for regional data centers. Let
several physical layer tasks such as IQ decompression, precoding, dig- denote the set of regional data center servers. Let boolean variable
ital beamforming, iFFT CP addition, and D2A. Using open front-haul 𝛽𝑖𝑏 denote if CU-CNF of the 𝑖th SFR is deployed on the 𝑏th server of
interfaces such as CPRI in LTE and eCPRI in 5G, O-RU connects to regional data center. CU-CNF of the 𝑖th SFR has capacity and storage
RAN distributed unit (DU). Virtualized high physical layer, MAC, and requirement from the regional data center denoted respectively by 𝑦𝑅 𝑖
RLC functions run on DU which is deployed on local data centers as and 𝑧𝑅 𝑖 . The resource constraints for servers of the regional data center
a single containerized network function (CNF). We call such function mandates that
as DU-CNF. The upper layer functions of the RAN in 5G includes RRC, ∑
𝛽𝑖𝑏 𝑦𝑅 𝑅
𝑖 ≤ 𝜌𝑏 , ∀𝑏 ∈ , (2)
PDCP-C, PDCP-U, and SDAP which run as one CNF on the regional data 𝑖∈
centers. We call this function CU-CNF. O-RAN Intelligent controllers ∑
𝛽𝑖𝑏 𝑧𝑅 𝑅
𝑖 ≤ 𝑆𝑏 , ∀𝑏 ∈ ,
(RIC) near-RealTime (RT) are also deployed in regional data center 𝑖∈
to be close to CU-CNF functions and are connected to them via an ∑
𝛽𝑖𝑏 𝜃𝑖 ≤ 𝑑𝑏𝑅 , ∀𝑏 ∈ ,
interface called E2. Cellular core functions runs on core data centers. 𝑖∈
Slicing is used to isolate different services and reach the required SLA
where 𝜌𝑅 , 𝑆𝑏𝑅 , and 𝑑𝑏𝑅 denote the total capacity, storage, and transmis-
levels [24]. 𝑏
sion rate of the 𝑏th server of regional data center.
Customers receive service by connecting to the RRHs which are
Similarly, we have constraints for core data centers. Let denote
connecting to RAN part, and requesting for services by attaching to
the set of core data center servers and boolean variable 𝛾𝑖𝑐 denote if
RRHs. In order to guarantee Quality of Service (QoS) in Cloud based
the core CNF of the 𝑖th SFR is deployed on the 𝑐th server of core data
architecture, QoS parameters such as throughput, jitter, and delay
center. Core CNF of the 𝑖th SFR has capacity and storage requirements
should be rephrased to resource allocation in terms of capacity, storage
from the core data centers denoted respectively by 𝑦𝐶 𝐶
𝑖 , 𝑧𝑖 . The resource
and processing capability [18,25].
constraints of the core data center mandates that
It is important to notice that there is a data base called R-NIB in RIC ∑
Near-RT, and some application layer functions are handled in it, such as 𝛾𝑖𝑐 𝑦𝐶 𝐶
𝑖 ≤ 𝜌𝑐 , ∀𝑐 ∈ (3)
3rd party, radio connection, mobility management, QoS, and interface 𝑖∈
∑
managements. RIC near-RT in regional data centers connects logically 𝛾𝑖𝑐 𝑧𝐶 𝐶
𝑖 ≤ 𝑆𝑐 ∀𝑐 ∈
via A1 interface to RIC near-RT in core data centers. Orchestration and 𝑖∈
∑
automation in terms of ONAP, MANO, and NMS are the functions added 𝛾𝑖𝑐 𝜃𝑖 ≤ 𝑑𝑐𝐶 , ∀𝑐 ∈
to next generation core rather than traditional core network. 𝑖∈
4
N. Kazemifard and V. Shah-Mansouri Computer Networks 188 (2021) 107809
Table 1
Simulation parameters.
Parameter name Value
Set of SFRs
Set of servers of local data center
Set of servers of regional data centers
Set of servers of core data centers
𝛼𝑖𝑎 Decision binary variable indicating if SFR 𝑖 using server 𝑎 ∈
𝛽𝑖𝑏 Decision binary variable indicating if SFR 𝑖 using server 𝑏 ∈
𝛾𝑖𝑐 Decision binary variable indicating if SFR 𝑖 using server 𝑐 ∈
𝑦𝐿𝑖 , 𝑦𝑅
𝑖
, 𝑦𝐶𝑖 Processing requirement of SFR 𝑖 on local, regional, and core data centers respectively.
𝑧𝐿𝑖 , 𝑧𝑅
𝑖
, 𝑧𝐶𝑖 Storage requirement of SFR 𝑖 on local, regional, and core data centers respectively.
𝜌𝐿𝑖 , 𝜌𝑅
𝑖
, 𝜌𝐶𝑖 Processing capacity of local, regional, and core data centers respectively.
𝑆𝑖𝐿 , 𝑆𝑖𝑅 , 𝑆𝑖𝐶 Storage limit of SFR 𝑖 on local, regional, and core data centers respectively.
𝑑𝑎𝐿 , 𝑑𝑏𝑅 , 𝑑𝑐𝐶 Rate capacity of local, regional, and core data centers server respectively.
𝜃𝑖 Data rate requirement of SFR 𝑖
Next, we consider the transmission delay between the data centers. We We notice that the solution of (10) and (11) are identical since at the
consider a constant transmission costs between the data centers. Let optimal point, the inequalities in (11b)–(11d) are active. In constraint
𝜀𝐿−>𝑅 (𝑎, 𝑏) and 𝜀𝑅−>𝐶 (𝑏, 𝑐) denote the cost of using links from local in (11b), if 𝛼𝑖𝑎 is zero, the constrain is always valid. If 𝛼𝑖𝑎 = 1, then the
data center server 𝑎 to regional data center server 𝑏 and from regional constraint becomes a linear constraint if we consider 1∕𝐷𝑖𝐷𝑈 as a new
data center server 𝑏 to core data center server 𝑐, respectively. The variable. First, we convert this constrain by adding a new variable 𝑀
transmission cost for SFR 𝑖 is non-zero between local server 𝑎 and which is large and multiplied by 1 − 𝛼𝑖𝑎 . For case of 𝛼𝑖𝑎 = 0 and 𝛼𝑖𝑎 = 1,
regional server 𝑏 if both 𝛼𝑖𝑎 and 𝛽𝑖𝑏 is non-zero. We introduce binary the following constraint acts similar to (11b)
𝑖 which is one if both variables 𝛼 𝑎 and 𝛽 𝑏 are one. We can
variable 𝜛𝑎,𝑏 𝑖 𝑖 1 ∑
relate 𝜛𝑎,𝑏 to 𝛼𝑖𝑎 and 𝛽𝑖𝑏 as
𝑖 − (1 − 𝛼𝑖𝑎 )𝑀 ≤ 𝜌𝐿𝑎 ×𝐹 −
𝐿
𝛼𝑖𝑎 𝜃𝑖 , (12)
𝐷𝑖DU
𝑖∈
𝑖
1 − 𝜛𝑎,𝑏 ≤ (1 − 𝛼𝑖𝑎 ) + (1 − 𝛽𝑖𝑏 ), ∀𝑖 ∈ , 𝑎 ∈ , 𝑏 ∈ . (9) where 𝑀 is a large number. Similarly, we have the followings for (11c)
and (11d)
𝑖 is one if 𝛼 𝑎 and 𝛽 𝑏 are one. If one of 𝛼 𝑎 and
Eq. (9) guarantees that 𝜛𝑎,𝑏 𝑖 𝑖 𝑖 1 ∑
𝑏 𝑖
𝛽𝑖 are zero, then 𝜛𝑎,𝑏 can be one and zero. Since we are minimizing − (1 − 𝛽𝑖𝑏 )𝑀𝜌𝑅 𝑅 𝐿
𝑏 × 𝐹 × 𝐹𝑏 − 𝛽𝑖𝑏 𝜃𝑖 , (13)
𝐷𝑖CU
𝑖 takes value 0 in case it has option between one and 𝑖∈
the delay, 𝜛𝑎,𝑏 ∑
1
zero. Considering the transmission delay, the end-to-end processing and − (1 − 𝛾𝑖𝑐 )𝑀 ≤ 𝜌𝐶 𝐶
𝑐 ×𝐹 − 𝛾𝑖𝑐 𝜃𝑖 , (14)
𝐷𝑖Core
transmission delay can be written as 𝑖∈
∑∑ Making a change of variables, as 𝐷̃ 𝑖DU = 1∕𝐷𝑖DU , 𝐷̃ 𝑖CU = 1∕𝐷𝑖CU , 𝐷̃ 𝑖Core =
𝑖
𝐷𝑖 = 𝜛𝑎,𝑏 𝜀𝐿−>𝑅 (𝑎, 𝑏)𝜃𝑖
𝑎∈ 𝑏∈ 1∕𝐷𝑖Core . Then, the problem can be written as
∑∑ ∑∑∑
𝑖
+ 𝜛𝑏,𝑐 𝜀𝑅−>𝐶 (𝑏, 𝑐)𝜃𝑖 min 𝑖
𝜛𝑎,𝑏 𝜀𝐿−>𝑅 (𝑎, 𝑏)𝜃𝑖
𝑏∈ 𝑐∈ 𝑖∈ 𝑎∈ 𝑏∈
𝛼𝑖𝑎 𝛽𝑖𝑏 ∑∑∑
𝑖
+ 𝐿 ∑ + + 𝜛𝑏,𝑐 𝜀𝑅−>𝐶 (𝑏, 𝑐)𝜃𝑖
𝜌𝑎 × 𝐹 𝐿 − 𝑖∈ 𝛼𝑖 𝜃𝑖 𝜌𝑅 × 𝐹 𝑅 − ∑𝐼 𝑐 𝛽 𝑏 𝜃𝑖
𝑎
𝑖∈ 𝑏∈ 𝑐∈
𝑏 𝑖=𝛾 𝑖 𝑖 ∑ 1 1 1
𝛾𝑖𝑐 + + + (15)
+ 𝐶 ∑ 𝑖∈ 𝐷̃ 𝑖DU 𝐷̃ 𝑖CU 𝐷̃ 𝑖Core
𝜌𝑐 × 𝐹 𝐶 − 𝑖∈ 𝛾𝑖𝑐 𝜃𝑖
subject to (1), (2), (3), (4), (5), (12), (13), (14). (16)
The minimum delay problem can be formulated as
∑ This is a problem with convex objective and linear binary constraints
min 𝐷total = 𝐷𝑖 (10) which can be solved using standard math tools. However, it is math-
𝑖∈
ematically intractable and does not have a known solution to achieve
subject to (1), (2), (3), (4), (5).
optimal value in polynomial time. In the next section, we propose a
This is a binary non-convex optimization problem which is known to be method with lower complexity.
non-trivial to solve optimally. We perform a series of changes to make
this problem a linear binary programming problem. For equations in 5. Splitting service flow traffics along servers
(6)–(8), we use inequality instead of equality. Then, we have
∑∑∑ In this section, we formulate minimum delay CNF placement in O-
𝑖
min 𝜛𝑎,𝑏 𝜀𝐿−>𝑅 (𝑎, 𝑏)𝜃𝑖 (11a)
RAN system but we allow the traffic of each SFR to split and traverse
𝑖∈ 𝑎∈ 𝑏∈
∑∑∑ the RAN and core in multiple paths. This means, traffic flow of an SFR
𝑖
+ 𝜛𝑏,𝑐 𝜀𝑅−>𝐶 (𝑏, 𝑐)𝜃𝑖
𝑖∈ 𝑏∈ 𝑐∈
leaving an RRH may split into several paths where each path follows
∑( ) 𝐿 denote the rate
different routes between CU-CNF and DU-CNFs. Let 𝜋𝑎,𝑖
+ 𝐷𝑖DU + 𝐷𝑖CU + 𝐷𝑖Core
𝑖∈
of SFR 𝑖 processed at DU-CNF of edge server 𝑎 ∈ . Since 𝜃𝑖 is the total
𝛼𝑖𝑎 rate of SFR 𝑖, we have
s. t. ∑ ≤ 𝐷𝑖DU , ∀𝑖 ∈ , 𝑎 ∈ , (11b) ∑
𝜌𝑎 × 𝐹 − 𝑖∈ 𝛼𝑖𝑎 𝜃𝑖
𝐿 𝐿
𝜃𝑖 = 𝐿
𝜋𝑎,𝑖 , (17)
𝛽𝑖𝑏 𝑎∈
∑ ≤ 𝐷𝑖CU , ∀𝑖 ∈ , 𝑏 ∈ , (11c) 𝑅 and 𝜋 𝐶 denote the rate of SFR 𝑖 processed at CU-CNF of local
𝜌𝑅 × 𝐹 𝑅 − 𝑖∈ 𝛽𝑖𝑏 𝜃𝑖 Let 𝜋𝑏,𝑖 𝑐,𝑖
𝑏
𝛾𝑖𝑐 server 𝑏 ∈ and at Core CNF of core server 𝑐 ∈ . For the local and
∑ 𝑐
≤ 𝐷𝑖Core , ∀𝑖 ∈ , 𝑐 ∈ , (11d) core data centers, we similarly have
𝜌𝐶 × 𝐹 𝐶 −
𝑖∈ 𝛾𝑖 𝜃𝑖
𝑐 ∑
𝑅
(1), (2), (3), (4), (5). 𝜃𝑖 = 𝜋𝑏,𝑖 , (18)
𝑏∈
5
N. Kazemifard and V. Shah-Mansouri Computer Networks 188 (2021) 107809
∑
𝐶
𝜃𝑖 = 𝜋𝑐,𝑖 . Proof. We use the proof by contradiction. Assume that these two terms
𝑐∈ are not equal and without loss of generality, assume that 𝜕𝐷(𝜓) 𝑝 | >
𝜕𝜙𝑖 𝜓̄
We next determine the rate at which departing rate of edge data center 𝜕𝐷(𝜓)
| . If we reduce small value of 𝛿 from 𝜙̄𝑝 (assuming that 𝜙̄𝑝 > 0)
𝑞 𝜓̄ 𝑖 𝑖
𝜕𝜙𝑖
𝑎 enters local data center 𝑏 and then core data center 𝑐. Let 𝑟𝐿−>𝑅 (𝑖, 𝑎, 𝑏)
and adds it to 𝜙̄ 𝑞𝑖 while the rest of the data rates are constant, the
and 𝑟𝑅−>𝐶 (𝑖, 𝑏, 𝑐) respectively denote the rate at which flow of SFR 𝑖
solution remains feasible. Let 𝜓̃ 𝑖 denote the updated vector. The change
moves from edge server 𝑎 to local server 𝑏 and moves from local server | |
vector for SFR 𝑖 is 𝛿𝑖 = [𝛿𝑖1 , … , 𝛿𝑖 𝑖 ] where all 𝛿𝑖 are zero except
𝑏 to core server 𝑐. Flow conservation constraint mandates that
𝛿𝑖𝑞 = −𝛿𝑖𝑝 = 𝛿. The total change vector is 𝛥 = [𝛿1 , … , 𝛿|| ]. For small
∑
𝐿
𝜋𝑎,𝑖 = 𝑟𝐿−>𝑅 (𝑖, 𝑎, 𝑏), (19) values of 𝛿, the change in 𝐷 can be first order approximated as
𝑏∈ ∑
∑ 𝐷(𝜓)̃ − 𝐷(𝜓)
̄ = 𝐷𝑖 (𝜓̃𝑖 ) − 𝐷𝑖 (𝜓̄𝑖 ) (28)
𝑅
𝜋𝑏,𝑖 = 𝑟𝐿−>𝑅 (𝑖, 𝑎, 𝑏). 𝑖∈
𝑎∈
𝜕𝐷(𝜓) | 𝑝 𝜕𝐷(𝜓) | 𝑞
≈ ∇𝐷 × 𝛥 = | 𝛿 + | 𝛿
And similarly 𝜕𝜙𝑝𝑖 |𝜓̄ 𝑖 𝜕𝜙𝑞𝑖 |𝜓̄ 𝑖
∑ ( )
𝑅
𝜋𝑏,𝑖 = 𝑟𝑅−>𝐶 (𝑖, 𝑏, 𝑐), (20) 𝜕𝐷(𝜓) | 𝜕𝐷(𝜓) |
= 𝛿 − | + | .
𝑐∈
∑ 𝜕𝜙𝑝 |𝜓̄𝑖 𝜕𝜙𝑞 |𝜓̄ 𝑖
𝐶
𝜋𝑐,𝑖 = 𝑟𝑅−>𝐶 (𝑖, 𝑏, 𝑐).
𝜕𝐷(𝜓) 𝜕𝐷(𝜓)
𝑏∈ Since we assume that 𝑝 | > 𝑞 | , the last term in (28) is negative
𝜕𝜙𝑖 𝜓̄ 𝜕𝜙𝑖 𝜓̄
The delay functions similarly can be written as which means 𝐷(𝜓) ̃ − 𝐷(𝜓)̄ < 0. It contradicts with optimality of 𝜓. ̄
𝜕𝐷 𝜕𝐷
If we repeat the case for 𝑝𝑖 |𝜓̄ < 𝑞𝑖 |𝜓̄ , we obtain the same results.
DU 1 𝐿 𝜕𝜙𝑖 𝜕𝜙𝑖
𝐷𝑖,𝑎 = ∑ 𝐿
, if 𝜋𝑎,𝑖 > 0. (21) Therefore, at the optimal point, (27) is valid. ■
𝑆𝑎𝐿 × 𝐹𝑎𝐿 − 𝑖∈ 𝜋𝑎,𝑖
CU 1 𝑅 Theorem 1 suggests an iterative gradient based algorithm as listed
𝐷𝑖,𝑏 = ∑ , if 𝜋𝑏,𝑖 > 0. (22)
𝑆𝑏𝑅 × 𝐹𝑏𝑅 − 𝑖∈ 𝜋𝑏,𝑖
𝑅 in Algorithm 1. The algorithm iterates on all the SFRs. For each SFR,
1 we deduct rate from a path with the highest derivative and we add it
Core 𝐶
𝐷𝑖,𝑐 = ∑ 𝐶
, if 𝜋𝑐,𝑖 > 0. (23) to the path with the lowest derivative. It makes sure that it is a descent
𝑆𝑐𝐶 × 𝐹𝑐𝐶 − 𝑖∈ 𝜋𝑐,𝑖
direction which results in lower value of the objective function.
Assuming constant transmission delay for the link between data centers.
Let 𝜀𝐿−>𝑅 (𝑎, 𝑏) and 𝜀𝑅−>𝐶 (𝑏, 𝑐) denote the cost of using links from local
Algorithm 1: Gradient Based Minimum Delay (GBMD) Algorithm
data center 𝑎 to regional data center 𝑏 and from regional data center 𝑏
to core data center 𝑐, respectively. The expected delay for SFR 𝑖 is the 1 Start with a feasible solution for 𝜙𝑝𝑖 , ∀𝑖 ∈ , 𝑝 ∈ 𝑖 . while Not
average delay on all paths which is obtained as Converged do
2 for any SFR 𝑖 ∈ do
𝐷𝑖 = 𝜀𝐿−>𝑅 (𝑎, 𝑏)𝑟𝐿−>𝑅 (𝑎, 𝑏) + 𝜀𝑅−>𝐶 (𝑏, 𝑐)𝑟𝑅−>𝐶 (𝑏, 𝑐) 3 Find path 𝑝 of SFR 𝑖 with highest 𝜕𝐷𝑝 ,
𝐿 𝜙𝑖
∑ 𝜋𝑎,𝑖 𝜕𝐷
+ ∑ 4 Find path 𝑞 of SFR 𝑖 with minimum 𝑞 ,
𝐿 𝐿 𝐿 𝜙𝑖
𝑎 𝑆𝑎 × 𝐹𝑎 − 𝑖∈ 𝜋𝑎,𝑖
5 if 𝜙𝑝𝑖
> 0,
𝑅
∑ 𝜋𝑏,𝑖 6 Set: 𝜙𝑝𝑖 = 𝜙𝑝𝑖 − 𝛿.
+ 𝑅 𝑅 ∑ 𝑅 7 Set: 𝜙𝑞𝑖 = 𝜙𝑞𝑖 + 𝛿.
𝑏 𝑆𝑏 × 𝐹𝑏 − 𝑖∈ 𝜋𝑏,𝑖
𝐶 8 Check if new values of 𝜙𝑝𝑖 and 𝜙𝑞𝑖 are feasible. If not,
∑ 𝜋𝑐,𝑖
reverse last steps.
+ ∑ 𝐶
. (24)
𝑐 𝑆𝑐𝐶 × 𝐹𝑐𝐶 − 𝑖∈ 𝜋𝑐,𝑖 9 end
10 end
The minimum delay problem can be written as
∑
min 𝐷total = 𝐷𝑖 (25)
𝜋,𝑟 GBMD algorithm has polynomial time complexity and can be solved
𝑖∈
6
N. Kazemifard and V. Shah-Mansouri Computer Networks 188 (2021) 107809
Table 2
Data centers resource settings.
DC type Processing capacity (no. of CPU cores) Transmission rate (Gbps) Storage (TB)
local 1000 1000 2000
Regional 5000 1000 3000
Core 8000 5000 4000
Fig. 3. The aggregated end-to-end delay versus number of SFRs for base, 1.2 times Fig. 5. The aggregated End-to-End delay versus variable amount of data center
base, and 1.5 times base capacity of DCs. resources.
add 900 GBs to the total capacity. The results are shown in Fig. 5 .
Increasing the capacity of the data centers reduces the aggregate End-
to-End delay substantially. This suggests that by marginally increasing
the capacity, one exponentially gains in lower End-to-End delay. Since
CAPEX and OPEX is associated by increasing the amount of resources,
there is inherently a tradeoff between delay and cost in choosing the
right amount of data center resources.GBMD algorithm has a direct
impact on resource allocation by decreasing End-to-End delay up to 72
percent.
Supporting different QCIs (Quality Class Identifier), which is a
mechanism used in 3GPP Long Term Evolution (LTE) networks to
ensure bearer traffic is allocated appropriate QoS, is important for
network providers. We consider QoS level for guaranteed bit rate (GBR)
services. A comparison between best effort and QoS based services
shows that adding QoS will increase End-to-End Delay in case of limited
resources. This is a trade-off between providing QoS based services for
high value customers and providing best effort services for all levering
GBMD algorithm. The simulation result is shown in Fig. 6.
Fig. 4. The aggregated End-to-End delay versus variable number of SFRs for different 8. Conclusion
amount of data center resources.
Architectural transformation is an inevitable step towards meeting
5G use case requirements. In this paper, End-to-End virtualization in
delay. Single path refers to the problem modeled in (15). We compare it access, IPBB, and core network using local, regional, and core data cen-
with GBMD algorithm in Algorithm 1. GBMD algorithm achieves lower ters is studied. It is a solution to reduce the End-to-End delay of service
delay since it has a wider feasible region compared to the single path requests by offering flexibility and increasing chance of selecting proper
algorithm. For a fixed amount of data center resources, the delay of resources. O-RAN proposes the use of open and virtualized RAN. In O-
single path increases exponentially while it is more close to linear for RAN, different layers of 5G RAN can run on different data centers at
GBMD Fig. 4 . different locations as VNFs/CNFs. A chain of those VNFs/CNFs provides
For the next simulation results shown in Fig. 5, we vary the data the basic telecom connectivity for a 5G network. A hierarchical three
center capacity. As it is mentioned in Table 2, we consider different layer data center architecture is deployed. To support multi-tenancy,
base values for Local, Regional, and Core data Centers. For this sim- a new chain of CNFs is created for each SFR. It provides isolation
ulation, we increase the total amount of capacity for all layers. More between the service management. We first model the CNF placement
precisely, at each step, we add 100 GB to the processing capacity of and resource allocation problem for the case that the traffic of an SFR
each data center at all layers. In this simulation, there are four local, traverses one chain via a single traffic path. Since it is not tractable,
four regional and one core data centers. Therefore, in each step, we we model the case that the traffic of an SFR can traverse multiple
7
N. Kazemifard and V. Shah-Mansouri Computer Networks 188 (2021) 107809
References
Nasim Kazemifard received the B.Sc., M.sc., and Ph.D
Degrees in Computer Architecture from Shahid Beheshti
[1] W. Kiess, X. An, S. Beker, Software-as-a-Service for the virtualization of mobile University of Tehran, and Tehran Science and Research
network gateways, in: IEEE Global Communications Conference, San Diego, CA, branch of Islamic Azad University, in 2005, 2008, and 2016
Dec. 2015. respectively. She primarily focused on Image Processing,
[2] L.J. Vora, Evolution of mobile generation technology: 1G to 5G and review of NoC (Network on chip), and QCA (Quantum Cellular Au-
upcoming wireless technology 5G, Int. J. Mod. Trends Eng. Res. 2 (10) (2015) tomata) during her study. From 2007, she started her career
281–290. in Telecom Operators focusing on Next Generation Net-
[3] E. Coronado, S.N. Khan, R. Riggio, 5G-empower: A software-defined networking works, Fiber to the X, and IPTV. Nasim Kazemifard, Ph.D, is
platform for 5G radio access networks, IEEE Trans. Netw. Serv. Manag. 16 (2) currently technical Knowledge manager in CTO department
(2019) 715–728. of MCI, the first Mobile Operator in Iran. Her research
[4] Md. Faizul Bari, Resource Orchestration in Softwarized Networks (Ph.D. thesis), interests include Network Architecture, Cloudification, 5G,
University of Waterloo, Waterloo, Ontario, Canada, 2018. NFV, SDN, and IoT.
[5] W. Xia, T.Q.S. Quek, J. Zhang, S. Jin, H. Zhu, Programmable hierarchical C-RAN:
From task scheduling to resource allocation, IEEE Trans. Wireless Commun. 18
(3) (2019) 2003–2016. Vahid Shah-Mansouri (S’02–M’13) received the B.Sc. de-
[6] M.D. Mauro, G. Galatro, M. Longo, F. Postiglione, M. Tambasco, Availability gree in electrical engineering from University of Tehran,
analysis of ip multimedia subsystem in cloud environments, in: 4th International Tehran, Iran, in 2003, the M.Sc. degree in electrical en-
Conference on System Reliability and Safety, ICSRS, Rome, Italy, 2019. gineering from Sharif University of Technology, Tehran,
[7] V.G. Nguyen, A. Brunstrom, K.J. Grinnemo, J. Taheri, SDN/NFV-based mobile Iran, in 2005, and the Ph.D. degree from The University
packet core network architectures: A survey, IEEE Commun. Surv. Tutor. 19 (3) of British Columbia, Vancouver, BC, Canada, in 2011. Since
(2017) 1567–1602. 2013, he has been an Assistant Professor with the School of
[8] S. Li, L.D. Xu, S. Zhao, 5G Internet of Things: A survey, J. Ind. Inf. Integr. 10 Electrical and Computer Engineering, University of Tehran,
(2018) 1–9. Tehran, Iran. His research interests include analysis and
[9] Q. Duan, N. Ansari, M. Toy, Software-defined network virtualization - An mathematical modeling of communication and computer
architectural framework for integrating SDN and NFV for service provisioning networks and next generation cellular systems.
in future networks, IEEE Netw. 30 (5) (2016) 10–16.