applied

sciences
Article
QoS-Guaranteed Radio Resource Management in
LTE-A Co-Channel Networks with Dual Connectivity
Ren-Hung Hwang * , Min-Chun Peng and Kai-Chung Cheng
Department of Computer Science & Information Engineering, National Chung Cheng University,
Chiayi 600, Taiwan
* Correspondence: rhhwang@cs.ccu.edu.tw; Tel.: +886-5-272-9396




Received: 26 June 2019; Accepted: 25 July 2019; Published: 26 July 2019


Featured Application: Dual connectivity increases data rate of user equipment (UE), suitable for
multimedia applications in cellular networks.

Abstract: Dual connectivity (DC) was first proposed in 3GPP Release 12 which allows one piece
of user equipment (UE) to connect to two base stations in heterogeneous networks (HetNet) at
the same time, to increase the flexibility of resource utilization. DC has been further extended to
multiple connectivity in 5G New Radio (NR). On the other hand, different UE tends to have different
bandwidth requirements. Thus, in DC, one of the challenging issues is how to integrate resources
from two base stations to enhance the quality of service (QoS) as well as the data transfer rate of
each UE. In this paper, we proposed novel resource management mechanisms to improve the QoS
of UE in the co-channel dual connectivity network. In terms of resource allocation, we designed
the Max–Min Threshold Scheduler (MTS) which, in principle, allocates a resource block to the UE
with the best channel quality while considering the issues of intercell resource allocation and the
QoS requirement of each UE. In order to balance the load of different cells, we designed a novel
cell selection scheme based on the HetNet Congestion Indicator (HCI) which considers not only the
signal quality of UE but also the remaining resources of each base station. To improve the QoS of cell
edge UE, cell range expansion (CRE) and the Almost Blank Subframe (ABS) were proposed in 3GPP.
In this paper, based on Q-learning, we designed an adaptive mechanism which dynamically adjusts
the ABS ratio according to the network condition to improve resource utilization. Our simulation
results showed that our MTS scheduler was able to achieve a 31.44% higher data rate than the
Proportional Fairness Scheduler; our HCI cell selection scheme yielded a 2.98% higher data rate than
the signal-to-interference plus noise ratio (SINR) cell selection scheme; the QoS satisfaction rate of
our Q-learning dynamic ABS scheme was 4.06% higher than that of the Static ABS scheme. Finally,
for the cell edge users who often suffer poor data transfer rate, by integrating the mechanisms of DC,
CRE, and ABS, our experimental results showed that the QoS satisfaction ratio of cell edge UEs could
be improved by 10.76% as compared to the single connectivity and no ABS situation.

Keywords: heterogeneous network; dual connectivity; quality of service; resource allocation;

cell selection

1. Introduction
Due to the rapidly growing mobile data traffic, 4G is not able to meet the current traffic demand.
With the limited wireless radio resource, dual connectivity (DC) technology and heterogeneous network
(HetNet) architecture can improve the data transfer rate beyond 4G. DC is based on carrier aggregation
(CA) and coordinated multi-point (CoMP), allowing user equipment (UE) to connect two different
base stations at the same time, which are referred to as Master eNodeB (MeNB) and Secondary

Appl. Sci. 2019, 9, 3018; doi:10.3390/app9153018 www.mdpi.com/journal/applsci

Appl. Sci. 2019, 9, 3018 2 of 22

eNodeB (SeNB) [1]. Recently, DC has been further extended to include multiple connectivity in 5G
New Radio (NR) [2]. The research studies on DC have attracted quite a lot of attention, e.g., in [3–5],
authors have explored the potential benefits and challenges, including buffer status reporting, power
headroom calculation, discontinuous reception, logical channel prioritization, bearer split, and so on.
In [6], authors indicated that MeNB and SeNB have different scheduling requirements, proposing the
coordination mechanism to prevent the user equipment from using beyond its maximum transmission
power. In [7], a study of DC was divided into control plane and user plane issues. Using the channel
state information reference signal (CSI-RS), which is unique to the Multi-input Multi-output (MIMO)
frame, SeNB no longer deals with handover issues and MeNB can update the Radio Resource Control
(RRC) of the user equipment to achieve handover between the different base stations. In summary,
DC can effectively improve throughput in HetNet by optimal resource allocation [8]. Thus, in this
paper, we focused on DC combined with the radio resource management to integrate resources across
base stations and do the best to satisfy the bandwidth requirements of UEs. We explored three
research topics in DC, including resource allocation, cell selection, and enhanced inter-cell interference
coordination (e-ICIC).
In the study of the resource allocation, most commonly adopted resource scheduling mechanisms are
Round Robin Scheduler (RR), Best Channel Quality Indicator (CQI) Scheduler (BCQI), and Proportional
Fair Scheduler (PF) [9–11]. RR, which is simple to implement, assigns equal portions of resource blocks
(RB) to each UE in a circular order. However, the disadvantage is that this scheduler does not consider
channel condition and can lead to low system throughput. In contrast, BCQI assigns resource blocks
based on the channel condition. Although the system throughput is increased, it may cause UEs with
low CQI suffer starvation. PF is a compromise between throughput and fairness, based on the balance
of the competing benefit among the UEs, trying to maintain fairness while maximizing throughput.
Therefore, in order to emphasize the system throughput and fairness, most of the literature adopted
Proportional Fair scheduling. In [12], the authors use Karush–Kuhn–Tucker optimization conditions to
reduce the complexity of Proportional Fair, assuming that the transmission power of the base station is
evenly distributed on each subcarrier. In [13], the geometric mean is used to replace the arithmetic
mean in the Proportional Fair, which yields quicker convergence and improves throughput and block
error rate (BLER). In DC, since a UE could connect to two base stations at the same time, the co-channel
interference factors must also be considered. Therefore, the Proportional Fair is not necessarily the best
scheduler. We proposed to estimate the channel quality of each subcarrier and assign resource block to
the UE with the best CQI while considering how to integrate resources from two stations to enhance
QoS as well as data transfer rate of each UE.
In the study of the cell selection, the 3GPP defines that the basic cell selection principle of the
user equipment is done according to the received reference signal reception power (RSRP). The higher
the transmission power of the base station, the wider the range of signals that can be covered and the
more UEs will connect to that base station [14,15]. Many studies indicated that the RSRP is conducive
to large base stations (e.g., Macro BS), but not obvious for small cells (e.g., pico BS). In [16], the cell
range expansion (CRE) scheme is investigated, in which the coverage of pico BS is extended by adding
additional reference signal such that more UEs could select the pico BS rather than Macro BS. However,
increasing the CRE bias value alone does not improve the performance of the small cell, as the cell edge
users are significantly affected by the intercell interference. In [17], the serving cell is selected according
to the signal-to-interference plus noise ratio (SINR) which considers not only the transmission power
of the base station, but also the interference between the base stations. Many other factors are also
considered in cell selection in the literature, such as load balance, utility function of proportional fair,
dynamic strategy, and data transfer rate [18–21]. In this paper, based on the special features of DC,
we consider load balancing and downlink data transfer rate in the cell selection. Specifically, we first
estimate CQI based on SINR and then estimate the number of resource blocks that a UE can be assigned,
and finally calculate the actual data transfer rate.
Appl. Sci. 2019, 9, 3018 3 of 22

Appl. Sci. 2019, 9, x 3 of 22

In the study of the enhanced intercell interference coordination, some previous works took the
the Macro
Almost BlankBS Subframe
were preserved for the
(ABS) into pico BS to achieve
consideration time
[22,23], in domain
which coordination.
a ratio of time slotsMost of previous
of a frame of the
works adopted static ABS setting and showed that the ratio of 0.5 yielded
Macro BS were preserved for the pico BS to achieve time domain coordination. Most of previous works the best performance.
However,
adopted it may
static ABS waste a lotand
setting of wireless
showedresources of the
that the ratio ofMacro BS. Inthe
0.5 yielded thisbest
paper, we proposed
performance. a better
However,
method to adjust the ABS ratio dynamically according to the network conditions.
it may waste a lot of wireless resources of the Macro BS. In this paper, we proposed a better method to It would decrease
the ABS
adjust theratio
ABSwhenratio the Macro BSaccording
dynamically is overloaded,
to theand increase
network the ABS ratio
conditions. to improve
It would decrease the throughput
the ABS ratio
when the Macro BS is overloaded, and increase the ABS ratio to improve the throughput theoretical
of pico BS when the load of Macro BS is low. Simulation results are given to substantiate of pico BS
findings
when the with
load aofcomparison
Macro BS is withlow. fixed ABS configuration.
Simulation results are given to substantiate theoretical findings with
Besides, there
a comparison are various
with fixed types of small cell and UEs with different bandwidth requirements in
ABS configuration.
HetNet. Mostthere
Besides, literature focused
are various on single
types of smalltraffic, which
cell and UEsis with
not able to reflect
different the real-world
bandwidth situation
requirements in
[24]. Thus, in this work, multiple types of applications with different QoS requirements
HetNet. Most literature focused on single traffic, which is not able to reflect the real-world situation [24]. were
considered.
Thus, in this By work,considering voice of
multiple types traffic and video
applications traffic,
with we classify
different two types of
QoS requirements theconsidered.
were traffic into
different QoS indicators: Guaranteed bit rate (GBR) and maximum bit
By considering voice traffic and video traffic, we classify two types of the traffic into different rate (MBR). The QoS
QoS
requirement
indicators: of GBR is bit
Guaranteed a guaranteed
rate (GBR) and fix transmission
maximum bitrate, while that
rate (MBR). The of
QoSMBR is a guaranteed
requirement of GBR of is
a guaranteed fix transmission rate, while that of MBR is a guaranteed of minimum rate and aschemes
minimum rate and a limit of maximum rate. The numerical results show that our proposed limit of
had significant
maximum rate. improvement
The numericalinresults the QoSshowsatisfaction ratio thanschemes
that our proposed previoushadworks in the literature.
significant improvement
in theThe
QoSrest of this paper
satisfaction ratioisthan
organized
previousas follows.
works inSection 2 introduced system design. The proposed
the literature.
Max–min Threshold Scheduler (MTS), HetNet Congestion Indicator (HCI), and Q-learning dynamic
The rest of this paper is organized as follows. Section 2 introduced system design. The proposed
ABS will be described in detail in Section 3. The system deployment and the simulation results
Max–min Threshold Scheduler (MTS), HetNet Congestion Indicator (HCI), and Q-learning dynamic
present in Section 4. Finally, in Section 5, conclusion and future works are discussed.
ABS will be described in detail in Section 3. The system deployment and the simulation results present
in Section 4. Finally, in Section 5, conclusion and future works are discussed.
2. System Model
2. System Model
2.1. Co-Channel Networks with Dual Connectivity
2.1. Co-Channel Networks with Dual Connectivity
For HetNets, the wireless radio resources not only determine the quality of the network but also
determine whether
For HetNets, thethe bandwidth
wireless radio requirements of UEs
resources not only are satisfied.
determine In theoflimited
the quality wireless
the network butradio
also
resources, whether
determine how to effectively
the bandwidthimprove the spectrum
requirements of UEs utilization is very
are satisfied. important;
In the therefore,
limited wireless co-
radio
channel HetNet
resources, how toiseffectively
currently improve
the mosttheefficient approach,
spectrum meaning
utilization that
is very Macro BS
important; and picoco-channel
therefore, BS co-use
the same
HetNet is frequency.
currently theHowever, the challenge
most efficient approach, of meaning
the co-channel network
that Macro is the
BS and picointerference between
BS co-use the same
each base station.
frequency. Based
However, theon the different
challenge downlink
of the transmission
co-channel network ispower of the basebetween
the interference station ineach
HetNets,
base
the low Based
station. poweron pico
theBS with the
different distancetransmission
downlink closer to the power
Macro of BSthe
willbase
suffer the greater
station interference,
in HetNets, the low
and lead
power picoto BS
thewith
decrease of the signal
the distance closercoverage of the
to the Macro BSpico
willBS, as shown
suffer in Figure
the greater 1 [25]. and lead to
interference,
the decrease of the signal coverage of the pico BS, as shown in Figure 1 [25].

Figure 1. Pico base station (BS) coverage area in the presence of a Macro BS.
Figure 1. Pico base station (BS) coverage area in the presence of a Macro BS.
In this paper, we combine the dual connectivity technology with CRE and ABS by integrating
resources from two base stations to enhance the data transfer rate of each UE. For example, in 3GPP
In this paper, we combine the dual connectivity technology with CRE and ABS by integrating
co-channel network scenario [1] as shown in Figure 2, it can be observed that UE 1 and UE 3 are close
resources from two base stations to enhance the data transfer rate of each UE. For example, in 3GPP
to the pico BS and Macro BS, respectively, and can receive better CQI than the UE 2. On the other
co-channel network scenario [1] as shown in Figure 2, it can be observed that UE 1 and UE 3 are close
hand, UE 2 is located at the signal junction of the base station and has a worse CQI due to the greater
to the pico BS and Macro BS, respectively, and can receive better CQI than the UE 2. On the other
interference. In order to enhance the data transfer rate, UE 2 with dual connectivity can access the
hand, UE 2 is located at the signal junction of the base station and has a worse CQI due to the greater
resources from Macro BS in non-ABS situations and can also access the resources from pico BS in ABS
interference. In order to enhance the data transfer rate, UE 2 with dual connectivity can access the
situations (see part (a) in Figure 2) where non-ABS means the ABS mechanism is not enabled.
resources from Macro BS in non-ABS situations and can also access the resources from pico BS in ABS
situations (see part (a) in Figure 2) where non-ABS means the ABS mechanism is not enabled.
Appl. Sci. 2019, 9, 3018 4 of 22
Appl. Sci. 2019, 9, x 4 of 22

We
Welearned
learnedthat
thatthe
thedifference
differencebetween
betweenthe thedual
dualconnectivity
connectivityandandthe
the original
original single
single connectivity
connectivity
(SC)
(SC) is that some UEs, such as UE 2, can access radio resources from the Macro BSBS
is that some UEs, such as UE 2, can access radio resources from the Macro and
and thethe pico
pico BS
BS simultaneously. In order to facilitate the management, as shown in Figure 2b,
simultaneously. In order to facilitate the management, as shown in Figure 2b, we define two we define two
categories
categories of
of the
the user
user equipment,
equipment, namely
namely Macro
Macro user
user equipment
equipment (MUE)
(MUE) and
and Pico
Pico user
user equipment
equipment
(PUE). The characteristics of each category are described as follows.
(PUE). The characteristics of each category are described as follows.
•• Macro
Macrouser
userequipment (MUE):Only
equipment(MUE): Onlyaccess
accessthe
theradio
radioresources
resourcesfrom
fromthetheMacro
MacroBS,
BS,and
andcannot
cannot
access
access the radio resources from the pico BS. Therefore, in our proposed mechanisms, wegive
the radio resources from the pico BS. Therefore, in our proposed mechanisms, we will will
the MUE
give a higher
the MUE priority
a higher than the
priority PUE
than thein
PUEthein
resource allocation
the resource of the of
allocation Macro BS. BS.
the Macro
•• Picouser
Pico userequipment
equipment(PUE):
(PUE):Besides
Besidesaccessing
accessingthetheradio
radioresources
resources from
from the
the pico
pico BS,
BS, aa UE
UE may
may
alsoaccess
also accessthe
theradio
radioresources
resourcesfrom
fromthe
theMacro
MacroBS.BS.

Figure2.2. Dual
Figure Dual connectivity
connectivity conditions.
conditions.
2.2. Optimal Resource Allocation
2.2. Optimal Resource Allocation
In LTE-A, the basic schedule unit is a resource block (RB). One RB consists of 12 subcarriers and
In LTE-A, the basic schedule unit is a resource block (RB). One RB consists of 12 subcarriers and
a time domain, and the base station may calculate the data transfer rate of an RB when it is allocated
a time domain, and the base station may calculate the data transfer rate of an RB when it is allocated
to a specific UE. Due to the difference in channel quality, the signal status of different UE is different,
to a specific UE. Due to the difference in channel quality, the signal status of different UE is different,
so the data transfer rate will be affected by the modulation and coding rate (see Table 1).
so the data transfer rate will be affected by the modulation and coding rate (see Table 1).
Table 1. Channel Quality Indicator (CQI) table.
Table 1. Channel Quality Indicator (CQI) table.
CQI Index Modulation Code Rate × 1024 Efficiency
CQI Index Modulation Code Rate × 1024 Efficiency
0 out of range
0 out of range
1 QPSK 78 0.1523
12 QPSK
QPSK 78
120 0.1523
0.2344
23 QPSK
QPSK 120
193 0.2344
0.3770
34 QPSK
QPSK 193
308 0.3770
0.6016
4 QPSK 308 0.6016
5 QPSK 449 0.8770
Appl. Sci. 2019, 9, 3018 5 of 22

Table 1. Cont.

CQI Index Modulation Code Rate × 1024 Efficiency

5 QPSK 449 0.8770
6 QPSK 602 1.1758
7 16QAM 378 1.4766
8 16QAM 490 1.9141
9 16QAM 616 2.4063
10 64QAM 466 2.7305
11 64QAM 567 3.3223
12 64QAM 666 3.9023
13 64QAM 772 4.5234
14 64QAM 873 5.1152
15 64QAM 948 5.5547

The impact of the resource scheduling on the data transfer rate is very significant. Since all UEs
share bandwidth of a base station, to optimize the system throughput it is intuitive to assign an RB
to the UE with the best channel quality. That is, we formulate a throughput optimization problem
as follows.
XX K
max TPu,k ∗ Xu,k (1)
Xu,k
u∈U k=1

where u is a UE belongs to the UE set U = {u1 , . . . , uU }, k is a RB of the RB set K = {1, 2, . . . , K}, TPu,k is
the throughput of RB k when it is allocated to the UE u, and Xu,k represents whether RB k is assigned
to UE u. TPu,k can be calculated by 12 (subcarriers) * 7 (symbols) * CQIe f f iciency . There is a positive
correlation between TPu,k and CQIe f f iciency ; however, CQIe f f iciency is not a linear function of CQIindex ,
which makes the objective function nonlinear. Therefore, we replace CQIe f f iciency with CQIindex and
make the optimization problem (1) to a 0–1 integer programming problem as follows:
X XK
max CQIu,k ∗ Xu,k . (2)
Xu,k k =1
u∈U

Subject to: X
Xu,k ≤ 1∀k (3)
u∈U

Xu,k ∈ { 0, 1}∀u, k (4)

where Equation (3) constrains each RB can only be assigned to a UE, and Equation (4) states that Xu,k
is an indication variable.
Two common approaches of solving 0–1 linear programming problems are “implicit enumeration”
and “branch and bound”. Unfortunately, the computational complexity of these two methods is
exponential time. In order to reduce the computational complexity, we can first relax the 0–1 integer
programming to linear programming. The relaxed linear programming can then be solved by many
existing solutions, such as “simplex method”, “ellipsoid method”, and “interior point method”.
For example, in MATLAB, the linprog library is provided which can obtain the optimal solution by the
“internal point method” according to the scale of the optimization problem.
After obtaining the optimal solution from linear programming, two methods can be used to
transfer the solution back to the 0–1 integer programming solution.

• Max method: For RB k, the sum of all variables is at most 1 under constraint (3). The max
method sorts these variables in descending order, assigns 1 to the variable with the largest value
(i.e., allocate the RB to the UE) and 0 to the rest variables.
P
• Random method: For RB k, let N = Xi,k . The random method allocates RB k to UE i with
Xi,k
probability pi where pi = N (i.e., set Xi,k =1 and Xu,k = 0 ∀u , i).
Appl. Sci. 2019, 9, 3018 6 of 22

As aforementioned, we will propose a max–min RB allocation method which will be described in

detail in the next section. In the following, we performed experiments to compare the performance
of three RB allocation methods, namely, the max method, the random method, and the max–min
method. We compared three performance metrics: The similarity between the optimal solution (of the
0–1 integer programming) and that of these methods, the difference of objective function (1) and
(2) of the optimal solution, and that of these methods. The similarity was defined as the pair-wise
comparison of each variable of the optimal solution, and that of the three methods. For example, if the
system had 50 RBs, and allocation of 40 RBs were the same between two methods, we would define the
similarity as 80%. For the objective function, we compared the degradation of the objective function
of one method as compared to the optimal solution. For example, 10% means the objective function
obtained by one approximate method only achieves 90% of that of the optimal solution.
The system parameters of the experiments are shown in Table 2, in which the system has seven
Macro BSs, each Macro BS contains three sectors, and each sector deploys five user equipments. We
simulate 500 TTIs, and for each TTI, the optimal solution as well as the solutions of the three methods
were calculated for each sector (i.e., 21 sectors).

Table 2. Simulation parameter of the optimal resource allocation.

Parameters Configuration
Simulation time 500 TTI
Cellular layout Wrap around, 7 Macros, 3 sectors per site
Carrier frequency 2.14 GHz
Bandwidth 10 MHz
Number of RBs 50
Number of UEs per sector 5
Macro transmit power 46 dBm
Thermal noise −174 dBm/Hz
Pathloss model As in 3GPP TS 36.942

Table 3 shows the experimental results. As can be observed, random method did not perform well,
neither yielded good similarity to the optimal solution nor achieved good objective value. The max
method yielded the closest solution to the optimal solution. On the other hand, the max–min method
also yielded a very competitive solution to the max method, in particular, both methods yielded very
high objective value with less than 1% degradation. However, the max-min method had much less
computation complexity as compared to the other methods which need to solve 0–1 integer linear
programming or linear programming problem. In summary, the max-min method is a good candidate
mechanism for RB allocation.

Table 3. Experimental results.

Similarity Degradation of Obj. Func. (1) Degradation of Obj. Func. (2)

Max method 93.96% 0.43% 0.67%
Random method 36.03% 37.99% 48.42%
Max–Min method 87.62% 0.47% 0.83%

2.3. Guarantee Quality of Service (QoS)

In order to evaluate the system performance such as throughput, most literature assume UEs
have full buffer. However, in reality, each UE has its own traffic characteristic and quality of service
requirement. Allocating resources more than a UE’s needs does not increase system throughput. In this
work, we considered two types of traffic, each with its own bandwidth requirement. By considering
voice traffic and video traffic, we classified two types of the traffic into two QoS categories: Guaranteed
bit rate (GBR) and maximum bit rate (MBR). The characteristics of each QoS requirement are described
as follows [26]:
Appl. Sci. 2019, 9, 3018 7 of 22

• Guaranteed bit rate (GBR): Some applications, such as audio, require a fixed amount of bandwidth
continuously during their lifetime. Thus, the traffic characteristic as well as QoS requirement of
this kind of traffic is a constant data rate.
• Maximum bit rate (MBR): Yet, other types of applications, such as video, require a variable
amount of bandwidth. The traffic may be bursty, and is usually characterized by a peak data
rate and a sustainable data rate. Thus, we set the QoS requirement of MBR with two parameters,
namely a minimum of the sustainable data rate (or average data rate) and a peak (maximum)
data rate.

We took the QoS requirement into consideration while scheduling RBs. That is, when a base
station allocates RBs to UEs under its coverage, it will try its best to guarantee the QoS of each UE
under the constraint of limited RBs. Specifically, to provide QoS guarantee to UE u, the number of RBs
to be allocated depends on its QoS requirement and channel condition. For GBR traffic, the required
data rate is its guaranteed data rate while for MBR traffic, it is its minimum data rate. The number of
required RBs for UE u can be calculated by following equation [24]:
& '
UEu,QoS
NRBu = (5)
RBdata_rate

where UEu,QoS is QoS requirement for UE u, RBdata_rate is data rate of a RB, and NRBu is the minimum
number of the required RBs to satisfy the QoS of UE u. If the base station has abundant RBs, it will not
allocate more RBs to UEs with GBR traffic, instead, it will allocate to UEs with MBR traffic, but only up
to their peak data rate.

3. Proposed Schemes
The radio resource management scheme of base stations affects the performance of applying dual
connectivity in HetNets. In this section, we propose three novel schemes in resource scheduling, cell
selection, and enhanced intercell interference coordination, which are named Max—min Threshold
Scheduler (MTS), HetNet Congestion Indicator (HCI), and Q-learning Dynamic Almost Blank Subframe
(QD-ABS), respectively. The details for these schemes are outlined in the following.

3.1. Max–Min Threshold Scheduler

The MTS approach can be split into two different views: max–min scheduling and Threshold
restrictions. The former is responsible for optimizing the resource allocation, the latter limits the
allocation of RBs according to the environmental conditions with dual connectivity.

3.1.1. Max–Min Scheduling

For one RB, the achievable data transfer rate can be calculated by base station when the RB is
assigned to a UE. In general, the base station will allocate an RB to the UE with the best channel
quality while considering the issues of intercell resource allocation and the UE’s QoS requirement.
We assumed that there are N RBs within a time slot and M UEs connected to the base station. Based on
UE’s CQI, the base station constructs an NxM matrix A as shown in Figure 3. The max–min scheduling
is then used to assign RBs to UEs to satisfy QoS of UEs as much as possible.
The max–min scheduling consists of five steps:

(1) Find the maximum (max) of the CQI value in the matrix A, and assume it is A(i, j). It assigns the
i-th RB to the j-th UE so that the UE can receive the highest data rate with this RB.
(2) When there are more than one maximum CQIs in the matrix A, it selects the RB with minimum
(min) impact to the other UEs in the same row. For example, if A(i1, j1) and A(i2, j2) are the
same maximum in the matrix A, then it calculates s(i1) = M
P PM
j=1 A(i1, j) and s(i2) = j=1 A(i2, j),
respectively. It then assigns the i1-th RB to the j1-th UE first if s(i1) < s(i2).
 Appl. Sci. 2019, 9, 3018 8 of 22

Appl. Sci. 2019, 9, x 8 of 22

(3) If s(i1) = s(i2), it sorts the vector A(i1,*) and A(i2,*) in descending order, compares the elements
(4) If all theone by one until
elements areit still
findsthe
the one withitless
same, randomly it is As (i1,j).
value, sayselects one,Itsay
thenA(i1,
assigns i1-thassigns
theand
j1), RB to thethe RB to
s
j-th UE (where A (i1,*) is the sorted vector).
a UE accordingly.
(4) If all the elements are still the same, it randomly selects one, say A(i1, j1), and assigns the RB to
(5) After allocating the i-th RB, the i-th row of the matrix A is cleared to 0 (indicating that the RB
a UE accordingly.
(5) After allocating the i-th RB, the i-th row of the matrix A is cleared to 0 (indicating that the RB had
had been allocated).
been allocated).
The max-min scheduling
The max-min repeats
scheduling the
repeats thestep
step from (1)toto
from (1) (5)(5) until
until all ofall
theofUEs
thehave
UEssatisfied
have satisfied
QoS QoS
requirement or all or
requirement ofall
theof RBs hadhad
the RBs been
beenallocated.
allocated.

FigureFigure 3. Table
3. Table of user
of user equipment (UE)’s
equipment (UE)’sCQI
CQIat each resource
at each block (RB).
resource block (RB).
Once the RB allocation is completed, the max–min scheduling then evaluates whether the QoS
Once the RB allocation
requirements is completed,
of the currently assigned UEs the
aremax–min
satisfied. Inscheduling then evaluates
LTE-A, the transport block (TB) whether
is used to the QoS
requirements
calculateofthe
the currently
throughput assigned
of the UEs
downlink. A TBare satisfied.
consists In LTE-A,
of several RBs, andthe transport
all RBs block (TB) is used
must be transmitted
using the
to calculate unified CQI encoding.
throughput Given
of the the number
downlink. A ofTBavailable
consists RBsofthat can beRBs,
several assigned
andtoalltheRBs
UE, must be
it calculates the maximum data rate it can receive from the base station using following equation:
transmitted using unified CQI encoding. Given the number of available RBs that can be assigned to
the UE, it calculates the maximum data∗ RB
TB = Nsymbol rate it can receive from the base station using(6)following
assigned ∗ CQIe f f iciency ∗ Tslot − CRC
equation:
where Nsymbols is the number of symbols per RB, RBassigned is the number of RBs which the UE can be
𝑇𝐵is=the 𝑁
assigned, CQIe f f iciency ∗ 𝑅𝐵
bits per RB, ∗ 𝐶𝑄𝐼 of two time
Tslots is the number ∗𝑇 − 𝐶𝑅𝐶
slots per millisecond, and CRC (6)
is the cyclic redundancy check in 24 bits.
where 𝑁 is the number of symbols per RB, 𝑅𝐵 is the number of RBs which the UE can
3.1.2. Threshold Restrictions
be assigned, 𝐶𝑄𝐼 is the bits per RB, 𝑇 is the number of two time slots per millisecond,
Recall that there are two types of UEs in the system, namely MUE and PUE. Thus, we let
and CRC U is= the
UM cyclic redundancy
∪ UP , where UM and Ucheck in set
P are the 24 of
bits.
MUEs and PUEs respectively. Since MUE can only
access RBs of the Macro BS, the proposed MTS scheme gives higher priority to MUE when allocating
3.1.2. Threshold
RBs of theRestrictions
Macro BS, as illustrated in Figure 4. The MTS adopts a threshold-based scheme which
allocates a certain number of RBs to MUEs before it exercises the max–min scheduling. The basic idea
Recall
is to that theretheare
guarantee QoStwo typesfirst.
of MUEs of It
UEs
thenin the system,
adopts namely
the max–min MUEtoand
scheduling PUE.
allocate the Thus, we let 𝑼 =
remaining
𝑴 𝑷RBs to both MUEs
𝑴 𝑷
𝑼 ∪ 𝑼 , where 𝑼 andand 𝑼 PUEs.
are the set other
On the of MUEs
hand,and PUEs
the pico BS respectively. Since MUE
exercises the max-min can only
scheduling to access
allocate
RBs of the Macrothe BS,
RBs the
to PUEs only. MTS scheme gives higher priority to MUE when allocating RBs of
proposed
the Macro BS, as illustrated in Figure 4. The MTS adopts a threshold-based scheme which allocates a
certain number of RBs to MUEs before it exercises the max–min scheduling. The basic idea is to
guarantee the QoS of MUEs first. It then adopts the max–min scheduling to allocate the remaining
RBs to both MUEs and PUEs. On the other hand, the pico BS exercises the max-min scheduling to
allocate the RBs to PUEs only.
 Appl. Sci. 2019, 9, 3018 9 of 22
Appl. Sci. 2019, 9, x 9 of 22

FigureFigure
4. Macro BS adopts
4. Macro threshold-based
BS adopts threshold-based RB allocationbefore
RB allocation before applying
applying the max-min
the max-min scheduling.
scheduling.

The number of RBs reserved for MUEs is based on solving following optimization problem,
The number of RBs reserved for MUEs is based on solving following optimization problem,
similar to the problem defined in Section 2, but two new constraints are added, as shown in the following.
similar to the problem defined in Section 2, but two new constraints are added, as shown in the
XK
following.
X
max CQI ∗ X . u,k u,k (7)
Xu,k u∈UM k =1

Subject to: max ∑ ∈𝑼𝑴 ∑ 𝐶𝑄𝐼 , ∗ 𝑋 , . (7)

, X
Xu,k ≤ 1 ∀k (8)
u∈UM
Subject to:
K
X
∀u ∈ UM
𝑋 X, u,k ≤
≤ NRBu
1 ∀𝑘 (9)
(8)
k =1
∈𝑼𝑴
CQIu,k ≥ 7 ∀k (10)

Xu,k ∈ { 0, 1} ∀u ∈ UM , k (11)
𝑋 , ≤of allocated
where Equation (9) means that the number
𝑁𝑅𝐵 RBs cannot 𝑼𝑴 the QoS requirement of the
∀𝑢 ∈exceed (9)
MUE, and Equation (10) means that the channel quality must be higher than CQI 7 to avoid wasting
RBs on low CQI MUEs. In summary, the threshold-based mechanism is expected to achieve similar
performance of the max-min𝐶𝑄𝐼 , ≥7
scheduling ∀𝑘 on similar optimization
because both methods are based (10)
objectives. The purpose of the threshold-based mechanism is to protect MUEs to access enough
resources before competing𝑋with ∈ 0, 1
, PUEs. ∀𝑢 ∈ 𝑼𝑴 , 𝑘 (11)

where3.2.
Equation (9) meansIndicator
HetNet Congestion that the number of allocated RBs cannot exceed the QoS requirement of
the MUE, and Equation (10) means that the channel quality must be higher than CQI 7 to avoid
Due to the dual connectivity, each piece of user equipment is connected to the Macro BS.
wasting RBs on lowproblem
The handover CQI MUEs. In summary,
between the HeNB willthebe
threshold-based
processed by themechanism
Macro BS. We is propose
expected to achieve
a new
similarHetNet
performance
Congestion Indicator (HCI) cell selection scheme to achieve the load balance, considering notsimilar
of the max-min scheduling because both methods are based on
optimization
only theobjectives.
signal qualityTheof purpose of also
the UE, but the threshold-based mechanism
the number of remaining is toofprotect
resources MUEs
each base to access
station.
enough In resources
addition to before
the cell competing
selection, wewith
also use HCI to adjust the type of the UE, either MUE or PUE.
PUEs.
3.2.1. Cell Selection Scheme
3.2. HetNet Congestion Indicator
When a UE has more than one BS to join, it selects the one with least HCI. The key idea of HCI is
Due to the the
to estimate dual connectivity,
congestion level ofeach
a basepiece ofbased
station user on
equipment is connected
the competitive to the proposed
online algorithm Macro BS. The
handover problem
in [27]. between
Specifically, thecell
the HCI HeNB willscheme
selection be processed
consists by thesteps:
of two Macro BS. We propose a new HetNet
Congestion Indicator (HCI) cell selection scheme to achieve the load balance, considering not only
the signal quality of the UE, but also the number of remaining resources of each base station. In
addition to the cell selection, we also use HCI to adjust the type of the UE, either MUE or PUE.
Appl. Sci. 2019, 9, x 10 of 22

(1) A UE continuously measures the SINR among neighboring cells. If the SINR of current serving
Appl. Sci. 2019, 9, 3018 10 of 22
BS becomes weaker and some neighboring BSs become stronger, then the UE enters the cell
selection procedure (step 2).
(2) The
(1) UE A UE selects a list ofmeasures
continuously candidatethe BSs
SINRwhich
amonghave higher cells.
neighboring SINRs. For
If the each
SINR of BS, the serving
current UE estimates
BS
becomes weaker and some neighboring BSs become stronger, then the
and reports its Channel Quality Indicator (CQI) to the BS. By taking account of the QoSUE enters the cell selection
procedure
requirement of(step 2). and the status of RB usage, the BS replies to the UE with its HCI which is
the UE
(2) The UE selects
calculated as follows. a listAssume
of candidate
the BSs which have
maximum higherofSINRs.
number Fortime
RBs per each slot
BS, the UE estimates
(system capacity)andis C,
number reports its Channel
of RBs which Quality Indicator
have been used (CQI) to the BS.
(allocated) is RBy taking account of the QoS requirement
i, the number of RBs required by the UE
of the UE and the status of RB usage, the BS replies to the UE with its HCI which is calculated as
(based on its CQI and QoS requirement) is b, the HCI is given by:
follows. Assume the maximum number of RBs per time slot (system capacity) is C, number of
RBs which have been used (allocated) is Ri , the number of RBs required by the UE (based on its
𝐻𝐶𝐼(𝑏) = 𝑐𝑜𝑠𝑡(𝑅 + 𝑏, 𝐶) − 𝑐𝑜𝑠𝑡(𝑅 , 𝐶) (12)
CQI and QoS requirement) is b, the HCI is given by:
where the cost() function is defined as follows.
HCI (b) = cost(Ri + b, C) − cost(Ri , C) (12)
( ) (13)
𝑐𝑜𝑠𝑡(𝑅 , 𝐶) = 𝜇
where the cost() function is defined as follows.
where 𝜇 is a constant parameter which is usually setRto C (in our case, it is set to 50).
i
cost(Ri , C) = µ( C −1) (13)
After obtaining the HCI of each candidate BS, it selects the BS with the least HCI to join. The HCI
function iswhere
meantµ istoaestimate the congestion
constant parameter whichlevel or traffic
is usually set toload
C (in of
oura case,
BS. it is set to 50).
After obtaining the HCI of each candidate BS, it selects the BS with the least HCI to join. The HCI
3.2.2. Classify a UE as a MUE or PUE Type
function is meant to estimate the congestion level or traffic load of a BS.
As mentioned in Section 2, a UE could be either MUE or PUE. If a UE receives signal from both
3.2.2. Classify a UE as a MUE or PUE Type
Macro BS and pico BS, and it selects the Macro BS as its serving BS as the BS has the least HCI, then
the UE is As mentioned
classified as ain Section
MUE; 2, a UE could
otherwise, it is abePUE.
eitherDue
MUE to or
thePUE. If a UEareceives
mobility, UE maysignal
change fromitsboth
serving
Macrowill
BS which BS andalsopico BS, and
change itsit type.
selectsAs theaforementioned,
Macro BS as its serving BS as the BS
the different hasof
type thethe
least
UEHCI,willthen the the
affect
UE is classified as a MUE; otherwise, it is a PUE. Due to the mobility,
priority of accessing resources. Specifically, a MUE can only access resources from the Macroa UE may change its serving BS BS,
which will also change its type. As aforementioned, the different type of the UE will affect the priority
while a PUE can access resources from two BSs simultaneously. Figure 5 illustrates how a UE may
of accessing resources. Specifically, a MUE can only access resources from the Macro BS, while a PUE
enter the dual connectivity range or transits from PUE to MUE. Initially, a UE may be within the
can access resources from two BSs simultaneously. Figure 5 illustrates how a UE may enter the dual
coverage range of a pico BS, but far from the Macro BS. In this stage, it is a PUE. As it moves closer
connectivity range or transits from PUE to MUE. Initially, a UE may be within the coverage range of
to thea Macro
pico BS,BS, butitfar
begins
fromtothereceive
Macro signal fromstage,
BS. In this Macroit isBS and enters
a PUE. the dual
As it moves connectivity
closer to the Macro range.
BS, At
this stage,
it beginsit is
toable to utilize
receive resources
signal from Macro of BSboth BSs. As
and enters theitdual
continues to move
connectivity closer
range. thestage,
At this Macro BS, it
it is
eventually
able to utilize resources of both BSs. As it continues to move closer the Macro BS, it eventually selects it
selects Macro BS as its serving BS based on the HCI cell selection scheme. At this stage,
becomes
Macro a MUE.
BS as its serving BS based on the HCI cell selection scheme. At this stage, it becomes a MUE.

Figure 5. 5.
Figure Signal
Signalcoverage
coveragein
in dual connectivitymode.
dual connectivity mode.

3.3. Q-Learning Dynamic Almost Blank Subframe

3.3. Q-Learning Dynamic Almost Blank Subframe
In order to explore the dual connectivity mechanism combined with CRE and ABS, the ABS
In order to explore
configuration learning the dual connectivity
procedure mechanism
is introduced in this section. combined
We proposewith CRE
a novel and ABS,
Q-learning the ABS
dynamic
configuration learning procedure is introduced in this section. We propose a novel Q-learning
dynamic ABS method based on reinforcement learning. Furthermore, it is effective to adjust the ABS
ratio and reduce the interference in the co-channel networks by coordinating the adjacent Macro BSs.
Appl. Sci. 2019, 9, x 11 of 22

Thus, we also propose a centralized decision to adjust the ABS ratio which requires all Macro BSs to
synchronize their ABS ratio.
Appl. Sci. 2019, 9, 3018 11 of 22

3.3.1. Related Works

ABS method based on reinforcement learning. Furthermore, it is effective to adjust the ABS ratio
Prior and works
reduceontheABS ratio caninbethe
interference classified
co-channel into fixed and
networks dynamic ABS
by coordinating the ratio settings.
adjacent Macro In the fixed
BSs.
dynamicThus,ABS we ratio
alsosetting
proposeschemes,
a centralized ABS ratio is
decision to set to athefixed
adjust value
ABS ratio based
which on some
requires experiments,
all Macro BSs to e.g.,
synchronize their ABS ratio.
[28,29]. On the other hand, the dynamic ABS ratio setting schemes adjust the ABS ratio according to
the system 3.3.1.status,
Relatedsuch
Works as traffic load and number of edge UEs. Quite a few of dynamic ABS ratio
setting schemes are based on reinforcement learning [30–32].
Prior works on ABS ratio can be classified into fixed and dynamic ABS ratio settings. In the fixed
Reinforcement
dynamic ABSlearning is aschemes,
ratio setting machine ABSlearning method
ratio is set which
to a fixed focuses
value based on onsome
issues such as learning
experiments,
with feedback or sequential
e.g., [28,29]. On the otherdecision-making.
hand, the dynamic ABS Theratio
purpose
setting is to letadjust
schemes the machine learn
the ABS ratio actively what
according
to the system
kind of action can bestatus, such aswith
achieved traffic load
the and number
maximal of edge UEs.
feedback, ratherQuite
thana few
tellofthe
dynamic
machineABS ratio
what action
setting schemes are based on reinforcement learning [30–32].
to take [30,31]. Different from the supervised learning, reinforcement learning can occur without a
Reinforcement learning is a machine learning method which focuses on issues such as learning
teacher by repeatedly interacting with the environment. It does not require accurate input and
with feedback or sequential decision-making. The purpose is to let the machine learn actively what
output, emphasizing
kind of action can a be
continuous
achieved with decision-making
the maximal feedback, action of than
rather the tell
online planning.
the machine what For
actionexample,
Figure 6 to illustrates theDifferent
take [30,31]. structurefrom ofthe
ABS learner–environment
supervised interaction.
learning, reinforcement learning It can
is difficult to adjust the
occur without
ABS ratio in the unknown environment, which consists of factors such as the designed cellular layout,
a teacher by repeatedly interacting with the environment. It does not require accurate input and output,
the densityemphasizing a continuous
of the connected UE, decision-making
interference between action ofthe theBSs,
online
andplanning. For example,
so on. However, in Figure
such a6complex
illustrates the structure of ABS learner–environment interaction. It is difficult to adjust the ABS ratio in
situation, reinforcement learning will allow the system to select an arbitrary ABS ratio first, observe
the unknown environment, which consists of factors such as the designed cellular layout, the density
the feedback of whether
of the connected the performance
UE, interference between theis good
BSs, and orsonot, and then
on. However, adjust
in such (learn)situation,
a complex the ABS ratio
according to the feedback.
reinforcement learningThe system
will allow continues
the system to an
to select observe
arbitraryand
ABSadjust until
ratio first, the most
observe suitable ABS
the feedback
ratio is learned.
of whether the performance is good or not, and then adjust (learn) the ABS ratio according to the
feedback.isThe
Q-learning system continues
a well-known to observe
algorithm and
for adjust until the learning.
reinforcement most suitableTheABS ratio is learned.
Q-learning consists of the
Q-learning is a well-known algorithm for reinforcement learning. The Q-learning consists of the
various states, action, and cost. The agent of Q-learning will continue to learn until a task is reached.
various states, action, and cost. The agent of Q-learning will continue to learn until a task is reached.
In other words, the agent
In other words, learns
the agent howhow
learns to choose
to choosethe theaction
action atatdifferent
different states
states to achieve
to achieve the bestthe best reward
reward
or the minimum punishment
or the minimum punishmentby repeatedly
by repeatedlyinteracting
interacting with withthethe environment.
environment.

Figure Figure The Almost

6. The6.Almost Blank
Blank Subframe(ABS)
Subframe (ABS) learner-environment
learner-environmentinteraction.
interaction.

3.3.2. Q-Learning-Based Dynamic ABS (QD-ABS) Scheme

In this work, we proposed a Q-learning-based dynamic ABS scheme to learn the appropriate
ABS configuration for a global ABS ratio setting. The key idea is inspired from the literature [33]. In
[33], the ABS ratio was dynamically adjusted based on the throughput of Macro BS and pico BS. Since
the UE distribution in each sector of a macro station is different, the ABS ratio will also be different.
Furthermore, the difference in ABS ratios will affect the throughput of each BS. Thus, we can define
the difference between the distribution of the throughput as the cost parameter in Q-learning. The
Appl. Sci. 2019, 9, 3018 12 of 22

3.3.2. Q-Learning-Based Dynamic ABS (QD-ABS) Scheme

In this work, we proposed a Q-learning-based dynamic ABS scheme to learn the appropriate
ABS configuration for a global ABS ratio setting. The key idea is inspired from the literature [33].
In [33], the ABS ratio was dynamically adjusted based on the throughput of Macro BS and pico BS.
Since the UE distribution in each sector of a macro station is different, the ABS ratio will also be
different. Furthermore, the difference in ABS ratios will affect the throughput of each BS. Thus, we can
define the difference between the distribution of the throughput as the cost parameter in Q-learning.
The agent finally chooses the best ABS ratio when Q-learning is converged to the minimum cost.
The detailed steps of Q-learning-based dynamic ABS scheme consists of two steps:
(1) It estimates the required ABS ratio in each sector of the Macro station first. Let the total
bandwidth requirement of the UEs in Macro BS and pico BS be NMUE and NPUE , respectively. Assume
that the sum of the throughput allocated by the Macro BS and pico BS in the previous frame to the UEs
∗
is ThMUE ∗ , respectively. Thus, the difference between the required bandwidth and the actual
and ThPUE
assigned throughput can be expressed as follows.

∆ThMUE = ThMUE ∗ − NMUE (14)

∆ThPUE = ThPUE ∗ − NPUE . (15)

In order to meet the QoS requirements of the UE, the Equation (16) illustrates the opportunity
of using ABS, which contains two conditions. The first one is that all of the MUEs must satisfy QoS
requirement, and there are remaining resources in the Macro BS (∆ThMUE > 0). The other one is that
some PUEs have not satisfied QoS requirement after resource allocation in pico BS (∆ThPUE < 0).
Let the set of Macro BSs be denoted by M. If the above two conditions are met, then we can first give
the parameter µb (b ∈ M) as the ABS ratio in each sector of the Macro station.

µb , i f ∆ThMUE > 0 and ∆ThPUE < 0

(
ABS ratio . (16)
0, others.

Assuming that the throughput of the radio resource is evenly distributed across each TTI in
a frame. For example, the allocation of 1000 bits by a frame is equivalent to an average transmission of
100 bits in 10 TTI. The formula of the ABS µb is described as follows:

(1 − µb )ThMUE ∗ − NMUE ≥ 0



µb [1 + µb (ε − 1)]ThPUE ∗ − NPUE ≤ 0

= argmax (17)

µb

0<µ <1


b

where ε is the gain factor and µb (ε − 1) is the ratio of increased data transfer rate of the PUE after
using ABS over the original data rate. Through the difference in the throughput and the restriction of
the QoS requirement, ABS µb can be calculated by solving the simultaneous inequalities. Equation (17)
consists of three conditions: (1) The decreased throughput of MUEs will not be less than their required
bandwidth (NMUE ); (2) the increased throughput of PUEs will not be more than their required
bandwidth (NPUE ); (3) the value of µb must be between 0 and 1. The aim is to avoid wasting the
resources of the MUE after using ABS.
(2) With the ABS µb of each sector of the Macro station, the next step is to learn the unified ABS
ratio by Q-learning algorithm. The state, action, and cost of the Q-learning algorithm based on the ABS
configuration are defined as follows:

• Agent: The system.

• State: The state is defined as s, which is the satisfaction ratio of PUEs in the system. The satisfaction
ratio of PUEs is defined as the number of PUEs whose QoS requirements can be met over the total
number of PUEs.
Appl. Sci. 2019, 9, 3018 13 of 22

• Action: The action is defined as at , which is the ABS ratio.

• Cost: The cost is defined as c, which is the new calculated throughput of each sector of the Macro
BS after using the unified ABS ratio.

The cost c estimates the immediate return incurred due to action at at state s. The cost function is
calculated as:
∆µ = at − µb (18)

i f ∆µ = 0


  0, 
∗ 
∆µTh  i f ∆µ > 0

Costb  , (19)

   MUE
 ∆µ (ε − 1) Th ∗ 
PUE , i f ∆µ < 0

X
c= Costb (20)

where Equation (18) represents the difference before and after adjusting the ABS ratio. In Equation
(19), ∆µ > 0 indicates that the actual ABS ratio chosen by the system (at ) is greater than that of the
sector of the Macro BS estimated by itself (µb ), thus losing the excessive throughput of MUEs. On the
contrary, ∆µ < 0 indicates that the actual ABS ratio chosen by the system is less than that of the sector
of the Macro BS estimated by itself, thus losing the expected throughput of PUEs. Finally, the sum of
these costs in each sector of the Macro BS is calculated in Equation (20).
Given aforementioned parameters, the Q-value Q(st , at ) can be defined as the expected cost of
taking action at at state st . Then Q-value is updated as follows:
" #
Q(st , at ) = (1 − ρ)Q(st , at ) + ρ c + min Q(st+1 , at+1 )

(21)
at + 1

where st+1 is the next state after taking the action at , and ρ is the learning rate, which denotes the willing
to learn from the environment. Eventually, a Q-table will be built as shown in Table 4. Once these
values in a Q-table have been learned and converged, the optimal action at each state is the one with
the lowest Q-value.

Table 4. Q-table in Q-Learning-Based Dynamic ABS (QD-ABS) scheme.

Dynamic ABS Ratio

a1 a2 a3 ... an
< 25% Q(s1 , a1 ) Q(s1 , a2 ) Q(s1 , a3 ) ... Q(s1 , an )
PUE satisfaction ratio 25~50% Q(s2 , a1 ) Q(s2 , a2 ) Q(s2 , a3 ) ... Q(s2 , an )
50~75% Q(s3 , a1 ) Q(s3 , a2 ) Q(s3 , a3 ) ... Q(s3 , an )
> 75% Q(s4 , a1 ) Q(s4 , a2 ) Q(s4 , a3 ) ... Q(s4 , an )

The detailed pseudo-code is given as follows:

Appl. Sci. 2019, 9, 3018 14 of 22

Algorithm: Q-learning dynamic ABS (QD-ABS)

Initialize:
1 for each b ∈ M do
2 calculate the initial ABS ratio µb based on Equations (23) and (24)
3 end for
4 for each s ∈ S, α ∈ A do
5 initialize the Q-value as the Q(st , αt )
6 end for
Learning:
7 loop
8 get the current state st ∈ S
9 select the action at ∈ A which has the minimum Q-value
10 calculate ∆µ based on Equation (18)
11 for each b ∈ M do
12 calculate the cost of changing the ABS ratio from µb to αt based on Equation (19)
13 sum the cost to c based on Equation (20)
14 end for
15 observe the next state st+1 and update
" the Q-table entry #as follows:
16 Q(st , at ) = (1 − ρ)Q(st , at ) + ρ c + min Q(st+1 , at+1 )

at+1
17 end loop
18 select the action αt which has the minimum Q-value for state s
QD-ABS algorithm

4. Evaluation

4.1. Simulation Assumption

Appl. Sci. 2019, 9, x 14 of 22
In this work, we adopted the system level simulation [34] to validate our proposed schemes.
Specifically, we used
In this work,Vienna LTE-A
we adopted theDownlink System
system level Level[34]
simulation Simulator
to validate version v2.0 Q3-2018
our proposed schemes.as our
simulator [35]. The
Specifically, we LTE-A downlink
used Vienna LTE-Amacro-pico HetNet
Downlink System with
Level seven cell
Simulator sitesv2.0
version wrapped
Q3-2018around
as our and
three simulator
hexagonal [35]. The LTE-A
sectors downlink
per cell site wasmacro-pico HetNet
considered with In
here. seven
eachcellcell,
sitestwo
wrapped
pico around
BSs withandfixed
three hexagonal sectors per cell site was considered here. In each cell, two pico BSs with
position were deployed at the cell edge with a distance of 8/15 ISD (Inter-site Distance) from the macro fixed position
BS, aswere deployed
shown at the7.
in Figure cellIn
edge with a distance
addition, mobile of
UEs8/15that
ISD were
(Inter-site Distance) from
nonuniform werethe macro BS, as
distributed in the
shown in Figure 7. In addition, mobile UEs that were nonuniform were distributed in the network.
network. The total number of UEs was 315 in the simulated scenario with 15 UEs deployed in each
The total number of UEs was 315 in the simulated scenario with 15 UEs deployed in each sector of a
sector of a Macro BS. Two traffic types were considered among 315 UEs, 158 UEs generating GBR
Macro BS. Two traffic types were considered among 315 UEs, 158 UEs generating GBR traffic and 157
trafficUEs
andgenerating
157 UEs MBR generating MBRsimulation
traffic. Each traffic. Each
was simulation
run for 10,000was
TTIsrun
andfor 10,000
30 runs TTIs and 30
of simulations runs of
were
simulations were conducted. The 95% confidence interval of each simulation
conducted. The 95% confidence interval of each simulation case was also calculated. System case was also calculated.
System parameters
parameters of simulation
of the the simulation are summarized
are summarized in Tablein5.Table 5.

Figure 7.7.Layout
Figure Layoutof
of network topology.
network topology.

Table 5. System parameters of the simulation.

Parameter Configuration
Cellular layout Wrap around, 7 Macros, 3 sectors per site
Carrier frequency 2.14 GHz
Appl. Sci. 2019, 9, 3018 15 of 22

Table 5. System parameters of the simulation.

Parameter Configuration
Cellular layout Wrap around, 7 Macros, 3 sectors per site
Carrier frequency 2.14 GHz
Bandwidth 10 MHz
ISD 500 m
Macro transmit power 46 dBm
Pico transmit power 30 dBm
Number of picos per sector 2
Number of UEs per sector 15
Cell selection HCI and SINR
Scheduler MTS and PF
ABS Q-learning dynamic and Static
Thermal noise −174 dBm/Hz
UE speed 3 km/hr
Pathloss model As in 3GPP TS 36.942
Traffic model VoIP (GBR for 1.5 Mbps) and
Video (MBR for 3 Mbps–7 Mbps)
Duplex model FDD
Number of RBs C = 50
Learning rate ρ = 0.5

4.2. Q-Learning-Based Dynamic Almost Blank Subframe

For performance comparison of our proposed schemes, the following simulation scenarios were
considered (see Table 6). The purpose of designing the five scenarios is illustrated as follows.

• Scenario 1: It is designed to verify the performance of the HCI cell selection scheme, which is
compared with the SINR scheme.
• Scenario 2: It is designed to verify the performance of the MTS resource allocation scheme,
which is compared with the Max-min scheme.
• Scenario 3: It is designed to verify the performance of the PF resource allocation scheme in dual
connectivity environment, which is compared with our proposed MTS scheme. In addition, it is
also compared with itself in single connectivity environment.
• Scenario 4: It is designed to verify the performance of the QD-ABS scheme in single connectivity,
which is compared with the static ABS scheme.
• Scenario 5: It is designed to verify the performance of dual connectivity combined with CRE
and ABS.
Table 6. Simulation scenarios.

Connection Cell Selection Scheduler E-ICIC CRE Bias

HCI
Scenario 1 DC MTS Non-ABS 0 dB
SINR
MTS
Scenario 2 DC HCI Non-ABS 0 dB
Max-min
MTS
DC HCI Non-ABS 0 dB
Scenario 3 PF
SC HCI PF Non-ABS 0 dB
Non-ABS 0 dB
Scenario 4 SC SINR PF QD-ABS 6 dB
Static ABS = 0.5 6 dB
SC SINR PF Non-ABS 0 dB
SC SINR PF Static ABS = 0.5 6 dB
Scenario 5
DC HCI MTS Non-ABS 0 dB
DC HCI MTS QD-ABS 6 dB
 Appl. Sci. 2019, 9, 3018 16 of 22

4.3. Output Indicators

In this work, the QoS requirement of each UE was considered. We analyzed the UE individual
satisfaction ratio per frame, by recording how many UEs met their QoS requirements in the simulation
time. The details are described as follows:

• System: The QoS satisfaction ratio of the system. The ratio was defined as the percentage of UEs
that met their QoS requirements.
• VoIP: The QoS satisfaction ratio of UEs generating voice traffic (GBR).
• Video: The QoS satisfaction ratio of UEs generating video traffic (MBR).
• MUE: The QoS satisfaction ratio of MUEs.
• PUE: The QoS satisfaction ratio of PUEs.
• RE: The QoS satisfaction ratio of UEs in the coverage of the CRE signal. A positive CRE bias 6 dB
was added to the downlink received signal strength from pico BS.

4.4. Simulation Results

4.4.1. Evaluation for the Cell Selection of HCI Scheme

Appl. Sci. 2019, 9, x 16 of 22
The initial distribution of UE population in Macro BSs and pico BSs were 60% and 40%, respectively.
To compare the HCI scheme with the SINR cell selection scheme, we let Macro BSs have higher load
to Macro BSs. Figure 8 shows the population ratio of MUEs and PUEs of the two cell selection
in order to assess whether pico BSs could offload the bandwidth requirements from UEs to Macro
schemes. In the HCI scheme, more UEs could be offloaded to pico BSs from Macro BSs due to the
BSs. Figure 8 shows the population ratio of MUEs and PUEs of the two cell selection schemes. In the
consideration
HCI scheme, of the
more cell
UEsload. Asbe
could a result of load
offloaded balancing
to pico BSs frombetween
Macro BSsMacro BSs
due to theand pico BSs, Figure
consideration of
9 shows that
the cell the As
load. HCI scheme
a result wasbalancing
of load able to achieve
betweenbetter
MacroQoS satisfactions
BSs and from different
pico BSs, Figure aspects,
9 shows that the in
particular the QoS
HCI scheme wassatisfaction
able to achieveof better
the whole system was
QoS satisfactions increased
from differentby 3%. Notably,
aspects, there
in particular the was
QoS one
exception, which
satisfaction waswhole
of the the PUE’s
systemQoS
was satisfaction. Clearly,
increased by 3%. since
Notably, pico
there wasBSs were
one less loaded,
exception, the HCI
which was
the PUE’s QoS satisfaction. Clearly, since pico BSs were less loaded, the HCI scheme associated
scheme associated more UEs to pico BSs which resulted in less QoS satisfaction for PUEs as compared more
to theUEs to pico
SINR BSs which resulted in less QoS satisfaction for PUEs as compared to the SINR scheme.
scheme.

Figure 8. Connection
Figure ratio
8. Connection of of
ratio thetheMacro
Macrouser
user equipment (MUE)and
equipment (MUE) and Pico
Pico user
user equipment
equipment (PUE).
(PUE).
 .
Appl. Sci. 2019, 9, 3018 17 of 22
Figure 8. Connection ratio of the Macro user equipment (MUE) and Pico user equipment (PUE).

Figure 9. 9.
Figure Satisfaction
Satisfactionratio
ratioof
of the
the different cellselection
different cell selection schemes.
schemes.

4.4.2.4.4.2. Evaluation
Evaluation for for
thethe ResourceAllocation
Resource Allocation of
of MTS
MTS Scheme
Scheme
In this scenario, the distribution of UE population was the same as that of the cell selection scenario.
In this scenario, the distribution of UE population was the same as that of the cell selection
We verified our proposed MTS scheme by comparing to the original max-min scheduling. As shown
scenario. We verified our proposed MTS scheme by comparing to the original max-min scheduling.
in Table 3, the max-min scheme can achieve almost optimal radio resource scheduling. In the MTS
As shown
scheme, inweTable 3, the
added max-min
a threshold schemethe
to protect can achieve
MUEs’ QoSalmost
in dual optimal radio
connectivity resource
mode. Figurescheduling.
10 shows In
the MTS scheme,
that the we added
MTS scheme a threshold
yielded higher QoSto protect the MUEs’
satisfactions from allQoS in dual
aspects. connectivity
Especially, the MTS mode. Figure
scheme
10 shows
yielded
Appl. Sci.
that the higher
2019, 5.07%
9, x
MTS scheme yielded of
QoS satisfaction higher QoS system
the whole satisfactions from all
as compared aspects.
to the Especially,
max-min scheme. the MTS
17 of 22
scheme yielded 5.07% higher QoS satisfaction of the whole system as compared to the max-min
scheme.

Figure 10. 10.

Figure Satisfaction ratio
Satisfaction ratioofofthe
thedifferent schedulingschemes
different scheduling schemesin in scenario
scenario 2. 2.

4.4.3. Evaluation for the Resource Allocation of PF Scheme

4.4.3. Evaluation for the Resource Allocation of PF Scheme
Scenario 3 was designed to show the effectiveness of the MTS scheme as compared to the PF
Scenario 3 was designed to show the effectiveness of the MTS scheme as compared to the PF
scheduling. In addition, it also tries to demonstrate that the PF scheduling is not suitable in dual
scheduling. In addition,
connectivity (DC) mode. it also tries
Figure to demonstrate
11 compares the QoS that the PFofscheduling
satisfaction is notinsuitable
the MTS scheme DC modeinasdual
connectivity (DC) mode. Figure 11 compares the QoS satisfaction of the MTS scheme in DC
well as that of the PF scheme in both DC and signal connectivity (SC) modes. Clearly, the MTS mode as
scheme
well as that of the PF scheme in both DC and signal connectivity (SC) modes. Clearly, the MTS scheme
outperformed the PF scheduling, in particular, it outperformed the PF scheme by at least 34.55% in
the QoS satisfaction of the whole system. Since the PF scheduling is based on the fairness principle
among UEs, it suffered the disadvantage that some UEs with poor CQI may have participated in
competition of the radio resources. As a consequence, it was interesting to observe that the simulation
 Scenario 3 was designed to show the effectiveness of the MTS scheme as compared to the PF
scheduling. In addition, it also tries to demonstrate that the PF scheduling is not suitable in dual
connectivity (DC) mode. Figure 11 compares the QoS satisfaction of the MTS scheme in DC mode as
well Appl.
as that of the
Sci. 2019, PF scheme in both DC and signal connectivity (SC) modes. Clearly, the MTS
9, 3018 scheme
18 of 22
outperformed the PF scheduling, in particular, it outperformed the PF scheme by at least 34.55% in
the QoS satisfaction
outperformed theof
PFthe whole system.
scheduling, Since itthe
in particular, PF scheduling
outperformed is based
the PF schemeonbythe fairness
at least principle
34.55% in
amongthe UEs, it sufferedofthe
QoS satisfaction thedisadvantage
whole system. that
Sincesome
the PFUEs with poor
scheduling CQIon
is based may have participated
the fairness principle in
competition
among UEs, of theit radio resources.
suffered As a consequence,
the disadvantage it was
that some UEs interesting
with poor CQItomay observe
have that the simulation
participated in
results showed that
competition of thethe system
radio satisfaction
resources. of the PF itscheme
As a consequence, was better
was interesting in the that
to observe SC mode than that in
the simulation
the DC mode.
results showed that the system satisfaction of the PF scheme was better in the SC mode than that in the
DC
Whilemode. the MTS scheme achieved better QoS satisfaction, we may wonder its fairness. The
While
fairness index of theaMTS scheme
scheme achievedas
is defined better QoS satisfaction, we may wonder its fairness. The fairness
follows.
index of a scheme is defined as follows.
(∑ 𝑥 )
Fairness index = Pn 2 (22)
𝑛× ∑
i=1 xi 𝑥
Fairness index = Pn 2
(22)
where 𝑥 is the throughput of the ith UE and n is the nnumber × i=1 xofi UEs in the system.

Table x7 is
where shows the fairness
the throughput index
of the of and
ith UE the nMTS
is theinnumber
DC mode of UEsas in
well
theas the PF scheduling in both
system.
i
DC and SC modes.
Table We
7 shows theobserved that of
fairness index PFthe
in MTS
SC mode
in DC had
modethe best as
as well fairness index, while
the PF scheduling the DC
in both MTS in
DC mode
and SChad the lowest
modes. fairness
We observed thatindex. However,
PF in SC mode had thethe
difference wasindex,
best fairness not very
whilesignificant.
the MTS in InDCother
words, thehad
mode MTS thescheme could achieve
lowest fairness much higher
index. However, QoS satisfaction
the difference was not very by significant.
sacrificingIna other
little words,
fairness as
the MTS scheme could achieve much higher QoS satisfaction by sacrificing a little fairness as a tradeoff.
a tradeoff.

Figure
Figure 11. 11. Satisfaction
Satisfaction ratioofofthe
ratio thedifferent
different scheduling
schedulingschemes in scenario
schemes 3. 3.
in scenario

Table 7. Fairness index of the Max–min Threshold Scheduler (MTS) scheme and the Proportional Fair
Scheduler (PF) scheduling.

MTS in DC Mode PF in DC Mode PF in SC Mode

Fairness index 0.50623 0.64226 0.68495

4.4.4. Evaluation for the E-ICIC of QD-ABS Scheme

Scenario 4 compares the performance of the Q-learning-based dynamic ABS (QD-ABS) scheme
with that of the static ABS scheme. For the static ABS scheme, the ABS ratio was set to 0.5 according to
most of the literature. Recall that the QD-ABS scheme adopts the Q-learning mechanism to dynamically
adjust the ABS ratio according to the feedback of the system. Two scenarios were simulated, we first
simulated the case where more UEs were MUEs. Specifically, the UE population of Macro BS was three
times more than that of pico BS. In the second case, all UEs were evenly distributed.
Figure 12 presents that the system QoS satisfaction ratio of the QD-ABS scheme was 4.06% higher
than that of the Static ABS scheme in the first scenario. In the second scenario where UEs were evenly
distributed, the proposed QD-ABS scheme was still better than the static ABS scheme, as shown in
Figure 13.
of
of Macro
Macro BS BS was
was three
three times
times more
more than
than that
that of
of pico
pico BS.
BS. In
In the
the second
second case,
case, all
all UEs
UEs were
were evenly
evenly
distributed.
distributed.
Figure
Figure 12 12 presents
presents that
that the
the system
system QoS
QoS satisfaction
satisfaction ratio
ratio of
of the
the QD-ABS
QD-ABS scheme
scheme was
was 4.06%
4.06%
higher
higherthan
thanthat
thatof ofthe
theStatic
StaticABS
ABSscheme
schemeininthe
thefirst
firstscenario.
scenario.InInthe
thesecond
secondscenario
scenariowhere
whereUEs
UEswere
were
evenly
evenly
Appl. distributed,
Sci. distributed, the proposed QD-ABS scheme was still better than the static
2019, 9, 3018 the proposed QD-ABS scheme was still better than the static ABS scheme, ABS scheme,19 ofas
as
22
shown
shownin inFigure
Figure13. 13.

Figure
Figure 12.
Figure12. Satisfactionratio
12.Satisfaction ratioof
ratio ofthe
thedifferent
differentenhanced
different enhancedinter-cell
enhanced inter-cellinterference
inter-cellinterferencecoordination
interferencecoordination(e-ICIC)
coordination(e-ICIC)
(e-ICIC)
schemes in the overload schemes
schemesin
situation inthe
of theoverload
Macro situation
situationof
BS.
overload ofthe
theMacro
MacroBS.
BS.

Figure
Figure13.
13.Satisfaction
Satisfactionratio
Satisfaction ratioof
ratio ofthe
of thedifferent
the differente-ICIC
different e-ICICschemes
schemeswhen
whenUEs
UEsare
areevenly
evenlydistributed.
distributed.

4.4.5. Evaluation for the Dual Connectivity Combined with CRE and ABS
Both the dual connectivity and the e-ICIC can improve the performance of the UE at the edge
of BSs. For this reason, we propose a comprehensive scheme to explore the feasibility of combining
the dual connectivity with CRE and ABS, and evaluate whether it can effectively improve the QoS
bandwidth requirements of edge UEs via a scenario 5 simulation. Since the comprehensive scheme is
based on the connection mode, cell selection, resource allocation, e-ICIC, and CRE bias, we compared
its performance with individual schemes, including SC, DC, non-ABS and Static ABS, and Q-learning
dynamic ABS.
In this scenario, we focused on the dual connectivity mode. Notably, the DC mode with CRE and
ABS did not yield better system QoS satisfaction, but it was useful to increase the QoS satisfaction
of edge UEs. Figure 14 shows the QoS satisfaction of different combinations of schemes of different
aspects where the labels RE, RE VoIP, RE Video on the x-axis denote the UEs in the CRE region, the UEs
of type GBR in the CRE region, and the UEs of the type MBR in the CRE region respectively. UEs in
CRE region are referred to as cell edge UEs. We can observe from Figure 14 that by integrating the
mechanisms of dual connectivity, CRE, and ABS, the QoS satisfaction ratio of cell edge UEs can be
improved by 10.76% as compared to the traditional approach.
of edge UEs. Figure 14 shows the QoS satisfaction of different combinations of schemes of different
aspects where the labels RE, RE VoIP, RE Video on the x-axis denote the UEs in the CRE region, the
UEs of type GBR in the CRE region, and the UEs of the type MBR in the CRE region respectively. UEs
in CRE region are referred to as cell edge UEs. We can observe from Figure 14 that by integrating the
mechanisms
Appl. Sci. 2019, 9,of dual connectivity, CRE, and ABS, the QoS satisfaction ratio of cell edge UEs can
3018 20 ofbe
22
improved by 10.76% as compared to the traditional approach.

Figure
Figure 14. Satisfaction
14. Satisfaction ratio ratio
of theofscheme
the scheme that integrates
that integrates dual connectivity
dual connectivity withrange
with cell cell range
expansion
(CRE) and ABS. expansion (CRE) and ABS.

5. Conclusions
5. Conclusions and
and Future
Future Works
Works
In this
In thiswork,
work, we we
discussed the development
discussed the developmentof spectrum utilization utilization
of spectrum in the future.inFor heterogeneous
the future. For
heterogeneous network research, we designed how to integrate dual connectivity, cell allocation,
network research, we designed how to integrate dual connectivity, cell selection, resource selection,
and e-ICIC
resource in the co-channel
allocation, and e-ICIC networks. On thenetworks.
in the co-channel other hand, On we studied
the other UEswe
hand, with different
studied QoS
UEs with
requirements
different QoS and traffic characteristics,
requirements such as voice traffic
and traffic characteristics, such(low data traffic
as voice transfer rate)
(low andtransfer
data video traffic
rate)
(high data transfer rate). In the resource allocation mechanism, we propose
and video traffic (high data transfer rate). In the resource allocation mechanism, we propose the MTSthe MTS scheduling to
meet the QoS requirement, considering the resource integration across the base
scheduling to meet the QoS requirement, considering the resource integration across the base stations stations to enhance the
data
to transferthe
enhance rate. In the
data cell selection
transfer rate. In mechanism, we propose
the cell selection the HCIwe
mechanism, scheme
proposeto consider
the HCInot only the
scheme to
signal quality received by the UE, but also the remaining RBs of each base
consider not only the signal quality received by the UE, but also the remaining RBs of each station, and further achieve
base
the loadand
station, balancing
further between
achieve the theload
basebalancing
stations. between
In the e-ICIC mechanism,
the base wethe
stations. In propose
e-ICICthe QD-ABS
mechanism,
scheme to adjust the ABS ratio dynamically according to system feedback.
we propose the QD-ABS scheme to adjust the ABS ratio dynamically according to system feedback.
The ultimate
The ultimate goal
goal ofof the
the proposed
proposed mechanisms
mechanisms is is to
to improve
improve the the QoS
QoS satisfaction
satisfaction ratio
ratio of
of UEs.
UEs.
Our simulation
Our results showed
simulation results showed that that our
our MTSMTS scheduler
scheduler waswas able
able to
to achieve
achieve 31.44%
31.44% higher
higher rate
rate than
than
the Proportional Fairness scheduler; our HCI cell selection scheme yielded 2.98%
the Proportional Fairness scheduler; our HCI cell selection scheme yielded 2.98% higher rate than higher rate than the
the
SINR cell selection scheme; the QoS satisfaction ratio of our Q-learning-based
SINR cell selection scheme; the QoS satisfaction ratio of our Q-learning-based dynamic ABS scheme dynamic ABS scheme
was
was 4.06%
4.06% higher
higher than
than that
that of
of the
the static
static ABS
ABS scheme. Finally, by
scheme. Finally, by integrating
integrating the
the mechanisms
mechanisms of of dual
dual
connectivity, CRE, and ABS, the QoS satisfaction ratio of cell edge UEs could
connectivity, CRE, and ABS, the QoS satisfaction ratio of cell edge UEs could be improved by 10.76% be improved by 10.76%
as compared
as compared to to the
the traditional
traditional approach.
approach.
We are expecting to see some small-scale commercial deployments for the fifth generation (5G)
networks by some leading operators in 2019 worldwide. The 5G air interface, called new radio (NR),
is expected to interwork with different wireless technologies where UEs will have multiconnectivity
capabilities. In particular, in the early stage of 5G deployment, interworking with LTE-A based on
dual connectivity will be necessary to boost the deployment and has been proposed by 3GPP as
a 5G operational requirement [2,36]. Extending the dual connectivity to multi-connectivity in 5G will
become an important technique and require further study [37].

Author Contributions: Conceptualization, R.-H.H. and M.-C.P.; methodology, R.-H.H. and K.-C.C.; software,
K.-C.C.; validation, R.-H.H. and M.-C.P.; formal analysis, R.-H.H. and K.-C.C.; investigation, R.-H.H. and M.-C.P.;
resources, R.-H.H.; data curation, K.-C.C.; writing—original draft preparation, K.-C.C.; writing—review and
editing, R.-H.H. and M.-C.P.; visualization, K.-C.C.; supervision, R.-H.H.; project administration, R.-H.H.; funding
acquisition, R.-H.H.
Funding: This research was funded by Ministry of Science and Technology, Taiwan, grant number MOST
105-2221-E-194-03 and MOST 106-2221-E-194-021-MY3.
Conflicts of Interest: The authors declare no conflict of interest.
Appl. Sci. 2019, 9, 3018 21 of 22

References
1. Study on Small Cell Enhancements for E-UTRA and E-UTRAN-Higher Layer Aspects, 3GPP TR
36.842. Available online: https://portal.3gpp.org/desktopmodules/Specifications/SpecificationDetails.aspx?
specificationId=2543 (accessed on 19 July 2019).
2. NR inter-band Carrier Aggregation/Dual connectivity for 2 bands DL with x bands UL (x = 1, 2), 3GPP TR
38.716-02-00. Available online: https://portal.3gpp.org/desktopmodules/Specifications/SpecificationDetails.
aspx?specificationId=3517 (accessed on 19 July 2019).
3. Jha, S.C.; Sivanesan, K.; Vannithamby, R.; Koc, A.T. Dual Connectivity in LTE Small Cell Networks.
In Proceedings of the 2014 IEEE Globecom Workshops, Austin, TX, USA, 8–12 December 2014; pp. 1205–1210.
4. Zhang, J.; Zeng, Q.; Mahmoodi, T.; Georgakopoulos, A.; Demestichas, P. LTE Small Cell Enhancement by
Dual Connectivity. Wirel. World Res. Forum 2014, 15, 1–20.
5. Kishiyama, Y.; Benjebbour, A.; Nakamura, T.; Ishii, H. Future steps of LTE-A: Evolution toward integration
of local area and wide area systems. IEEE Wirel. Commun. 2013, 20, 12–18. [CrossRef]
6. Liu, J.; Liu, J.; Sun, H. An Enhanced Power Control Scheme for Dual Connectivity. In Proceedings of the IEEE
80th Vehicular Technology Conference (VTC Fall), Vancouver, BC, Canada, 14–17 September 2014; pp. 1–5.
7. Zakrzewska, A.; Lopez-Perez, D.; Kucera, S.; Claussen, H. Dual connectivity in LTE HetNets with split control
and user plane. In Proceedings of the 2013 IEEE Globecom Workshops, Atlanta, GA, USA, 9–13 December
2013; pp. 391–396.
8. Ramamoorthi, Y.; Kumar, A. Performance comparison of dual connectivity with CoMP in heterogeneous
cellular networks. In Proceedings of the 9th International Conference on Communication Systems and
Networks (COMSNETS), Bengaluru, India, 4–8 January 2017; pp. 1–6.
9. Jabbar, A.I.A.; Abdullah, F.Y. Long Term Evolution (LTE) Scheduling Algorithms in Wireless Sensor Networks
(WSN). Int. J. Comput. Appl. 2015, 121, 12–16.
10. Trivedi, R.D.; Trivedi, M.C. Comparison of Different Scheduling Algorithm for LTE. Int. J. Emerg. Technol.
Adv. Eng. 2014, 4, 1–6.
11. Habaebi, M.H.; Chebil, J.; Al-sakkaf, A.G.; Dahawi, T.H. Comparison between Scheduling Techniques in
Long Term Evolution. IIUM Eng. J. 2013, 14, 67–76. [CrossRef]
12. Sun, Z.; Yin, C.; Yue, G. Reduced-Complexity Proportional Fair Scheduling for OFDMA Systems.
In Proceedings of the International Conference on Communications, Circuits and Systems, Guilin, China,
25–28 June 2006; pp. 1–5.
13. Ayhan, M.; Zhao, Y.; Choi, H.A. Utilizing Geometric Mean in Proportional Fair Scheduling: Enhanced
Throughput and Fairness in LTE DL. In Proceedings of the IEEE Global Communications Conference
(GLOBECOM), Washington, DC, USA, 4–8 December 2016; pp. 1–6.
14. Qu, T.; Xiao, D.; Yang, D. A novel cell selection method in heterogeneous LTE-advanced systems.
In Proceedings of the 3rd IEEE International Conference on Broadband Network and Multimedia Technology
(IC-BNMT), Beijing, China, 26–28 October 2010; pp. 510–513.
15. Qian, C.; Zhang, S.; Zhou, W. A Novel Cell Selection Strategy with Load Balancing for Both Idle and
RRC-Connected Users in 3GPP LTE Network. In Proceedings of the International Conference on Wireless
Communications & Signal Processing (WCSP), Xi’an, China, 23–25 October 2012; pp. 1–6.
16. Qu, T.; Xiao, D.; Yang, D.; Jin, W.; He, Y. Cell selection analysis in outdoor Heterogeneous Networks.
In Proceedings of the 3rd international Conference on Advanced Computer Theory and Engineering
(ICACTE), Chengdu, China, 20–22 August 2010; pp. 554–557.
17. Guvenc, I.; Jeong, M.-R.; Demirdogen, I.; Kecicioglu, B.; Watanabe, F. Range Expansion and Inter-Cell
Interference Coordination (ICIC) for Picocell Networks. In Proceedings of the IEEE Vehicular Technology
Conference (VTC Fall), San Francisco, CA, USA, 5–8 September 2011; pp. 1–6.
18. Wang, J.; Liu, J.; Wang, D.; Pang, J.; Shen, G. Optimized Fairness Cell Selection for 3GPP LTE-A Macro-Pico
HetNets. In Proceedings of the IEEE Vehicular Technology Conference (VTC Fall), San Francisco, CA, USA,
5–8 September 2011; pp. 1–5.
19. Tian, P.; Tian, H.; Zhu, J.; Chen, L.; She, X. An adaptive bias configuration strategy for range extension in LTE
advanced heterogeneous networks. In Proceedings of the IET international Conference on Communication
Technology and Application (ICCTA), Beijing, China, 14–16 October 2011; pp. 336–340.
Appl. Sci. 2019, 9, 3018 22 of 22

20. Kamal, A.; Mathai, V. A Novel Cell Selection method for LTE HetNet. In Proceedings of the International
Conference on Communication and Signal, Tamilnadu, India, 3–5 April 2014.
21. Yasir, B.A.; Su, G.; Bachache, N. Range Expansion for Pico Cell in Heterogeneous LTE- A Cellular Networks.
In Proceedings of the 2nd international Conference on Computer Science and Network Technology,
Changchun, China, 29–31 December 2012; pp. 1–6.
22. Oh, J.; Han, Y. Cell Selection for Range Expansion with Almost Blank Subframe in Heterogeneous
Networks. In Proceedings of the IEEE 23rd International Symposium on Personal, Indoor and Mobile Radio
Communications (PIMRC), Sydney, Australia, 9–12 September 2012; pp. 653–657.
23. Daeinabi, A.; Sandrasegaran, K.; Zhu, X. Performance Evaluation of Cell Selection Techniques for Pico
cells in LTE-Advanced Networks, 10th international Conference on Electrical Engineering/Electronics.
In Proceedings of the Computer, Telecommunications and Information Technology (ECTiCON), Krabi,
Thailand, 15–17 May 2013; pp. 1–6.
24. Huang, Z.; Liu, J.; Shen, Q.; Wu, J.; Gan, X. A threshold-based multi-traffic load balance mechanism in LTE-A
networks. In Proceedings of the IEEE Wireless Communications and Networking Conference (WCNC),
New Orleans, LA, USA, 9–12 March 2015; pp. 1–6.
25. Lopez-Perez, D.; Chu, X. Inter-Cell Interference Coordination for Expanded Region Picocells in Heterogeneous
Networks. In Proceedings of the 20th International Conference on Computer Communications and Networks
(ICCCN), Maui, HI, USA, 31 July–4 August 2011; pp. 1–6.
26. Nam, H.; Kim, K.H.; Kim, B.H.; Calin, D.; Schulzrinne, H. Towards dynamic QoS-aware over-the-top
video streaming, World of Wireless, Mobile and Multimedia Networks (WoWMoM). Proceeding of IEEE
International Symposium on a World of Wireless, Sydney, Australia, 19 June 2014; pp. 1–9.
27. Kamath, A.; Palmon, O.; Plotkin, S. Routing and Admission Control in General Topology Networks with
Poisson Arrivals. J. Algor. 1998, 27, 236–258. [CrossRef]
28. Wang, Y.; Pedersen, K.I. Time and Power Domain Interference Management for LTE Networks with
Macro-cells and HeNBs. In Proceedings of the IEEE Vehicular Technology Conference (VTC Fall),
San Francisco, CA, USA, 5–8 September 2011.
29. Barbieri, A.; Damnjanovic, A.; Ji, T.; Montojo, J.; Wei, Y.; Malladi, D.; Song, O.; Horn, G. LTE Femtocells:
System Design and Performance Analysis. IEEE J. Sel. Areas Commun. 2012, 30, 586–594. [CrossRef]
30. Simsek, M.; Bennis, M.; Czylwik, A. Dynamic Inter-Cell Interference Coordination in HetNets:
A Reinforcement Learning Approach. In Proceedings of the IEEE Global Communications Conference
(GLOBECOM), Anaheim, CA, USA, 3–7 December 2012; pp. 5446–5450.
31. Li, Q.; Xia, H.; Zeng, Z.; Zhang, T. Dynamic enhanced Inter-Cell Interference Coordination using reinforcement
learning approach in Heterogeneous Network. In Proceedings of the 15th IEEE International Conference on
Communication Technology (ICCT), Guilin, China, 17–19 November 2013; pp. 1–5.
32. Daeinabi, A.; Sandrasegaran, K. A fuzzy Q-learning approach for enhanced intercell interference coordination
in LTE-Advanced heterogeneous networks. In Proceedings of the 20th Asia-Pacific Conference on
Communication (APCC2014), Pattaya city, Thailand, 1–3 October 2014; pp. 139–144.
33. Gao, L.; Tian, H.; Tian, P.; Zhang, J.; Wang, M. A distributed dynamic ABS ratio setting scheme for macro-femto
heterogeneous networks. In Proceedings of the IEEE International Conference on Communications
Workshops (ICC), Budapest, Hungary, 9–13 June 2013; pp. 1–5.
34. Alcatel-Lucent Shanghai Bell. DL Pico-Macro HetNet Performance: Cell Selection. In Proceedings of the
3GPP TSG-RAN WG1 #60bis, Beijing, China, 12–16 April 2010.
35. Vienna LTE-A Downlink System Level Simulator. Available online: https://www.nt.tuwien.ac.at/research/
mobile-communications/vccs/vienna-lte-a-simulators/lte-a-downlink-system-level-simulator/ (accessed on
19 July 2019).
36. Study on Scenarios and Requirements for Next Generation Access Technologies. 3GPP TR. 38.913. Available online:
http://www.3gpp.org/DynaReport/38913.htm (accessed on 19 July 2019).
37. Monteiro, V.F.; Sousa, D.A.; Maciel, T.F.; Cavalcanti, F.R.P.; Silva, C.F.M.; Rodrigues, E.B. Distributed RRM for
5G Multi-RAT Multiconnectivity Networks. IEEE Syst. J. 2019, 13, 192–203. [CrossRef]

© 2019 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access
article distributed under the terms and conditions of the Creative Commons Attribution
(CC BY) license (http://creativecommons.org/licenses/by/4.0/).