Cognitive Internet of Vehicles
Cognitive Internet of Vehicles
Cognitive Internet of Vehicles
Computer Communications
journal homepage: www.elsevier.com/locate/comcom
A R T I C L E I N F O A B S T R A C T
Keywords: To fully realize future autonomous driving scenarios, Internet of Vehicles (IoV) has attracted wide attention from
CIoV both academia and industry. However, suitable cost and stable connectivity cannot be strongly guaranteed by
Internet of Vehicles existing architectures such as cellular networks, vehicular ad hoc networks, etc. With the prosperous develop-
Cognitive computing ment of artificial intelligence, cloud/edge computing and 5G network slicing, a more intelligent vehicular
Vehicular network
network is under deliberation. In this paper, an innovative paradigm called Cognitive Internet of Vehicles (CIoV)
is proposed to help address the aforementioned challenge. Different from existing works, which mainly focus on
communication technologies, CIoV enhances transportation safety and network security by mining effective
information from both physical and network data space. We first present an overview of CIoV including its
evolution, related technologies, and architecture. Then we highlight crucial cognitive design issues from three
perspectives, namely, intra-vehicle network, inter-vehicle network and beyond-vehicle network. Simulations are
then conducted to prove the effect of CIoV and finally some open issues will be further discussed. Our study
explores this novel architecture of CIoV, as well as research opportunities in vehicular network.
⁎
Corresponding authors at: School of Computer Science and Technology, Huazhong University of Science and Technology, China.
E-mail addresses: minchen2012@hust.edu.cn (M. Chen), yuanwentian@hust.edu.cn (Y. Tian), g.fortino@unical.it (G. Fortino), zhangjing@hust.edu.cn (J. Zhang),
iztok.humar@fe.uni-lj.si (I. Humar).
https://doi.org/10.1016/j.comcom.2018.02.006
Received 15 October 2017; Received in revised form 8 January 2018; Accepted 10 February 2018
0140-3664/ © 2018 Published by Elsevier B.V.
Please cite this article as: Chen, M., Computer Communications (2018), https://doi.org/10.1016/j.comcom.2018.02.006
M. Chen et al. Computer Communications xxx (xxxx) xxx–xxx
measured by millisecond. Once a network congestion or long delay making process of autonomous vehicles will be more thorough and
happens, due to slow computation or limited band width, a series of reliable through the cognitive cycle of perception, training, learning
life-threatening traffic accidents would take place. and feedback.
(3) Seamless connectivity: in the future, user’s requirements on net- (3) Efficient utilization of resources: with perception of network
work quality and service continuity would be much higher. traffic status and real-time road circumstance, the decisions derived
Specifically, many computation-intense tasks need to be processed by analytic technologies such as machine learning and deep
in real time. Therefore, QoS of many vehicular application can be learning, can help resource cognitive engine to conduct more ef-
satisfied only if a stable and uninterrupted network connection is fective control over vehicles, and to enhance information sharing
guaranteed. efficiency within vehicular networks.
(4) Data privacy: large amounts of private information on vehicle (4) Rich market potentiality: in terms of market opportunities, the
owners are involved in vehicular networks, and it cannot be pro- benefits brought by CIoV are not limited to vehicle market, they are
tected by traditional network protection mechanism. Additionally, also closely linked to many other aspects in people’s life, such as
the urban traffic system needs a secure and robust network en- entertainment, healthcare, agenda and so on. Such a feature will
vironment to guarantee the orderly conduct of autonomous driving. also drive many traditional application devices to be transformed
(5) Resource constraints: though self-organizing networks of vehicles into intelligence embedded application devices.
may provide real-time communication, the computing resources
and network resources possessed by a single vehicle is still limited, The remainder of this paper is organized as follows. We first present
especially during a transition period of a semi-autonomous driving the evolution of CIoV and review its related technologies in Section 2.
scenario. Resources need to be precisely scheduled in real time, Then in Section 3, the five-layered CIoV architecture is presented, with
based on actual driving course of massive vehicular networks. a particular emphasis of interaction between data cognitive engine and
resource cognitive engine under cloud/edge framework. In Section 4 we
In order to solve those problems, the intelligence of IoV need be explore critical cognitive design issues from three perspectives (intra-
strengthened in comprehensive directions. Therefore, Cognitive vehicle network, inter-vehicle network and beyond-vehicle network),
Internet of Vehicles (CIoV) is proposed in this paper to realize in- aiming to enhance user experience and performance of traffic system. In
telligent cognition, control and decision-making for future autonomous Section 5, we simulate a vehicular edge scenario in CIoV to prove the
driving scenarios. In contrast to existing works on IoV, the human- effectiveness of our proposed architecture. Finally, we discuss some
centric CIoV utilizes hierarchical cognitive engines and conduct joint open issues related with the implementation of CIoV in Section 6 and
analysis in both physical and network data space. To grasp a concrete draw our conclusions in Section 7.
idea, we divided main participants in CIoV into intra-vehicle network,
inter-vehicle network and beyond-vehicle network in Fig. 1, different 2. Background and related work
scaled networks also focus on different cognitive functions. Main ad-
vantages of CIoV are listed below: CIoV is proposed as the advanced solution to strengthen cognitive
intelligence of IoV. In order to better understand the development of
(1) Cognitive Intelligence: CIoV enables IoV to bear more accurate vehicular networks, this section will explain the differences between
perceptive ability, through cognition in intra-vehicle network CIoV and three related concepts, i.e., ITS, VANET and IoV. Fig. 2 il-
(driver, passengers, smart devices, etc.), inter-vehicle network lustrates the evolution process of CIoV. Furthermore, key technologies
(adjacent intelligent vehicles) and beyond-vehicle network (road that enables CIoV, including self-driving technology, cloud/edge hybrid
environment, cellular network, edge nodes, remote cloud, etc), it framework and 5G network slicing, is further presented.
can also provide macrocosmic information and scheduling strate-
gies to the whole transportation system.
(2) Reliable decision-making: by introducing cognitive computing 2.1. ITS, VANET, IoV and CIoV
into autonomous driving systems, learning ability of autonomous
vehicles can be effectively improved. Moreover, the decision- Intelligent transportation system (ITS) is an extensive conception,
put forth before 2000. ITS involves a series of application systems:
2
M. Chen et al. Computer Communications xxx (xxxx) xxx–xxx
vehicle management system, automatic license plate recognizing other. On one hand, large amounts of data generated during driving
system, traffic signal control system. A typical example is that, in order process of vehicles can provide sufficient learning and training basis to
to realize extraction and utilization of static and dynamic information AI. On the other hand, with rapid development of electronic circuits
of vehicles on information network platform, electronic tags carried on such as GPU, TPU, FPGA, ASIC etc., the performance of deep learning
vehicles can be recognized by technologies such as wireless radio-fre- algorithm in aspect of real-time processing has been improved sig-
quency identification (RFID). nificantly, ensuring real-time business guarantee to environment per-
The rapid development of wireless mobile communication tech- ception, decision-making and control on CIoV. At present, great pro-
nology attracts researchers’ attention to communication between ve- gress has been made in research on self-driving with introduction of
hicles as a way to enhance road safety and improve transportation ef- algorithms such as AI-based path optimization algorithm [10] and ob-
ficiency. For a long time, vehicular ad hoc network (VANET) has been stacle and road identification algorithm [11].
under the spotlight. VANET mainly utilizes DSRC communication In CIoV, information cognition and interaction of intelligent au-
technology [9]. Yet a few problems remain unsolved: due to the high tonomous vehicles, which are able to obtain more information with
speed mobility of vehicles and currently incomplete infrastructure, the adjacent vehicles, road and infrastructure, are brought into considera-
reliability of service connection in VANET is vulnerable. tion. Thus, the perceptive ability of CIoV is greatly increased in contrast
As a result, VANET alone could not meet requirements for future to a single autonomous vehicle.
autonomous driving scenarios. The emergence of big data and IoT leads
to the concept of IoV. Under agreed communication protocol and data
interaction standard, wireless communication and information ex- 2.3. Cloud/edge hybrid framework
change may be conducted on IoV between vehicle-to-anything (V2X),
such as other vehicles, road, pedestrian, etc. Characterized by powerful computing and storage capacity, cloud
In the macro-framework of IoV, CIoV aims to solve the top-level computing platform can reduce the deployment cost of software ser-
problem of IoV, i.e., enhancing intelligence of IoV comprehensively. To vices [44]. However, with more and more accessed mobile devices and
be specific, how to fully mine information of all participants (Fig. 1) to high-quality local processing requirements, edge computing, as a fra-
reach the following objectives based on joint cognition of physical data mework much closer to the user end, supplements the functions of
space and network data space: (1) enhancing user experience as per cloud computing effectively. Authors in [12] present the advantages of
private demand; (2) improving driving safety in traffic system; (3) edge computing pattern in background of IoT, from angles such as delay
strengthening data safety in network environment; (4) comprehensively constraint, bandwidth constraint and limited resources constraint.
optimizing network resource allocation. The cognitive design issues will In [13], a comparison is made between feature of Cloud Radio Access
be explored in detail in Section 4. In order to present a complete and Network (C-RAN) and that of Mobile Edge Computing (MEC), and the
clear understanding of CIoV, the architecture of CIoV will be provided importance of mutual collaboration of MEC in 5G network is elabo-
in next section. rated.
Cloud/edge hybrid framework is a rational solution for CIoV.
Specifically, intelligent application services may be provided in the
2.2. Self-driving technology neighbourhood through collaboration of edge nodes, thus to meet de-
mands of many delay-sensitive vehicular applications that require local
In recent years, artificial intelligence (AI) technology is again in the processing, such as real-time road condition analysis and real-time
ascendant, and deep learning gradually becomes the most important behavior analysis for driver. However, with limited storage and com-
part in AI technology. As a vertical application of AI, self-driving/au- puting capacity, edge computing could not meet the requirements for
tonomous driving technology attracts much attention in vehicle in- long-term cognition of users and environment. In this case, it is ne-
dustry. The investigation report published by Eno Center for cessary to unload tasks to cloud for further analysis in idle time of non-
Transportation reveals that self-driving technology may significantly driving condition. In addition, communication between vehicular edge
reduce the quantity of traffic accidents caused by driving errors [2]. and cloud is also important, especially in emergent situations.
AI-based self-driving technology and IoV can complement each
3
M. Chen et al. Computer Communications xxx (xxxx) xxx–xxx
2.4. 5G network slicing cleaning, formatting and normalization for data, thus to extract useful
information preliminarily.
With the evolution of mobile communication industry, 5G network
service is booming recently with special features such as closer to user 3.2. Communication layer
requirements, enhanced customization capability, deep integration
between network and business, and more friendly services. Due to its In order to be adapted to requirements of applications with different
elasticity and expandability, 5G network slicing becomes the research timeliness, cloud/edge hybrid architecture is mainly adopted in com-
focus in network communication field. Network slicing may explore munication layer of CIoV. Related radio technologies in communication
and release the potentiality of telecommunication technology, enhance layer (such as Wi-Fi, DSRC, LTE etc.) are shown in Table 1. At the scale
efficiency and reduce cost. On the other hand, there is potential market of intra-vehicle network, most of the driving data need timely local
demand for network slicing in fields such as vehicles, smart city and processing and computing, exploiting the real-time communication
industrial manufacturing. between intelligent devices on intra-vehicle network and edge cloud.
5G network slicing can meet IoV requirements on ultra-low delay On premise of meeting connectivity demand, the main objective of
and high reliability and other specific applications [15]. In nature, it inter-vehicle network is resource optimization. On one hand, real-time
divides the physical network of operators into multiple virtual net- information interaction can be realized through self-organizing network
works, each virtual network corresponds to one of service requirements between vehicles or star network between vehicles and RSUs. On the
such as delay, bandwidth, security and reliability, thus it can cope with other hand, when there is no inter-operable units near a demand, cel-
different network application scenarios flexibly. Moreover, with in- lular network may be adopted for communication. At large scale, the
troduction of network slice broker [14], 5G network slicing technology cloud needs to conduct centralized control over the whole traffic in-
may realize network resources sharing, and integration & allocation formation, and to establish the feature model for network topology,
will be conducted to network resources that were mutually independent road condition information and space-time service of group autono-
originally, thus to realize real-time and dynamical scheduling of net- mous moving objects of the whole IoV. Furthermore, the communica-
work resources corresponding to special requirements. tion between one vehicle and another vehicle is necessary, e.g., those
At present, attempts have been made in some research projects to in-vehicle services that are not delay-sensitive can be gradually un-
introduce network slicing technology into IoV. In [16], a network loaded to cloud for computing and analysis.
computing resource allocation algorithm on IoV with cluster as unit is
put forth, but the part of core network including road-side unit is not 3.3. Cognition layer
involved. In [17], the network resources in the space and in air seg-
ments are provided to vehicles in ground segments for shared use, but In order to combine specific service demands, and to enhance in-
the delay indicator, which is most important in service quality of 5G telligence of CIoV, cognitive engines are arranged at cloud/edge, di-
network slicing, has not yet been taken into consideration. In CIoV, vided into data cognitive engine at cognition layer and resource cog-
double cognitive engines are introduced to conduct cognition, control nitive engine at control layer. Physical data space and network data
and scheduling to network resources, and the cognitive design issue on space provide the data to data cognitive engine.
resource allocation of 5G network slicing will be discussed in Section 4. As for cognition in physical data space, the data cognitive engine
processes and analyzes heterogeneous data flows through cognitive
3. Architecture of cognitive internet of vehicles analysis methods (machine learning, deep learning, data mining, patten
recognition etc.). In detail, data cognitive engine is able to conduct
When compared with traditional sensor network, there are higher cognition of user tasks by use of data collected, e.g., driving behavior
requirements on perception accuracy, stability in data transmission, model analysis, emotion analysis, road condition investigation and etc.
real-time analysis, intelligent decisions and network reliability for Based on cognition of user tasks, these can be divided into real-time
CIoV, demanding for more complex architectures. In this section, an vehicle area network services and non-real-time vehicle area network
architecture that meets requirements of CIoV is put forth, as shown in services. Generally speaking, real-time vehicle area network services
Fig. 3, comprising the sensing layer, communication layer, cognition are generally deployed on edge closer to user terminals, and non-real-
layer, control layer and application layer. time vehicle area network services can be deployed on the cloud, even
far away from user.
3.1. Sensing layer In network data space, the data cognitive engine can realize dy-
namic cognition of data such as computing, storage and network re-
The sensing layer of CIoV is in charge of collecting and pre-pro- sources, based on the resource allocation feedback on cloud/edge net-
cessing for multi-source heterogeneous big data [45]. These data come work, provide network optimization methods & real-time resource
from multidimensional space-time data in physical space on one hand, allocation strategies, and send analysis results to resources cognitive
and from network traffic and resource distribution data in network engine to guide network resource allocation. To be specific, when there
space on the other hand. is a delay-sensitive task, the edge will firstly check whether it has suf-
Compared with big data set in traditional fields, big data in physical ficient resources to complete this task, if no, those tasks not sensitive to
space are often unstructured. To be specific, driver related information delay may be transferred to cloud to realize reallocation of resources,
should be described through driving video, facial expression data and thus to meet the delay demand of the delay-sensitive task. On this basis,
etc. Route related information such as accurate position and environ- different engines may be deployed as per different business types and
ment should be described through multiple sensors collecting real-time application scenarios, e.g., engine oriented to collection and storage of
data from ambient pedestrians, vehicles and environment, and it should mass data, engine oriented to driving behavior analysis and engine
be transformed into multidimensional space-time data. oriented to network security, and etc.
The data that are current in network space are mainly operator data,
for example, RSU, information on resource occupation by base station 3.4. Control layer
etc., and service request information of users, basic data information of
user, etc. Generally speaking, the original data set collected may be As the scale of IoV constantly extends, exponentially increased data
unclean, redundant and inconsistent. Therefore, in order to enhance need to be processed and corresponding strategies need to be provided.
effective usage rate of resources in edge devices, appropriate data Thus, control layer is the key factor determining system performance.
analysis algorithms should be adopted in perception layer to conduct Since the traditional method of centralized control in data center
4
M. Chen et al. Computer Communications xxx (xxxx) xxx–xxx
Table 1
Related radio technologies in communication layer.
cannot guarantee low delay constraint of interactions between auton- network where different business requirements need to be satisfied.
omous moving objects, it is not suitable for delay sensitive applications Resource cognitive engines deployed on edge support delay sensitive
on the edge of IoV. Enabled by technologies like NFV, SDN, SON and data management. Although the storage, processing and bandwidth
network slicing, the main function of resource cognitive engines are to resources available for edge are limited, distributed decision-making
manage and dispatch network resources. can be realized to process data from bottom layer. To meet QoS re-
Resource cognitive engines are deployed at different locations of quirements of intra-vehicle cognitive applications, the resource cogni-
cloud/edge in order to strengthen stability and reliability of the tive engine on intra-vehicle edge are responsible for real-time
5
M. Chen et al. Computer Communications xxx (xxxx) xxx–xxx
Fig. 4. (a) Distracted driving activities distribution in 2010 and (b) distracted driving activities distribution in 2012.
processing of driving data. In this way, the fastest decision-making can heterogeneity and complexity of CIoV network, we divide CIoV into
be guaranteed. Resource cognitive engines deployed on cloud conduct intra-vehicle network, inter-vehicle network, and beyond-vehicle net-
network optimization in a centralized way through effective utilization work from the perspective of scale and main functions. This section will
of the global information of IoV. Execution on cloud is at the cost of focus on the cognitive design issues of these three networks. Three
large centralized data storage, processing and bandwidth resources. networks own their independent characteristics respectively. However,
Specifically, the most important work of cloud is to monitor resource we also lay emphasis on the interaction and cooperation of these three
utilization on edge network and to conduct dynamic scheduling for networks to optimize the overall performance.
resources in real time. In addition, cloud receives emergent messages
sent by edge and conducts a series of emergency treatment through 4.1. Cognition in intra-vehicle network
high-performance computing. With cooperative control of resource
cognitive engines at different levels, safety of the whole traffic system The edge cognitive computing (ECC) of 5G cognitive network en-
can be improved. ables the communication between wearable and vehicular embedded
computing devices to be faster, more intelligent and more stable. In the
3.5. Application layer scale of intra-vehicle space, safety and comfortableness of driver and
passenger become the main concern. Furthermore, through cognition of
At the operation level, CIoV involves coordination and cooperation intra-vehicle network, we can promote the safety of the whole traffic
among multiple parties, including manufacturer of automatic driving system, which will be discussed in detail in the interaction with beyond-
vehicles, mobile communication operator, social networking services vehicle network.
provider, manufacturer of intelligent devices, software services pro-
vider and etc. Among plenty of applications provided by CIoV, two 4.1.1. Driving guidance based on long-term behavioral cognition
main categories are customized application services and intelligent Among the existing methods of driving guidance, many articles
transportation applications, it is noted that such services can be defined consider reducing the traffic accidents by the real-time driver mon-
as Opportunistic since they are markedly dynamic, context-aware and itoring from two aspects. The first method is through detecting the
co-located [18]. driving behavior of drivers. According to the survey of National
Highway Traffic Safety Administration in 2012 [19], several typical
– Customized application services are aimed to reduce safety risks distracted driving activities are shown in Fig. 4. By supervising and
during driving; typical examples are driver fatigue detection, controlling these driving behaviors [20], proper warning beforehand
driving guidance, driver emotion monitoring and etc. Since many can be provided and the response time of drivers can be increased to
intelligent devices have access to CIoV, cognitive applications (such reduce the occurrence of traffic accidents. The second method is
as mobile health monitoring) can be customized based on the dif- through monitoring the fatigue state and negative emotion of drivers in
ferent features of users. Cognitive applications will be elaborated in real time, vigilant measures will be taken for the tired drivers [21]. A
next section. novel method of detecting cardiac defense response using ECG
– Intelligent transportation applications include intelligent driving, signal [22] is promising since it can be well combined with wearable
intelligent transportation management, etc. To be specific, (1) in- computing, effectively monitoring drivers without giving rise to their
telligent driving helps drivers with accurate judgment on road uncomfortableness.
conditions through communication between one vehicle and an- These two monitoring methods are feasible, however, both focus on
other vehicle/road, in combination with cognition of personal intervention on the driver. Besides, information of intra-vehicle net-
driving behavior. (2) intelligent transportation management means work is not excavated fully. In fact, intra-vehicle network is a relatively
helping traffic management department to analyze service condition private environment, and the behavior and mental state of the driver in
of road and vehicles by the use of information analyzed in cognition the intra-vehicle network can reflect his recent living status. American
layer, thus to relieve traffic jam and improve road condition. Time Use Survey (ATUS) reports the average time of American people
in the vehicle [23] and the result indicates that vehicle has become the
4. Cognitive design issues in multi-scale networks essential part in people’s life. Thus, the long-term cognitive analysis of
intra-vehicle network data is a significant work. In addition, Pope et al.
The key issues of CIoV lies in how to excavate the information of all [24] discusses the influence of basic information (age, gender and ex-
participants (Fig. 1) and enhance safety of physical space and security ecutive capability) of the owner. Zhang et al. [25] discusses the in-
of network space; in other words, CIoV aims to (1) promote user ex- fluence of the living status of the owner such as stress disorder on
perience based on personal demands, (2) improve driving safety of driving. Chen et al. [26] discusses machine learning methods on dis-
traffic system, (3) strengthen data security of network environment, (4) ease prediction problems. These works show that the long-term data
optimize the distribution of network resources. Due to the collection, processing and cognition of personal information can further
6
M. Chen et al. Computer Communications xxx (xxxx) xxx–xxx
guide the driving behaviors of the driver to avoid the occurrence of the safety of personnel in the vehicle. To improve such situation, the
traffic accidents, and also can improve the user experience in all aspects cognitive intra-vehicle network carries out the emotion analysis,
with customized service. In short, cognitive intra-vehicle network can driving behavior surveillance, and physical health surveillance. The
help to adjust the living status of the user and provide guidance in many camera of the intra-vehicle network can entrust the facial expression
aspects (such as driving, health, work, amusement and diet) to the user. data of the driver to the vehicle-mounted edge device for analysis. As
Specifically, the vehicular embedded sensors (such as camera, na- for the driving behavior detection, the camera detects the eyelid state
vigator, and speedometer) collect the intra-vehicle network data mainly and micro-nod of the driver, to discover the micro-sleep behavior ef-
involving picture, voice, video, physical health data, and driving re- fectively, analyze in combination with the data collected by the devices
lated data. Then the vehicular edge cloud carries out data processing such as steering wheel and intelligent odometer embedding in the
and real-time data analysis. The vehicular terminal can finish the real- sensor, remind and give an early warning to the driver, and prevent the
time computing of most tasks locally. However, the storage space of occurrence of traffic accidents. In addition, the healthy and physiolo-
local device is limited, so the vehicular edge need to upload the local gical index data of each passenger and driver can be collected by the
data to user’s private cloud for processing and storage in the time when smart clothing and other wearable device, and uploaded to the vehicle-
user leaves the vehicle. The cloud carries out the analysis training of the mounted edge for real-time analysis. D2D can be applied in this situa-
private data involving basic information, driving behavior, emotion, tion [31]. The vehicle-mounted edge assesses the health grade of each
health condition, then converts original data to personalized rules by user by the data cognitive engine, and reports the analysis result to
cognitive computing. These rules reflect past and current living status user’s smart phone. Users in the same intra-vehicle network can select
records of the owner (involving driving habits, health history and travel the visible window sharing the health status. If the driver suddenly feels
period). In other words, the cooperation of vehicular edge and remote unwell (such as burst of acute disease) during the driving activity, the
cloud maps the history record and real-time information of the user into vehicle-mounted edge will perceive the critically ill condition of the
an iterative living habit document for the purpose of further guiding driver timely from the data collected by the smart clothing, adopt the
specific user. Particularly in the case of emergence, for example, if the safety automatic driving mode timely, and give an alarm to the nearby
driver is detected of fatigue driving, the vehicular edge will send the vehicles and cloud. The cloud will dispatch more resources (commu-
abnormal message to the cloud. Due to small data size but high real- nication resources of cellular mobile network, and computing resources
time performance of such message, communication mode of mobile of remote data center, nearby vehicles and RSU) to carry out deeper and
network will be adopted. The cloud will make the emergency treatment more comprehensive condition analysis for the ill driver. At the same
in view of different message types, involving taking measures such as time, the cloud rapidly contacts the ambulance, doctor and driver’s
playing the prompt voice or music to stimulate the emotion and fatigue home [32]; and delivers the analysis result to the doctor, so as to make
state of the driver, and adjusting the vehicular device into the safe the diagnostic analysis for patients by the time of the ambulance is on
autonomous driving mode forcibly. The safety driving can be guaran- the way and enhance the survival rate of the ill driver.
teed by the cooperative effect of long-term behavioral cognition and
real-time behavioral detection. 4.2. Cognition in inter-vehicle network
4.1.2. Mobile cognitive application based on interactions among multiple Inter-vehicle network is composed of all vehicles that can commu-
smart devices nicate and share flexible resources. There are various communication
In recent years, due to rapid development of artificial intelligence modes of inter-vehicle network, involving the road edge communica-
and chip design, the quantity of mobile intelligent device is increased. tion, V2V communication, and mobile network communication. The
For example, it is estimated that 325 million devices in 2016 will be intelligent autonomous moving object is the most important element in
significantly increased to about 929 million devices in 2021 glob- CIoV. Therefore, we take the inter-vehicle network into consideration,
ally [27]. The mobile intelligent devices include smart phone, aug- and solve the issues of data instability collected by the vehicle and 5G
mented reality helmet, smart clothing, and smart watch, etc. These network resource optimization distribution by the cooperation cogni-
enhanced devices have been commonly defined as Smart Objects tion of inter-vehicle network.
(SOs) [28,29] and represent fundamental building block for all IoT
scenarios, including IoV. 4.2.1. IoV stable service modeling based on group cognition
Under the environment of vehicle-mounted edge cloud, the strict Due to diversity of real road environment, the environmental data
requirements on latency and reliability of the majority of mobile in- reported by the vehicle has the definite error. At the same time, the
telligent devices can be met. NB-IoT technology can also enhance the high-speed mobility characteristic of vehicle and the lower stability of
seamless connection among numerous devices [30]. Meanwhile the wireless channel result in the vehicle data failing to reach timely and
mobile intelligent device can enhance the user experience of vehicle- the larger data delay variation. Excavating the useful information from
mounted environment, provide the convenient channel of information, the data with the definite error and delay variation becomes the key
and facilitate other aspects of people’s life based on different applica- issue of realizing the IoV landing. Next, the density of vehicle data
tions. We explain the mobile cognitive application based on multi-in- service is characterized by the non-uniformity of spatial and time do-
telligent device interaction with the example of mobile health surveil- mains. In the spatial domain, the service volume of vehicles in the rural
lance (Fig. 5). area is very large, while the vehicle data service volume in rural area
The health status of the driver not only influences his/her own and expressway is relatively low. In the time domain, the change in
safety, but also influences the safety of passenger in the vehicle, safety data traffic is drastic. The data traffic is large in the traffic peak period,
of other drivers, and even the traffic system security. In the case of poor while it is opposite in the work and rest time. The dynamic change of
health or fatigue driving of the driver, the attention of the drive will be vehicle data traffic in the spatial and time domains is very large, thus it
significantly reduced, and the response time will be increased, which is very urgent to meet the flexibility and intelligent demand of IoV in
often results in the occurrence of traffic accidents. Therefore, it is very the issues of site deployment, hotspot coverage, and resource dis-
important to monitor the physical health of the driver during the tribution. In a word, the data instability and the high requirements on
driving process. latency and reliability of communication link are two challenges of IoV
Under the traditional driving environment, the passenger and driver service modeling.
fail to understand the mutual healthy conditions before. Due to wea- The cooperation cognition of inter-vehicle network can enhance the
kened state of consciousness, the tired driver even fails to know his/her stability of IoV service modeling. The theoretical model of vehicle data
own status but selects to drive continuously, which greatly threatens service built by CIoV includes three aspects of space, time and mobility.
7
M. Chen et al. Computer Communications xxx (xxxx) xxx–xxx
5G network slice can create the special slice for the service with
different requirements in CIoV. Specifically, it places the virtual net-
work function in different positions (i.e., edge cloud or core cloud) in
accordance with different service characteristics. The operator can
customize different network slices (such as billing and strategy control)
in the way as needed by the service to meet the user demands, and also
it is the most cost-effective mode. Chen et al. [33] addresses green and
Fig. 6. 5G network slicing resource cognition. mobility-aware caching issues in 5G networks. However, more research
is required to discuss 5G resource allocation problems under the fra-
mework of Artificial Intelligence. In CIoV, we set forth the design idea
The space model depicts the spatial position of the service data flow.
of solving and realizing the closed loop optimization by double cogni-
The time model depicts the dynamic change of each data flow with the
tive engine (Fig. 6).
time. The mobility model depicts the change of service spatial position.
Based on different demands (latency, reliability and flexibility) for
When the cognitive IoV bears the data service, the space-time dis-
different cognitive applications [40,41], the network slice service re-
tribution of data service and the IoV transport service are closely cou-
quest types of IoV are also different. The data cognitive engine will
pled. By utilizing cognitive computing to tackle waving error and delay
combine the current resource distribution and real-time request of
variation problems of vehicle data, useful information is extracted, the
lessee, and carry out the fusion cognitive analysis of isomerous data in
generalization ability of theoretical model is improved and the gen-
the methods of machine learning and deep learning. Then, the data
eralization error is under control. From the perspective of group co-
cognitive engine reports the dynamic flow pattern analyzed to the re-
operation of inter-vehicle network, the mutual cognition between the
source cognitive engine. Resource cognitive engine jointly optimizes
intelligent moving objects in the group can greatly enhance the pure
the comprehensive benefits and resource efficiency. Firstly, resource
vision-based environmental perception. Next, the sharing map mod-
cognitive engine controls and screens the access request, then conducts
eling information by the group cooperation is more reliable. Finally by
the dynamic scheduling distribution of resources based on cognition to
cognition of vehicle behavior in the time and space, the transport me-
network resources, and feeds back the scheduling result to the data
chanism of vehicle data service with space-time random characteristics
cognitive engine, to realize the closed loop optimization. By utilizing
in IoV is established, and prediction accuracy of IoV service is en-
double cognitive engines for the dynamic scheduling of 5G network
hanced, to enable the limited resources to provide an outstanding ser-
slice technology, different service qualities in IoV can be satisfied, total
vice.
costs can be saved and the operation efficiency of network resources
can be enhanced. Besides that, in order to further improve QoE of 5G
4.2.2. Optimization of 5G network slicing resources distribution based on network slicing, a delay announcement method is a new research di-
dynamic demand rection [34].
There are different vehicle types on the traffic roads, such as private
car, public car, freight wagon, ambulance, and police car. The vehicle- 4.3. Cognition in beyond-vehicle network
mounted device abilities (ratio of computing resource and flexible re-
source) and actual service demands (driving speed and passenger car- From the large scale, CIoV can collect and analyze the physical
rying capacity, whether being the special service) in different vehicle space data, involving data of intra-vehicle network driver, driving data
types are different. There are many different intelligent application of adjacent vehicles, and beyond-vehicle road environmental data. We
demands inside CIoV, such as personal customizing information service, provide the scheme of strengthening the cooperation of beyond-vehicle
safety unmanned system, and real-time health surveillance system. In network, intra-vehicle network, and inter-vehicle network to achieve
addition, the resource demand in driving is often changed dynamically. the total improvement of road traffic safety in CIoV in Fig. 7. On the
Thus, the traditional fixed resource distribution mode fails to satisfy the other hand, due to its complexity, IoV has very strict requirements for
future driving environment. the reliability of network safety. For the avoidance of events such as
8
M. Chen et al. Computer Communications xxx (xxxx) xxx–xxx
personal information disclosure, traffic accident and breakdown of road conditions, classifying in accordance with the characteristics such as
system, CIoV realizes the network safety protection by the joint cog- gray level, edge, texture, color and gradient histogram of target in the
nition of physical space and network space. follow-up cognition process, identifying the pedestrian target, ana-
lyzing the height, age and other information of pedestrian, and pre-
dicting the dangerous acts of target. Started in the 1990s [35], the re-
4.3.1. Road traffic safety enhancement mechanism based on cooperation of
search of pedestrian behavior identification had been extensively
intra-vehicle network, inter-vehicle network, and beyond-vehicle network
applied to the aspects of medical rehabilitation and virtual reality, and
As the most important element in CIoV, the intelligent autonomous
had mainly analyzed and inferred the object behavior by monitoring
moving object (represented by autonomous vehicle) has the correct
the activist behavior or surroundings. The behavior identification is
perception and comprehension ability for the surroundings, and also
cored by behavior classification. There are many researches focusing on
has the automatic decision-making ability. The unmanned vehicle
the performance of classification algorithm, such as [36] combining the
should recognize the driving behavior of the driver within the scope of
decision tree and discrete hidden markov model (DHMM) for distin-
intra-vehicle network. If the autonomous vehicle production, the people
guishing the behavior.
and the unmanned vehicle will coexist for a long term in the future, and
The mutually adjacent intelligent autonomous moving objects can
the driving behavior of the people should be understood by the un-
share the environmental map in real time based on the group cognition,
manned vehicle, so it is very important to research the cooperation
to obtain more detailed and comprehensive perception. In addition,
driving mechanism between the unmanned vehicle and the driver. The
each intelligent autonomous moving object sends the collected traffic
driving state of each driver is the factor to be considered in the cog-
route information to the cloud, and the cloud gives the report of traffic
nitive IoV. Through the cognition of intra-vehicle network to the driver
road conditions based on the vehicle route. The traditional graph theory
state, the nearby inter-vehicle network shares the information, and
method (such as Dijkstra algorithm) and mathematical programming
gives an early warning to the driver in the fatigue state. The cloud will
method have large computing amount, to result in long computing
also distribute more resources to the driver in the fatigue state, and
time. Moreover, according to the geometrical distance and road quality,
promote the ability of traffic system in dealing with the emergency
it computes the optimal route and fails to describe the time variation of
situation.
real-time traffic network objectively. CIoV makes the real-time de-
For the cognition of beyond-vehicle environment, the intelligent
scription of traffic network possible, and also has the function of traffic
autonomous moving object perceives the environment and collects the
flow prediction, to enable the large-scale topological network modeling
data by a series of self-equipped sensors such as radar, camera, navi-
to be more exquisite. The dynamic route planning optimization based
gator, and intelligent speedometer, and then finishes the environment
on real-time traffic update of CIoV and the mechanism and algorithm of
modeling and builds the real-time 2D/3D map in accordance with the
exploring the distributed solution of group intelligent moving object
observed data and existing map, and automatically decides how to
will be an important research direction of IoV.
control the action of moving object by the motion trajectory planning
modeling. At present, MODAT (Moving Object Detection and Tracking)
technology of the intelligent autonomous object can solve the detection 4.3.2. Network data security strengthening mechanism based on joint
and trajectory tracking of (multiple) dynamic objects, and meanwhile analysis of physical space and network space
predict their future trajectory, and accordingly provide the necessary The network environment of IoV is different from the traditional
information for the real-time obstacle avoidance planning. In addition, network environment. Once the attacker invades, the remote control is
in the beyond-vehicle network environment, predicting the behavior of conducted for the autonomous vehicle, to result in inestimable harms
pedestrians is the most important category in the obstacle behavior and threaten the life and property of the driver, and the massive net-
prediction. The pedestrian detection and behavior prediction are aimed work attack even affects the breakdown of overall traffic system.
at detecting the nearby pedestrian target in the complex road Therefore, the network security of IoV is very important. However, the
9
M. Chen et al. Computer Communications xxx (xxxx) xxx–xxx
10
M. Chen et al. Computer Communications xxx (xxxx) xxx–xxx
6. Open issues
11
M. Chen et al. Computer Communications xxx (xxxx) xxx–xxx
12
M. Chen et al. Computer Communications xxx (xxxx) xxx–xxx
Commun. 15 (4) (2016) 2756–2769. [31] L. Zhou, Mobile device-to-device video distribution: theory and application, ACM
[17] N. Zhang, S. Zhang, P. Yang, O. Alhussein, W. Zhuang, X.S. Shen, Software defined Trans. Multimedia Comput. Commun. 12 (3) (2016) 1253–1271.
space-air-ground integrated vehicular networks: challenges and solutions, IEEE [32] M. Chen, J. Yang, X. Zhu, X. Wang, M. Liu, J. Song, Smart home 2.0: innovative
Commun. Mag. 55 (7) (2017) 101–109. smart home system powered by botanical iot and emotion detection, Mobile
[18] G. Fortino, W. Russo, C. Savaglio, M. Viroli, M. Zhou, Modeling opportunistic iot Networks Appl. 22 (2017) 1159–1169.
services in open iot ecosystems, 17th Workshop From Objects to Agents WOA 2017, [33] M. Chen, Y. Hao, L. Hu, K. Huang, V. Lau, Green and mobility-aware caching in 5g
Scilla, Italy, 2017, pp. 90–95. networks, IEEE Trans. Wireless Commun. 16 (12) (2017) 8347–8361.
[19] P. Schroeder, M. Meyers, L. Kostyniuk, National survey on distracted driving atti- [34] L. Zhou, Qoe-driven delay announcement for cloud mobile media, IEEE Trans.
tudes and behaviors - 2012, 2013. Circuits Syst. Video Technol. 27 (1) (2017) 84–94.
[20] A. Jain, H.S. Koppula, B. Raghavan, S. Soh, A. Saxena, Car that knows before you [35] T. Darrell, A. Pentland, Space-time gestures, Proceedings of IEEE Conference on
do: anticipating maneuvers via learning temporal driving models, Proceedings of Computer Vision and Pattern Recognition, (1993), pp. 335–340.
the IEEE International Conference on Computer Vision, (2015), pp. 3182–3190. [36] S. Reddy, M. Mun, J. Burke, D. Estrin, M. Hansen, M. Srivastava, Using mobile
[21] Y. Huo, W. Tu, Z. Sheng, V.C.M. Leung, A survey of in-vehicle communications: phones to determine transportation modes, ACM Trans. Sens. Netw. 6 (2) (2010)
requirements, solutions and opportunities in iot, Internet of Things (2015) 1–27.
132–137. [37] X. Hou, Y. Li, M. Chen, D. Wu, D. Jin, S. Chen, Vehicular fog computing: a view-
[22] R. Covello, G. Fortino, R. Gravina, A. Aguilar, J.G. Breslin, Novel method and real- point of vehicles as the infrastructures, IEEE Trans. Veh. Technol. 65 (6) (2016)
time system for detecting the cardiac defense response based on the ECG, 3860–3873.
Proceedings of IEEE International Symposium on Medical Measurements and [38] G. Aloi, J. Netw, et al., Enabling iot interoperability through opportunistic smart-
Applications, (2013), pp. 53–57. phone-based mobile gateways, Comput. Appl. 81 (2017) 74–84.
[23] B.o. L. Statistics, American time use survey, 2017, https://www.bls.gov/news. [39] C. Szegedy, et al., Going deeper with convolutions, 2015 IEEE Conference on
release/pdf/atus.pdf. Computer Vision and Pattern Recognition (CVPR), (2015), pp. 1–9.
[24] C.N. Pope, T.R. Bell, D. Stavrinos, Mechanisms behind distracted driving behavior: [40] Y. Zhang, et al., Temporec: temporal-topic based recommender for social network
the role of age and executive function in the engagement of distracted driving, services, Mobile Networks Appl. 22 (6) (2017) 1182–1191.
Accid. Anal. Prev. 98 (2017) 123–129. [41] Y. Zhang, Grorec: a group-centric intelligent recommender system integrating so-
[25] Y. Zhang, M. Chen, N. Guizani, D. Wu, V. Leung, SOVCAN: safety-oriented vehicular cial, mobile and big data technologies, IEEE Trans. Serv. Comput. 9 (5) (2016)
controller area network, IEEE Commun. 55 (8) (2017) 94–99. 786–795.
[26] M. Chen, Y. Hao, K. Hwang, L. Wang, L. Wang, Disease prediction by machine [42] K.K. Venkatasubramanian, A. Banerjee, S.K.S. Gupta, PSKA: usable and secure key
learning over big healthcare data, IEEE Access 5 (1) (2017) 8869–8879. agreement scheme for body area networks, IEEE Trans. Inf. Technol. Biomed. 14 (1)
[27] C.W. paper, Cisco visual networking index: global mobile data traffic forecast up- (2010) 60–68.
date, 2016–2021, White Paper (2017). [43] H. Rahimi-Eichi, U. Ojha, F. Baronti, M.Y. Chow, Battery management system: an
[28] G. Fortino, A. Rovella, W. Russo, C. Savaglio, On the classification of cyberphysical overview of its application in the smart grid and electric vehicles, IEEE Ind.
smart objects in the internet of things, Proceedings of International Workshop on Electron. Mag. 7 (2) (2013) 4–16.
Networks of Cooperating Objects Smart Cities, Berlin, Germany, (2014), pp. 3–9. [44] Y. Zhang, et al., Health-CPS: healthcare cyber-physical system assisted by cloud and
[29] G. Fortino, A. Guerrieri, W. Russo, C. Savaglio, Towards a development metho- big data, IEEE Syst. J. 11 (1) (2017) 88–95.
dology for smart object-oriented iot systems: a metamodel approach, Proceedings of [45] Q. Liu, Y. Ma, M. Alhussein, Y. Zhang, L. Peng, Green data center with iot sensing
IEEE Systems, Man and Cybernetics (SMC 2015), (2015), pp. 1297–1302. and cloud-assisted smart temperature controlling system, Comput. Networks 101
[30] M. Chen, Y. Miao, K. Hwang, Narrow band internet of things, IEEE Access 5 (2017) (2016) 104–112.
20557–20577.
13