{'fraction_non_alphanumeric': 0.026206786681535765, 'fraction_numerical': 0.004221761988210929, 'mean_word_length': 5.014131736526946, 'pattern_counts': {'":': 0, '<': 1, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 1, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Connected Vehicular Transportation: Data Analytics and Traffic-dependent Networking

With onboard operating systems becoming increasingly common in vehicles, the real-time broadband infotainment and Intelligent Transportation System (ITS) service applications in fast-motion vehicles become ever demanding, which are highly expected to significantly improve the efficiency and safety of our daily on-road lives. The emerging ITS and vehicular applications, e.g., trip planning, however, require substantial efforts on the real-time pervasive information collection and big data processing so as to provide quick decision making and feedbacks to the fast moving vehicles, which thus impose the significant challenges on the development of an efficient vehicular communication platform. In this article, we present TrasoNET, an integrated network framework to provide realtime intelligent transportation services to connected vehicles by exploring the data analytics and networking techniques. TrasoNET is built upon two key components. The first one guides vehicles to the appropriate access networks by exploring the information of realtime traffic status, specific user preferences, service applications and network conditions. The second component mainly involves a distributed automatic access engine, which enables individual vehicles to make distributed access decisions based on access recommender, local observation and historic information. We showcase the application of TrasoNET in a case study on real-time traffic sensing based on real traces of taxis.


II. DATA ANALYTICS FOR CVTS APPLICATIONS
The continuous monitoring on movements (e.g., safety services by short range v2x communications), mobility applications and vehicles condition monitoring would result in exponential growth of diverse source data which provides a wealth of information that is valuable to traffic management parties, drivers, repair shops and automakers.
In the following, we list some representative CVTS applications to describe their dependance on data analytics. 1) Real-time traffic estimation: By using the moving vehicles or smartphones on-board vehicles for data sensing, and uploading the sensing reports (such as time, location and heading direction of vehicles) to the data center, the realtime traffic conditions of the roads, such as average running speed and traffic density, can be achieved by data analysis.
The taxi/bus management system in Shanghai, China, represents a practical deployment of the CVTS platform.
Around 40, 000 taxis and buses of Shanghai are now equipped with the on-board GPS and sensors which periodically report the vehicle information (GPS location, velocity, heading direction, passengers on/off) in cycles ranging from 30 seconds to 5 minutes. This results in 65 million records transmitted to the traffic management center everyday through cellular networks, and enables multiple management purposes. Road traffic conditions can be estimated efficiently by sparse sensing and advanced estimation methods. For example, compressive sensing and matrix completion based methods are reported in [6,7] based on the GPS dataset of Shanghai.
2) Online navigation for connected vehicles: Traffic prediction is more difficult than traffic estimation. Fortunately, through correlation analysis of big data, traffic patterns can be gleaned more easily, faster and clearly than before. For example, a social proximity mobility pattern of vehicles is adopted in [8], i.e., each vehicle has a restricted mobility region around a specific social spot such as a financial and sport center. By using data analysis, the social spots of vehicles in the real-world can be identified. The traffic peak probably appear at the rush hour around the social spot (SP), which makes traffics predictable. Five SPs has been shown in Fig. 1. In another example, the traffic can also be predicted from the message published in social networks, such as network group events (e.g., big show and football game) information including time, place, and number of attendees. The traffic can be predicted to influx to the social places before the event and outflow after the event.
The traffic also has strong correlations with online navigation services: the more people in a particular geographic place search for the routes to a particular destination online, the more probably the traffic congestion happens on the route to the destination. In this case, the traffic can be predicted with the data from search engine. The online navigation server can provide more feasible path plans for vehicles.
3) Remote vehicle diagnostics and road condition warning: Based on the data collected from cars, the drivers can arrange better service interval by taking their own driving habits, and predicted wear and tear into account rather than conventional means based on defined number of kilometers. More importantly, valuable information on potential vehicle or road hazards can be delivered to drivers in realtime. For example, if a number of vehicles' traction control systems are activated at the same time and place, the cars in this area can be warned about "icy conditions", fuel-efficient driving in heavy traffic, and etc.

4) Fuel up or charge:
During travelling in the city, the electric vehicles/hybrid electric vehicles (EVs/HEVs) may make decision on what time and where to fuel up or load at location-specific charging piles by estimating the driving mileage according to real-time traffic conditions. The information provided by CVTS is valuable to design efficient ways of resource management in smart grid system in the city [9]. 5) Dynamic urban planning: Sensory data from vehicles and mobile devices provide a pervasive way to understand how people use the city's infrastructure and affect the city, including urban dynamics, energy consumption and environment impacts such as noise and pollution. The big data related to real-time traffic can be used to improve city's services. For example, the planning and management parties can estimate how residential and working areas in cities are connected temporally, what the dynamic correlation of traffic density and pollution level appears, and how to reduce operational costs by optimizing planning. It also creates feedback loops with vehicles to reduce energy consumption and environmental impact.

III. FEATURES OF DATA ANALYTICS IN CVTS
For the aforementioned applications, to explore the strong correlation among multi-source data is the key, which helps us capture the present road conditions and predict the future. The data collection for connected vehicles and related applications of CVTS distinguish with the traditional ones from the following three aspects: • From static sensing to dynamic sensing: As a large amount of traffic data can now be harnessed through ubiquitous roadside sensors, vehicles and mobile devices, static traffic sampling no longer makes as much sense.
Moreover, due to the complicated traffic conditions, accurate traffic estimation and prediction can hardly be achieved on small random samples, and require as much data as possible. Connected vehicles make it possible.
• From precise data to messy data [10]: In the applications, allowing for imprecision (for messiness) of data may be an advantage, rather than a shortcoming. We can infer the vehicles' direction, speed and position with messy GPS data thus traffic estimation can be improved to the level of predicting the traffic congestion in a particular road rather than a region in the city by using 65 millions of "dirty" (or "noisy") taxi GPS data rather than small precise samples from digital camera and loop detector [6]. The data could be messy if they cover as many streets as possible.
• From parametric data to nonparametric data: Correlations are useful in a small-data world, but they really shine in the context of large volume data and/or big data. For traffic sensing and prediction, different (complicated) traffic models can be assumed to parameterize the relationship of traffic flows. However, multi-source data in CVTS allows us to pick a nonparametric model with simpler algorithms, and it results in more accurate than the sophisticated solution [6,10].
To summarize, for data analytics of connected vehicles, we can relax the standards of allowable errors and increase messiness by combining different types of data information from different sources. In dealing with even more comprehensive datasets, we no longer need to worry so much about individual data points, but biasing the overall analysis. Through them we can glean insights more easily, faster, and more clearly than before. Correlations of multi-source data let us analyze the city traffic not by shedding light on its inner working but by identifying a useful proxy (e.g., the online navigation requests implies the possible congestion) for it. It is foreseeable that big data enhances the data analytics in CVTS to change the way of services provisioning in and enables connected vehicles from multi-dimensions of traffic, vehicular network and ITS.

IV. TRAFFIC-DEPENDENT NETWORKING FOR CVTS
To engineer an efficient and economic network architecture is the foremost issue to facilitate the data collection, decision feedbacks and traffic related services.

A. Network Framework
The network framework is challenged by following issues: • From a traffic sensing perspective, even with the broad mobility of vehicles and the dense deployment of static sensors on the road, it cannot be guaranteed that the traffic information for all the roads in all the time could be sensed. It therefore calls for an efficient and economical sampling way for traffic sensing. Crowdsensing by more vehicles could be one of the solutions in the very near future. cover the whole city, and the communication link between mobile device and eNB is more stable than that between mobile device with RSU. RSU is equipped with a wireless transceiver operating on DSRC and/or WiFi, and hence the transmission range is small compared with eNB. But it provides high-rate transmission for mobile devices. Due to the explosive growth of mobile data traffic, the cellular network nowadays is straining to meet the current mobile data demand and faces an increasingly severe overload problem. RSU is not only an alternative for V2I (vehicular to infrastructure) communications, but also enables an offloading for cellular networks [8].
Mobile Devices: We do not discriminate what kind of mobile devices they are, but care about what network they access to. Normally, smartphones can connect to cellular network through LTE/5G and VANET infrastructures through WiFi, while vehicles can additionally connect to VANET infrastructure and other vehicles through DSRC.
Since WiFi and DSRC technologies can be applied to drive-thru connection when they are moving on the road [12], we propose an automatic network access engine in the mobile devices to offload data originally targeted for cellular networks, which is referred to as the automatic offloading engine.
Central Controller: The central controller is connected to base stations (e.g., eNBs for LTE), RSUs and Internet backbones. It allocates the network radio resources based on the realtime traffic estimated by TMC, and service demands requested by mobile devices. It acts as an interface between the physical network routers and the network operators to specify network services. The controller builds a logical control plane separated from data plane.
Different from Internet Protocol (IP) based networks, such a frame enables mobile devices to move between different access interfaces without changing identities or violating specifications. The control function can be implemented by a protocol known as OpenFlow which enable controller to drive the access network edge hardware in order to create an easily programmable identity-based overlay on the traditional IP core.
Cloud: As the data analytics center for TMC and other service providers, the cloud receives data from static traffic sensors and mobile devices, and analyzes them for traffic estimation and prediction. Other traffic-related services are then analyzed based on the realtime traffic and data from other service providers. One key feature provided by cloud is the access guidance for the mobile devices to facilitate the automatic offloading engine.
As it is shown in Fig. 3, TrasoNET builds the connection of data collection, analytics and traffic-dependent networking from the following three aspects: 1) Firstly, the traffic big data are collected from static and mobile sensors through access network of TrasoNET.
The aforementioned static sensors (e.g., cameras and inductive loops) transmit the traffic data to Regional Traffic Management Center through wired networks. Ubiquitous data from Mobile Devices (e.g., embedded, tethered or integrated on-board units, and smartphone) could be transmitted through wireless access network.
For example, the probe vehicles (such as Taxis and buses) and floating cars (such as police cars from Public Security Bureau) in the city could provide sparse GPS data for preliminary traffic estimation. Then the trafficdependent networking mechanism to be introduced in Subsection IV-B could facilitate big data collection from ubiquitous Mobile Devices. More data improves the traffic estimation and other traffic related services.
2) Secondly, on the aggregation layer and application layer of TrasoNET, the data analytics provides the real-time regional and global traffic conditions. It facilitates the Central Controller to allocate wide-area network radio resources (e.g., base stations, RSUs and Internet backbones) according to the estimated traffic density, speed, Another key feature dependent on real-time traffic condition is the regional network access guidance for Mobile Devices, which realizes locally network resource management. As for Mobile Devices, the decision-making of network selection and handover can be given locally by the guidance-based access mechanism for efficiency and offloading purposes. In this sense, the data analytics take effects on the macroscopic, midscopic and microscopic network resource allocation and network access.

B. Traffic-dependent Network Access Mechanism
The access control of networks is one of the key mechanism to guarantee the real-time CVTS applications. In the framework of TrasoNET, we give a guidance-based automatic access mechanism for efficient and offloading purposes. From the perspective of network access, the aforementioned four components map the phases of guidance, information push and distributed decision-making into Access Recommender Console, Broadcasting and Automatic offloading Engine.
Access recommender console: To recommend an "optimum network" to vehicles based on multiple criteria, the cloud could apply intelligent computation methods to set the priority of network access for a specific region under realtime traffic condition. The well-known Analytic Hierarchy Process (AHP) for multi-criteria decision [13] is one Establish the Hierarchy Structure

Construct judgment matrix
Establish priorities among the elements of hierarchy  Fig. 4: Analytic Hierarchy Process for network selection of good solutions. The logical flowchart of AHP algorithm is given in Fig. 4. The key steps are introduced in the following.
• Model the network recommendation problem as a hierarchy which contains the goal, alternatives for reaching the goal, and criteria for evaluating alternatives.
• Establish priorities among the elements of the hierarchy by making a series of judgments based on pair-wise comparisons of these elements. The compared results construct a pair-wise comparison matrix A = a i j, i, j = 1, 2, · · · , n, where n is the number of criteria of second level,and every element a ij is based on a standardized comparison scale from equal importance to dominance.
• The pair-wise comparison matrix should satisfy transitive preference and strength relations, it is necessary to check its consistency. Calculate consistency indicators C.I., random consistency indicators RI, and get the consistency ratio CR = CI/RI. For example, consistency of judgement matrix is acceptable for the case of CR < 0.1.
• Synthesize these priority vectors to construct an overall priority vector and check the consistency again.
The final priorities of alternative networks for the "Optimum Network" can be got through the above algorithm.
Traffic density is the critical factor and reflects the feature of vehicles' mobility.

A. Data analytics for traffic sensing in CVTS
The taxies in Shanghai generate sensing reports every 30 seconds and report the readings of GPS, i.e. location, time of report, current speed and headings, to TMC through cellular networks. The TMC collects all the traffic reports and constructs a huge traffic matrix X = {x ij }, in which each entry x ij represents the traffic condition of the i-th road (e.g. average speed based on all the reports from PVs on the road) at the j-th duty cycle of a day.
For example, the location of each report is matched to one road by map matching algorithm, data from different PVs are fused to get the traffic matrix X. Since the PVs cannot cover all the roads for all the time, TMC needs to estimate the traffic of un-sampled road in the traffic matrix. Matrix completion is applied in [6,7] with the low rank property of the traffic matrix. The matrix completion based estimation could be computed in the cloud. The estimation result is then sent to TMC for traffic management and message publishing to the vehicles in the city through the traffic bulletin board or information push through cellular network.
The main idea of the real trace analytics is as follows. Firstly, estimate the values of average speed in the unsampled roads [15] with the constraints of the temporal continuity and bound of the traffic data (the speed limit), respectively. Secondly, use the sampled data, together with the estimated data, to solve the optimization problem by minimizing the rank of the traffic matrix. The so-called HaTTEM algorithm is presented in [7].  However, the integrity analysis of sensing report about specific roads tells the fact that only 69% of over 35, 000 roads in Shanghai have sensing reports for only 30% time of the day. There are not any GPS reports of taxies or buses in 17% roads within a whole day [6]. The coverage of taxies' traces in the city is quite uneven due to the aforementioned social proximity. So we need floating cars (FCs, e.g. police cars from Public Security Bureau and patrol cars from Traffic Management Department) to provide more data. These cars don't need to change the patrolling area, but just adjust the patrolling path for better traffic sensing. It is well-known that the disorder of samples can be expressed by entropy. The relation between the entropy and the estimation error is seen in Fig. 7.
By planning the paths of only 260 controllable FCs for the whole city of Shanghai, China, even with the 15% of current PV samples, the average entropy is reduced from 0.233 nats (unity of entropy) to 0.05 nats. Thus, it is seen from Fig. 7 that the estimation error could be reduced from 35% to 15% with the complimentary GPS data of FCs.

B. Recommendation algorithm for Traffic-dependent networking
The PVs and FCs connect not only to TMC for management, but also frequently to Internet for providing more emerging services, (e.g., the new free taxi calling services in Shanghai, China with mobile APPs called Diditaxi and Kuaidadi 1 ). Access to Internet would become a standard feature of future motor vehicles. However, simply using the cellular infrastructure for vehicle Internet access may result in an increasingly severe data overloading issue, which eventually would degrade the communication service performance of both traditional smartphone and vehicular mobile users. This advances of citywide free WLAN access in Shanghai make it possible to serve vehicular users in the near future. This article provides an access network recommendation mechanism for different network applications based on the estimated traffic which could be achieved by the method in Subsection V-A.
In order to demonstrates the feasibility of traffic-dependent networking, we provide in Fig. 8    for voice and video service, respectively. The data rates are RMB1/Mb for cellular network and RMB10/2Gb per month, respectively. Based on the aforementioned AHP method, the network access recommendation can be given based on the comparison matrices in Table I. It implies that Voice Service is sensitive to network access delay, while Video Service need more priority for bandwidth. Furthermore, we show the access network recommendation result in Fig. 9 for different service types according to the traffic condition (vehicle density) demonstrated on the bottom X-Y layer of Fig. 9.
For the traffic density, its second level pair-wise comparison matrix is formed as A 2×2 . The simulation result It is noted that Fig. 9 represents the average index values for the recommendation of cellular network and VANET, respectively. If there are only two accessible networks, the priority can be normalized. It is easily seen from Fig.   9 that cellular network is recommended for voice service in a large region around SP where the vehicular traffic density is relatively high. Therefore, cellular network is still the first choice for voice service, especially at SPs.
On the other hand, VANET is recommended to offload the cellular network for video service in the region close to SP (except SP due to QoS requirement). It indicates that the network selection/ handover is closely related to vehicular traffic condition, which is demonstrated the necessity of traffic-dependent networking.
With the access network recommendation, the procedure of distributed automatic network access decision-making could be shown in Fig. 5.There are 4 premise variables S, A, O, R , which represent Speed of vehicle, Application of network (i.e. voice or vedio), current Option of access network, and Recommendation of access network, respectively. The output is achievable QoS represented by Q. The fuzzy sets and corresponding membership functions for each premise variable can be seen in Fig. 5. S is in the range of 0 ∼ 80km/h. The fuzzifier for A, O and R is singleton. Hence, we have the following 16 fuzzy rules: • Rule 1: If S is Low, A is V oice, O is Cellular and R is Cellular, then Q could be level h ; • Rule 2: If S is Low, A is V oice, O is Cellular and R is V AN ET , then Q could be level h ; • · · · ; • Rule 16: If S is High, A is V ideo, O is V AN ET and R is V AN ET , then Q could be level l .
With defuzzifier of fuzzy inference result, the distributed automatic network access engine determines the network selection and handover. In order to avoid ping-pong handover due to the mobility and perturbation of QoS, set two thresholds for QoS and delay, respectively. Calculate the QoS improvement by switching the current network to the other. Only if the improvement exceeds the QoS threshold for the time longer than the delay threshold, the handover happens.

VI. CONCLUSION AND FUTURE RESEARCH TOPICS
This article describes an architecture called TransoNET for data analytics and networking in connected vehicles enabled transportation systems. To efficient manage network resources for CVTS applications, we describe the features of data analytics and subsequently introduce the traffic-dependent networking approach for data collecting.
It shows how vehicular traffic can be estimated by matrix completion and how the recommendation-automatic integrated method provides efficient guidance to vehicles for network accessing. The data analysis based on real traces of taxies gives an exemplary study on traffic sensing. In particular, we study a case of multiple-network selection by the combination of network access recommendation from cloud and automatic access engine in vehicles.
It has been demonstrated the necessity to explore the relationship between vehicular traffic and networking for providing real-time services in CVTS.
Based on the proposed CVTS architecture, potential research directions can be envisioned to improve the data analytics and networking performance from both cloud and vehicle sides. On the cloud side, big data processing algorithm can be incorporated, e.g. crowdsourcing technologies, for ubiquitous traffic sensing such that more vehicles could take the roles of PV and FC. The social patterns of the vehicles may be considered to improve the traffic crowdsensing. On the vehicle side, automatic network access engine needs low-complexity decisionmaking algorithms for explosively increasing infotainment services through vehicles to Internet connection. As the terminals of crowdsensing, the vehicles could be more intelligent by automatically adapting the cycles of sensing and reporting according to local vehicular traffic. We believe CVTS will attract enormous attention from academia and industry in the near future.