A MapReduce-Based Nearest Neighbor Approach for Big-Data-Driven Traffic Flow Prediction

Abstract— In big-data-driven trafc ow prediction systems, the robustness of prediction performance
depends on accuracy and timeliness. This paper presents a new MapReduce-based nearest neighbor (NN) approach for trafc ow prediction using correlation analysis (TFPC) on a Hadoop platform. In particular, we develop a real-time prediction system including two key modules, i.e., ofine distributed training (ODT) and online parallel prediction (OPP). Moreover, we build a parallel k-nearest neighbor optimization classier, which incorporates correlation information among trafc ows into the classication process. Finally, we propose a novel prediction calculation method, combining the current data observed in OPP and the classication results obtained from large-scale historical data in ODT, to generate trafc ow prediction in real time. The empirical study on real-world trafc ow big data using the leave-oneout cross validation method shows that TFPC signicantly outperforms four state-of-the-art prediction approaches, i.e., autoregressive integrated moving average, Naïve Bayes, multilayer perceptron neural networks, and NN regression, in terms of accuracy, which can be improved 90.07% in the best case, with an average mean absolute percent error of 5.53%. In addition, it displays excellent speedup, scaleup, and sizeup. < final year projects >