Optimized Resource Allocation on Hybrid Cloud Network For Big Data Applications Using Statistical Analysis

Abstract-The stunning growth in big data platforms has transformed the process of analyzing terabytes of data generated for creating business values in large enterprises. The use of cloud computing platforms triggers this process by providing flexible, scalable and on demand computing resources. But these resources are limited to an organization and it is very important to allocate them wisely by improving its performance and availability at a minimized cost without violating service-level agreement (SLA). Therefore this proposed research focuses on simplifying resource allocation through two statistical algorithms namely Big Data Application Resource Allocation (BRA) algorithm and Get Availability (GA)  algorithm. BRA algorithm will take input parameters such as cost, performance, availability from hybrid cloud network to get the best optimal solution for resource allocation and GA algorithm is used to and total failure probability of the solution to calculate total availability. The proposed implementation plan use Open Nebula for setting up the hybrid infrastructure for capturing trace generated by dummy big data operations and use Apache Spark for statistical modeling of algorithms and real time monitoring of network trace. The outcome of the research is new recommendations on optimizing processors, data storage, network and availability.