Machine Learning Based Big Data Analytics Framework For Discovering Of Patterns From Multiple Data Sources

Abstract

Organization that scatters over different regions that perform multi-state business transaction are always interested I identifying novel patterns of interest given augmenting nosiness volumes. On a daily basis, we encounter an unparalleled increase in data from many sources, which adds to the concept of big data in terms of its size, speed, and diversity. These datasets provide significant obstacles to analytics frameworks and processing resources, so making the total study arduous for extracting useful information promptly. Therefore, in order to overcome these types of difficulties, it is crucial to create a very effective framework for analyzing large amounts of data. Therefore, in order to tackle these difficulties by harnessing non-linear connections from extensive and complex information, analytics frameworks are using machine  (ML) and  (DL) methods. Apache Spark is widely recognized as the most efficient tool for processing large amounts of data. It is particularly useful for solving complex machine learning problems that need several iterations. Spark MLlib, a distributed machine learning library, is used for this purpose. When dealing with research problems in the real world, architectures like Long Short-Term Memory  in deep learning are a useful method for addressing practical challenges like as decreased accuracy, long-term sequence dependence, and the problem of disappearing and expanding gradients in traditional deep architectures. This study proposes a very efficient analytics system that combines a progressive machine learning method with Spark-based linear models, Multilayer Perceptron , and Long Short-Term Memory. The accuracy of the system's predictions is improved by the use of a two-stage cascade structure. The architecture that we provide makes it possible to organize statistical analysis of large amounts of data in a way that is not only scalable but also effective. We applied the cascading structure on two different real-world datasets in order to illustrate the effectiveness of our framework. Furthermore, the results of the experiments show that our analytical framework is superior to the approaches that are considered to be state-of-the-art in terms of classification accuracy.