Data Mining In Big Data Analytics: Exploring Machine Learning Techniques For Pattern Recognition

Abstract

As the amount of data keeps adding at an exponential rate, Big Data Analytics is an increasingly critical field that needs such advanced machine learning-based data mining methods to efficiently find patterns. In this study, deep learning architectures, namely Convolutional Neural Networks (CNN) and Fully Connected Neural Networks (FCNN), are evaluated and compared regarding high dimensional feature extractions and classification with traditional Support Vector Machine (SVM) techniques. The implementation of the above-proposed framework was presented by training validated models on a high-dimensional dataset in TensorFlow and PyTorch. Classification effectiveness was assessed using performance metrics of accuracy, precision, recall, and F1-score. A PCA-based visualization was performed to analyze whether each model would extract the features well. Also CNN model has the highest accuracy i.e 93.5% compared to the accuracy of FCNN i.e 89.1 and SVM i.e 85.2 which proves its better hierarchical feature learning. It was also found that CNNs converged faster with 25 epochs, with SVM taking too long to converge and offering bad separability of the features, thus CNN towards FCNN models proved to be more effective for complex pattern recognition tasks for Big Data Analytics. Nevertheless, more research is needed to create computationally viable XAI and hybrid models for their real-world use.