Efficient Data Mining in Big Data using Machine Learning Algorithms
DOI:
https://doi.org/10.15680/IJCTECE.2019.0202001Keywords:
Big Data, Data Mining, Machine Learning Algorithms, Supervised Learning, Unsupervised Learning, Scalability, Pattern Recognition, Data Analytics, Feature Selection, Model OptimizationAbstract
The exponential growth of data in various domains necessitates the development of efficient data mining techniques to extract meaningful patterns and insights. Traditional data mining methods often struggle to handle the volume, variety, and velocity characteristic of big data. Machine learning (ML) algorithms have emerged as powerful tools to address these challenges, offering scalable and adaptive approaches to data analysis. This paper explores the integration of ML algorithms in big data analytics, focusing on their efficiency in processing large datasets and uncovering hidden patterns. We examine various ML techniques, including supervised and unsupervised learning methods, and their applications in big data scenarios. The paper also discusses the challenges associated with implementing these algorithms at scale and the strategies employed to overcome them. Through a comprehensive review, we highlight the significance of efficient data mining in big data using ML algorithms and the potential for future advancements in this field.
References
1. Han, J., Kamber, M., & Pei, J. Data Mining: Concepts and Techniques. Morgan Kaufmann.
2. Aggarwal, C. C. Data Mining: The Textbook. Springer.
3. Dean, J., & Ghemawat, S. MapReduce: Simplified data processing on large clusters. Communications of the ACM, 51(1), 107-113.
4. Zaharia, M., Chowdhury, M., Franklin, M. J., Shenker, S., & Stoica, I.. Spark: Cluster computing with working sets. HotCloud, 10(10-10), 95.
5. Chen, T., & Guestrin, C. XGBoost: A scalable tree boosting system. Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 785-794.
6. Lecun, Y., Bengio, Y., & Hinton, G. Deep learning. Nature, 521(7553), 436-444.
7. Friedman, J., Hastie, T., & Tibshirani, R. The Elements of Statistical Learning. Springer.
