Big Data Mining Using K-Means and DBSCAN Clustering Techniques

The World Wide Web industry generates big and complex data such as web server log files. Many data mining techniques can be used to analyze log files to extract knowledge and valuable information for both organizations and web developers. Large amounts of heterogeneous data are generated by websites, performing effective analysis on these data and transforming them into useful information using the existing traditional techniques is a challenging process. Therefore, this paper aims to analyze and cluster the log file data to get useful information that helps understand the users' behavior. A variety of data mining techniques were used to address the problem; three steps of data pre-processing were applied, namely the cleaning of data, the identification of users, and the identification of sessions. Results obtained after pre-processing phase showed that the data quality will improve when the number of records reduced by (51.45). The density-based spatial clustering of applications with noise (DBSCAN) and the K-means algorithm were used to develop clustering algorithms. Density-based clustering with three clusters outperformed the K-Means algorithm with three clusters in terms of accuracy. © 2022, The Author(s), under exclusive license to Springer Nature Switzerland AG.