Khan, Adam and Ali, Asad and Khan, Jahangir and Ullah, Fasee and Faheem, Muhammad (2025) Using Permutation-Based Feature Importance for Improved Machine Learning Model Performance at Reduced Costs. IEEE Access, 13. 36421 – 36435. ISSN 21693536
Full text not available from this repository.Abstract
In Software Quality Assurance (SQA), predicting defect-prone software modules is essential for ensuring software reliability and consistency. This task is commonly achieved through Machine Learning (ML) techniques, but improving model performance typically incurs significant computational costs. These high computational costs and uncertain payoffs make most Software engineering researchers reluctant to optimize ML models. This creates a need for novel techniques that can achieve near-optimal performance of hyperparameter settings while maintaining the computational efficiency of default settings. To address this, we employed five ML models, Decision Tree, Ranger, Random Forest, Support Vector Machine, and k-nearest Neighbors, and optimized their parameters using the random search technique. Our experiments covered six diverse Software Fault Prediction (SFP) datasets, encompassing various software features, application domains, and defect patterns, to evaluate the approach’s generalizability and effectiveness. Moreover, the Permutation Feature Importance (PFI)-based model-agnostic method was employed to identify the top ten features most critical for model accuracy and efficiency. These selected features were used to retrain the ML models without hyperparameters (default settings) to determine whether similar performance could be achieved at low computational cost. The results show an average accuracy improvement of 77.39 and a 92.02 reduction in computational cost. The most important case attained a 99.25 accuracy improvement and a 96.77 cost reduction. Such results clearly show that PFI-based feature selection is capable of high performance at a fraction of computational cost, offering an efficient solution for software engineers to optimize ML models. © 2013 IEEE.
Item Type: | Article |
---|---|
Impact Factor: | Cited by: 1; All Open Access, Gold Open Access |
Uncontrolled Keywords: | Application programs; Computer software selection and evaluation; Cost engineering; Nearest neighbor search; Software quality; Support vector machines; Computational costs; Default setting; Hyper-parameter; Machine learning; Machine-learning; Model-agnostic technique; Permutation feature importance; Predictive accuracy; Software fault prediction; Software reliability |
Depositing User: | Mr Ahmad Suhairi Mohamed Lazim |
Date Deposited: | 08 Jul 2025 16:28 |
Last Modified: | 08 Jul 2025 16:28 |
URI: | http://scholars.utp.edu.my/id/eprint/38940 |