Year 2020, Volume 4 , Issue 1, Pages 11 - 21 2020-06-15

A Comparison of Software Defect Prediction Metrics Using Data Mining Algorithms

Zeynep Behrin GÜVEN AYDIN [1] , Rüya ŞAMLI [2]


Data mining is an interdisciplinary field that uses methods such as machine learning, artificial intelligence, statistics, and deep learning. Classification is an important data mining technique as it is widely used by researchers. Generally, statistical methods or machine learning algorithms such as Decision Trees, Fuzzy Logic, Genetic Programming, Random Forest, Artificial Neural Networks and Logistic Regression have been used in software defect prediction in the literature. Performance measures such as Accuracy, Precision, Mean Absolute Error (MAE) and Root Mean Squared Error (RMSE) are used to examine the performance of these classifiers. In this paper, 4 data sets entitled JM1, KC1, CM1, PC1 in the PROMISE repository, which are created within the scope of the publicly available NASA institution's Metric Data Program, are examined as in the other software defect prediction studies in the literature. These datasets include Halstead, McCabe method-level, and some other class-level metrics. Data sets are used with Wakiato Environment for Knowledge Analysis (WEKA) data mining software tool. By this tool, some classification algorithms such as Naive Bayes, SMO, K *, AdaBoost1, J48 and Random Forest were applied on NASA error datasets in PROMISE repository and their accuracy rates were compared. The best value among the accuracy rates was obtained in the Bagging algorithm in the PC1 data set with the values of %94.13.


Keywords: Software Defect Prediction, McCabe, Halstead, Data Mining, Accuracy, Random Forest


Cite this paper as:
GÜVEN AYDIN, Z.B., SAMLI, R. (2020). A Comparison of Software Defect Prediction Metrics Using Data Mining Algorithms. Journal of Innovative Science and Engineering. 4(1): 11-21

*Corresponding author: Zeynep Behrin GÜVEN AYDIN
E-mail: zeynepguven@maltepe.edu.tr


Received Date: 24/02/2020
Accepted Date: 05/05/2020
© Copyright 2020 by
Bursa Technical University. Available online at http://jise.btu.edu.tr/


The works published in Journal of Innovative Science and Engineering (JISE) are licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.


Software Defect Prediction, McCabe, , Halstead, Data Mining, Accuracy, Random Forest
  • [1] Gayatri, M. and Sudha, A. (2014). Software Defect Prediction System using Multilayer Perceptron Neural Network with Data Mining. International Journal of Recent Technology and Engineering (IJRTE), 3(2): 54-59.
  • [2] Menzies, T., Greenwald, J., and Frank, A. (2006). Data Mining Static Code Attributes to Learn Defect Predictors. IEEE Transactions on Software Engineering, 33(1): 2-13.
  • [3] Elish, K.O. and Elish, M.O. (2008). Predicting Defect-Prone Software Modules Using Support Vector Machines, Journal of Systems and Software, 81: 649-660.
  • [4] Lessmann, S., Baesens, B., Mues, C., and Pietsch, S. (2008). Benchmarking Classification Models for Soft ware Defect Prediction: A Proposed Framework and Novel Findings. IEEE Transactions on Software Eng-ineering, 34(4): 485-496.
  • [5] Moeyersoms, J., de Fortuny, E. J., Dejaeger, K., Baesens, B., and Martens, D. (2015). Comprehensible Software Fault and Effort Prediction: A Data Mining Approach. Journal of Systems and Software, 100: 80-90.
  • [6] Gyimothy, T., Ferenc, R., and Siket, I. (2005). Empirical validation of object-oriented metrics on open source software for fault prediction. IEEE Transactions on Software engineering, 31(10): 897-910.
  • [7] Dhankhar, S., Rastogi, H., and Kakkar, M. (2015) Software fault prediction performance in software engineering, 2nd International Conference on Computing for Sustainable Global Development (INDIACom), New Delhi, 11-13 March 2015, pp. 228-232. [8] Koru, A. G. and Liu, H. (2005). Building effective defect-prediction models in practice. IEEE software, 22(6): 23-29.
  • [9] Ma, Y., Guo, L., and Cukic, B. (2007). A Statistical Framework for the Prediction of Fault-Proneness. In Advances in Machine Learning Applications in Software Engineering IGI Global, 237-263.
  • [10] Wang, T. and Li, W. (2010). Naive Bayes Software Defect Prediction Model, 2010 International Conference on Computational Intelligence and Software Engineering, Wuhan, pp. 1-4.
  • [11] Wang, H., Khoshgoftaar, T. M., and Napolitano, A. (2011). An Empirical Study of Software Metrics Selection Using Support Vector Machine. In SEKE July, pp. 83-88.
  • [12] Choudhary, G. R., Kumar, S., Kumar, K., Mishra, A., and Catal, C. (2018). Empirical Analysis of Change Metrics for Software Fault Prediction. Computers & Electrical Engineering, 67: 15-24.
  • [13] Pandey, A. K. and Goyal, N. K. (2010). Predicting Fault-Prone Software Module Using Data Mining Technique and Fuzzy Logic. International Journal of Computer and Communication Technology, 2(2):56-63.
  • [14] Khoshgoftaar, T.M. and Seliya, N. (2002). Software Quality Classification Modeling Using the SPRINT Decision Tree Algorithm, In the proceedings of the 4th IEEE International Conference on Tools with Artificial Intelligence, Washington, DC, pp. 365-374.
  • [15] Thwin, M.M. and Quah, T. (2003). Application of Neural Networks for Software Quality Prediction Using Object-Oriented Metrics, In the proceedings of the 19th International Conference on Software Maintenance, Amsterdam, The Netherlands, pp. 113-122.
  • [16] Elish K.O. and Elish M.O. (2008). Predicting defect-prone software modules using support vector machines, Journal of Systems and Software, 81:649-660.
  • [17] Pai, G.J. and Dugan, J.B. (2007). Empirical Analysis of Software Fault Content and Fault Proneness Using Bayesian Methods, IEEE Transactions on Software Engineering, 33: 675-686.
  • [18] Yu, Menzies, T., Greenwald, J., and Frank, A. (2007). Data Mining Static Code Attributes to Learn Defect Predictors, IEEE Transactions on Software Engineering, 33: 2-13.
  • [19] https://machinelearningmastery.com/what-is-the-weka-machine-learning-workbench/ Accessed: 29 January 2020.
  • [20] Chaudhary, N., Mehta, G., and Bajaj, K. (2015). Comparison Of Classification Algorithms And Design Of A Percentage-Split Based Method For Data Classification, IJCSIT, 2(5):1-6.
  • [21] Aydilek, İ . (2018). Yazılım Hata Tahmininde Kullanılan Metriklerin Karar Ağaçlarındaki Bilgi Kazançlarının İncelenmesi ve İyileştirilmesi. Pamukkale Üniversitesi Mühendislik Bilimleri Dergisi, 24(5):906-914.
  • [22] http://promise.site.uottawa.ca/SERepository/datasets-page.html/Accessed:09.04.2020
Primary Language en
Subjects Engineering
Journal Section Research Articles
Authors

Orcid: 0000-0002-4121-8220
Author: Zeynep Behrin GÜVEN AYDIN (Primary Author)
Institution: MALTEPE ÜNİVERSİTESİ
Country: Turkey


Orcid: 0000-0002-8723-1228
Author: Rüya ŞAMLI
Institution: İSTANBUL ÜNİVERSİTESİ - CERRAHPAŞA
Country: Turkey


Supporting Institution TÜBİTAK
Project Number 118E682.
Thanks This research work was supported by The Scientific and Technological Research Council of Turkey (TÜBİTAK), Project Number: 118E682. Also, we are thankful to the PROMISE software engineering repository for providing free and easy access to the NASA defect data sets for use in our research.
Dates

Publication Date : June 15, 2020

Bibtex @research article { jise693098, journal = {Journal of Innovative Science and Engineering (JISE)}, issn = {}, eissn = {2602-4217}, address = {ursa Technical University, Mimar Sinan Campus, Mimar Sinan Mah. Mimar Sinan Blv. Eflak Cad. No:177 16310 Yıldırım, Bursa / Turkey}, publisher = {Bursa Technical University}, year = {2020}, volume = {4}, pages = {11 - 21}, doi = {10.38088/jise.693098}, title = {A Comparison of Software Defect Prediction Metrics Using Data Mining Algorithms}, key = {cite}, author = {Güven Aydın, Zeynep Behrin and Şamlı, Rüya} }
APA Güven Aydın, Z , Şamlı, R . (2020). A Comparison of Software Defect Prediction Metrics Using Data Mining Algorithms . Journal of Innovative Science and Engineering (JISE) , 4 (1) , 11-21 . DOI: 10.38088/jise.693098
MLA Güven Aydın, Z , Şamlı, R . "A Comparison of Software Defect Prediction Metrics Using Data Mining Algorithms" . Journal of Innovative Science and Engineering (JISE) 4 (2020 ): 11-21 <http://jise.btu.edu.tr/en/pub/issue/53898/693098>
Chicago Güven Aydın, Z , Şamlı, R . "A Comparison of Software Defect Prediction Metrics Using Data Mining Algorithms". Journal of Innovative Science and Engineering (JISE) 4 (2020 ): 11-21
RIS TY - JOUR T1 - A Comparison of Software Defect Prediction Metrics Using Data Mining Algorithms AU - Zeynep Behrin Güven Aydın , Rüya Şamlı Y1 - 2020 PY - 2020 N1 - doi: 10.38088/jise.693098 DO - 10.38088/jise.693098 T2 - Journal of Innovative Science and Engineering (JISE) JF - Journal JO - JOR SP - 11 EP - 21 VL - 4 IS - 1 SN - -2602-4217 M3 - doi: 10.38088/jise.693098 UR - https://doi.org/10.38088/jise.693098 Y2 - 2020 ER -
EndNote %0 Journal of Innovative Science and Engineering (JISE) A Comparison of Software Defect Prediction Metrics Using Data Mining Algorithms %A Zeynep Behrin Güven Aydın , Rüya Şamlı %T A Comparison of Software Defect Prediction Metrics Using Data Mining Algorithms %D 2020 %J Journal of Innovative Science and Engineering (JISE) %P -2602-4217 %V 4 %N 1 %R doi: 10.38088/jise.693098 %U 10.38088/jise.693098
ISNAD Güven Aydın, Zeynep Behrin , Şamlı, Rüya . "A Comparison of Software Defect Prediction Metrics Using Data Mining Algorithms". Journal of Innovative Science and Engineering (JISE) 4 / 1 (June 2020): 11-21 . https://doi.org/10.38088/jise.693098
AMA Güven Aydın Z , Şamlı R . A Comparison of Software Defect Prediction Metrics Using Data Mining Algorithms. JISE. 2020; 4(1): 11-21.
Vancouver Güven Aydın Z , Şamlı R . A Comparison of Software Defect Prediction Metrics Using Data Mining Algorithms. Journal of Innovative Science and Engineering (JISE). 2020; 4(1): 11-21.