15.5 Prediction and Accuracy

  • Using testing set and confusion matrix to compare accuracies

15.5.1 Model-1

pred1 = predict(cv_tree1, newdata = d_test1)
confusionMatrix(pred1, d_test1$Creditability)
Confusion Matrix and Statistics

          Reference
Prediction   0   1
         0  33  10
         1  27 130
                                          
               Accuracy : 0.815           
                 95% CI : (0.7541, 0.8663)
    No Information Rate : 0.7             
    P-Value [Acc > NIR] : 0.0001479       
                                          
                  Kappa : 0.5207          
                                          
 Mcnemar's Test P-Value : 0.0085289       
                                          
            Sensitivity : 0.5500          
            Specificity : 0.9286          
         Pos Pred Value : 0.7674          
         Neg Pred Value : 0.8280          
             Prevalence : 0.3000          
         Detection Rate : 0.1650          
   Detection Prevalence : 0.2150          
      Balanced Accuracy : 0.7393          
                                          
       'Positive' Class : 0               
                                          

15.5.2 Model-2

pred2 = predict(cv_tree2, newdata = d_test1)
confusionMatrix(pred2, d_test1$Creditability)
Confusion Matrix and Statistics

          Reference
Prediction   0   1
         0  32   9
         1  28 131
                                          
               Accuracy : 0.815           
                 95% CI : (0.7541, 0.8663)
    No Information Rate : 0.7             
    P-Value [Acc > NIR] : 0.0001479       
                                          
                  Kappa : 0.5157          
                                          
 Mcnemar's Test P-Value : 0.0030846       
                                          
            Sensitivity : 0.5333          
            Specificity : 0.9357          
         Pos Pred Value : 0.7805          
         Neg Pred Value : 0.8239          
             Prevalence : 0.3000          
         Detection Rate : 0.1600          
   Detection Prevalence : 0.2050          
      Balanced Accuracy : 0.7345          
                                          
       'Positive' Class : 0