Compare Classification Models
TModC.Rd
For the comparison of several classification models, the AUC values and BrierScore values of the models are determined and tabulated. Both the absolute values and the relative values are reported, each related to the model with the highest corresponding value.
Usage
TModC(..., newdata = NULL, reference = NULL, ord = NULL)
# S3 method for class 'TModC'
plot(x, col = NULL, args.legend = NULL,...)
Arguments
- ...
the models to be compared
- x
TModC
object to plot- newdata
the data to use for predicting. If not provided, the
model.frame
will be used.- reference
the reference values
- ord
character defining the order of the results table, can be any of
"auc"
,"bs"
,"auc_p"
,"bs_p"
,"bs_rnk"
,"auc_rnk"
,"ensemble"
(using the mean of"auc_p"
and"bs_p"
for the ranking).- col
the color for the lines in the ROC plot
- args.legend
the legend to be placed in the ROC plot
Value
a matrix with the columns
- auc
absolute value of area under the ROC curve (AUC)
- auc_p
percentage of the auc based on the best observerd AUC
- auc_rnk
the rank of the auc
- bs
absolute value of the Brier score
- bs_p
percentage of the Brier score based on the best observed BS
- bs_rnk
the rank of the BS
- auc_grnk
character representation of the AUC rank
- bs_grnk
character representation of the BS rank
See also
TMod
, BrierScore
, AUC
, ROC
Examples
d.pim <- SplitTrainTest(d.pima, p = 0.2)
mdiab <- formula(diabetes ~ pregnant + glucose + pressure + triceps +
insulin + mass + pedigree + age)
r.glm <- FitMod(mdiab, data=d.pim$train, fitfn="logit")
r.rp <- FitMod(mdiab, data=d.pim$train, fitfn="rpart")
mods <- list(glm=r.glm, rp=r.rp)
# the table with the measures
(tm <- TModC(mods, ord="auc"))
#> Setting levels: control = neg, case = pos
#> Setting direction: controls < cases
#> Setting levels: control = neg, case = pos
#> Setting direction: controls < cases
#> Error in eval(x$call$formula): object 'mdiab' not found
# plotting the ROC curves
plot(tm, col=c("darkmagenta", "dodgerblue"))
#> Error: object 'tm' not found