Classification with Reject OptionAuthor: Radu Herbei
Under the classic binary supervised learning framework, the statistician's task is to form a classifier which represents his/her guess of the label of a future observation. However, allowing for the reject option (I don‚'t know) besides taking a hard decision (0 or 1) is of great importance in practice. In a medical setting, for instance, when classifying whether a disease is present or absent, the reject option is sometimes preferred and leads to making the right decision after additional data is collected. We study binary classification that allows for a reject option, in which case no decision is made. This reject option is to be used for those observations for which the conditional class probabilities are close, and as such are hard to classify. We generalize existing theory for both plug-in rules and empirical risk minimizers to this setting. We also extend our approach to the situation when the type I and type II errors have distinct costs.