Ask for More Than Bayes Optimal: A Theory of Indecisions for Selective Hypothesis Testing

arXiv:2412.12807v4 Announce Type: replace-cross Abstract: Selective classification is a powerful tool for automated decision-making in high-risk scenarios, allowing classifiers to act only when confident and abstain when uncertainty is high. Given a target accuracy, our goal is to minimize the number of indecisions, which are observations that we do not automate. For difficult problems, the target accuracy may be unattainable without abstaining from making a decision. By using indecisions, we can target a misclassification rate below the Bayes error rate, while minimizing overall indecision ma