Choose evaluation metrics that match the cost of false positives and false negatives.
The core idea
Accuracy can be misleading when classes are imbalanced. Precision asks how many predicted positives were correct; recall asks how many actual positives were found. F1 balances precision and recall, while ROC-AUC measures ranking across thresholds.
A practical way to approach it
Write down the error that matters most. For fraud review, false positives may waste analyst time; for a safety alert, missing a true case may be more costly. Inspect the confusion matrix and evaluate several thresholds instead of reporting one score without context.
Try it in a small project
Use an imbalanced toy dataset and compare accuracy with precision and recall. Then change the decision threshold and note how the error trade-off changes.
Common mistakes to avoid
No single metric captures every requirement. Include calibration, operational costs, subgroup performance, and data drift where relevant.
Key takeaway
Keep the first version small, make your assumptions explicit, and verify the result with a repeatable test. For production use, check the current official documentation for the tools and versions you choose.