Class Imbalance: When 99% Accuracy Means Nothing

Here's my favourite machine learning magic trick. I'll build a fraud detection model with 99% accuracy in one line:
def predict(transaction):
return "legitimate"
Applause, please. If 99% of transactions are legitimate, this model is right 99% of the time. It also catches exactly zero fraud, which is the only thing it was built to do.

Why accuracy lies
Accuracy counts all correct predictions equally. When one class dominates, a model can score highly just by always predicting the majority. This happens everywhere: fraud, disease screening, equipment failure, spam on a quiet day, rare defects in manufacturing.
Better questions, better metrics
Instead of "how often are we right?", ask:
- Recall: of all the actual fraud, how much did we catch?
- Precision: of everything we flagged as fraud, how much really was fraud?
from sklearn.metrics import classification_report
print(classification_report(y_test, y_pred, digits=3))

There's always a trade-off. Flag more transactions and recall goes up, but precision falls and your customers get annoying calls about their perfectly normal coffee purchase. Which trade-off is right depends on the cost of each mistake: a missed cancer diagnosis and a false spam alert are not equally bad.
Other good tools:
- Precision-recall curves and the area under them, which focus on the rare class.
- F1 score, a single number balancing precision and recall (useful, but don't hide behind it).
- A confusion matrix, the humble table that shows exactly what kinds of mistakes you're making.
Ways to handle imbalance
Adjust the decision threshold. Many models output a probability. You don't have to flag at 0.5; maybe 0.2 is right for your costs. This is often the cheapest and most effective fix.
proba = model.predict_proba(X_test)[:, 1]
y_pred = (proba >= 0.2).astype(int)
Use class weights, so mistakes on the rare class cost more during training:
from sklearn.linear_model import LogisticRegression
model = LogisticRegression(class_weight="balanced", max_iter=1000)
Resample carefully. Undersample the majority or oversample the minority (techniques like SMOTE create synthetic examples). Only ever resample the training data; your test set must look like the real world.
Get more examples of the rare class, if you can. Sometimes the best model improvement is a conversation with the team that labels the data.
The takeaway
When someone shows you a model with 99% accuracy, your first question should be "what percentage is the majority class?" If the answer is 99%, you're looking at the one-line magic trick, just with more GPUs.