What is precision and recall in machine learning?

Precision and recall are commonly used metrics to measure the performance of machine learning models or AI solutions in general. This helps us understand how well the models are making predictions.

Let’s use the example of email SPAM prediction. Say you have a model that looks at an email and decides if it’s spam or not. To see how well it performs, you’ll want to compare it to human-generated labels, which we’ll call Real labels.

To demonstrate this, the table below shows some actual decals and predicted decals for the car (model). Now we assume that Spam Prediction is positive, not spam prediction is negative.

What is precision in ML?

Given this, intuitively, accuracy Measures the proportion of correct positive predictions.

As you can see from the above table, from 2 Spam (Positive) machine predictions, only 1 is correct. So the accuracy is 0.5 or 50%.

What is Recall in ML?

Let’s remember Measures the proportion of true positive labels correctly identified by the model.

Notice from the table above that we have 3 real labels that are positive and only one of them is captured correctly by the model. So the recall is 0.33 or 33%.

Overall, in the SPAM prediction example, the precision is 50% and the recall is 33%.

What message does precision and recall convey?

What are the exact measurements at a high level? Correctness. What are the recall measures at a high level? coverage. For example, if the accuracy is 98%, it means that when the model says the prediction is positive, the prediction is probably accurate. A model can be overly conservative and make only limited positive predictions, resulting in high accuracy. In other words, it fails to make enough positive predictions. That’s why you also need to consider recall – to ensure you have enough real positives.

Regarding recall, high recall means that the model is able to capture most of the positive predictions. But if the model says everything is positive, regardless of the underlying reasoning, the recall will be artificially high and close to perfect. Therefore, a balance needs to be struck between precision and recall. You want accurate predictions, but at the same time not at the cost of missing too many positive predictions (false negative predictions). Ideally, you want sufficiently high precision and recall.

Summary

in this accuracy measures the proportion of correct positive predictions and recall Measures coverage of actual positive labels. For a model to be considered “good”, the precision and recall must be at an acceptable level. Ultimately, what is acceptable depends on the application.

Recommended reading:

connected