Machine Learning Metric Choosing Framework

Machine learning metrics are crucial for evaluating the performance of models. This framework provides a comprehensive guide to choosing the right metrics for different scenarios.

Common Metrics

• Accuracy: The ratio of correctly predicted observations to the total observations.
  • Formula: ( \text{Accuracy} = \frac{\text{Number of Correct Predictions}}{\text{Total Number of Predictions}} )
• Precision: The ratio of correctly predicted positive observations to the total predicted positive observations.
  • Formula: ( \text{Precision} = \frac{\text{True Positives}}{\text{True Positives} + \text{False Positives}} )
• Recall: The ratio of correctly predicted positive observations to the all observations in actual class.
  • Formula: ( \text{Recall} = \frac{\text{True Positives}}{\text{True Positives} + \text{False Negatives}} )
• F1 Score: The weighted average of Precision and Recall.
  • Formula: ( \text{F1 Score} = 2 \times \frac{\text{Precision} \times \text{Recall}}{\text{Precision} + \text{Recall}} )

Choosing the Right Metric

• Binary Classification: Use Accuracy, Precision, Recall, or F1 Score depending on the business context.
• Multi-Class Classification: Use Accuracy, Precision, Recall, or F1 Score for each class.
• Regression: Use Mean Absolute Error (MAE), Mean Squared Error (MSE), or R-squared.

Example

Suppose you are working on a binary classification problem to predict whether an email is spam or not. In this case, you might want to focus on Precision and Recall since missing a spam email (False Negative) can be more costly than incorrectly marking a non-spam email as spam (False Positive).

For more information on machine learning metrics, check out our Machine Learning Metrics Guide.