What are various matrices in machine learning classification algorithm?

In classification tasks in machine learning, evaluating model performance involves various metrics. The F1 Score is one of the most commonly used, especially when dealing with imbalanced classes. Here's an overview of the F1 Score and other key metrics:

Accuracy: 

Formula: (TP + TN) / (TP + TN + FP + FN ) 

Usecase: when classes are balanced 

Limitation: this will be misleading if the classes are imbalanced 

Precision: 

Forumula: Precision = TP / (TP + FP)

Use case: When false positives are costly (e.g., spam detection).

Recall (Sensitivity or True Positive Rate)

Formula is TP / (TP + FN)

Use case: When false negatives are costly (e.g., medical diagnosis).

F1 Score: 

Formula: Recall = (Precision (.) Recall) / ( Precision + Recall)  

Use cases: When you want a balance between precision and recall, especially for imbalanced datasets.

ROC-AUC (Receiver Operating Characteristic - Area Under Curve)

Use Case: Measures model’s ability to distinguish between classes across all thresholds.

Higher AUC → Better model.

Confusion Matrix 

A confusion matrix is a performance measurement tool used in classification tasks in machine learning. It gives a detailed breakdown of how well a classification model is performing by showing actual vs. predicted class results.

For a binary classification, the confusion matrix looks like:

                                Predicted: Positive Predicted: Negative

Actual: Positive True Positive (TP) False Negative (FN)

Actual: Negative False Positive (FP) True Negative (TN)

True Positive (TP): Correctly predicted positive class

True Negative (TN): Correctly predicted negative class

False Positive (FP): Incorrectly predicted positive (Type I error)

False Negative (FN): Incorrectly predicted negative (Type II error)

Code for getting confusion matrix is given below 

from sklearn.metrics import confusion_matrix, classification_report

y_true = [1, 0, 1, 1, 0, 1, 0]

y_pred = [1, 0, 1, 0, 0, 1, 1]

# Confusion Matrix

cm = confusion_matrix(y_true, y_pred)

print("Confusion Matrix:\n", cm)

# Classification Report

print("\nClassification Report:\n", classification_report(y_true, y_pred))

When Is It Especially Beneficial?

When dealing with imbalanced datasets (e.g., fraud detection, medical diagnosis).

When overall accuracy is misleading — confusion matrix shows where the model fails.

In multi-class classification, the confusion matrix shows errors across all classes.

Sample code for various Accuracy, Precision, Recall and AOC is below 

from sklearn.metrics import (

accuracy_score,

precision_score,

recall_score,

f1_score,

roc_auc_score,

confusion_matrix,

classification_report,

RocCurveDisplay

)

from sklearn.model_selection import train_test_split

from sklearn.datasets import make_classification

from sklearn.ensemble import RandomForestClassifier

import matplotlib.pyplot as plt

# 1. Create a binary classification dataset

X, y = make_classification(

n_samples=1000, n_features=20, n_informative=2,

n_redundant=10, n_classes=2, weights=[0.7, 0.3], random_state=42

)

# 2. Split into train and test

X_train, X_test, y_train, y_test = train_test_split(

X, y, test_size=0.3, stratify=y, random_state=42

)

# 3. Train a classifier

clf = RandomForestClassifier(random_state=42)

clf.fit(X_train, y_train)

# 4. Predict

y_pred = clf.predict(X_test)

y_proba = clf.predict_proba(X_test)[:, 1] # Probabilities for ROC-AUC

# 5. Compute metrics

print("Accuracy:", accuracy_score(y_test, y_pred))

print("Precision:", precision_score(y_test, y_pred))

print("Recall:", recall_score(y_test, y_pred))

print("F1 Score:", f1_score(y_test, y_pred))

print("ROC-AUC Score:", roc_auc_score(y_test, y_proba))

# 6. Confusion matrix

print("\nConfusion Matrix:")

print(confusion_matrix(y_test, y_pred))

# 7. Full classification report

print("\nClassification Report:")

print(classification_report(y_test, y_pred))

# 8. Optional: ROC Curve

RocCurveDisplay.from_estimator(clf, X_test, y_test)

plt.title("ROC Curve")

plt.show()

references:

OpenAI