Quality metrics

Quality metrics (score/loss functions) are used to quantify the performance of a given model. Score functions are typically maximized (e.g. accuracy, concordance, ...) whereas loss functions should be minimized (e.g. mean squared error, error rate, ...). Optunity provides common score/loss functions for your convenience.

We use the following calling convention:

  • y (iterable): the true labels/function values
  • yhat (iterable): the predicted labels/function values
  • we assume y and yhat are of the same length (though we do not assert this).
  • potential parameters of the score function must be added by keyword

All functions listed here are available in optunity.metrics.

Score functions

Score functions are typically maximized (e.g. optunity.maximize()).

Classification

Score Associated Optunity function
accuracy accuracy()
area under ROC curve roc_auc()
area under PR curve pr_auc()
\(F_\beta\) fbeta()
precision/PPV precision()
recall/sensitivity recall()
specificity/NPV npv()
PU score pu_score()

Regression

Score Associated Optunity function
R squared r_squared()

Loss functions

Loss functions are typically minimized (e.g. optunity.minimize()).

Classification

Score Associated Optunity function
Brier score brier()
error rate error_rate()
log loss logloss()

Regression

Score Associated Optunity function
mean squared error mse()
absolute error absolute_error()