Quality metrics¶
Quality metrics (score/loss functions) are used to quantify the performance of a given model. Score functions are typically maximized (e.g. accuracy, concordance, ...) whereas loss functions should be minimized (e.g. mean squared error, error rate, ...). Optunity provides common score/loss functions for your convenience.
We use the following calling convention:
- y (iterable): the true labels/function values
- yhat (iterable): the predicted labels/function values
- we assume y and yhat are of the same length (though we do not assert this).
- potential parameters of the score function must be added by keyword
All functions listed here are available in optunity.metrics
.
Score functions¶
Score functions are typically maximized (e.g. optunity.maximize()
).
Classification¶
Score | Associated Optunity function |
---|---|
accuracy | accuracy() |
area under ROC curve | roc_auc() |
area under PR curve | pr_auc() |
\(F_\beta\) | fbeta() |
precision/PPV | precision() |
recall/sensitivity | recall() |
specificity/NPV | npv() |
PU score | pu_score() |
Regression¶
Score | Associated Optunity function |
---|---|
R squared | r_squared() |
Loss functions¶
Loss functions are typically minimized (e.g. optunity.minimize()
).
Classification¶
Score | Associated Optunity function |
---|---|
Brier score | brier() |
error rate | error_rate() |
log loss | logloss() |
Regression¶
Score | Associated Optunity function |
---|---|
mean squared error | mse() |
absolute error | absolute_error() |