Module prefeitura_rio.metrics

Metrics implementations for the prefeitura_rio package.

In this module, we define the metrics that will be used to evaluate the performance of the models. The metrics are defined as functions that take the true and predicted values as input and return a single value.

In the prefeitura_rio.metrics.agnostic module, we define metrics that are agnostic to any framework.

Expand source code
# -*- coding: utf-8 -*-
"""
Metrics implementations for the prefeitura_rio package.

In this module, we define the metrics that will be used to evaluate the
performance of the models. The metrics are defined as functions that take
the true and predicted values as input and return a single value.

In the `agnostic` module, we define metrics that are agnostic to any
framework.
"""
from typing import Callable, Dict

from prefeitura_rio.utils import assert_dependencies

from .agnostic import brier, carabetta, fbeta, mse, nash_sutcliffe, pet, rmse, sp


@assert_dependencies(["numpy", "sklearn"], extras=["metrics"])
def evaluate(
    y_true,
    y_pred,
    *,
    metrics: Dict[str, Callable] = None,
    beta=1,
    threshold=0.5,
) -> Dict[str, float]:
    """
    Evaluates the performance of a model.

    Args:
        y_true (array-like): The true values.
        y_pred (array-like): The predicted values.
        metrics (dict, optional): A dictionary of metrics to be evaluated.
            Defaults to `None`.
        beta (int, optional): The beta value for the F-beta score. Defaults to
            `1`.
        threshold (float, optional): The threshold for the F-beta score.
            Defaults to `0.5`.

    Returns:
        dict: A dictionary with the results of the metrics.
    """
    if metrics is None:
        metrics = {
            "Brier": brier,
            "Carabetta": carabetta,
            "F-Beta": lambda y_true, y_pred: fbeta(y_true, y_pred, beta=beta, threshold=threshold),
            "MSE": mse,
            "Nash-Sutcliffe": nash_sutcliffe,
            "PET": pet,
            "RMSE": rmse,
            "SP": lambda y_true, y_pred: sp(y_true, y_pred, threshold=threshold),
        }
    return {name: func(y_true, y_pred) for name, func in metrics.items()}

Sub-modules

prefeitura_rio.metrics.agnostic

Framework-agnotic metrics implementations for the prefeitura_rio package. Most of them can usually be imported from the sklearn.metrics module.