cascade.metrics#

class cascade.metrics.Accuracy(value: SupportsFloat | None = None, name: str = 'accuracy', dataset: str | None = None, split: str | None = None, interval: Tuple[SupportsFloat, SupportsFloat] | None = None, extra: Dict[str, SupportsFloat] | None = None, **kwargs: Any)[source]#

Accuracy metric - the number of correct answers divided by the number of all

By default name is accuracy, can be changed Direction is always up

Can be computed iteratively using compute_add

__init__(value: SupportsFloat | None = None, name: str = 'accuracy', dataset: str | None = None, split: str | None = None, interval: Tuple[SupportsFloat, SupportsFloat] | None = None, extra: Dict[str, SupportsFloat] | None = None, **kwargs: Any) → None[source]#

Creates Metric

Parameters:

name (str) – Name of the metric
value (Optional[MetricType]) – Scalar value of the metric, by default None
dataset (Optional[str]) – Dataset on which metric was computed, by default None
split (Optional[str]) – The split of the dataset for example train or test, by default None
direction (Literal["up", "down", None]) – Is metric better when it is greater or less, by default None
interval (Optional[Tuple[MetricType, MetricType]]) – Upper and lower boundaries of value, by default None
extra (Optional[Dict[str, Any]]) – Extra values that needs to be stored with metric, by default None

compute(gt: Sequence[Any], pred: Sequence[Any]) → SupportsFloat[source]#: The method to compute metric’s value. Should always populate the internal self.value field and return it.

compute_add(gt: Sequence[Any], pred: Sequence[Any]) → SupportsFloat[source]#: The method to compute the metric incrementally. Should always populate the internal self.value field and return it.

class cascade.metrics.Loss(value: SupportsFloat | None = None, name: str = 'loss', dataset: str | None = None, split: str | None = None, **kwargs: Any)[source]#

Loss is the convenience metric which by default has name loss and is always directed down

__init__(value: SupportsFloat | None = None, name: str = 'loss', dataset: str | None = None, split: str | None = None, **kwargs: Any) → None[source]#: See also

cascade.metrics.Metric

Base class for every metric, defines the way of computation and serves as the value and metadata storage

Metrics should always be scalar. If your metric returns some complex types like dicts or lists consider using multiple Metric objects. Or if values are connected, use extra keyword.

Creates Metric

Parameters:

name (str) – Name of the metric
value (Optional[MetricType]) – Scalar value of the metric, by default None
dataset (Optional[str]) – Dataset on which metric was computed, by default None
split (Optional[str]) – The split of the dataset for example train or test, by default None
direction (Literal["up", "down", None]) – Is metric better when it is greater or less, by default None
interval (Optional[Tuple[MetricType, MetricType]]) – Upper and lower boundaries of value, by default None
extra (Optional[Dict[str, Any]]) – Extra values that needs to be stored with metric, by default None

compute(*args: Any, **kwargs: Any) → SupportsFloat[source]#: The method to compute metric’s value. Should always populate the internal self.value field and return it.

compute_add(*args: Any, **kwargs: Any) → SupportsFloat[source]#: The method to compute the metric incrementally. Should always populate the internal self.value field and return it.

to_dict() → Dict[str, Any][source]#

Converts metric object into dict, omits fields that are None

Return type:: Dict[str, Any]

cascade.metrics.MetricType#: alias of SupportsFloat