cascade.metrics#

class cascade.metrics.Accuracy(value: Optional[SupportsFloat] = None, name: str = 'accuracy', dataset: Optional[str] = None, split: Optional[str] = None, interval: Optional[Tuple[SupportsFloat, SupportsFloat]] = None, extra: Optional[Dict[str, SupportsFloat]] = None, **kwargs: Any)[source]#

Accuracy metric - the number of correct answers divided by the number of all

By default name is accuracy, can be changed Direction is always up

Can be computed iteratively using compute_add

__init__(value: Optional[SupportsFloat] = None, name: str = 'accuracy', dataset: Optional[str] = None, split: Optional[str] = None, interval: Optional[Tuple[SupportsFloat, SupportsFloat]] = None, extra: Optional[Dict[str, SupportsFloat]] = None, **kwargs: Any) → None[source]#

Creates Metric

Parameters:

  • name (str) – Name of the metric

  • value (Optional[MetricType]) – Scalar value of the metric, by default None

  • dataset (Optional[str]) – Dataset on which metric was computed, by default None

  • split (Optional[str]) – The split of the dataset for example train or test, by default None

  • direction (Literal["up", "down", None]) – Is metric better when it is greater or less, by default None

  • interval (Optional[Tuple[MetricType, MetricType]]) – Upper and lower boundaries of value, by default None

  • extra (Optional[Dict[str, Any]]) – Extra values that needs to be stored with metric, by default None

compute(gt: Sequence[Any], pred: Sequence[Any]) → SupportsFloat[source]#

The method to compute metric’s value. Should always populate the internal self.value field and return it.

compute_add(gt: Sequence[Any], pred: Sequence[Any]) → SupportsFloat[source]#

The method to compute the metric incrementally. Should always populate the internal self.value field and return it.

class cascade.metrics.Loss(value: Optional[SupportsFloat] = None, name: str = 'loss', dataset: Optional[str] = None, split: Optional[str] = None, **kwargs: Any)[source]#

Loss is the convenience metric which by default has name loss and is always directed down

__init__(value: Optional[SupportsFloat] = None, name: str = 'loss', dataset: Optional[str] = None, split: Optional[str] = None, **kwargs: Any) → None[source]#

See also

cascade.metrics.Metric

class cascade.metrics.Metric(name: str, value: Optional[SupportsFloat] = None, dataset: Optional[str] = None, split: Optional[str] = None, direction: Optional[Literal['up', 'down']] = None, interval: Optional[Tuple[SupportsFloat, SupportsFloat]] = None, extra: Optional[Dict[str, Any]] = None)[source]#

Base class for every metric, defines the way of computation and serves as the value and metadata storage

Metrics should always be scalar. If your metric returns some complex types like dicts or lists consider using multiple Metric objects. Or if values are connected, use extra keyword.

__init__(name: str, value: Optional[SupportsFloat] = None, dataset: Optional[str] = None, split: Optional[str] = None, direction: Optional[Literal['up', 'down']] = None, interval: Optional[Tuple[SupportsFloat, SupportsFloat]] = None, extra: Optional[Dict[str, Any]] = None) → None[source]#

Creates Metric

Parameters:

  • name (str) – Name of the metric

  • value (Optional[MetricType]) – Scalar value of the metric, by default None

  • dataset (Optional[str]) – Dataset on which metric was computed, by default None

  • split (Optional[str]) – The split of the dataset for example train or test, by default None

  • direction (Literal["up", "down", None]) – Is metric better when it is greater or less, by default None

  • interval (Optional[Tuple[MetricType, MetricType]]) – Upper and lower boundaries of value, by default None

  • extra (Optional[Dict[str, Any]]) – Extra values that needs to be stored with metric, by default None

compute(*args: Any, **kwargs: Any) → SupportsFloat[source]#

The method to compute metric’s value. Should always populate the internal self.value field and return it.

compute_add(*args: Any, **kwargs: Any) → SupportsFloat[source]#

The method to compute the metric incrementally. Should always populate the internal self.value field and return it.

to_dict() → Dict[str, Any][source]#

Converts metric object into dict, omits fields that are None

Return type:

Dict[str, Any]

cascade.metrics.MetricType#

alias of SupportsFloat