seqme.metrics.Fold

class seqme.metrics.Fold(metric, *, deviation='std', estimate='unbiased', n_splits=None, split_size=None, drop_last=False, strict=True, shuffle=False, seed=0)

A wrapper for any metric, which splits the sequences into non-overlapping subsets, computes the metric on each split and aggregates the results.

Fold splits the data into k-folds or fixed-size splits, with optional shuffling, and then aggregates the results.

__init__(metric, *, deviation='std', estimate='unbiased', n_splits=None, split_size=None, drop_last=False, strict=True, shuffle=False, seed=0)

Initialize the Fold wrapper.

Parameters:

• metric (Metric) – The underlying metric to evaluate per fold.

• deviation (Literal['std', 'se', 'var']) –

  Type of deviation to compute:

  • 'std': Standard deviation

  • 'se': Standard error

  • 'var': Variance

• estimate (Literal['biased', 'unbiased']) – How to estimate the deviation.

• n_splits (Optional[int]) – Number of folds to create (exclusive with split_size).

• split_size (Optional[int]) – Fixed size for each fold (exclusive with n_splits).

• drop_last (bool) – Drop final fold if smaller than split_size.

• strict (bool) – Error on any non-null fold deviation.

• shuffle (bool) – Shuffle data before splitting.

• seed (int) – Seed for deterministic shuffling of sequences when creating folds.

__call__(sequences)

Call the wrapped metric on each fold of sequences and aggregate the results.

Parameters:

sequences (list[str]) – Sequences to split into folds.

Returns:

Aggregated mean value and standard deviation, standard error or variance across folds.

Return type:

MetricResult

Methods

__init__(metric, *[, deviation, estimate, ...])	Initialize the Fold wrapper.
__call__(sequences)	Call the wrapped metric on each fold of sequences and aggregate the results.

Attributes

name	Name of the metric.
objective	Whether lower or higher scores indicate better performance.