Skip to content

Validation

Validation dataclass

Configuration to perform validation during model training.

Parameters:

Name Type Description Default
dataset Dataset

The validation data, as a TabularDataset or TextDataset object.

 required
lr_schedule LrSchedule | None

A LrSchedule object. Leave to None for a constant learning rate.

 None
early_stop EarlyStop | EarlyStopMode | str | None

An EarlyStop object, an EarlyStopMode object, or a string representing an EarlyStopMode. If None, early stop will not be performed.

 None
save_best Path | str | None

A path to save the best model checkpoint. If None, the best model will not be saved.

 None
dp DpValid | None

A DpValid object to perform validation in a Differentially Private (DP) manner with respect to the validation set. Importantly, only the global loss has DP guarantees.

 None
tensorboard Path | str | None

A path to a TensorBoard log directory. If None, TensorBoard data will not be recorded.

 None
each int

The validation frequency.

 200
trigger str | HookEvent

The event triggering validation. Can be a HookEvent object or a string representing a HookEvent.

 STEP

EarlyStop dataclass

Early stop configuration.

Parameters:

Name Type Description Default
delta float

The smallest variation which is considered an improvement: if the relative improvement of the monitored quantity is less than this value, the early stopping will be triggered.

 required
patience int

The number of consecutive times that the early stopping criterion must be triggered before actually failing.

 3

EarlyStopMode

Enumeration class representing different early stop modes. Supported modes are: PRECISE, NORMAL and QUICK.

LrSchedule dataclass

Learning rate scheduler that reduces the learning rate of a given factor when the monitored quantity reaches a plateau.

Parameters:

Name Type Description Default
factor float

The factor by which the learning rate will be reduced, must be between 0 and 1.

 0.8
patience int

The number of steps for which the condition must be triggered before actually reducing the learning rate.

 3
threshold float

The threshold for measuring the new optimum, to only focus on significant changes.

 0.001
cooldown int

The number of steps to wait before resuming normal operation after the learning rate has been reduced.

 4
lr_min_ratio float

The ratio to compute the minimal learning rate, i.e. lr * self.lr_min_ratio. The learning rate will never go below such value.

 0.25

DpValid dataclass

Configuration to perform validation during model training with Differential Privacy (DP) guarantees on the validation set. Importantly, only the global loss has DP guarantees.

Parameters:

Name Type Description Default
eps float

The epsilon value of the DP guarantee on the validation data.

 required
loss_clipping float | tuple[float, float]

The clipping value for the loss. If a tuple of two float, it is interpreted as (min, max). If a single float, it is interpreted as (0, max). While max must be positive, min must be non-positive.

 1.0