domainlab.algos.msels package

Submodules

domainlab.algos.msels.a_model_sel module

Abstract Model Selection

class domainlab.algos.msels.a_model_sel.AMSel(val_threshold=None)

Bases: object

Abstract Model Selection

accept(trainer, observer4msel)

Visitor pattern to trainer accept trainer and tr_observer

abstract base_update(clear_counter=False)

observer + visitor pattern to trainer if the best model should be updated return boolean

property best_te_metric

decoratee best test metric

property best_val_acc

decoratee best val acc

early_stop()

check if trainer should stop return boolean

if_stop(acc_val=None)

check if trainer should stop and additionally tests for validation threshold return boolean

property max_es

maximum early stop

property model_selection_epoch

the epoch when the model was selected

property observer4msel

the observer from trainer

reset()

reset observer via reset model selector

property sel_model_te_acc

the selected model test accuaracy

update(epoch, clear_counter=False)

level above the observer + visitor pattern to get information about the epoch

property val_threshold

the treshold below which we don’t stop early

domainlab.algos.msels.c_msel_oracle module

Model Selection should be decoupled from

class domainlab.algos.msels.c_msel_oracle.MSelOracleVisitor(msel=None, val_threshold=None)

Bases: AMSel

save best out-of-domain test acc model, but do not affect how the final model is selected

accept(trainer, observer4msel)

Visitor pattern to trainer accept trainer and tr_observer

base_update(clear_counter=False)

if the best model should be updated

early_stop()

if should early stop oracle model selection does not intervene how models get selected by the innermost model selection

property oracle_last_setpoint_sel_te_acc

last setpoint acc

domainlab.algos.msels.c_msel_tr_loss module

AMSel.accept —> Trainer

class domainlab.algos.msels.c_msel_tr_loss.MSelTrLoss(max_es, val_threshold=None)

Bases: AMSel

1. Model selection using sum of loss across training domains

2. Visitor pattern to trainer

base_update(clear_counter=False)

if the best model should be updated

early_stop()

if should early stop

property max_es

maximum early stop

reset()

reset observer via reset model selector

domainlab.algos.msels.c_msel_val module

Model Selection should be decoupled from

class domainlab.algos.msels.c_msel_val.MSelValPerf(max_es, val_threshold=None)

Bases: MSelTrLoss

1. Model selection using validation performance

2. Visitor pattern to trainer

base_update(clear_counter=False)

if the best model should be updated

property best_te_metric

decoratee best test metric

property best_val_acc

decoratee best val acc

reset()

reset observer via reset model selector

property sel_model_te_acc

the selected model test accuaracy

Module contents