domainlab.tasks package

Submodules

domainlab.tasks.a_task module

Base class for Task

class domainlab.tasks.a_task.NodeTaskDG(succ=None)

Bases: AbstractChainNodeHandler

Domain Generalization Classification Task

conf_without_args()

configuration without init_business

get_list_domains()

1. get list of domain names

2. better use method than property so new domains can be added

get_list_domains_tr_te(tr_id, te_id)

For static DG task, get train and test domains list.

Parameters:

• tr_id – training domain ids; int or str, or a list of int or str, or None; if None, then assumed to be the complement of te_id.

• te_id – test domain ids; int or str, or a list of int or str; required.

Returns:

list of training domain names, list of test domain names.

get_na(na_tr, na_te)

task name appended with configurations :param na_tr: training domain names :param na_te: test domain names

abstract init_business(args, trainer=None)

construct task data loader

is_myjob(request)

Parameters:

request – string

property isize

isize

Type:

getter for input size

property list_domain_tr

property getter of list of domains for this task

property loader_te

loader of mixed test domains

property loader_tr

loader of mixed train domains

property loader_val

loader of validation dataset on the training domains

set_list_domains(list_domains)

setter for self._list_domains

property task_name

The basic name of the task, without configurations

domainlab.tasks.a_task_classif module

Abstract class for TaskClassif

class domainlab.tasks.a_task_classif.NodeTaskDGClassif(succ=None)

Bases: NodeTaskDG

abstract class for classification task

property dim_y

classification dimension

property list_str_y

getter for list_str_y

sample_sav(root, batches=5, subfolder_na='task_sample')

sample data from task and save to disk

domainlab.tasks.b_task module

Use dictionaries to create train and test domain split

class domainlab.tasks.b_task.NodeTaskDict(succ=None)

Bases: NodeTaskDG

Use dictionaries to create train and test domain split

decorate_dset(model, args)

dispatch re-organization of data flow to model

get_dset_by_domain(args, na_domain, split=False)

each domain correspond to one dataset, must be implemented by child class

init_business(args, trainer=None)

create a dictionary of datasets

domainlab.tasks.b_task_classif module

Use dictionaries to create train and test domain split

class domainlab.tasks.b_task_classif.NodeTaskDictClassif(succ=None)

Bases: NodeTaskDict, NodeTaskDGClassif

Use dictionaries to create train and test domain split

count_domain_class()

iterate all domains and count the class label distribution for each return a double dictionary {“domain1”: {“class1”:3, “class2”: 4,…}, ….}

init_business(args, trainer=None)

create a dictionary of datasets

domainlab.tasks.task_dset module

Use dictionaries to create train and test domain split

domainlab.tasks.task_dset.mk_task_dset(isize, taskna='task_custom', dim_y=None, list_str_y=None, parent=<class 'domainlab.tasks.b_task_classif.NodeTaskDictClassif'>, succ=None)

make a task via a dictionary of dataset where the key is domain value is a tuple of dataset for training and dataset for validation (can be identical to training)

domainlab.tasks.task_folder module

When class names and numbers does not match across different domains

class domainlab.tasks.task_folder.NodeTaskFolder(succ=None)

Bases: NodeTaskDictClassif

create dataset by loading files from an organized folder then each domain correspond to one dataset

property dict_domain2imgroot

“xx/yy/zz”}

Type:

{“domain name”

property extensions

return allowed extensions

get_dset_by_domain(args, na_domain, split=False)

each domain correspond to one dataset, must be implemented by child class

class domainlab.tasks.task_folder.NodeTaskFolderClassNaMismatch(succ=None)

Bases: NodeTaskFolder

when the folder names of the same class from different domains have different names

get_dset_by_domain(args, na_domain, split=False)

each domain correspond to one dataset, must be implemented by child class

domainlab.tasks.task_folder_mk module

When class names and numbers does not match across different domains

domainlab.tasks.task_folder_mk.mk_task_folder(extensions, list_str_y, dict_domain_folder_name2class, dict_domain_img_trans, img_trans_te, isize, dict_domain2imgroot, taskna, succ=None)

Make task by specifying each domain with folder structures :param extensions: Different Options: 1. a python dictionary with key as the domain name and value (str or tuple[str]) as the file extensions of the image. 2. a str or tuple[str] with file extensions for all domains. 3. None: in each domain all files with an extension in (‘jpg’, ‘jpeg’, ‘png’) are loaded. :param list_str_y: a python list with user defined class name where the order of the list matters. :param dict_domain_folder_name2class: a python dictionary, with key as the user specified domain name, value as a dictionary to map the sub-folder name of each domain’s class folder into the user defined common class name. :param dict_domain_img_trans: a python dictionary with keys as the user specified domain name, value as a user defined torchvision transform. This feature allows carrying out different transformation (composition) to different domains at training time. :param img_trans_te: at test or inference time, we do not have knowledge of domain information so only a unique transform (composition) is allowed. :isize: domainlab.tasks.ImSize(image channel, image height, image width) :dict_domain2imgroot: a python dictionary with keys as user specified domain names and values as the absolute path to each domain’s data. :taskna: user defined task name

domainlab.tasks.task_mini_vlcs module

test task for image size 224

domainlab.tasks.task_mini_vlcs.addtask2chain(chain)

given a chain of responsibility for task selection, add another task into the chain

domainlab.tasks.task_mnist_color module

Color MNIST with palette

class domainlab.tasks.task_mnist_color.NodeTaskMNISTColor10(succ=None)

Bases: NodeTaskDictClassif

Use the deafult palette with 10 colors

get_dset_by_domain(args, na_domain, split=True)

get_dset_by_domain. :param args: :param na_domain: :param split: for test set, no need to split args.split: by default, split is set to be zero which in python can be evaluated in if statement, in which case, no validation set will be created. Otherwise, this argument is the split ratio

get_list_domains()

1. get list of domain names

2. better use method than property so new domains can be added

property isize

image channel, height, width

property list_str_y

getter for list_str_y

domainlab.tasks.task_pathlist module

The class TaskPathList provides the user an interface to provide a file with each line consisting of a pair, where the first slot contains the path (either absolute or relative if the user knows from where this package is executed) of an image and the second slot contains the class label as a numerical string.

class domainlab.tasks.task_pathlist.NodeTaskPathListDummy(succ=None)

Bases: NodeTaskDictClassif

typedef class so that other function can use isinstance

get_dset_by_domain(args, na_domain, split=False)

each domain correspond to one dataset, must be implemented by child class

domainlab.tasks.task_pathlist.mk_node_task_path_list(isize, img_trans_te, list_str_y, img_trans_tr, dict_class_label_ind2name, dict_domain2imgroot, dict_d2filepath_list_img_tr, dict_d2filepath_list_img_val, dict_d2filepath_list_img_te, succ=None)

mk_node_task_path_list.

Parameters:

• isize –

• list_str_y –

• img_trans_tr –

• dict_class_label_ind2name –

• dict_domain2imgroot –

• dict_d2filepath_list_img_tr –

• dict_d2filepath_list_img_val –

• dict_d2filepath_list_img_te –

• succ –

domainlab.tasks.task_utils module

convert ids to a list of domain names

domainlab.tasks.task_utils.parse_domain_id(list_domain_id, list_domains)

Convert ids to a list of domain names. :param list_domain_id: domain id or ids provided as an int or str, or a list of int or str. :param list_domains: list of available domains :return: list of domain names

domainlab.tasks.utils_task module

Task wraps around datasets, this file provide utilities

class domainlab.tasks.utils_task.DsetClassVecDecorator(dset, dict_folder_name2class_global, list_str_y)

Bases: Dataset

decorate a pytorch dataset with a new class name

fun_class_local_na2vec_new(k)

local class name within one domain, to one-hot vector of new representation

property targets

return a list of all targets so class sample count is straight forward

class domainlab.tasks.utils_task.DsetClassVecDecoratorImgPath(dset, dict_folder_name2class_global, list_str_y)

Bases: DsetClassVecDecorator

class domainlab.tasks.utils_task.DsetDomainVecDecorator(dset, vec_domain, na_domain)

Bases: Dataset

decorate a pytorch dataset with a fixed vector representation of domain

property targets

return a list of all targets so class sample count is straight forward

class domainlab.tasks.utils_task.DsetDomainVecDecoratorImgPath(dset, vec_domain, na_domain)

Bases: DsetDomainVecDecorator

Except returning x, y, d, additionally, the path of x is returned currently not in use since it is mostly important to print predictions together with path for the test domain

class domainlab.tasks.utils_task.ImSize(i_c, i_h, i_w)

Bases: object

ImSize.

property c

image channel

property h

image height

property w

image width

class domainlab.tasks.utils_task.LoaderDomainLabel(batch_size, dim_d)

Bases: object

wraps a dataset with domain label and into a loader

domainlab.tasks.utils_task.img_loader2dir(loader, folder, test=False, list_domain_na=None, list_class_na=None, batches=5)

save images from loader to directory so speculate if loader is correct :param loader: pytorch data loader :param folder: folder to save images :param test: if true, the loader is assumend to be a test loader; if false (default) it is assumed to be a train loader :param list_domain_na: optional list of domain names :param list_class_na: optional list of class names :param batches: number of batches to save

domainlab.tasks.utils_task.mk_loader(dset, bsize, drop_last=True, shuffle=True, num_workers=0)

Parameters:

bs – batch size

domainlab.tasks.utils_task.mk_onehot(dim, ind)

Parameters:

• dim – dimension of representation vector

• ind – index

domainlab.tasks.utils_task.tensor1hot2ind(tensor_label)

tensor1hot2ind.

Parameters:

tensor_label –

domainlab.tasks.utils_task_dset module

task specific dataset operation

class domainlab.tasks.utils_task_dset.DsetIndDecorator4XYD(dset)

Bases: Dataset

For dataset of x, y, d, decorate it wih index

class domainlab.tasks.utils_task_dset.DsetZip(dset1, dset2, name=None)

Bases: Dataset

enable zip return in getitem: x_1, y_1, x_2, y_2 to avoid always the same match, the second dataset does not use the same idx in __get__item() but instead, a random one

domainlab.tasks.zoo_tasks module

all available tasks for domainlab

class domainlab.tasks.zoo_tasks.TaskChainNodeGetter(args)

Bases: object

1. Hardcoded chain 3. Return selected node

domainlab.tasks.zoo_tasks.get_task(name=None)

get build in task from DomainLab

Module contents