dataprofiler.labelers.base_model module

class dataprofiler.labelers.base_model.AutoSubRegistrationMeta(clsname, bases, attrs)

Bases: abc.ABCMeta

mro()

Return a type’s method resolution order.

register(subclass)

Register a virtual subclass of an ABC.

Returns the subclass, to allow usage as a class decorator.

class dataprofiler.labelers.base_model.BaseModel(label_mapping, parameters)

Bases: object

Base Model Initializer. Only model and model parameters are stored here :param parameters: Contains all the appropriate parameters for the model.

Must contain num_labels.

Returns

None

requires_zero_mapping = False

property label_mapping

mapping of labels to their encoded values

Type

return

property reverse_label_mapping

Reversed order of current labels, useful for when needed to extract Labels via indices

Type

return

property labels

Retrieves the label :return: list of labels

property num_labels

classmethod get_class(class_name)

get_parameters(param_list=None)

Returns a dict of parameters from the model given a list. :param param_list: list of parameters to retrieve from the model. :type param_list: list :return: dict of parameters

set_params(**kwargs)

Given kwargs, set the parameters if they exist.

set_label_mapping(label_mapping)

Sets the labels for the model

Parameters

label_mapping (dict) – label mapping of the model

Returns

None

classmethod help()

Help function describing alterable parameters.

Returns

None

abstract reset_weights()

Reset the weights of the model.

Returns

None

abstract predict(data, batch_size, show_confidences, verbose)

Predict the data with the current model :param data: model input data to predict on :type data: iterator of data to process :param batch_size: number of samples in the batch of data :type batch_size: int :param show_confidences: whether user wants prediction confidences :type show_confidences: bool :param verbose: Flag to determine whether to print status or not :type verbose: bool :return: char level predictions and confidences :rtype: dict

abstract classmethod load_from_disk(dirpath)

Loads whole model from disk with weights :param dirpath: directory path where you want to load the model from :type dirpath: str :return: None

abstract save_to_disk(dirpath)

Saves whole model to disk with weights :param dirpath: directory path where you want to save the model to :type dirpath: str :return: None

class dataprofiler.labelers.base_model.BaseTrainableModel(label_mapping, parameters)

Bases: dataprofiler.labelers.base_model.BaseModel

Base Model Initializer. Only model and model parameters are stored here :param parameters: Contains all the appropriate parameters for the model.

Must contain num_labels.

Returns

None

abstract fit(train_data, val_data, batch_size=32, epochs=1, label_mapping=None, reset_weights=False)

Train the current model with the training data and validation data :param train_data: Training data used to train model :type train_data: Union[pd.DataFrame, pd.Series, np.ndarray] :param val_data: Validation data used to validate the training :type val_data: Union[pd.DataFrame, pd.Series, np.ndarray] :param batch_size: Used to determine number of samples in each batch :type batch_size: int :param epochs: Used to determine how many epochs to run :type epochs: int :param label_mapping: Mapping of the labels :type label_mapping: dict :param reset_weights: Flag to determine whether or not to reset the

model’s weights

Returns

None

classmethod get_class(class_name)

get_parameters(param_list=None)

Returns a dict of parameters from the model given a list. :param param_list: list of parameters to retrieve from the model. :type param_list: list :return: dict of parameters

classmethod help()

Help function describing alterable parameters.

Returns

None

property label_mapping

mapping of labels to their encoded values

Type

return

property labels

Retrieves the label :return: list of labels

abstract classmethod load_from_disk(dirpath)

Loads whole model from disk with weights :param dirpath: directory path where you want to load the model from :type dirpath: str :return: None

property num_labels

abstract predict(data, batch_size, show_confidences, verbose)

Predict the data with the current model :param data: model input data to predict on :type data: iterator of data to process :param batch_size: number of samples in the batch of data :type batch_size: int :param show_confidences: whether user wants prediction confidences :type show_confidences: bool :param verbose: Flag to determine whether to print status or not :type verbose: bool :return: char level predictions and confidences :rtype: dict

requires_zero_mapping = False

abstract reset_weights()

Reset the weights of the model.

Returns

None

property reverse_label_mapping

Reversed order of current labels, useful for when needed to extract Labels via indices

Type

return

abstract save_to_disk(dirpath)

Saves whole model to disk with weights :param dirpath: directory path where you want to save the model to :type dirpath: str :return: None

set_label_mapping(label_mapping)

Sets the labels for the model

Parameters

label_mapping (dict) – label mapping of the model

Returns

None

set_params(**kwargs)

Given kwargs, set the parameters if they exist.