dataprofiler.labelers.character_level_cnn_model module¶
-
dataprofiler.labelers.character_level_cnn_model.
build_embd_dictionary
(filename)¶ Returns a numpy embedding dictionary from embed file with GloVe-like format
- Parameters
filename (str) – Path to the embed file for loading
-
dataprofiler.labelers.character_level_cnn_model.
create_glove_char
(n_dims, source_file=None)¶ Embeds GloVe chars embeddings from source file to n_dims principal components in a new file
- Parameters
n_dims (int) – Final number of principal component dims of the embeddings
source_file (str) – Location of original embeddings to factor down
-
class
dataprofiler.labelers.character_level_cnn_model.
NoV1ResourceMessageFilter
(name='')¶ Bases:
logging.Filter
Removes TF2 warning for using TF1 model which has resources.
Initialize a filter.
Initialize with the name of the logger which, together with its children, will have its events allowed through the filter. If no name is specified, allow every event.
-
filter
(record)¶ Determine if the specified record is to be logged.
Returns True if the record should be logged, or False otherwise. If deemed appropriate, the record may be modified in-place.
-
-
class
dataprofiler.labelers.character_level_cnn_model.
CharacterLevelCnnModel
(label_mapping=None, parameters=None)¶ Bases:
dataprofiler.labelers.base_model.BaseTrainableModel
CNN Model Initializer. initialize epoch_id
- Parameters
label_mapping (dict) – maps labels to their encoded integers
parameters (dict) –
Contains all the appropriate parameters for the model. Must contain num_labels. Other possible parameters are:
max_length, max_char_encoding_id, dim_embed, size_fc dropout, size_conv, num_fil, optimizer, default_label
- Returns
None
-
requires_zero_mapping
= True¶
-
set_label_mapping
(label_mapping)¶ Sets the labels for the model
- Parameters
label_mapping (dict) – label mapping of the model
- Returns
None
-
save_to_disk
(dirpath)¶ Saves whole model to disk with weights
- Parameters
dirpath (str) – directory path where you want to save the model to
- Returns
None
-
classmethod
load_from_disk
(dirpath)¶ Loads whole model from disk with weights
- Parameters
dirpath (str) – directory path where you want to load the model from
- Returns
None
-
reset_weights
()¶ Reset the weights of the model.
- Returns
None
-
fit
(train_data, val_data=None, batch_size=32, label_mapping=None, reset_weights=False, verbose=True)¶ Train the current model with the training data and validation data
- Parameters
train_data (Union[list, np.ndarray]) – Training data used to train model
val_data (Union[list, np.ndarray]) – Validation data used to validate the training
batch_size (int) – Used to determine number of samples in each batch
label_mapping (Union[dict, None]) – maps labels to their encoded integers
reset_weights (bool) – Flag to determine whether to reset the weights or not
verbose (bool) – Flag to determine whether to print status or not
- Returns
None
-
predict
(data, batch_size=32, show_confidences=False, verbose=True)¶ Run model and get predictions
- Parameters
data (Union[list, numpy.ndarray]) – text input
batch_size (int) – number of samples in the batch of data
show_confidences – whether user wants prediction confidences
verbose (bool) – Flag to determine whether to print status or not
- Returns
char level predictions and confidences
- Return type
dict
-
details
()¶ Prints the relevant details of the model (summary, parameters, label mapping)
-
classmethod
get_class
(class_name)¶
-
get_parameters
(param_list=None)¶ Returns a dict of parameters from the model given a list. :param param_list: list of parameters to retrieve from the model. :type param_list: list :return: dict of parameters
-
classmethod
help
()¶ Help function describing alterable parameters.
- Returns
None
-
property
label_mapping
¶ mapping of labels to their encoded values
- Type
return
-
property
labels
¶ Retrieves the label :return: list of labels
-
property
num_labels
¶
-
property
reverse_label_mapping
¶ Reversed order of current labels, useful for when needed to extract Labels via indices
- Type
return
-
set_params
(**kwargs)¶ Given kwargs, set the parameters if they exist.