Text Column Profile

class dataprofiler.profilers.text_column_profile.TextColumn(name, options=None)

Bases: dataprofiler.profilers.numerical_column_stats.NumericStatsMixin, dataprofiler.profilers.base_column_profilers.BaseColumnPrimitiveTypeProfiler

Text column profile subclass of BaseColumnProfiler. Represents a column in the dataset which is a text column.

Initialization of column base properties and itself.

Parameters
  • name (String) – Name of the data

  • options (TextOptions) – Options for the Text column

type = 'text'
property profile

Property for profile. Returns the profile of the column.

Returns

diff(other_profile, options=None)

Finds the differences for text columns

Parameters

other_profile (TextColumn Profile) – profile to find the difference with

Returns

the text columns differences

Return type

dict

property data_type_ratio

Calculates the ratio of samples which match this data type. NOTE: all values can be considered string so always returns 1 in this case.

Returns

ratio of data type

Return type

float

update(df_series)

Updates the column profile.

Parameters

df_series (pandas.core.series.Series) – df series

Returns

None

col_type = None
static is_float(x)

For “0.80” this function returns True For “1.00” this function returns True For “1” this function returns True

Parameters

x (str) – string to test

Returns

if is float or not

Return type

bool

static is_int(x)

For “0.80” This function returns False For “1.00” This function returns True For “1” this function returns True

Parameters

x (str) – string to test

Returns

if is integer or not

Return type

bool

property kurtosis
property mean
property median

Estimates the median of the data.

Returns

the median

Return type

float

property median_abs_deviation
Get median absolute deviation estimated from the histogram of the data

Subtract bin edges from the median value Fold the histogram to positive and negative parts around zero Impose the two bin edges from the two histogram Calculate the counts for the two histograms with the imposed bin edges Superimpose the counts from the two histograms Interpolate the median absolute deviation from the superimposed counts

Returns

median absolute deviation

property mode

Finds an estimate for the mode(s) of the data.

Returns

the mode(s) of the data

Return type

list(float)

static np_type_to_type(val)

Converts numpy variables to base python type variables

Parameters

val (numpy type or base type) – value to check & change

Return val

base python type

Rtype val

int or float

property skewness
property stddev
property variance