Numerical Column Stats¶
Build model for dataset by identifying col type along with its respective params.
- class dataprofiler.profilers.numerical_column_stats.abstractstaticmethod(function: Callable)¶
Bases:
staticmethod
For making function an abstract method.
Initialize abstract static method.
- class dataprofiler.profilers.numerical_column_stats.NumericStatsMixin(options: Optional[dataprofiler.profilers.profiler_options.NumericalOptions] = None)¶
Bases:
object
Abstract numerical column profile subclass of BaseColumnProfiler.
Represents column in the dataset which is a text column. Has Subclasses itself.
Initialize column base properties and itself.
- Parameters
options (NumericalOptions) – Options for the numerical stats.
- type: Optional[str] = None¶
- profile() Dict ¶
Return profile of the column.
- Returns
- report(remove_disabled_flag: bool = False) Dict ¶
Call the profile and remove the disabled columns from profile’s report.
“Disabled column” is defined as a column that is not present in self.__calculations but is present in the self.profile.
- Variables
remove_disabled_flag – true/false value to tell the code to remove values missing in __calculations
- Returns
Profile object pop’d based on values missing from __calculations
- Return type
Profile
- diff(other_profile: dataprofiler.profilers.numerical_column_stats.NumericStatsMixin, options: Optional[Dict] = None) Dict ¶
Find the differences for several numerical stats.
- Parameters
other_profile (NumericStatsMixin Profile) – profile to find the difference with
- Returns
the numerical stats differences
- Return type
dict
- property mean: float¶
Return mean value.
- property mode: List[float]¶
Find an estimate for the mode[s] of the data.
- Returns
the mode(s) of the data
- Return type
list(float)
- property median: float¶
Estimate the median of the data.
- Returns
the median
- Return type
float
- property variance: float¶
Return variance.
- property stddev: float¶
Return stddev value.
- property skewness: float¶
Return skewness value.
- property kurtosis: float¶
Return kurtosis value.
- property median_abs_deviation: float¶
Get median absolute deviation estimated from the histogram of the data.
Subtract bin edges from the median value Fold the histogram to positive and negative parts around zero Impose the two bin edges from the two histogram Calculate the counts for the two histograms with the imposed bin edges Superimpose the counts from the two histograms Interpolate the median absolute deviation from the superimposed counts
- Returns
median absolute deviation
- abstract update(df_series: pandas.core.series.Series) dataprofiler.profilers.numerical_column_stats.NumericStatsMixin ¶
Update the numerical profile properties with an uncleaned dataset.
- Parameters
df_series (pandas.core.series.Series) – df series with nulls removed
- Returns
None
- static is_float(x: str) bool ¶
Return True if x is float.
For “0.80” this function returns True For “1.00” this function returns True For “1” this function returns True
- Parameters
x (str) – string to test
- Returns
if is float or not
- Return type
bool
- static is_int(x: str) bool ¶
Return True if x is integer.
For “0.80” This function returns False For “1.00” This function returns True For “1” this function returns True
- Parameters
x (str) – string to test
- Returns
if is integer or not
- Return type
bool
- static np_type_to_type(val: Any) Union[int, float] ¶
Convert numpy variables to base python type variables.
- Parameters
val (numpy type or base type) – value to check & change
- Return val
base python type
- Rtype val
int or float