Base Data

Contains abstract class for data loading and saving.

class dataprofiler.data_readers.base_data.BaseData(input_file_path: Optional[str], data: Any, options: Dict)

Bases: object

Abstract class for data loading and saving.

Initialize Base class for loading a dataset.

Options can be specified and maybe more specific to the subclasses.

Parameters

  • input_file_path (str) – path to the file being loaded or None

  • data (multiple types) – data being loaded into the class instead of an input file

  • options (dict) – options pertaining to the data type

Returns

None

data_type: str
info: Optional[str] = None
property data

Return data.

property data_format: Optional[str]

Return data format.

property is_structured: bool

Determine compatibility with StructuredProfiler.

property file_encoding: Optional[str]

Return file encoding.

get_batch_generator(batch_size: int) → Generator[Union[pandas.core.frame.DataFrame, List], None, None]

Get batch generator.

classmethod is_match(input_file_path: str, options: Optional[Dict]) → bool

Return true if match, false otherwise.

reload(input_file_path: Optional[str], data: Any, options: Optional[Dict]) → None

Reload the data class with a new dataset.

This erases all existing data/options and replaces it with the input data/options.

Parameters

  • input_file_path (str) – path to the file being loaded or None

  • data (multiple types) – data being loaded into the class instead of an input file

  • options (dict) – options pertaining to the data type

Returns

None

property length: int

Return the length of the dataset which is loaded.

Returns

length of the dataset