Examples

These examples provide a more in-depth look into the details of the Data Profiler library.

Basics

• Overview of Data Profiler
  • Basic Usage of the Data Profiler
  • Data reader class
  • Structured Profiler vs. Unstructured Profiler
  • Profiler options
  • Update profiles
  • Merge profiles
  • Conclusion
• Data Reader
  • Automatically reading and detecting data
  • Specifying detection options of Data and loading pandas.DataFrame
  • Accessing data and attributes
  • Checking data file types with is_match
  • Reloading data after altering options with reload
  • A deeper dive into CSVData
  • Intro to the CSVData data reader
  • CSVData Options
  • Options: delimiter and quotechar
  • Options: header
  • Options: data_format
  • Options: selected columns
  • Intro to GraphData data reader
• Structured Profiler
  • Reporting
  • Profiler Type
  • Profiler options
  • Updating Profiles
  • Merging Profiles
  • Differences in Data
  • Graphing a Profile
  • Saving and Loading a Profile
• Unstructured Profiler
  • Reporting
  • Profiler Type
  • Profiler options
  • Updating Profiles
  • Merging Profiles
  • Saving and Loading a Profile
• Graph Profiler
  • Profile
  • Saving and Loading a Profile
  • Difference in Data
  • Conclusion
• Labeler
  • What is the Labeler
  • Structured Data Prediction
  • Unstructured Data Prediction
  • Train the Labeler from Scratch
  • Transfer Learning a Labeler
  • Saving and Loading a Labeler
  • Building a Labeler from the Ground Up
• Adding Models to a Labeler Pipeline
  • Dataset
  • Implement a new character-level LSTM model
  • Integrate the new LSTM model to the DataLabeler
• Creating a Regex Labeler
  • Loading and using the pre-existing regex data labeler
    • Predicting with the regex labeler
  • Replacing the regex rules in the existing labeler
    • Predicting with the new regex labels
    • Adjusting postprocessor properties
  • Integrating the new Regex labeler into Structured Profiling
    • Setting the Structuredprofiler’s DataLabeler
  • Saving the Data Labeler for future use
  • Loading the saved Data Labeler
• Creating a ColumnName Labeler
  • Loading and predicting using a pre-existing model using load_from_library
  • Loading and using the pre-existing column name labeler using load_with_components
    • Predicting with the ColumnName labeler
  • Replacing the parameters in the existing labeler
    • Predicting with the new labels
  • Saving the Data Labeler for future use
  • Loading the saved Data Labeler
• Merge Profile List
  • Imports
  • Setup the Data and Profiler
  • Run Merge on List of Profiles
• Dataloader with Popmon Reports
  • How to Install Popmon
  • Comparison of Dataloaders
  • Popmon’s usage example using both dataloaders
  • Examples of data
  • Dataprofiler’s examples using both dataloaders
  • Report comparison