For every data project, we can confirm the “health” of our data by conducting data health checks
Checking for design integrity is a health check
confirming that data is not being duplicated is a health check
Consistency checks of the data aim to ensure:
that data is in alignment with the data governance planning for systems
and that it follows the designed flows as intended
can prevent a lot of downstream issues by ensuring the quality and completeness of the data
Data health checks can be a mix of manual and automated checks
For machine learning algorithms,
data health checks are essential in order to ensure we are getting the best results based on the data the organization consumes
often necessary to detect data drift
A phenomenon where the statistical properties of input data change over time, potentially reducing the accuracy and reliability of machine learning models or analytical systems
occurs when your data is statistically different than the data that was originally inputted