SUMMARY: The project aims to construct a data validation flow using TensorFlow Data Validation (TFDV) and document the end-to-end steps using a template. The Chicago Taxi Trips dataset is a regression situation where we attempt to predict the value of a continuous variable.

INTRODUCTION: The City of Chicago collects taxi trip data in its role as a regulatory agency. This example notebook illustrates how we can use TensorFlow Data Validation (TFDV) to investigate and visualize datasets. The data validation process includes examining descriptive statistics, inferring a schema, checking for and fixing anomalies, and detecting drift and skew in the dataset.

Additional Notes: I adapted this workflow from the TensorFlow Data Validation tutorial on TensorFlow.org (https://www.tensorflow.org/tfx/tutorials/data_validation/tfdv_basic). I also plan to build a TFDV script for validating future datasets and building machine learning models.

CONCLUSION: In this iteration, the data validation workflow helped to validate the features and structures of the training, validation, and test datasets. The workflow also generated statistics over different slices of data which can help track model and anomaly metrics.

Dataset Used: Chicago Taxi Trips Dataset, with modifications by TensorFlow.org

Dataset ML Model: Binary classification with numerical and categorical attributes

Dataset Reference: https://storage.googleapis.com/artifacts.tfx-oss-public.appspot.com/datasets/chicago_data.zip

The HTML formatted report can be found here on GitHub.