Template Credit: Adapted from a template made available by Dr. Jason Brownlee of Machine Learning Mastery.
SUMMARY: The purpose of this project is to construct a predictive model using various machine learning algorithms and to document the end-to-end steps using a template. The MNIST dataset is a multi-class classification situation where we are trying to predict one of several (more than two) possible outcomes.
INTRODUCTION: The MNIST problem is a dataset developed by Yann LeCun, Corinna Cortes, and Christopher Burges for evaluating machine learning models on the handwritten digit classification problem. The dataset was constructed from many scanned document datasets available from the National Institute of Standards and Technology (NIST). The MNIST handwritten digit classification problem has become a standard dataset used in computer vision and deep learning.
Images of digits were taken from a variety of scanned documents, normalized in size and centered. Each image is a 28 by 28-pixel square (784 pixels total). A standard split of the dataset is used to evaluate and compare models, where 60,000 images are used to train a model, and a separate set of 10,000 images are used to test it. It is a digit recognition task, so there are ten classes (0 to 9) to predict.
For the Take1 iteration, we constructed a simple dense-layer model to classify the images. This model serves as the baseline for future iterations.
For this iteration, we will convert the dense-layer model into a convolutional model to examine whether we can improve its performance.
ANALYSIS: For the Take1 iteration, the performance of the Take1 model with the default parameters achieved an accuracy score of 98.05% on the validation dataset after training for five epochs. After tuning the hyperparameters, the Take1 model processed the training dataset with an accuracy rate of 98.82%. The same model processed the test dataset with an accuracy of 98.17%. If we were to train the model much further than three epochs, we could run into the over-fitting issue.
For this iteration, the performance of the Take2 model with the default parameters achieved an accuracy score of 98.77% on the validation dataset after training for five epochs. After tuning the hyperparameters, the Take2 model processed the training dataset with an accuracy rate of 99.46%. The same model processed the test dataset with an accuracy of 98.61%.
Adding the convolutional layers improved the model’s prediction performance slightly. The increasing difference between training and validation accuracy after each epoch indicated that we potentially have a high variance problem. We will need more data to train the model before we can bring up the prediction accuracy of the validation dataset.
CONCLUSION: For this dataset, the model built using Keras and TensorFlow achieved a satisfactory result and should be considered for future modeling activities.
Dataset Used: MNIST Handwritten Digits Dataset
Dataset ML Model: Multi-class classification with numerical attributes
Dataset Reference: https://www.tensorflow.org/datasets/catalog/mnist
One potential source of performance benchmarks: https://machinelearningmastery.com/how-to-develop-a-convolutional-neural-network-from-scratch-for-mnist-handwritten-digit-classification/
The HTML formatted report can be found here on GitHub.