Template Credit: Adapted from a template made available by Dr. Jason Brownlee of Machine Learning Mastery.

SUMMARY: This project aims to construct a text classification model using a neural network and document the end-to-end steps using a template. The Movie Review Sentiment Analysis dataset is a binary classification situation where we attempt to predict one of the two possible outcomes.

Additional Notes: This script is a replication, with some small modifications, of Dr. Jason Brownlee’s blog post, Deep Convolutional Neural Network for Sentiment Analysis. I plan to leverage Dr. Brownlee’s tutorial and build a TensorFlow-based text classification notebook template for future modeling of similar datasets.

INTRODUCTION: The Movie Review Data is a collection of movie reviews retrieved from the imdb.com website in the early 2000s by Bo Pang and Lillian Lee. The reviews were collected and made available as part of their research on natural language processing. The dataset comprises 1,000 positive and 1,000 negative movie reviews drawn from an archive of the rec.arts.movies.reviews newsgroup hosted at IMDB. The authors refer to this dataset as the ‘polarity dataset.’

We will use the last 100 positive reviews and the last 100 negative reviews as a test set (200 reviews) and the remaining 1,800 reviews as the training dataset. This is a 90% train, 10% split of the data.

In iteration Take1, we constructed the necessary code modules to handle the tasks of loading text, cleaning text, and vocabulary development.

In iteration Take2, we constructed a bag-of-words model and analyzed it with a simple multi-layer perceptron network.

In iteration Take3, we fine-tuned the model using different Tokenizer modes in the Keras API.

In this Take4 iteration, we will fine-tune the model using word embedding as part of fitting a deep learning model.

ANALYSIS: From iteration Take2, the baseline model’s performance achieved an average accuracy score of 85.81% after 25 epochs with ten iterations of cross-validation. Furthermore, the final model processed the test dataset with an accuracy measurement of 91.50%.

From iteration Take3, the best model’s performance (using the frequency Tokenizer mode) achieved an average accuracy score of 86.06% after 25 epochs with ten iterations of cross-validation. Furthermore, the final model processed the test dataset with an accuracy measurement of 86.50%.

In this Take4 iteration, the best model’s performance (using the word embedding with Keras) achieved an average accuracy score of 84.17% after ten epochs with ten iterations of cross-validation. Furthermore, the final model processed the test dataset with an accuracy measurement of 88.50%.

CONCLUSION: In this iteration, the best TensorFlow model appeared to be suitable for modeling this dataset. We should consider experimenting with TensorFlow for further modeling.

Dataset Used: Movie Review Sentiment Analysis Dataset

Dataset ML Model: Binary class text classification with text-oriented features

Dataset Reference: https://www.cs.cornell.edu/home/llee/papers/cutsent.pdf and http://www.cs.cornell.edu/people/pabo/movie-review-data/review_polarity.tar.gz

One potential source of performance benchmarks: https://machinelearningmastery.com/deep-learning-bag-of-words-model-sentiment-analysis/

The HTML formatted report can be found here on GitHub.