Template Credit: Adapted from a template made available by Dr. Jason Brownlee of Machine Learning Mastery.

SUMMARY: The purpose of this project is to construct a prediction model using various machine learning algorithms and to document the end-to-end steps using a template. The Superconductivity Temperature dataset is a regression situation where we are trying to predict the value of a continuous variable.

INTRODUCTION: The research team wishes to create a statistical model for predicting the superconducting critical temperature based on the features extracted from the superconductor’s chemical formula. The model seeks to examine the features that can contribute the most to the model’s predictive accuracy.

In iteration Take1, we established the baseline mean squared error for comparison with the future rounds of modeling.

In iteration Take2, we examined the feature selection technique of attribute importance ranking by using the Gradient Boosting algorithm. By selecting only the most important attributes, we decreased the modeling time and still maintained a similar level of RMSE compared to the baseline model.

In this iteration, we will examine the feature selection technique of recursive feature elimination (RFE) by using the Random Forest algorithm. By selecting no more than 50 attributes, we hope to maintain a similar level of RMSE compared to the baseline model.

ANALYSIS: The baseline performance of the machine learning algorithms achieved an average RMSE of 14.84. Two algorithms (Extra Trees and Random Forest) achieved the top RMSE metrics after the first round of modeling. After a series of tuning trials, Extra Trees turned in the best overall result and achieved an RMSE metric of 9.56. By using the optimized parameters, the Extra Trees algorithm processed the test dataset with an RMSE of 9.32, which was even better than the prediction from the training data.

From iteration Take2, the average performance of the machine learning algorithms achieved an RMSE of 15.29. Extra Trees achieved an RMSE metric of 9.57 with the training dataset and processed the test dataset with an RMSE of 9.34. At the importance level of 99%, the attribute importance technique eliminated 31 of 81 total attributes. The remaining 50 attributes produced a model that achieved a comparable RMSE to the baseline model. The modeling time went from 2 minutes 39 seconds down to 1 minute 37 seconds, a reduction of 35.8%.

From iteration Take3, the average performance of the machine learning algorithms achieved an RMSE of 15.27. Extra Trees achieved an RMSE metric of 9.57 with the training dataset and processed the test dataset with an RMSE of 9.30. The recursive feature selection technique eliminated 31 of 81 total attributes. The remaining 50 attributes produced a model that achieved a comparable RMSE to the baseline model. The modeling time went from 2 minutes 39 seconds down to 1 minute 34 seconds, a reduction of 40.8%.

CONCLUSION: For this iteration, the Extra Trees algorithm achieved the best overall results using the training and testing datasets. For this dataset, Extra Trees should be considered for further modeling.

Dataset Used: Superconductivity Data Set

Dataset ML Model: Regression with numerical attributes

Dataset Reference: https://archive.ics.uci.edu/ml/datasets/Superconductivty+Data

One potential source of performance benchmarks: https://doi.org/10.1016/j.commatsci.2018.07.052

The HTML formatted report can be found here on GitHub.