A Single Layer Neural Network coded from scratch that performs digit recognition in images with more than 95% accuracy.
The goal of the project was to compare different hand-coded optimization algorithms in terms of accuracy, speed of convergence and computational cost. The ones tried were Gradient Method (GM), Stochastic Gradient Method (SGM) and the Broyden–Fletcher–Goldfarb–Shanno algorithm (BFGS).
The data used was a subset of 500 images from the MNIST database, split in half for training and testing. The pixels were flattened before being inputted to the network, which was trained to output a binary answer on whether the image contained X digit or not.
The digit 3 was the hardest to identify, where the model obtained only 86% accuracy, probably due to its resemblance to the shape of the 8. In the image above, pink and red images are false positives and negatives, respectively; while green and blue ones are correct answers.