Howdy, Deep learning is really hot right now and people seem to like iPhones. Let’s put them together :). A while back there weren’t many tutorials on how to do this, now there are quite a few more. I’d like to add to the space with a maximally efficient minimal working example. We’re going to build a deep learning iOS app which classifies CIFAR10 in 5 minutes. If you’d like to see a live code demonstration of this tutorial please checkout my youtube video.
Make sure you have keras, coremltools, and xcode before getting started. You can install them by buying a mac and then running:
pip install keras & coremltools
Train the Model
We don’t have time to design a model from scratch! We only have 5 minutes! Let’s just steal this one. If you’re familiar with keras you’ll see that it’s just loading in the CIFAR dataset and then training a simple CNN on it. We need to make a few adjustments to complete this tutorial quickly. First, change the number of epochs on line 18 to only 1. This will mean that our model should finish training in about 3 minutes on my Macbook Pro. Next change the data_augmentation parameter on line 19 to False. Lastly, get rid of lines 103-113 we don’t need to save our model and we don’t need to (want to) know how good (bad?) it is. We just need a fast model!
One last thing before we start training. We need to get this trained model in to a form that Steve Jobs can understand. To do that we go to Apple for guidance. At the bottom of the page you will see a Convert Model Section steal those lines of code pasted them at the bottom of our stolen keras model file. We need to make a few small changes to have them work with keras models. Update them as below.
If you saved this in a file called mlcore_convert.py then to start the model cooking you just have to run:
Build an iOS App
While our model is simmering let’s build an iOS app for classifiying images. I don’t have time to design an App so I’m just going to steal one that Apple approves of. This command will download an iOS Xcode project that uses a model for image classification.
People seem to like Apple so it can’t be too bad right? Now just open up this fancy new Xcode project and we’re done with our App!
(Note sometimes Apple likes to change the name of their links for no reason so if the above wget command doesn’t work you can also download this Xcode project here)
Fix the Broken Stuff
There are a few things with this app that we need to deal with.
We need to add our newly trained model to the Xcode project and update the code to use our model instead.
The picture size used in CIFAR is different than that taken by your iPhone. We need to scale the images.
1. Add the Model
Let’s incorporate our new model! Change line 30 of ImageClassificationViewController.swift to:
let model = try VNCoreMLModel(for: CIFAR().model)
Now copy the new model we just trained with keras to
Vision+ML Example/Model/CIFAR.mlmodel and link
it using Xcode’s build phases.
2. Fix Sizing Issues
Now we need to add a function to resize the image the following function stolen from stackoverflow will do the trick:
Now we just need to call this new function in our App. Change the first few lines of updateClassifications to:
This will resize all images before running them through our model.
We’re done! Just build the app now : ). You can put any simple keras model in to an iOS application. Try making the model better or classifying other things.