For April Fool’s day, I wanted to do something different, so I planned to pan the very tool that served me through undergrad: Excel97. As luck would have it, some other folks had the same idea and scooped me! So, I decided to go through my usual paces of learning a tool, but this time with Deep Excel (and still the MNIST model, of course)

Mostly, I malign spreadsheets because they’re the wrong tool for the data science and machine learning job. They are great for some tasks, like tabularizing and doing simple computations on data. And Excel gives you the ability to easily see your variables as you manipulate them. But I still remember how much I fought Excel trying to implement something like a “for loop”. This was before I knew Python or even MatLab; if only I’d known there was a better way.

I missed posting about last week’s successful video defining custom Keras activations! This wasn’t nearly as painful as figuring out how to access the weights in an skflow model, but it’s about as much work after the fact. Like everything in life, no tool is perfect.

And just tonight I started looking at Recurrent Neural Networks. These are cool ways to introduce the element of time in your dataset. Essentially, you’re adding extra features from the previous data point into your current time step’s neural net. It gets more complicated than that with things like LSTM (use those features to decide when to “forget” old information and the like), but we’re sticking to the basics.

Unfortunately, I didn’t see a tutorial on using RNNs with Keras, so we’re winging it off of a Theano tutorial. But it’s a little light on the explanations and details. Hopefully my video starts to clear the waters.

In the end, I couldn’t get the model working by the 1 hour mark. That’s ok, this just gives us another topic to cover next week: Keras RNN 2, Electric Boogaloo.

It’s time for a whole new library, Keras! This is actually another high level wrapping langauge, similar to skflow. It was originally built on top of Theano, but now it works with both Theano and TensorFlow. The interface isn’t as simple as skflow’s one line deep neural network, but it still makes common things easy. Oh! And it has beautiful progress bars while going through training. So soothing.

The basic construction in Keras is the model built of layers. After creating an empty model, you can add densely connected layers (and then their activation functions, like sigmoids), convolutional layers, or whatever. Then you tie it up with your output layer, typically softmax for classification. To actually train the model, you pick one of the many optimizers, set a couple parameters, and crank through your data.

The one downside is that it’s not clear how to add custom components. skflow included examples with custom convolutional layers, and eventually I figured out how to access its weights. But Keras is still opaque; I think I can hack in some custom activation functions, but it won’t be trivial.

After spending most of last video figuring out how to properly import the data for CIFAR10 image recognition, this time I blitzed through the modeling and visualization. You can download the binary data yourself from Alex Krizhevsky. Once you do that and read it in with np.fromfile('/path/to/extracted/data/bin'), it’s pretty easy to manipulate and use with skflow.

This video also made good use of pulling out the weights from skflow as discovered previously. In fact, it was nice to look at the first layer weights by properly coloring them red, green, or blue according to the channel.

In this case, I was only using 1/5 of the data anyway (forgot to read all the globs), but it still demonstrates how to setup and run the model with skflow. I really appreciate how easy skflow makes setting up a typical neural net. If only extracting the weights were a little more natural. Maybe that’s a future pull request.

This week we started looking at a new dataset (no more MNIST, rejoice!), the CIFAR10 image recognition dataset. Well, “looking” might be a strong verb. I wanted to build a simple CNN to try this in skflow. There’s a TensorFlow tutorial on this benchmark, but they develop a custom method of actually reading in the data. So it took me a while to decipher what they were doing before I could just read the images into my own simple numpy array. But I got it eventually and built a basic logistic regression classifier (not very good, but shows that the setup works). Next time (March 15 as I’m off next week), we’ll be at it in full force.