With the the 1.0 release of TensorFlow, I thought it’d be nice to check out the new high-level interface, TFLearn. While the old TF included skflow, that’s been superceded. I know TFLearn co-existed with it for awhile, but I believe they’ve now merged or skflow was totally subsumed. Either way, TFLearn is even more abstract than Keras, letting specify a simple neural network in one line.

While the basic example worked flawlessly, I ran into trouble trying to use the slightly more advanced tf.contrib.learn.DNNLinearCombinedClassifier. In such a “wide and deep” model, some of the input features go through a traditional neural network, while others are passed directly to the output later. Easy enough in concept, right? And sometimes there’s features that you’re pretty sure are linear, like certainscale parameters. But the API for this wasn’t very clear, and the worked example only works with Pandas dataframes, not NumPy arrays. Add to this that it’s no longer a “one line model” since it requires a bunch of setup variables and training functions, and it’s not so simple. Using Keras, I’m pretty sure I could do this with a simple Functional Model; not trivial, but reasonably straightforward, and the code would reflect the shape of the “wide and deep” model. So I’m still sticking to Keras for professional work, but TFLearn is hard to beat for the most common approaches.

It’s a very special stream this week because it’s Valentine’s Day. To celebrate, let’s explore data from a data website. While I couldn’t find recommendation dating on short notice, I did find a general set of anonymized profiles. This provides a framework for test-driving less flashy machine learning python libraries like Pandas and Sci-kit Learn.

Because we’re looking at more “traditional” libraries, the analysis sticks with exploratory data mining and some light regression. No Deep Learning here. That doesn’t mean the models are weak; you should use the simplest model you can get away with, and often, linear regression fits the bill.

With a special thanks to NerdFarmer, we have another sponsored episode on classifying leaf diseases. This time, we focused on trying to simplify the data and the model to make it a little more approachable. Previously, I squared up images by cutting to the smaller dimension and rescaling to 224x224 (thinking maybe transfer learning would happen later). This time, I decided not to throw away any part of images. So I padded the smaller dimension to equal the larger. To get them all on the same size, I still rescaled, now to 64x64. So rather than holding 2GB of data in memory, it was only about 256 MB; much faster to work with. During the stream itself, we even reached 66% accuracy.

After the stream, a fan commented that he used the same model to get about 100% accuracy! While this is great to hear, we may be overfitting the model. At this point, we should select a random subset of the training data to use for validation. I fear the accuracy may not be so stellar.

Ah, Autoencoders, previously my nemesis, but now just another tool in the toolbox. This time, I wanted to sort out how to handle convolutional autoencoders when handling strange sizes. UpSampling2D is the opposite of MaxPooling2D except for odd dimensions. Upsampling can’t know that you removed an odd-numbered row in the encoding step. To handle this, you need to add an empty row of zeros back with ZeroPadding2D. This seems incongrous at first, but the more I thought about it, adding fake zeros is pretty much the antithesis of removing a row of pixels.

The model itself, however, takes about 5 minutes per epoch, so I didn’t wait around to see how it would train. One alternative might be to train on the first 16 columns, and use that for each slice. Maybe another time.

Today, we look at another machine learning software library, PyTorch. This is obviously inspired by the Torch project, with the main difference being that PyTorch is made for Python, while Torch is based in Lua. I know all the cool kids are learning Lua, but I already know Python and several other scripting languages, so the marginal utility of another one just isn’t there for me. So it’s nice to see PyTorch available in my environment of choice.

Overall, PyTorch seems pretty decent. It’s similar to TensorFlow or Theano in that it offers fine-grained control (and specification) over graph-based computations. I did encounter a bug (already reported and couldn’t get the CUDA stuff working, but that might be due to legacy hardware. I did enjoy the interactivity of the computations; you can see right away what’s inside various nodes without having to eval them all the time. This comes with a downside, however, as the “value” inside nodes doesn’t get updated unless you force it, so you could get confused about what value is supposed to be in there. Building a neural network is also unintuitive to me. It seems you have to use a Python class; I tried to work around this, but it’s built-in at a low level. I suspect this is behavior or style inherited from traditional Torch, but I’m not a fan. I strongly prefer Keras’ ability to let me muck around with a model, or TensorFlow/Theano’s generality. If I want to put those models in classes, I can, but only PyTorch seems to force me. For now, I’ll stick with Theano/Keras for regular work since they support my hardware and workflow.