Machine Learning Microservices

There's something about machine learning and image based machine learning in particular that screams microservice to me. Nothing is complete in of itself. The output is very task orientated: are there people in this image? are they smiling? what are they looking at? is there deforestation occuring?

Each graph answers a question - question can follow from one to the other. Some questions are easier to answer than others, simple questions can reduce the need to answer harder questions.

So I started building a toy model - I'd read about c++ addons in nodejs, tensorflow had a c++ api.
I didn't know nodejs or c++ so this seemed a good way to experiment.

Tensorflow's computational graph approach makes transfer from one environment to another pretty simple. Since you're creating the graph definition and it's turning this into a program, it doesn't matter where you do it. Though on some levels I regard python as the devil's playground, it allows you to mess around, jupyter notebooks make this nicer still. It's ideal for building and running ml graphs. The beauty is that at the end you can just save the graph for use in any of the supported languages, environments: Android, iOs, java, c++, etc...
That's the starting point and by and large things work just as I'd hoped. Tensorflow does a really good job here.

The other half of this is nodejs and the addon mechanism - again I really don't have much to complain about.

There's information on glueing the two together here

Ok I guess there are some things to complain about, there is a general lack of documentation, api changes without deprecation... but what really matters is that they work.

Now that we've got an approach we can play around. Let's provide access to a few ml toys: inception, multibox, face detection and alignment.
The first two are pretty pure networks - feed in an image, it's scaled and a result falls out the other end. So we can just import a graph.pb and go. The face detection and alignment took a bit more work but there's nothing hard here, it's all just glue to connect the graph output from our python playground.

The results are nice.

That's pretty satisfying, the proposed figure positions are good, two excellent face identifications together with correct alignment. Inception says we're in the mountains. Nice.

Of course things don't always work out this well and this is one of the reasons a service based approach can seem very appealing.

The inception model is great but it's trained on the imagenet set - that's fine for competitions but there's a whole lot of stuff you're probably not interested in most of the time. A thousand classes is a lot, when most of them are used up with obscure breeds of dog, fish, monkeys, spiders it doesn't leave room for the everyday.

Which, brings us to a question that a whole load of people ask themselves: "What exactly do we want this stuff to do?".
We probably want a whole load of inception size services pre-qualified by simpler networks, context etc... Not as interested that there's a guitar in the image but who is playing it? Then you want a facenet service and not an inception service.

If this is interesting there's an evolving code base in github