C++ Tensorflow NodeJs Addon
I was playing around with NodeJs and its Addon architecture.
It's nice and I thought it would make for a very simple way to provide an Image Classification service using
Tensorflow.
Tensorflow
The tensorflow docs have improved a bit with version 1 but things are still a little thin. Particularly on the
C++ side. Even so you can get quite far by adapting the example code provided.
You'll want to follow the install from source
instructions. You'll need google build tool bazel
for this. I found the install process for both seemless (if lengthy) on both OSX and Linux - The windows install
is described as experimental and I haven't tried it.
I used WebStorm which has a nodejs express plugin which makes creating projects pretty simple but there are loads of
other ways to write the node side.
Start Simple
In this post we'll make a very simple c++ tensorflow lib which does a matrix multiply. Tensorflow is a general
maths tool, not only for machine learning, so it's nice to show it's not just about big deep learning architectures.
float example::MatrixMul() {
Scope root = Scope::NewRootScope();
auto vectorConstant = Const(root, { {2.0, 7.0 } });
auto matrixMultipy = MatMul(root, vectorConstant, { {7.0 }, {4.0 } });
ClientSession session(root);
std::vector outputs;
session.Run( {matrixMultipy }, &outputs);
float score = outputs[0].flat()(0);
return score;
}
The codes pretty simple and we only return a single float so we don't have to worry about types on
the node side for the moment.
It should all make sense if you've used tensorflow through python.
If you haven't then look up some python tutorials (I'll put some up here soon too):
The basic thing you need to grasp is that
tensorflow works by first creating a computational graph (like a program) so matrixMultipy above does
not return a value, it return the node in the computational graph (like a method in a program).
To get
an answer you have to run the graph (program), in session.Run we feed in the node we want to get an answer
from and a address for where we want the result stored.
If we add a main() we can build it as a binary first to see things are working
before placing another layer of possible error with our addon code.
Have a look at the source file here
example.cc
There's a simple method in there (called Simple) you can call from nodejs to see communication is working
without using the tensorflow libraries. That should always work, so if the node call fails you know something
is badly wrong...
The h file is here
example.h
The build file looks like this
# Description:
# TensorFlow C++ example
cc_binary(
name = "libexample.so",
srcs = ["example.cc", "example.h"],
linkshared = 1,
deps = [
"//tensorflow/cc:cc_ops",
"//tensorflow/cc:client_session",
"//tensorflow/core:tensorflow",
],
visibility = ["//visibility:public"],
)
It's pretty simple but I spent time messing around with cc_library which seemed like a good way to
go but I was getting unrecognized symbols when my node c++ code tried to connect.
This works so it's good enough for the moment.
Remove the linkshared line and change * name = "libexample.so" * to * name = "example" * if you
want to test the output as a binary first (you can just double click on the resulting example file
in OSX to see the result)
I've followed the tensorflow tutorials example and put these in
$pathToTensorFlow/tensorflow/cc/example/ where $pathToTensorFlow is where you git cloned Tensorflow.
To build it from $pathToTensorFlow run
bazel build tensorflow/cc/example/...
This takes
time so don't worry - future builds are
much faster.
Node
Let's move to the node side: I created an Express project in Webstorm but do whatever you
feel comfortable with. Create an addons folder, I put mine at root level:
At the root level create binding.gyp, this is where you'll build the addon
{
'targets': [{
'target_name': 'example',
'sources': ["addons/hellotf.cc", "addons/example.h"],
'libraries': [
'$pathToTensorflow/tensorflow/bazel-bin/tensorflow/cc/example/libexample.so'
],
'include_dirs' : [
"<!(node -e \"require('nan')\")"
]
}]
}
Again replacing the $pathToTensorflow.
target_name is the name you where the code will be build in this case: ../build/Release/example.node
Lets create hellotf.cc in the addons folder you created
// hello.cc
#include <node.h>
#include "example.h"
namespace demo {
using v8::FunctionCallbackInfo;
using v8::Isolate;
using v8::Local;
using v8::Object;
using v8::String;
using v8::Value;
using examples::example;
auto simpleExample = example();
void Method(const FunctionCallbackInfo<Value> args) {
Isolate* isolate = args.GetIsolate();
auto b = simpleExample.MatrixMul();
char buffer [100];
int cx;
cx = snprintf(buffer, 100, "The answer to ltuae is %e", b);
args.GetReturnValue().Set(String::NewFromUtf8(isolate, buffer));
}
void init(Local<Object> exports) {
NODE_SET_METHOD(exports, "hellotf", Method);
}
NODE_MODULE(addons, init)
} //
This is a standard addon outline - the init method exports our method as "hellotf" - so from javascript we
can call foo.hellotf() after we bring it in using require (see below)
Add the example.h header file to the addons directory as well but remove the tensorflow includes from the top
Now build the addon by going to the root directory.
Install
node gyp
npm install -g node-gyp
now call
node-gyp configure
to create the build files and finally
node-gyp build
Look at the node
addon page if you get lost.
In the routes/index.js (Webstorm will create this if you choose an Express project) paste the following
const example = require('../build/Release/example.node');
console.log(example.hellotf());
router.get('fs', function(req, res, next) {
res.render('index', { title: 'DeepThought' , text:
example.hellotf()});
});
Everything should work now - we just render using a jade (pug) file.
Fire up the node server and see if it works.
It didn't?
Did you get something like "Library not loaded /...../libexample.so", Reason: image not found
If the bazel build went well then all that's missing is node knowing where the lib is: The binding.gyp might
have the correct root but this isn't necessarily the shared lib address.
On OSX use otool -D on the .so file, it might give you bazel-out/local-opt/bin/.. so if the tensorflow directory
is not on the path searched for shared libraries it won't work.
You can fix it in OSX with
sudo install_name_tool -id
'$pathToTensorflow/tensorflow/bazel-bin/tensorflow/cc/example/libexample.so' libexample.so
Like before change the $pathToTensorflow.
If you're not in the libexample.so directory you'll need the full path. You'll also need to call node-gyp
build again (and change something in the .cc or it'll say nothing has changed and do nothing, I'm guessing
there should be a flag to force the build but I didn't find anything).
Getting images across
Let's start from the node side, we'll write a simple template for an upload form and connect it to a handler,
I used Busboy but if you have a preference go for it.
Here's the route:
var router = express.Router();
router
.get('fs', function(req, res, next) {
res.render('upload', { title: 'Upload' });
});
and the template:
extends layout
block content
form(action="/classify", method="post", cnctype="multipart/form-data")
input(type="text", name="title")
input(type="file", name="image")
button(type="submit", value="Upload") Upload File
The handling code is in
common.js and we also have a little helper
for dealing with the incoming file
util.js
If you've used a file uploader it should all look pretty standard - we base64 encode the data so we can send
it to the classify page to display.
This time our addon is going to have a method that accepts the buffer and an image type we get from mimetype.
We wait for the addon to return a value but obviously you don't have to block.
console.log('Done parsing form!');
var img = new Buffer(buffer, 'base64');
var imgString = img.toString('base64');
console.log('Image string to parse: ' + imgString);
console.time("classify");
var returnValue = image.classify(img, fileType);
console.timeEnd("classify");
console.log('Returned value: ' + returnValue);
res.render('classify', {
title: 'Classify', imageData: imgString, imageJson: returnValue},
function(err, html) {
res.send(html);
});
Let's skip across to the new addon now. It's going to look a lot like the other one but with code for handling
the Uint8Array we're sending it so we can pass it on to Tensorflow.
You'll find it in
classify.cc
The business part is here:
assert(args[0] -> IsUint8Array());
assert(args[1] -> IsNumber());
int imageType = args[1]->NumberValue();
auto imageArray = args[0].As<Uint8Array>();
//We want the length to create the string
auto length = imageArray->Length();
Nan::TypedArrayContents dataPtr(imageArray);
std::string charString = std::string(*dataPtr, *dataPtr + length);
There may be a shorter way to create this but I find v8 pretty short on detail. I've use native abstractions
for node
NAN to help out a bit after getting lost trying to pull
stuff out of v8's local wrapper.
Back to tensorflow
Tensorflow is expecting... tensors - what's a tensor? If you're familiar with numpy then it's just an NDArray.
For a color image we are going to have 3
channels RGB or whatever ordering (this can matter - it's fine
if you train from scratch but if you use a trained model with the channels in the wrong order things break) and
a batch dimension - Tensorflow expects everything in batches, rolling things up into big matrices is efficient.
So our tensor will be have size (numberOfImage, channels, numberOfPixels)
numberOfPixels might be 2 dimensions (width * height), futher down the line convolutions can change this again, lots
of things can change the tensor shape as we move through the graph.
This is why an NDArray, tensor type representation is one of the first things you should check if a library
supports fully, they are useful.
We are sending a string from our Node code so we'll want to decode it on the Tensorflow side. The library has
some helper code for this DecodeJpeg, DecodePng, DecodeGif, which expect a Tensorflow Input, luckily a string
is a valid input so there's no work to do.
Status classifier::StringToImage(tensorflow::Scope root, tensorflow::Input encodedImage,
int imageType, tensorflow::Output* imageOutput) {
const int wantedChannels = 3;
//TODO put in some peaking code here
tensorflow::Output decodedImage;
if (imageType == 1) {
auto imageReader = DecodeJpeg(root, encodedImage, DecodeJpeg::Channels(wantedChannels));
decodedImage = imageReader.image;
}
else if (imageType == 2) {
auto imageReader = DecodePng(root, encodedImage, DecodePng::Channels(wantedChannels));
decodedImage = imageReader.image;
}
else if (imageType == 3) {
auto imageReader = DecodeGif(root, encodedImage);
decodedImage = imageReader.image;
}
auto floatCaster = Cast(root.WithOpName("float_caster"), decodedImage, tensorflow::DT_FLOAT);
// Turn our image into a batch
auto dimsExpander = ExpandDims(root, floatCaster, 0);
auto resized = ResizeBilinear(
root, dimsExpander,
Const(root.WithOpName("resized"), {inputHeight, inputWidth }));
// Give us the image squashed into a 1 range
*imageOutput = Div(root.WithOpName(outputName), Sub(root, resized, {inputMean }), {inputStd });
return Status::OK();
}
This shouldn't need too much explanation. We found the imageType on the node side, we could do some peeking at
the headers to verify the value sent but we don't need it here.
The decoded image is a tensor, also a tensorflow::Output, it's a single image so we turn it into a batch tensor
of batch size 1 and then we resize it.
CNNs will either need a fixed size input or a way to window over an input with the fixed window size feeding
into the network below. So it's good to have this here - for images these are usually square so you can
consider cropping and padding though these don't have as much effect on the output as you might naively assume.
You've seen the Scope object before, right back at the start when we did the trivial matrix multiply. It's
where we build the graph before we run it in a session. Here's the Session code.
//This is what our service calls - returns negative for failure
int classifier::ReadAndRun(std::string byteString, int encoding, std::string* json) {
timestamp_t t0 = timer::get_timestamp();
auto root = tensorflow::Scope::NewRootScope();
tensorflow::Output imageTensor;
Status imageStatus = StringToImage(root, byteString, encoding, &imageTensor);
std::unique_ptr<ClientSession> inputSession(new ClientSession(root));
std::vector<Tensor> imageOutputs;
LOG(INFO) << "about to get image";
Status inputStatus = inputSession -> Run( { }, {imageTensor }, &imageOutputs);
LOG(INFO) << "image session loc: " << &inputSession;
LOG(INFO) << "image tensor loc: " << &imageOutputs;
if (!inputStatus.ok()) {
LOG(ERROR) << "Running check failed: " << inputStatus;
return -1;
}
else {
LOG(INFO) << "Got Image!";
}
timestamp_t t1 = timer::get_timestamp();
double timeTaken = (t1 - t0);
LOG(INFO) << "Image adjust time: " << timeTaken;
We put some timer code in there and some generous logging to assure ourselves that things aren't running out
of control