TensorflowModel & gRPC

[ratulray]

With reference to this comment explaining the difference between TensorflowModel and Model, I can not find the implementation of the server side code for TensorflowModel (the github links given are broken).
I see that tf_container/serve.py was removed in the following change:
aws/sagemaker-tensorflow-training-toolkit@12fd7ef
And I can not find where the latest container-side implementation lies in github.

Similarly it'll be useful to be able to check out container-side implementation corresponding to older container/sagemaker/serve.py.

The reason I'm asking is:
I want to be able to export a model with build_parsing_serving_input_receiver_fn (which expects a serialized tf.Example while serving) and this doesn't work well with TF REST api as it's not possible to put raw bytes into JSON payload. So, it works with TensorflowModel and not with Model (example). But I don't understand why we can not use py3 with TensorflowModel as TensorFlow serving api library supports both py2 & py3.
Ideally I would like to do something like what this example does, sending a serialized gRPC request (e.g. PredictionRequest proto) directly from client, but can not figure out how to do that cleanly (and hopefully without involving py2).

[nadiaya]

The old TensorflowModel as well as the image have been deprecated and no longer getting updated. It doesn't support py3 because the official corresponding version of TensorFlow Serving did not support py3 at the time.

Here is the latest serving container implementation which just uses TF serving and isn't bound to any python version: https://github.com/aws/sagemaker-tensorflow-serving-container/

And the actual serving files: https://github.com/aws/sagemaker-tensorflow-serving-container/tree/master/docker/build_artifacts/sagemaker

It's actually more flexible than old implementation and you can do both REST and gRPC requests. Here is the documentation on how to use it:
https://github.com/aws/sagemaker-python-sdk/blob/master/src/sagemaker/tensorflow/deploying_tensorflow_serving.rst#deploying-to-tensorflow-serving-endpoints

[nadiaya]

Also here is an example on how to pass binary data:
https://github.com/awslabs/amazon-sagemaker-examples/blob/master/sagemaker_batch_transform/tensorflow_open-images_jpg/tensorflow-serving-jpg-python-sdk.ipynb

[laurenyu]

closing due to inactivity