docker+tensorflow/serving

发表于 更新于 阅读次数：

Tensorflow在部署上也提供的许多多方便的方法。其中最为方便的，就是通过Docker+Tensorflow_Serving 进行部署。在这里仅仅只对docker + tensorflow/serving + rest api 进行简单介绍。

在进行部署前，先要安装Docker。

Docker

Install Docker

点击该链接进行安装 Install Docker

安装成功后，打开终端，输入

1
$ docker info

若能正确得到docker信息，代表安装成功。

常用命令

基本上了解 一下命令就行了。至于构建自己的Docker image等高级操作 请参阅Docker 以及docker_practice

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
#显示当前正在运行的容器
$ docker ps  

#显示所有容器，包括未运行的容器
$ docker ps --all			

#显示本地所有镜像
$ docker images		

#删除本地指定iamge_name镜像
$ docker image rm image_name	

#kill掉当前正在运行的容器
$ docker kill container_id/container_name

#使当前的容器停止运行  stop是等容器完成后续操作后停止，而kill是立即停止，会使得当前操作丢失
$ docker stop container_name

#使停止的容器重新运行
$ docker start container_name

#docker通过镜像 创建容器，并允许  代表参数省略 需要补齐
$ docker run ··· image_name ···
# 如   --name指定创建的容器名字,-d后台运行,-p 指定端口映射,8500:80,其中8500是外部端口，80是容器内部端口
$ docker run --name my_image_name -d -p 8500:80 image_name

#进行正在运行的容器中，并进入交互式bash
$ docker exec -it container_name bash

tensorflow/serving

在进行介绍前，首先确保已经安装了Docker。

这里我们以最简单的MINIST 进行介绍。下面是Tensorflow2.0版本的代码。

MINIST Model Save

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
import tensorflow as tf
from tensorflow.keras.layers import Dense, Flatten, Conv2D
from tensorflow.keras import Model
import numpy as np

mnist = tf.keras.datasets.mnist
(x_train, y_train), (x_test, y_test) = mnist.load_data()
x_train, x_test = x_train / 255.0, x_test / 255.0

# 增加 一个 维度
x_test = x_test[..., None]
x_train = x_train[..., None]


# 进行数据处理
train_ds = tf.data.Dataset.from_tensor_slices((x_train, y_train)).shuffle(10000).batch(32)
test_ds = tf.data.Dataset.from_tensor_slices((x_test, y_test)).shuffle(10000).batch(32)



# 构建模型
class MyModel(Model):
    def __init__(self):
        super(MyModel, self).__init__()
        self.conv1 = Conv2D(32, 3, activation='relu')
        self.flatten = Flatten()
        self.d1 = Dense(128, activation='relu')
        self.d2 = Dense(10, activation='softmax')

    def call(self, inputs, training=None, mask=None):
        x = self.conv1(inputs)
        x = self.flatten(x)
        x = self.d1(x)
        return self.d2(x)


model = MyModel()

# 选择优化器和损失函数
loss_object = tf.keras.losses.SparseCategoricalCrossentropy()
optimizer = tf.keras.optimizers.Adam()

train_loss = tf.keras.metrics.Mean(name='train_loss')
train_accuracy = tf.keras.metrics.SparseCategoricalAccuracy(name='train_accutacy')

test_loss = tf.keras.metrics.Mean(name='test_loss')
test_accuracy = tf.keras.metrics.SparseCategoricalAccuracy(
    name='test_accuracy')


@tf.function
def train_step(images, labels):
    with tf.GradientTape() as tape:
        predictions = model(images)
        loss = loss_object(labels, predictions)

    gradients = tape.gradient(loss, model.trainable_variables)
    optimizer.apply_gradients(zip(gradients, model.trainable_variables))

    train_loss(loss)
    train_accuracy(labels, predictions)


@tf.function
def test_step(images, labels):
    predictions = model(images)
    t_loss = loss_object(labels, predictions)

    test_loss(t_loss)
    test_accuracy(labels, predictions)

# 进行训练
EPOCHS = 1
for epoch in range(EPOCHS):
    for images, labels in train_ds:
        train_step(images, labels)

    for test_images, test_labels in test_ds:
        test_step(test_images, test_labels)

    template = "Epoch {}, loss: {}, Accuracy: {}, test Loss: {}, test Accuracy: {}"
    print(template.format(
        epoch + 1,
        train_loss.result(),
        train_accuracy.result() * 100,
        test_loss.result(),
        test_accuracy.result() * 100
    ))

 # 保存模型
model.save('./mode_out')

通过上面的model.save(‘./mode_out’)对模型进行保存，保存后，产生的文件目录如下:

/Users/lollipop/Documents/tf2/learn/mode_out
└── 00000123
├── assets
├── saved_model.pb
└── variables
├── variables.data-00000-of-00001
└── variables.index

有了模型后，我们就要开始进行部署了。

Server

通过下面docker命令 获取tensorflow/serving 镜像

1
$ docker pull tensorflow/serving

获取后，通过上面讲的常用Docker命令进行查看

1
$ docker images

可以看到 我们确实获取了tensorflow/serving

1
2
3
4
REPOSITORY           TAG                 IMAGE ID            CREATED             SIZE
nginx                latest              231d40e811cd        8 days ago          126MB
tensorflow/serving   latest              048f8669e211        5 weeks ago         214MB
hello-world          latest              fce289e99eb9        11 months ago       1.84kB

然后在终端进行输入下方命令进行部署。其中

source指定model_out的位置.

1
$ docker run -p 8501:8501 --name MINIST --mount source=/Users/lollipop/Documents/tf2/learn/mode_out,type=bind,target=/models/MINIST -e MODEL_NAME=MINIST -t tensorflow/serving

部署成功后终端会显示 如下信息：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
2019-12-01 06:26:46.997159: I tensorflow_serving/model_servers/server.cc:85] Building single TensorFlow model file config:  model_name: MINIST model_base_path: /models/MINIST
2019-12-01 06:26:46.997581: I tensorflow_serving/model_servers/server_core.cc:462] Adding/updating models.
2019-12-01 06:26:46.998092: I tensorflow_serving/model_servers/server_core.cc:573]  (Re-)adding model: MINIST
2019-12-01 06:26:47.375516: I tensorflow_serving/core/basic_manager.cc:739] Successfully reserved resources to load servable {name: MINIST version: 123}
2019-12-01 06:26:47.380496: I tensorflow_serving/core/loader_harness.cc:66] Approving load for servable version {name: MINIST version: 123}
2019-12-01 06:26:47.384253: I tensorflow_serving/core/loader_harness.cc:74] Loading servable version {name: MINIST version: 123}
2019-12-01 06:26:47.388339: I external/org_tensorflow/tensorflow/cc/saved_model/reader.cc:31] Reading SavedModel from: /models/MINIST/00000123
2019-12-01 06:26:47.410994: I external/org_tensorflow/tensorflow/cc/saved_model/reader.cc:54] Reading meta graph with tags { serve }
2019-12-01 06:26:47.415580: I external/org_tensorflow/tensorflow/core/platform/cpu_feature_guard.cc:142] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA
2019-12-01 06:26:47.482041: I external/org_tensorflow/tensorflow/cc/saved_model/loader.cc:202] Restoring SavedModel bundle.
2019-12-01 06:26:47.747514: I external/org_tensorflow/tensorflow/cc/saved_model/loader.cc:151] Running initialization op on SavedModel bundle at path: /models/MINIST/00000123
2019-12-01 06:26:47.790868: I external/org_tensorflow/tensorflow/cc/saved_model/loader.cc:311] SavedModel load for tags { serve }; Status: success. Took 402421 microseconds.
2019-12-01 06:26:47.798769: I tensorflow_serving/servables/tensorflow/saved_model_warmup.cc:105] No warmup data file found at /models/MINIST/00000123/assets.extra/tf_serving_warmup_requests
2019-12-01 06:26:47.944096: I tensorflow_serving/core/loader_harness.cc:87] Successfully loaded servable version {name: MINIST version: 123}
2019-12-01 06:26:47.955334: I tensorflow_serving/model_servers/server.cc:353] Running gRPC ModelServer at 0.0.0.0:8500 ...
[warn] getaddrinfo: address family for nodename not supported
[evhttp_server.cc : 238] NET_LOG: Entering the event loop ...
2019-12-01 06:26:47.965905: I tensorflow_serving/model_servers/server.cc:373] Exporting HTTP/REST API at:localhost:8501 ...

Client

服务端已经部署成功，我们来看看client 端怎么编写。这里我用的是REST api。 

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
import tensorflow as tf
import numpy as np
import json
import requests

#加载数据
mnist = tf.keras.datasets.mnist
(x_train, y_train), (x_test, y_test) = mnist.load_data()
x_train, x_test = x_train / 255.0, x_test / 255.0

# 增加 一个 维度
x_test = x_test[..., None]


# 将数据转换为Rest要求的Json格式
data = json.dumps({"instances": x_test[0:30].tolist()})
json_response = requests.post('http://localhost:8501/v1/models/MINIST:predict', data=data)
# json_response = requests.post('http://ip:端口/v1/models/MINIST:predict', data=data)

# 得到返回的预测结果
predictions = json.loads(json_response.text)['predictions']
print(np.argmax(predictions, axis=-1))
print(tf.argmax(predictions, axis=-1))

输出：

[7 2 1 0 4 1 4 9 6 9 0 6 9 0 1 5 9 7 3 4 9 6 6 5 4 0 7 4 0 1]

tf.Tensor([7 2 1 0 4 1 4 9 6 9 0 6 9 0 1 5 9 7 3 4 9 6 6 5 4 0 7 4 0 1], shape=(30,), dtype=int64)

在此可以看到Server端正确的给出了预测。

通过这种docker+tensorflow/serving 的方式，我们可以轻松的将我们的模型在服务器进行部署，部署后就像开了一个API接口一样，可供其他应用进行调用。