{"action":"created","issue":{"url":"https://api.github.com/repos/tensorflow/serving/issues/401","repository_url":"https://api.github.com/repos/tensorflow/serving","labels_url":"https://api.github.com/repos/tensorflow/serving/issues/401/labels{/name}","comments_url":"https://api.github.com/repos/tensorflow/serving/issues/401/comments","events_url":"https://api.github.com/repos/tensorflow/serving/issues/401/events","html_url":"https://github.com/tensorflow/serving/issues/401","id":220678815,"number":401,"title":"Is it normal that it costs around 15 seconds when I am using the exported inception v3 model in tensorflow serving to predict an image from imagenet?","user":{"login":"MWPLab","id":18246130,"avatar_url":"https://avatars1.githubusercontent.com/u/18246130?v=3","gravatar_id":"","url":"https://api.github.com/users/MWPLab","html_url":"https://github.com/MWPLab","followers_url":"https://api.github.com/users/MWPLab/followers","following_url":"https://api.github.com/users/MWPLab/following{/other_user}","gists_url":"https://api.github.com/users/MWPLab/gists{/gist_id}","starred_url":"https://api.github.com/users/MWPLab/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/MWPLab/subscriptions","organizations_url":"https://api.github.com/users/MWPLab/orgs","repos_url":"https://api.github.com/users/MWPLab/repos","events_url":"https://api.github.com/users/MWPLab/events{/privacy}","received_events_url":"https://api.github.com/users/MWPLab/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":6,"created_at":"2017-04-10T15:19:08Z","updated_at":"2017-06-06T16:10:51Z","closed_at":null,"body":"I have exported inception v3 model in .pb+variables format using the inception saved model python script in the example folder and deployed it in my tensorflow serving machine using bazel. But when I use the inception client in the example folder to send an image from imagenet (like a rose image) to serving  for prediction, it costs 15 seconds for end to end time. \r\nAlso, I have added some code to monitor only the one sentence result = stub.Predict(request, request_timeout) and it turns out that it costs more than 14 seconds. I have another Alexnet model and it costs about 2.5 seconds just to predict one image. Is this normal or there is some wrong with the sample code or model, or the prediction time is proportional to the depth of the model, like deep models would cost way much time than the not that deep models when predicting only one image? Would it cost less time on average if I predict a batch of images at one time in some way? By the way, my machine is a normal virtual machine, Linux+Intel machine 4 core cpu and 8GB memory. \r\nAnother question is, are there any best practices or performance tuning methods for tensorflow serving? As this latency (I am not sure if the prediction time counts the latency) is not that acceptable. Or is it possible that the prediction time would decrease like 5-10 times if I use a more powerful machine, like 16 cores and 32 GB memory, or even more that 10 times with GPU's acceleration? \r\nThanks a lot for answering those many questions! If the questions are not that proper here, I could also post them to StackOverflow, but I am just not sure whether someone would answer them to the point."},"comment":{"url":"https://api.github.com/repos/tensorflow/serving/issues/comments/306536530","html_url":"https://github.com/tensorflow/serving/issues/401#issuecomment-306536530","issue_url":"https://api.github.com/repos/tensorflow/serving/issues/401","id":306536530,"user":{"login":"markusnagel","id":9336024,"avatar_url":"https://avatars3.githubusercontent.com/u/9336024?v=3","gravatar_id":"","url":"https://api.github.com/users/markusnagel","html_url":"https://github.com/markusnagel","followers_url":"https://api.github.com/users/markusnagel/followers","following_url":"https://api.github.com/users/markusnagel/following{/other_user}","gists_url":"https://api.github.com/users/markusnagel/gists{/gist_id}","starred_url":"https://api.github.com/users/markusnagel/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/markusnagel/subscriptions","organizations_url":"https://api.github.com/users/markusnagel/orgs","repos_url":"https://api.github.com/users/markusnagel/repos","events_url":"https://api.github.com/users/markusnagel/events{/privacy}","received_events_url":"https://api.github.com/users/markusnagel/received_events","type":"User","site_admin":false},"created_at":"2017-06-06T16:10:51Z","updated_at":"2017-06-06T16:10:51Z","body":"@vaklyuenkov see my explanation in that issue. Next time please tag me there, than we don't spam this issue with non issue related things. Thanks."}}
{"actor":{"display_login":"markusnagel"}}