{"action":"created","issue":{"url":"https://api.github.com/repos/dmlc/nnvm/issues/199","repository_url":"https://api.github.com/repos/dmlc/nnvm","labels_url":"https://api.github.com/repos/dmlc/nnvm/issues/199/labels{/name}","comments_url":"https://api.github.com/repos/dmlc/nnvm/issues/199/comments","events_url":"https://api.github.com/repos/dmlc/nnvm/issues/199/events","html_url":"https://github.com/dmlc/nnvm/issues/199","id":268299112,"number":199,"title":"use nnvm is much slower than not use it","user":{"login":"HaoLiuHust","id":7233091,"avatar_url":"https://avatars3.githubusercontent.com/u/7233091?v=4","gravatar_id":"","url":"https://api.github.com/users/HaoLiuHust","html_url":"https://github.com/HaoLiuHust","followers_url":"https://api.github.com/users/HaoLiuHust/followers","following_url":"https://api.github.com/users/HaoLiuHust/following{/other_user}","gists_url":"https://api.github.com/users/HaoLiuHust/gists{/gist_id}","starred_url":"https://api.github.com/users/HaoLiuHust/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/HaoLiuHust/subscriptions","organizations_url":"https://api.github.com/users/HaoLiuHust/orgs","repos_url":"https://api.github.com/users/HaoLiuHust/repos","events_url":"https://api.github.com/users/HaoLiuHust/events{/privacy}","received_events_url":"https://api.github.com/users/HaoLiuHust/received_events","type":"User","site_admin":false},"labels":[],"state":"closed","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":6,"created_at":"2017-10-25T07:51:32Z","updated_at":"2017-11-13T10:31:33Z","closed_at":"2017-10-25T15:44:18Z","author_association":"NONE","body":"I run the script below:\r\n```\r\n\"\"\"\r\nCompile MXNet Models\r\n====================\r\n**Author**: `Joshua Z. Zhang <https://zhreshold.github.io/>`_\r\n\r\nThis article is an introductory tutorial to deploy mxnet models with NNVM.\r\n\r\nFor us to begin with, mxnet module is required to be installed.\r\n\r\nA quick solution is\r\n```\r\npip install mxnet --user\r\n```\r\nor please refer to offical installation guide.\r\nhttps://mxnet.incubator.apache.org/versions/master/install/index.html\r\n\"\"\"\r\n# some standard imports\r\nimport mxnet as mx\r\nimport nnvm\r\nimport tvm\r\nimport numpy as np\r\nimport os\r\nimport time\r\nimport cv2\r\n######################################################################\r\n# Download Resnet18 model from Gluon Model Zoo\r\n# ---------------------------------------------\r\n# In this section, we download a pretrained imagenet model and classify an image.\r\nfrom mxnet.gluon.model_zoo.vision import get_model\r\nfrom mxnet.gluon.utils import download\r\nfrom PIL import Image\r\nfrom matplotlib import pyplot as plt\r\nblock = get_model('resnet101_v1', pretrained=True, ctx=mx.cpu())\r\n\r\nimagepath='/home/liuhao/dataset/tinyimagenet/tiny-imagenet-200/test/images/'\r\nimages = os.listdir(imagepath)\r\nimages.sort()\r\n# img_name = 'cat.jpg'\r\nsynset_url = ''.join(['https://gist.githubusercontent.com/zhreshold/',\r\n                      '4d0b62f3d01426887599d4f7ede23ee5/raw/',\r\n                      '596b27d23537e5a1b5751d2b0481ef172f58b539/',\r\n                      'imagenet1000_clsid_to_human.txt'])\r\nsynset_name = 'synset.txt'\r\n# download('https://github.com/dmlc/mxnet.js/blob/master/data/cat.png?raw=true', img_name)\r\n# download(synset_url, synset_name)\r\nwith open(synset_name) as f:\r\n    synset = eval(f.read())\r\n# image = Image.open(img_name).resize((224, 224))\r\n# plt.imshow(image)\r\n# plt.show()\r\n\r\ndef transform_image(image):\r\n    image = np.array(image) - np.array([123., 117., 104.])\r\n    image /= np.array([58.395, 57.12, 57.375])\r\n    image = image.transpose((2, 0, 1))\r\n    image = image[np.newaxis, :]\r\n    return image\r\n\r\ntotal_time=0\r\n\r\nimagesarray = np.zeros(shape=(100,3,224,224))\r\nn=0\r\nfor imagename in images[:100]:\r\n    fullpath = os.path.join(imagepath,imagename)\r\n    #print fullpath\r\n    image = cv2.imread(fullpath)\r\n    image=cv2.resize(image,(224,224))\r\n    x = transform_image(image)\r\n    imagesarray[n]=x\r\n    n+=1\r\n    time_s=time.time()\r\n    output = block(mx.nd.array(x))\r\n    time_e=time.time()\r\n    total_time+=(time_e-time_s)\r\n\r\nprint total_time\r\n\r\n\r\n\r\n######################################################################\r\n# Compile the Graph\r\n# -----------------\r\n# Now we would like to port the Gluon model to a portable computational graph.\r\n# It's as easy as several lines.\r\n# We support MXNet static graph(symbol) and HybridBlock in mxnet.gluon\r\nsym, params = nnvm.frontend.from_mxnet(block)\r\n# we want a probability so add a softmax operator\r\nsym = nnvm.sym.softmax(sym)\r\n\r\n######################################################################\r\n# now compile the graph\r\nimport nnvm.compiler\r\ntarget = 'llvm'\r\nshape_dict = {'data': x.shape}\r\ngraph, lib, params = nnvm.compiler.build(sym, target, shape_dict, params=params)\r\n\r\n######################################################################\r\n# Execute the portable graph on TVM\r\n# ---------------------------------\r\n# Now, we would like to reproduce the same forward computation using TVM.\r\nfrom tvm.contrib import graph_runtime\r\nctx = tvm.cpu()\r\ndtype = 'float32'\r\nm = graph_runtime.create(graph, lib, ctx)\r\ntotal_time=0\r\n\r\nfor image in imagesarray:\r\n    # set inputs\r\n    m.set_input('data', tvm.nd.array(x.astype(dtype)))\r\n    m.set_input(**params)\r\n\r\n    time_s=time.time()\r\n    # execute\r\n    m.run()\r\n    time_e=time.time()\r\n    total_time+=(time_e-time_s)\r\n    # get outputs\r\n    tvm_output = m.get_output(0, tvm.nd.empty((1000,), dtype))\r\n    top1 = np.argmax(tvm_output.asnumpy())\r\n    #print('TVM prediction top-1:', top1, synset[top1])\r\n\r\nprint total_time\r\n# ######################################################################\r\n# # Use MXNet symbol with pretrained weights\r\n# # ----------------------------------------\r\n# # MXNet often use `arg_prams` and `aux_params` to store network parameters\r\n# # separately, here we show how to use these weights with existing API\r\n# def block2symbol(block):\r\n#     data = mx.sym.Variable('data')\r\n#     sym = block(data)\r\n#     args = {}\r\n#     auxs = {}\r\n#     for k, v in block.collect_params().items():\r\n#         args[k] = mx.nd.array(v.data().asnumpy())\r\n#     return sym, args, auxs\r\n# mx_sym, args, auxs = block2symbol(block)\r\n# # usually we would save/load it as checkpoint\r\n# mx.model.save_checkpoint('resnet18_v1', 0, mx_sym, args, auxs)\r\n# # there are 'resnet18_v1-0000.params' and 'resnet18_v1-symbol.json' on disk\r\n#\r\n# ######################################################################\r\n# # for a normal mxnet model, we start from here\r\n# mx_sym, args, auxs = mx.model.load_checkpoint('resnet18_v1', 0)\r\n# # now we use the same API to get NNVM compatible symbol\r\n# nnvm_sym, nnvm_params = nnvm.frontend.from_mxnet(mx_sym, args, auxs)\r\n# repeat the same steps to run this model using TVM\r\n\r\n```\r\n\r\nit turns out use nnvm will much slower than not use it, about 200x slow. is that only use it on Raspberry Pi would help?"},"comment":{"url":"https://api.github.com/repos/dmlc/nnvm/issues/comments/343877631","html_url":"https://github.com/dmlc/nnvm/issues/199#issuecomment-343877631","issue_url":"https://api.github.com/repos/dmlc/nnvm/issues/199","id":343877631,"user":{"login":"zt706","id":7455663,"avatar_url":"https://avatars2.githubusercontent.com/u/7455663?v=4","gravatar_id":"","url":"https://api.github.com/users/zt706","html_url":"https://github.com/zt706","followers_url":"https://api.github.com/users/zt706/followers","following_url":"https://api.github.com/users/zt706/following{/other_user}","gists_url":"https://api.github.com/users/zt706/gists{/gist_id}","starred_url":"https://api.github.com/users/zt706/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/zt706/subscriptions","organizations_url":"https://api.github.com/users/zt706/orgs","repos_url":"https://api.github.com/users/zt706/repos","events_url":"https://api.github.com/users/zt706/events{/privacy}","received_events_url":"https://api.github.com/users/zt706/received_events","type":"User","site_admin":false},"created_at":"2017-11-13T10:31:33Z","updated_at":"2017-11-13T10:31:33Z","author_association":"NONE","body":"Hi:\r\nI encountered the same problem, this is my nnvm/tvm test code:\r\n\r\n```python\r\n 87 def compile_and_exe_graph(sym, params, out_num):\r\n 88     \"\"\"\r\n 89         sym: MXNet static graph(symbol) or HybridBlock in mxnet.gluon\r\n 90     \"\"\"\r\n 91     # now compile the graph\r\n 92     import nnvm.compiler\r\n 93     target = 'cuda'\r\n 94     #target = 'llvm'\r\n 95     shape_dict = {'data': x.shape}\r\n 96     graph, lib, params = nnvm.compiler.build(sym, target, shape_dict, params=params)\r\n 97\r\n 98     # Execute the portable graph on TVM\r\n 99     from tvm.contrib import graph_runtime\r\n100     tvm_ctx = tvm.gpu(0)\r\n101     #tvm_ctx = tvm.cpu(0)\r\n102     dtype = 'float32'\r\n103     m = graph_runtime.create(graph, lib, tvm_ctx)\r\n104     # set inputs\r\n105     m.set_input('data', tvm.nd.array(x.astype(dtype)))\r\n106     m.set_input(**params)\r\n107\r\n108     # execute\r\n109     import time\r\n110     t = time.time()\r\n111     for i in range(100):\r\n112         m.run()\r\n113         # get outputs\r\n114         #tvm_output = m.get_output(0, tvm.nd.empty((1000,), dtype))\r\n115\r\n116         tvm_output = []\r\n117         for i in range(out_num):\r\n118             output = m.get_output(i, tvm.nd.empty(output_shape[i], dtype))\r\n119             #tvm_output.append(output.asnumpy())\r\n120             pass\r\n121     print('100 img total time == ', 1000*(time.time() -t))\r\n122\r\n123     ftimer = m.module.time_evaluator(\"run\", tvm_ctx, 100)\r\n124     for i in range(10):\r\n125         prof_res = ftimer()\r\n126         print(prof_res)\r\n127         # sleep for avoiding device overheat\r\n128         if i + 1 != 10:\r\n129             time.sleep(45)\r\n```\r\n\r\nand I modified the source code of ./src/runtime/rpc/rpc_session.cc:1039 to get the total running time of 100 pictures\r\n\r\n```c++\r\n1025 PackedFunc WrapTimeEvaluator(PackedFunc pf, TVMContext ctx, int nstep) {\r\n1026   auto ftimer = [pf, ctx, nstep](TVMArgs args, TVMRetValue *rv) {\r\n1027     TVMRetValue temp;\r\n1028     // skip first time call, to activate lazy compilation components.\r\n1029     pf.CallPacked(args, &temp);\r\n1030     DeviceAPI::Get(ctx)->StreamSync(ctx, nullptr);\r\n1031     // start timing\r\n1032     auto tbegin = std::chrono::high_resolution_clock::now();\r\n1033     for (int i = 0; i < nstep; ++i) {\r\n1034       pf.CallPacked(args, &temp);\r\n1035     }\r\n1036     DeviceAPI::Get(ctx)->StreamSync(ctx, nullptr);\r\n1037     auto tend = std::chrono::high_resolution_clock::now();\r\n1038     double speed = std::chrono::duration_cast<std::chrono::duration<double> >(\r\n1039         // tend - tbegin).count() / nstep;\r\n1040         tend - tbegin).count() / 1;\r\n1041     // return the time.\r\n1042     std::cout << \"all nstep == \" << nstep << std::endl;\r\n1043     *rv = speed;\r\n1044   };\r\n1045   return PackedFunc(ftimer);\r\n1046 }\r\n```\r\nthe log:\r\n('100 img total time == ', 2514.7600173950195)     <--- tvm run + getoutput time(ms)\r\nall nstep == 100\r\nProfileResult(mean=1.227199633)                           <--- _RPCTimeEvaluator total time(s)\r\nall nstep == 100\r\nProfileResult(mean=1.267423419)\r\nall nstep == 100\r\nProfileResult(mean=1.230426497)\r\nall nstep == 100\r\nProfileResult(mean=1.247696642)\r\nall nstep == 100\r\nProfileResult(mean=1.262426838)\r\nall nstep == 100\r\nProfileResult(mean=1.236824973)\r\nall nstep == 100\r\nProfileResult(mean=1.231287099)\r\nall nstep == 100\r\nProfileResult(mean=1.255299454)\r\nall nstep == 100\r\nProfileResult(mean=1.25883313)\r\nall nstep == 100\r\nProfileResult(mean=1.274763094)\r\n\r\ncomment line 118 the log:\r\n('100 img total time == ', 2099.6410846710205)      <--- tvm run time(ms)\r\nall nstep == 100\r\nProfileResult(mean=1.211560377)\r\nall nstep == 100\r\nProfileResult(mean=1.23746981)\r\nall nstep == 100\r\nProfileResult(mean=1.24266985)\r\nall nstep == 100\r\nProfileResult(mean=1.234779892)\r\nall nstep == 100\r\nProfileResult(mean=1.233300374)\r\nall nstep == 100\r\nProfileResult(mean=1.256389641)\r\nall nstep == 100\r\nProfileResult(mean=1.242223571)\r\nall nstep == 100\r\nProfileResult(mean=1.245141501)\r\nall nstep == 100\r\nProfileResult(mean=1.232106527)\r\nall nstep == 100\r\nProfileResult(mean=1.244905194)\r\n\r\nthe pure mxnet test code:\r\n```python\r\n 37 symbol = mobilenet_v1.get_symbol(1, use_global_stats=True)\r\n 38 print('symbol: ', symbol)\r\n 39\r\n 40 mod = mx.mod.Module(symbol, label_names=None, context=ctx)\r\n 41 data_shape = 512\r\n 42 mod.bind(data_shapes=[('data', (1, 3, data_shape, data_shape))], for_training=False)\r\n 43\r\n 44 mod.set_params(args, auxs)\r\n 45 Batch = namedtuple('Batch', ['data'])\r\n 46\r\n 47 nd_list = [mx.nd.array(x, dtype=np.float32)]\r\n 48\r\n 49 import time\r\n 50 t = time.time()\r\n 51 for i in range(100):\r\n 52     mod.forward(Batch(nd_list), is_train=False)\r\n 53     detections = mod.get_outputs()\r\n 54     for d in detections:\r\n 55         d = d.asnumpy()\r\n 56 print('mxnet 100 img time: ', 1000*(time.time() - t))\r\n```\r\nthe pure mxnet code log:\r\n('mxnet 100 img time: ', 1498.6538887023926)\r\n\r\n\r\nI have two questions\r\n1   Is there any problem with this test code?\r\n2   If the test code is no problem and how can I get the same total time in python code (tvm run + tvm getoutput time) as _RPCTimeEvaluator."}}
{"actor":{"display_login":"zt706"}}