title: Yocto Warrior Bitbake Recipe for PyTorch for NVIDIA Jetson Nano
body: <p>I am trying to create a simple Yocto Python Recipe for Python 3 PyTorch. The target is an SD Card Image for the NVIDIA Jetson Nano produced by Yocto from the meta-tegra layer. I can successfully compile and boot an image from meta-tegra without these recipes.</p>
  
  <p>NVIDIA themselves have compiled and released the ".whl" Python packages and they are found here: <a href="https://devtalk.nvidia.com/default/topic/1048776/official-tensorflow-for-jetson-nano-/" rel="noreferrer">https://devtalk.nvidia.com/default/topic/1048776/official-tensorflow-for-jetson-nano-/</a></p>
  
  <p>Using their build instructions, I tried to write a Bitbake recipe to install PyTorch as follows:</p>
  
  <pre><code>SUMMARY = "Facebook PyTorch AI"
  DESCRIPTION = "Facebook PyTorch AI"
  HOMEPAGE = "https://pytorch.org/"
  LICENSE = "MIT"
  LIC_FILES_CHKSUM = "file://LICENSE;md5=acf4d595f99e159bf31797aa872aef57"
  
  inherit pypi
  inherit setuptools3
  
  #BBCLASSEXTEND = "native nativesdk"
  
  DEPENDS += "python3-pytest-runner-native python3-pyyaml-native cmake-native"
  
  do_configure_prepend() {
    USE_NCCL=0
    USE_DISTRIBUTED=0
    TORCH_CUDA_ARCH_LIST="5.3;6.2;7.2"
  }
  
  do_compile_prepend() {
    USE_NCCL=0
    USE_DISTRIBUTED=0
    TORCH_CUDA_ARCH_LIST="5.3;6.2;7.2"
  }
  
  SRC_URI = "gitsm://github.com/pytorch/pytorch.git;protocol=https"
  SRCREV = "a3346e100e7f4e7ec90f18b7befcccc47d5a1c82"
  
  S = "${WORKDIR}/git"
  </code></pre>
  
  <p>The problem is that I hit the following error. I think it has to do with my environment variables not being picked up becuase the error includes "-DUSE_CUDA=False", however when I look at the bitbake environment ( bitbake -e ), it seems to be there. </p>
  
  <p>Here is the error:</p>
  
  <pre><code>**NOTE: Executing SetScene Tasks
  NOTE: Executing RunQueue Tasks
  ERROR: python3-torch-1.1.0-r0 do_compile: 'python3 setup.py build ' execution failed.
  ERROR: python3-torch-1.1.0-r0 do_compile: Function failed: do_compile (log file is located at /home/ubuntu/Desktop/comp-jetson-yocto/build/tmp/work/aarch64-poky-linux/python3-torch/1.1.0-r0/temp/log.do_compile.27625)
  ERROR: Logfile of failure stored in: /home/ubuntu/Desktop/comp-jetson-yocto/build/tmp/work/aarch64-poky-linux/python3-torch/1.1.0-r0/temp/log.do_compile.27625
  Log data follows:
  | DEBUG: Executing shell function do_compile
  | -- The CXX compiler identification is GNU 7.2.1
  | -- The C compiler identification is GNU 7.2.1
  | -- Check for working CXX compiler: /home/ubuntu/Desktop/comp-jetson-yocto/build/tmp/work/aarch64-poky-linux/python3-torch/1.1.0-r0/recipe-sysroot-native/usr/bin/aarch64-poky-linux/aarch64-poky-linux-g++
  | -- Check for working CXX compiler: /home/ubuntu/Desktop/comp-jetson-yocto/build/tmp/work/aarch64-poky-linux/python3-torch/1.1.0-r0/recipe-sysroot-native/usr/bin/aarch64-poky-linux/aarch64-poky-linux-g++ -- works
  | -- Detecting CXX compiler ABI info
  | -- Detecting CXX compiler ABI info - done
  | -- Detecting CXX compile features
  | -- Detecting CXX compile features - done
  | -- Check for working C compiler: /home/ubuntu/Desktop/comp-jetson-yocto/build/tmp/work/aarch64-poky-linux/python3-torch/1.1.0-r0/recipe-sysroot-native/usr/bin/aarch64-poky-linux/aarch64-poky-linux-gcc
  | -- Check for working C compiler: /home/ubuntu/Desktop/comp-jetson-yocto/build/tmp/work/aarch64-poky-linux/python3-torch/1.1.0-r0/recipe-sysroot-native/usr/bin/aarch64-poky-linux/aarch64-poky-linux-gcc -- works
  | -- Detecting C compiler ABI info
  | -- Detecting C compiler ABI info - done
  | -- Detecting C compile features
  | -- Detecting C compile features - done
  | -- Not forcing any particular BLAS to be found
  | -- Performing Test COMPILER_WORKS
  | -- Performing Test COMPILER_WORKS - Failed
  | CMake Error at cmake/MiscCheck.cmake:52 (message):
  |   Could not run a simple program built with your compiler.  If you are trying
  |   to use -fsanitize=address, make sure libasan is properly installed on your
  |   system (you can confirm if the problem is this by attempting to build and
  |   run a small program.)
  | Call Stack (most recent call first):
  |   CMakeLists.txt:271 (include)
  | 
  | 
  | -- Configuring incomplete, errors occurred!
  | See also "/home/ubuntu/Desktop/comp-jetson-yocto/build/tmp/work/aarch64-poky-linux/python3-torch/1.1.0-r0/git/build/CMakeFiles/CMakeOutput.log".
  | See also "/home/ubuntu/Desktop/comp-jetson-yocto/build/tmp/work/aarch64-poky-linux/python3-torch/1.1.0-r0/git/build/CMakeFiles/CMakeError.log".
  | Building wheel torch-1.2.0a0+a3346e1
  | -- Building version 1.2.0a0+a3346e1
  | cmake -DBUILD_AR=ar -DBUILD_AS=as  -DBUILD_CC=gcc  -DBUILD_CCLD=gcc  -DBUILD_CFLAGS=-isystem/home/ubuntu/Desktop/comp-jetson-yocto/build/tmp/work/aarch64-poky-linux/python3-torch/1.1.0-r0/recipe-sysroot-native/usr/include -O2 -pipe -DBUILD_CPP=gcc  -E -DBUILD_CPPFLAGS=-isystem/home/ubuntu/Desktop/comp-jetson-yocto/build/tmp/work/aarch64-poky-linux/python3-torch/1.1.0-r0/recipe-sysroot-native/usr/include -DBUILD_CXX=g++  -DBUILD_CXXFLAGS=-isystem/home/ubuntu/Desktop/comp-jetson-yocto/build/tmp/work/aarch64-poky-linux/python3-torch/1.1.0-r0/recipe-sysroot-native/usr/include -O2 -pipe -DBUILD_FC=gfortran  -DBUILD_LD=ld  -DBUILD_LDFLAGS=-L/home/ubuntu/Desktop/comp-jetson-yocto/build/tmp/work/aarch64-poky-linux/python3-torch/1.1.0-r0/recipe-sysroot-native/usr/lib                         -L/home/ubuntu/Desktop/comp-jetson-yocto/build/tmp/work/aarch64-poky-linux/python3-torch/1.1.0-r0/recipe-sysroot-native/lib                         -Wl,--enable-new-dtags                         -Wl,-rpath-link,/home/ubuntu/Desktop/comp-jetson-yocto/build/tmp/work/aarch64-poky-linux/python3-torch/1.1.0-r0/recipe-sysroot-native/usr/lib                         -Wl,-rpath-link,/home/ubuntu/Desktop/comp-jetson-yocto/build/tmp/work/aarch64-poky-linux/python3-torch/1.1.0-r0/recipe-sysroot-native/lib                         -Wl,-rpath,/home/ubuntu/Desktop/comp-jetson-yocto/build/tmp/work/aarch64-poky-linux/python3-torch/1.1.0-r0/recipe-sysroot-native/usr/lib                         -Wl,-rpath,/home/ubuntu/Desktop/comp-jetson-yocto/build/tmp/work/aarch64-poky-linux/python3-torch/1.1.0-r0/recipe-sysroot-native/lib                         -Wl,-O1 -Wl,--allow-shlib-undefined -Wl,--dynamic-linker=/home/ubuntu/Desktop/comp-jetson-yocto/build/tmp/sysroots-uninative/x86_64-linux/lib/ld-linux-x86-64.so.2 -DBUILD_NM=nm -DBUILD_PYTHON=True -DBUILD_RANLIB=ranlib -DBUILD_STRIP=strip -DBUILD_TEST=True -DCMAKE_BUILD_TYPE=Release -DCMAKE_CXX_FLAGS= -O2 -pipe -g -feliminate-unused-debug-types -fdebug-prefix-map=/home/ubuntu/Desktop/comp-jetson-yocto/build/tmp/work/aarch64-poky-linux/python3-torch/1.1.0-r0=/usr/src/debug/python3-torch/1.1.0-r0                      -fdebug-prefix-map=/home/ubuntu/Desktop/comp-jetson-yocto/build/tmp/work/aarch64-poky-linux/python3-torch/1.1.0-r0/recipe-sysroot=                      -fdebug-prefix-map=/home/ubuntu/Desktop/comp-jetson-yocto/build/tmp/work/aarch64-poky-linux/python3-torch/1.1.0-r0/recipe-sysroot-native=   -DCMAKE_C_FLAGS= -O2 -pipe -g -feliminate-unused-debug-types -fdebug-prefix-map=/home/ubuntu/Desktop/comp-jetson-yocto/build/tmp/work/aarch64-poky-linux/python3-torch/1.1.0-r0=/usr/src/debug/python3-torch/1.1.0-r0                      -fdebug-prefix-map=/home/ubuntu/Desktop/comp-jetson-yocto/build/tmp/work/aarch64-poky-linux/python3-torch/1.1.0-r0/recipe-sysroot=                      -fdebug-prefix-map=/home/ubuntu/Desktop/comp-jetson-yocto/build/tmp/work/aarch64-poky-linux/python3-torch/1.1.0-r0/recipe-sysroot-native=   -DCMAKE_EXE_LINKER_FLAGS=-Wl,-O1 -Wl,--hash-style=gnu -Wl,--as-needed -fstack-protector-strong -Wl,-z,relro,-z,now -DCMAKE_INSTALL_PREFIX=/home/ubuntu/Desktop/comp-jetson-yocto/build/tmp/work/aarch64-poky-linux/python3-torch/1.1.0-r0/git/torch -DCMAKE_PREFIX_PATH=/home/ubuntu/Desktop/comp-jetson-yocto/build/tmp/work/aarch64-poky-linux/python3-torch/1.1.0-r0/recipe-sysroot/usr/lib/python3.7/site-packages -DCMAKE_SHARED_LINKER_FLAGS=-Wl,-O1 -Wl,--hash-style=gnu -Wl,--as-needed -fstack-protector-strong -Wl,-z,relro,-z,now -DINSTALL_TEST=True -DPYTHON_EXECUTABLE=/home/ubuntu/Desktop/comp-jetson-yocto/build/tmp/work/aarch64-poky-linux/python3-torch/1.1.0-r0/recipe-sysroot-native/usr/bin/python3-native/python3 -DPYTHON_INCLUDE_DIR=/home/ubuntu/Desktop/comp-jetson-yocto/build/tmp/work/aarch64-poky-linux/python3-torch/1.1.0-r0/recipe-sysroot/usr/include/python3.7m -DPYTHON_LIBRARY=/home/ubuntu/Desktop/comp-jetson-yocto/build/tmp/work/aarch64-poky-linux/python3-torch/1.1.0-r0/recipe-sysroot/usr/lib/libpython3.7m.so.1.0 -DTORCH_BUILD_VERSION=1.2.0a0+a3346e1 -DUSE_CUDA=False -DUSE_DISTRIBUTED=True -DUSE_FBGEMM=True -DUSE_NCCL=False -DUSE_NUMPY=True -DUSE_QNNPACK=True -DUSE_SYSTEM_EIGEN_INSTALL=OFF -DUSE_SYSTEM_NCCL=False /home/ubuntu/Desktop/comp-jetson-yocto/build/tmp/work/aarch64-poky-linux/python3-torch/1.1.0-r0/git
  | Traceback (most recent call last):
  |   File "setup.py", line 749, in &lt;module&gt;
  |     build_deps()
  |   File "setup.py", line 323, in build_deps
  |     cmake=cmake)
  |   File "/home/ubuntu/Desktop/comp-jetson-yocto/build/tmp/work/aarch64-poky-linux/python3-torch/1.1.0-r0/git/tools/build_pytorch_libs.py", line 61, in build_caffe2
  |     rerun_cmake)
  |   File "/home/ubuntu/Desktop/comp-jetson-yocto/build/tmp/work/aarch64-poky-linux/python3-torch/1.1.0-r0/git/tools/setup_helpers/cmake.py", line 318, in generate
  |     self.run(args, env=my_env)
  |   File "/home/ubuntu/Desktop/comp-jetson-yocto/build/tmp/work/aarch64-poky-linux/python3-torch/1.1.0-r0/git/tools/setup_helpers/cmake.py", line 93, in run
  |     check_call(command, cwd=self.build_dir, env=env)
  |   File "/home/ubuntu/Desktop/comp-jetson-yocto/build/tmp/work/aarch64-poky-linux/python3-torch/1.1.0-r0/recipe-sysroot-native/usr/lib/python3.7/subprocess.py", line 347, in check_call
  |     raise CalledProcessError(retcode, cmd)
  | subprocess.CalledProcessError: Command '['cmake', '-DBUILD_AR=ar', '-DBUILD_AS=as ', '-DBUILD_CC=gcc ', '-DBUILD_CCLD=gcc ', '-DBUILD_CFLAGS=-isystem/home/ubuntu/Desktop/comp-jetson-yocto/build/tmp/work/aarch64-poky-linux/python3-torch/1.1.0-r0/recipe-sysroot-native/usr/include -O2 -pipe', '-DBUILD_CPP=gcc  -E', '-DBUILD_CPPFLAGS=-isystem/home/ubuntu/Desktop/comp-jetson-yocto/build/tmp/work/aarch64-poky-linux/python3-torch/1.1.0-r0/recipe-sysroot-native/usr/include', '-DBUILD_CXX=g++ ', '-DBUILD_CXXFLAGS=-isystem/home/ubuntu/Desktop/comp-jetson-yocto/build/tmp/work/aarch64-poky-linux/python3-torch/1.1.0-r0/recipe-sysroot-native/usr/include -O2 -pipe', '-DBUILD_FC=gfortran ', '-DBUILD_LD=ld ', '-DBUILD_LDFLAGS=-L/home/ubuntu/Desktop/comp-jetson-yocto/build/tmp/work/aarch64-poky-linux/python3-torch/1.1.0-r0/recipe-sysroot-native/usr/lib                         -L/home/ubuntu/Desktop/comp-jetson-yocto/build/tmp/work/aarch64-poky-linux/python3-torch/1.1.0-r0/recipe-sysroot-native/lib                         -Wl,--enable-new-dtags                         -Wl,-rpath-link,/home/ubuntu/Desktop/comp-jetson-yocto/build/tmp/work/aarch64-poky-linux/python3-torch/1.1.0-r0/recipe-sysroot-native/usr/lib                         -Wl,-rpath-link,/home/ubuntu/Desktop/comp-jetson-yocto/build/tmp/work/aarch64-poky-linux/python3-torch/1.1.0-r0/recipe-sysroot-native/lib                         -Wl,-rpath,/home/ubuntu/Desktop/comp-jetson-yocto/build/tmp/work/aarch64-poky-linux/python3-torch/1.1.0-r0/recipe-sysroot-native/usr/lib                         -Wl,-rpath,/home/ubuntu/Desktop/comp-jetson-yocto/build/tmp/work/aarch64-poky-linux/python3-torch/1.1.0-r0/recipe-sysroot-native/lib                         -Wl,-O1 -Wl,--allow-shlib-undefined -Wl,--dynamic-linker=/home/ubuntu/Desktop/comp-jetson-yocto/build/tmp/sysroots-uninative/x86_64-linux/lib/ld-linux-x86-64.so.2', '-DBUILD_NM=nm', '-DBUILD_PYTHON=True', '-DBUILD_RANLIB=ranlib', '-DBUILD_STRIP=strip', '-DBUILD_TEST=True', '-DCMAKE_BUILD_TYPE=Release', '-DCMAKE_CXX_FLAGS= -O2 -pipe -g -feliminate-unused-debug-types -fdebug-prefix-map=/home/ubuntu/Desktop/comp-jetson-yocto/build/tmp/work/aarch64-poky-linux/python3-torch/1.1.0-r0=/usr/src/debug/python3-torch/1.1.0-r0                      -fdebug-prefix-map=/home/ubuntu/Desktop/comp-jetson-yocto/build/tmp/work/aarch64-poky-linux/python3-torch/1.1.0-r0/recipe-sysroot=                      -fdebug-prefix-map=/home/ubuntu/Desktop/comp-jetson-yocto/build/tmp/work/aarch64-poky-linux/python3-torch/1.1.0-r0/recipe-sysroot-native=  ', '-DCMAKE_C_FLAGS= -O2 -pipe -g -feliminate-unused-debug-types -fdebug-prefix-map=/home/ubuntu/Desktop/comp-jetson-yocto/build/tmp/work/aarch64-poky-linux/python3-torch/1.1.0-r0=/usr/src/debug/python3-torch/1.1.0-r0                      -fdebug-prefix-map=/home/ubuntu/Desktop/comp-jetson-yocto/build/tmp/work/aarch64-poky-linux/python3-torch/1.1.0-r0/recipe-sysroot=                      -fdebug-prefix-map=/home/ubuntu/Desktop/comp-jetson-yocto/build/tmp/work/aarch64-poky-linux/python3-torch/1.1.0-r0/recipe-sysroot-native=  ', '-DCMAKE_EXE_LINKER_FLAGS=-Wl,-O1 -Wl,--hash-style=gnu -Wl,--as-needed -fstack-protector-strong -Wl,-z,relro,-z,now', '-DCMAKE_INSTALL_PREFIX=/home/ubuntu/Desktop/comp-jetson-yocto/build/tmp/work/aarch64-poky-linux/python3-torch/1.1.0-r0/git/torch', '-DCMAKE_PREFIX_PATH=/home/ubuntu/Desktop/comp-jetson-yocto/build/tmp/work/aarch64-poky-linux/python3-torch/1.1.0-r0/recipe-sysroot/usr/lib/python3.7/site-packages', '-DCMAKE_SHARED_LINKER_FLAGS=-Wl,-O1 -Wl,--hash-style=gnu -Wl,--as-needed -fstack-protector-strong -Wl,-z,relro,-z,now', '-DINSTALL_TEST=True', '-DPYTHON_EXECUTABLE=/home/ubuntu/Desktop/comp-jetson-yocto/build/tmp/work/aarch64-poky-linux/python3-torch/1.1.0-r0/recipe-sysroot-native/usr/bin/python3-native/python3', '-DPYTHON_INCLUDE_DIR=/home/ubuntu/Desktop/comp-jetson-yocto/build/tmp/work/aarch64-poky-linux/python3-torch/1.1.0-r0/recipe-sysroot/usr/include/python3.7m', '-DPYTHON_LIBRARY=/home/ubuntu/Desktop/comp-jetson-yocto/build/tmp/work/aarch64-poky-linux/python3-torch/1.1.0-r0/recipe-sysroot/usr/lib/libpython3.7m.so.1.0', '-DTORCH_BUILD_VERSION=1.2.0a0+a3346e1', '-DUSE_CUDA=False', '-DUSE_DISTRIBUTED=True', '-DUSE_FBGEMM=True', '-DUSE_NCCL=False', '-DUSE_NUMPY=True', '-DUSE_QNNPACK=True', '-DUSE_SYSTEM_EIGEN_INSTALL=OFF', '-DUSE_SYSTEM_NCCL=False', '/home/ubuntu/Desktop/comp-jetson-yocto/build/tmp/work/aarch64-poky-linux/python3-torch/1.1.0-r0/git']' returned non-zero exit status 1.
  | ERROR: 'python3 setup.py build ' execution failed.
  | WARNING: exit code 1 from a shell command.
  | ERROR: Function failed: do_compile (log file is located at /home/ubuntu/Desktop/comp-jetson-yocto/build/tmp/work/aarch64-poky-linux/python3-torch/1.1.0-r0/temp/log.do_compile.27625)
  ERROR: Task (/home/ubuntu/Desktop/comp-jetson-yocto/layers/meta-comp/recipes-python/python3-torch/python3-torch_1.1.0.bb:do_compile) failed with exit code '1'
  NOTE: Tasks Summary: Attempted 1770 tasks of which 1763 didn't need to be rerun and 1 failed.
  
  Summary: 1 task failed:
    /home/ubuntu/Desktop/comp-jetson-yocto/layers/meta-comp/recipes-python/python3-torch/python3-torch_1.1.0.bb:do_compile
  Summary: There was 1 WARNING message shown.
  Summary: There were 2 ERROR messages shown, returning a non-zero exit code.**
  </code></pre>