title: How to make the most of SIMD in OpenCL?
body: <p>In the <a href="http://www.freedesktop.org/wiki/Software/Beignet/optimization-guide/" rel="noreferrer">optimization guide of Beignet, an open source implementation of OpenCL targeting Intel GPUs</a></p>
  <blockquote>
  <p><strong>Work group Size should be larger than 16 and be multiple of 16.</strong></p>
  <p>As two possible SIMD lanes on Gen are 8 or 16. To not waste SIMD
  lanes, we need to follow this rule.</p>
  </blockquote>
  <p>Also mentioned in the <a href="https://software.intel.com/en-us/file/compute-architecture-of-intel-processor-graphics-gen7dot5-aug4-2014pdf" rel="noreferrer">Compute Architecture of Intel Processor Graphics Gen7.5</a>:</p>
  <blockquote>
  <p>For Gen7.5 based products, <strong>each EU has seven threads</strong> for a total of 28 Kbytes of general purpose register file (GRF).</p>
  <p>...</p>
  <p>On Gen7.5 compute architecture, most SPMD programming models employ
  this style code generation and EU processor execution. Effectively,
  each <strong>SPMD kernel instance appears to execute serially and independently within its own SIMD lane</strong>.</p>
  <p>In actuality, each thread executes a SIMD-Width number of kernel instances &gt;concurrently. <strong>Thus for a SIMD-16 compile of a compute</strong>
  <strong>kernel, it is possible for SIMD-16 x 7 threads = 112 kernel instances</strong>
  <strong>to be executing concurrently on a single EU.</strong> Similarly, for SIMD-32 x
  7 threads = 224 kernel instances executing concurrently on a single
  EU.</p>
  </blockquote>
  <p>If I understand it correctly, using the <code>SIMD-16 x 7 threads = 112 kernel instances</code> as a example, in order to run 224 threads on one EU, the work group size need to be 16. Then the OpenCL compiler will fold 16 kernel instances into a 16 lane SIMD thread, and do this 7 times on 7 work groups, and run them on a single EU?</p>
  <p><strong>Question 1:</strong> am I correct until here?</p>
  <p>However <a href="https://www.khronos.org/registry/cl/specs/opencl-1.1.pdf#page=161" rel="noreferrer">OpenCL spec</a> also provide vector data types. So it's feasible to make full use of the SIMD-16 computing resources in a EU by conventional SIMD programming(as in NEON and SSE).</p>
  <p><strong>Question 2:</strong> If this is the case, using vector-16 data type already makes explicit use of the SIMD-16 resources, hence removes the at-least-16-item-per-work-group restrictions. Is this the case?</p>
  <p><strong>Question 3:</strong> If all above are true, then how does the two approach compare with each other: <strong>1)</strong> 112 threads fold into 7 SIMD-16 threads by OpenCL compiler; <strong>2)</strong> 7 native threads coded to explicitly use vector-16 data types and SIMD-16 operations?</p>