cd /home/janfrode/code/NPB3.3.1/NPB3.3-OMP for O in O2 O3 O4 O5 ; do for i in bt cg ep ft is lu mg sp ua ; do echo ========= ./${O}/${i}.C.x ============ jsrun -n 1 -a 1 -c 22 -r 1 -d cyclic -b rs ./${O}/${i}.C.x done done $ bsub < jobscript Job <195755> is submitted to queue <excl>.  Note:  In several cases, performance results that are presented deviate significantly from  the general behavior within the same plot. The plot bars with deviations can be ignored  because the tests can experience unusual conditions (OS jitters, parasitic external  workload, and so on). The POWER9 processor-based core can run instructions from up to four application threads  simultaneously. This capability is known as  SMT . The POWER9 processor-based architecture  supports the following three multithreading levels: The SMT mode, which can be used to obtain the optimal performance, depends on the  characteristics of the application. Compilation parameters and mapping between application  threads and logical processors can also affect the timing. Execution units of a core are shared by all logical processors of a core (two logical processors  in SMT-2 mode and four logical processors in SMT-4 mode). Execution units are available with  multiple instances (for example, load and store units and fixed-point units) and single  instances (for example, branch execution unit). In ideal conditions, application threads do not compete for execution units. This configuration  results in each of the four logical processors of a core that is running in SMT-4 mode being  almost as fast as a core that is running in ST mode. Depending on the application threads instruction flow, some execution units become fully  saturated with instructions that come from different threads. As a result, the progress of the  dependent instructions is postponed. This postponement limits the overall performance of the  four logical processors of a core that is running in SMT-4 to the performance of a core that is  running in ST mode. It is also possible for even a single thread to saturate fully the resources of a whole core.  Therefore, adding threads to a core can result in performance degradation. For example, see  the performance of the  mg.C  benchmark that is shown in Figure 9-9 on page 268. 262  IBM High-Performance Computing Insights with IBM Power System AC922 Clustered Solution  2 Single-thread mode is referred sometimes as SMT-1. The reasons behind a conscious choice of an SMT mode GLYPH<SM590000> ST (single-thread) 2 GLYPH<SM590000> SMT-2 (two-way multithreading) GLYPH<SM590000> SMT-4 (four-way multithreading) 9.1.1 Performance effect of a rational choice of SMT mode