Frame Level Parallelism

Definition: Frame-level parallelism means a set of tools of processing multiple frames at the same time.

If all frames are I-frames then they can be processed at the same time (provided that the frames are available) due to lack of temporal dependencies. In general case due to temporal dependencies between frames the processing of some frames are lagged.

We describe two shemas of the frame-level parallelism: slice-based and tile-based.


Slice-based Picture Level Parallelism

The first thread starts the k-th frame, the second thread waits until a several mb-rows of the k-th frame have been completed. Then the second thread commences encoding of (k+1)th frame, search area is already available.











Disadvantage of slice-based parallelism is that vertical motion estimation is restricted. 

Use case: x265

To enable frame-level parallelism you need disable WPP (use ‘--no-wpp’) and apply ‘--frame-threads’ (no co-existence of WPP and frame-threads) by setting ‘--frame-threads N’, where N is the number of threads. 

 Example (2 concurrently encoded frames):

x265 --input a.yuv --input-res 3840x1744 --fps 24   --b-adapt 0  -b 0 --ref 1  --frame-threads 2     --no-wpp    --rc-lookahead 2 -o test.h265


Tile-based Picture Level Parallelism

Each picture is divided in same grid of tiles, each tile is self-contained to enable parallel processing (at each stage a tile is assigned to its own thread). The first thread completes several top-left tiles of frame 0 and then the second thread starts the frame 1 with motion search area resting on already processed tiles of the frame 0 and so on:






the picture is taken from the white paper:

"Scalable Video Technology for the Visual Cloud"













More formally, at the first stage the first thread processes top-left 2x2 tiling grid of the frame 0:


















At the second stage the first thread takes next layer of tiles of the frame 0 while the second thread starts processing the tile 0 (i.e. the top-left tile) of the frame 1, using already processed tiles from the frame 0 as reference:



















Unlike to Slice-level parallelism, the search area is square and vertical motion estimation is not restricted.