{'fraction_non_alphanumeric': 0.023796347537354733, 'fraction_numerical': 0.00387382401770891, 'mean_word_length': 5.191780821917808, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Evaluating Effect of Write Combining on PCIe Throughput to Improve HPC Interconnect Performance

HPC interconnect is a very crucial component of any HPC machine. Interconnect performance is one of the contributing factors for overall performance of HPC system. Most popular interface to connect Network Interface Card (NIC) to CPU is PCI express (PCIe). With denser core counts in compute servers and increasingly maturing fabric interconnect speeds, there is need to maximize the packet data movement throughput between system memory and fabric interface network buffers, such that the rate at which applications (CPU) generating data is matched with rate at which fabric consumes the same. Thus PCIe throughput for small and medium messages (Programmed Input/Output) needs to be improved, to synchronize with core processing rate and fabric speeds. And there is scope for this improvement by increasing the payload size in Transaction Layer Packet (TLP) of PCIe. Traditionally, CPU issues memory writes in 8 bytes (payload for TLP), underutilizing the PCIe bus since overhead compared to payload is more. Write combining can increase the payload size in TLP, leading to more efficient utilization of available bus bandwidth, thereby improving the overall throughput.This work evaluates the performance that could be gained by using Write Combine Buffers (WCB) available on Intel CPU, for send side interface of HPC interconnect. These buffers are used for aggregating the small (usually 8 bytes) memory mapped I/O stores, to form the bigger PCIe Transaction Level Packets (TLP), which leads to better bus bandwidth utilization. It is observed that, this technique improves peak PIO bandwidth by 2x compared to normal PIO. It is also observed that till 4096 bytes write combine enabled PIO outperforms DMA.