{'fraction_non_alphanumeric': 0.03499794569155493, 'fraction_numerical': 0.011336047510551676, 'mean_word_length': 4.632968651378077, 'pattern_counts': {'":': 0, '<': 1, '<?xml version=': 0, '>': 1, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Reliability-aware Garbage Collection for Hybrid HBM-DRAM Memories

Emerging workloads in cloud and data center infrastructures demand high main memory bandwidth and capacity. Unfortunately, DRAM alone is unable to satisfy contemporary main memory demands. High-bandwidth memory (HBM) uses 3D die-stacking to deliver 4–8× higher bandwidth. HBM has two drawbacks: (1) capacity is low, and (2) soft error rate is high. Hybrid memory combines DRAM and HBM to promise low fault rates, high bandwidth, and high capacity. Prior OS approaches manage HBM by mapping pages to HBM versus DRAM based on hotness (access frequency) and risk (susceptibility to soft errors). Unfortunately, these approaches operate at a coarse-grained page granularity, and frequent page migrations hurt performance. This article proposes a new class of reliability-aware garbage collectors for hybrid HBM-DRAM systems that place hot and low-risk objects in HBM and the rest in DRAM. Our analysis of nine real-world Java workloads shows that: (1) newly allocated objects in the nursery are frequently written, making them both hot and low-risk, (2) a small fraction of the mature objects are hot and low-risk, and (3) allocation site is a good predictor for hotness and risk. We propose RiskRelief, a novel reliability-aware garbage collector that uses allocation site prediction to place hot and low-risk objects in HBM. Allocation sites are profiled offline and RiskRelief uses heuristics to classify allocation sites as DRAM and HBM. The proposed heuristics expose Pareto-optimal trade-offs between soft error rate (SER) and execution time. RiskRelief improves SER by 9× compared to an HBM-Only system while at the same time improving performance by 29% compared to a DRAM-Only system. Compared to a state-of-the-art OS approach for reliability-aware data placement, RiskRelief eliminates all page migration overheads, which substantially improves performance while delivering similar SER. Reliability-aware garbage collection opens up a new opportunity to manage emerging HBM-DRAM memories at fine granularity while requiring no extra hardware support and leaving the programming model unchanged.

remaining objects are placed in DRAM to utilize its large capacity. Reliability-aware garbage collection overcomes the disadvantages of the state-of-the-art OS approach. Specifically, prediction enables pro-active allocation of objects in HBM as opposed to reactive page migrations. Moreover, placing objects using GC eliminates the overhead of costly page migrations.
In this article, we propose two reliability-aware garbage collectors. RiskRelief-Nursery (RR-N) places the nursery in HBM and the mature space in DRAM. It requires minimal changes to the Java runtime but is highly effective in delivering low soft error rates compared to an HBM-only system, while improving performance compared to a DRAM-only system. RiskRelief-Mature (RR-M) places the nursery in HBM and exploits offline program profiling to place hot and low-risk nursery survivors in HBM. We show that mature object hotness and risk are predictable on a per allocation-site basis. Surprisingly perhaps, we find that object hotness and risk are weakly correlated. Hence, placing objects in HBM based solely on hotness significantly hurts reliability. The insight is to place objects in HBM versus DRAM based on hotness and risk.
Based on these observations, we propose heuristics to classify allocation sites as DRAM and HBM. Allocation sites are classified as HBM if most objects they allocate are hot and low-risk. All other allocation sites default as DRAM. We generate this per allocation-site advice offline and feed it to RR-M. In turn, RR-M uses the advice during runtime to place nursery survivors in HBM or DRAM. Our proposed heuristics expose previously unseen Pareto-optimal trade-offs between execution time and soft error rate. A single profiling run generates a range of advices for the GC runtime. Thus, depending upon factors such as environmental conditions, available HBM capacity and performance goals, a system operator can adjust the advice fed to RR-M to meet specific constraints.
Our experimental results show that RR-N reduces the overall soft error rate by 18× on average compared to an HBM-only system, while improving performance over a homogeneous DRAMonly system by 20%. The state-of-the-art OS solution by Gupta et al. [27] achieves similar SER as RR-N, however, performance is substantially worse (even worse than the DRAM-only system) because of the high cost of TLB shootdowns on modern x86 multicores [51]. Both RR-N and the prior OS approach use a modest 128 MB of HBM on a 32-core platform. RR-M uses an additional 18% of HBM capacity but delivers 29% higher performance compared to a DRAM-only system. Higher HBM capacity impacts overall SER and RR-M reduces SER by 9× over HBM-only.
In summary, the main contributions of this article are: • hotness (access frequency) and risk (susceptibility to soft errors) characterization of objects in Java applications, showing that hotness and risk are only weakly correlated; • showing that allocation site is a good predictor for object hotness and risk; • the design and implementation of reliability-aware garbage collection for hybrid HBM-DRAM memories to minimize soft error rate while maximizing overall application performance-in contrast, performance-optimized HBM-DRAM management significantly hurts reliability; • profile-driven RiskRelief reliability-aware collectors that exploit allocation-site prediction to place hot and low-risk objects in HBM and the rest in DRAM; • a profiling framework to measure object hotness and risk on a per allocation-site basis; two heuristics to generate the allocation advice for GC; and a compilation framework that exploits the advice to steer allocation of objects in HBM and DRAM. • simulation and real hardware emulation results motivating hybrid HBM-DRAM memory for Java applications, and showing that RiskRelief collectors manage hybrid HBM-DRAM memory significantly better than state-of-the-art OS approaches.

EXPLOITING HIGH-BANDWIDTH MEMORY
In this section, we discuss the motivation for HBM, and we describe its distinct performance and reliability characteristics. We also review existing approaches to manage HBM.

3D-Stacked Memory
Disruptive approaches to mitigate the memory bandwidth wall are needed [56]. The bandwidth between conventional DRAM and the processor is limited by pin count, which increases by roughly 10% every year [33]. However, compute power grows much more rapidly. Furthermore, having enough pins to stream a 1024-bit word every cycle to the processor would require 40 Watt just for memory I/O [48]. High-bandwidth memory vertically stacks DRAM chips in a 3D arrangement to deliver higher bandwidth than conventional DRAM. Through-silicon vias (TSVs) interconnect the vertically stacked chips using wide communication lanes. Conventional DRAM technology, e.g., DDR4, places two 64-bit words on the data bus every cycle. Several DRAM chips work in tandem to produce the word. For example, 16 ×4 chips each provide 4 bits every cycle to render a 64-bit word. In contrast, the state-of-the-art HBM standard allows up to 12 dies per stack, and each stack has 8 unique 128-bit channels per stack, leading to a much wider, 1,024-bit memory interface [35]. Internally, each DRAM chip consists of many banks. A 64-byte cache line is striped across banks in different DRAM chips to maximize parallelism. Hardware employs error correction codes (ECC) to shield against soft errors. Typically, an additional chip provides ECC protection to the data word. Most commonly, DRAM employs single-error correcting, double-error detecting (SECDED) codes.
HBM inherits the failure modes of conventional DRAM, because it uses a similar cell technology and array layout. Unfortunately, new failure modes exist in HBM, for instance, due to TSV failures [34]. HBM also exhibits higher bit density increasing susceptibility to soft errors [7,27,34,36]. Furthermore, HBM employs weaker error correction due to cost and complexity constraints [27,36]. Put together, HBM reliability is a major concern that necessitates hardware and software approaches to mitigate the vulnerability to soft errors in HBM and improve the overall reliability of the memory system.

Managing HBM in Hardware
Exploiting HBM as a last-level DRAM cache is predominant. In particular, prior work proposes new organizations for DRAM caches [38], intelligent tag placement (for example, co-locating tags with data) [28,41], new techniques to reduce the bandwidth consumed by cache operations [18,19], and techniques to enable set associativity in giga-scale DRAM caches [71]. Prior work also attempts to mitigate the performance overhead of DRAM caches for capacity-limited applications [17]. Although transparent to the software stack, DRAM caches have two drawbacks: (1) they limit the available memory capacity, and (2) they require extensive hardware support, because conventional SRAM-based cache organizations are suboptimal for DRAM technology. Moreover, none of this prior work considers the low reliability of HBM, thus rendering program data in HBM highly vulnerable to transient faults. Liu et al. [44] propose Binary Star that coordinates the reliability schemes in the 3D DRAM LLC versus main memory to improve the reliability of the overall memory hierarchy. Binary Star achieves high reliability for the overall memory system with limited performance loss, while requiring modifications to both system software and hardware. RiskRelief does not require any hardware changes.
ECC codes are the first line of defense against transient faults. DRAM scaling relies on ECC hardware, because smaller DRAM cells are more susceptible to soft errors. Several works study DRAM soft error rates in the field [59,63,64]. Weaker ECC backs die-stacked memory due to implementation costs [36] and thus requires soft error mitigation from other sources, e.g., through software, as we discuss next.

Managing HBM in the OS
Existing OS approaches to manage hybrid HBM-DRAM memory aim at either maximizing performance or balancing performance and reliability. Performance-focused approaches hurt reliability [55], because they place all hot pages in HBM while being agnostic to soft error vulnerability. Gupta et al. [27] propose a dynamic page migration scheme that estimates page hotness and risk using performance counters and that migrates (every 100 ms) cold and high-risk pages to DRAM, and hot and low-risk pages to HBM. In contrast, we estimate hotness and risk at a much finer granularity of objects. Our solution pro-actively places objects in HBM versus DRAM, and does not require dynamic monitoring nor additional performance counter hardware. We compare to the OS page migration approach in this work.
Oskin and Loh [51] propose OS-managed DRAM caches. Their work shows the high cost of page migrations due to TLB shootdowns. They also explore statically partitioning program data in C applications in DRAM and HBM, albeit with negligible benefits. Their proposal does not consider the heterogeneity in reliability in a hybrid HBM-DRAM memory system. We expose both DRAM and HBM to the OS to exploit full memory capacity. Furthermore, this is the first work to expose 3D-stacked memory to garbage collection in the managed runtime for fine-grained object placement.

BACKGROUND
Before describing how RiskRelief predicts hotness and risk and leverages these predictions to manage hybrid HBM-DRAM systems, we first provide additional background in soft error reliability and managed runtimes.

Soft Error Reliability
RiskRelief builds upon two notions, namely, hotness and risk. Intuitively, hotness refers to how frequently an object is accessed, whereas risk refers to how susceptible an object is to soft errors. We now define both concepts and focus on risk more, because it is a less well-known metric.
Hotness. Hotness is a well-known concept and typically refers to how frequently a particular code segment executes. Analogously, we define the hotness of an object as to how frequently the object is accessed through read or write operations. We define an object's hotness as the sum of reads and writes to the object. Our analysis shows that of all accesses to objects, 54% of the accesses on average are reads, and 46% are writes. The high percentage of writes motivates our hotness criteria as the sum of reads and writes.

Risk.
Quantifying the risk of an object in HBM is more involved. We build upon the mechanistic notion of architectural vulnerability factor (AVF) to quantify susceptibility to soft errors. AVF is the probability that a transient fault leads to an observable program error. To compute AVF, Mukherjee et al. [57] categorize all bits in a hardware structure into two types: (1) those necessary for architecturally correct execution (ACE), and (2) the remaining un-ACE bits. A fault in the ACE bits results in an observable program error (assuming the fault evades ECC hardware), and a fault in un-ACE bits has no bearing on program correctness. A bit can be ACE for only a fraction of the total execution time. The AVF of a hardware structure is the fraction of all bits that are in ACE state during each cycle.
Precisely computing AVF of an object requires tracking every read and write operation. Consider an object O, stored at memory location M, is written at time t 1 and read at times t 2 and t 3 , after which O is dead from the program's point of view (i.e., no other memory location points to O). O is ACE for t 3 − t 1 time units, namely, between the write at t 1 and its last read at t 3 . In case the object would have been written at times t 2 and t 3 , the object would be un-ACE throughout. In other words, to precisely compute the AVF of an object, one needs to track all reads and all writes, which is too high overhead to do online in the context of a managed runtime.
Instead, we build upon prior work [27] and use proxy metrics that are easier to collect while correlating well with AVF. The proxies considered are the writes to reads ratio (W r /R d ) and the writes-squared to reads ratio (W r 2 /R d ). The intuition behind these proxies is that an object that is written a lot is more likely to lead to more un-ACE periods. We use the writes-squared to reads ratio in this work, because it places extra emphasis on the absolute number of accesses [27]. Since writes-squared to reads ratio is inversely proportional to AVF, we refer to it as AVF-X. In other words, a high writes-squared to reads ratio (high AVF-X) means low risk, and vice versa. SER is defined as the product of a device's failure-in-time (FIT-Rate) and AVF. FIT-Rate is defined as the raw failure rate due to single event faults, and depends on environmental factors and circuit characteristics.

Managed Runtimes
Java Virtual Machine. This work uses the language runtime to improve system reliability in hybrid memory systems. Our work generalizes to languages with garbage collection, but we use the Java Virtual Machine (JVM) in this work. Exposing HBM to the JVM entails extending the OS NUMA interface [3]. We use the open-source Jikes Research VM (RVM) as our platform. Jikes RVM's modular design makes it easy to modify [5,6,12,25]. Jikes RVM is a meta-circular VM written in Java. It has both a baseline and an optimizing compiler, along with several garbage collectors [11,14,60]. The object layout and metadata, and a variety of reference barriers can be changed quickly because of the clean interface between the compiler and garbage collector [25,69].
Generational Garbage Collection. Despite other differences, garbage collectors in modern languages have converged on a generational heap organization. The generational organization delivers high performance, because many objects die young [65]. The application (mutator) allocates new objects contiguously into a nursery. When the nursery memory is full, a minor collection first identifies live roots that point into the nursery, e.g., from global variables, the stack, registers, and the mature space. It then identifies reachable objects by tracing references from these roots. It copies reachable objects to a mature space. The nursery space is claimed en masse for fresh allocation.
Nursery size. Nursery size is critical to overall performance, pause time, and space efficiency [8,11,66,72]. A nursery collection incurs a fixed cost to scan the root set and a variable cost depending upon the number of objects that survive a minor collection. Large nurseries sometimes improve performance, because objects have more time to die. They, however, increase the overall memory footprint, often unnecessarily retaining dead short-lived objects, and they incur high pause times [50,72]. We use a 4 MB nursery, because prior work establishes that it performs well for our applications [13,58].
GenImmix. We build on the best-performing collector in Jikes RVM: generational Immix (Gen-Immix) [14]. We use it as the baseline and modify it to create the RiskRelief collectors. GenImmix uses a copying nursery and a mark-region mature space. The mark-region mature space consists of a hierarchy of blocks and lines. Blocks are multiples of page sizes and constitute multiple lines. Lines are multiples of cache line sizes. Objects can span lines but not blocks. Nursery collections copy nursery objects consecutively in space into free lines within blocks in the mature space by incrementing a bump pointer equal to the size of the object. This contiguous allocation outperform free-list allocators due to its locality benefits [11,14,32]. Immix reclaims memory at a line and block granularity by marking lines and blocks live when it marks objects live during tracing. To defragment blocks, it combines marking with copying based on runtime heuristics. We use the default settings for the maximum object size (8 KB), for line size (256 bytes), and block size (32 KB). The JVM manages objects larger than an 8 KB threshold separately, allocating them directly into a non-copying large object space [40].

HOTNESS AND RISK PREDICTION
This section motivates allocation-site prediction for object hotness and risk.

Distribution of Hotness and Risk
We start by quantifying hotness and risk across allocation sites for three benchmarks that are representative for the entire benchmark suite, namely, Fop, Bloat, and Pmd. Figure 1 (left column) shows the cumulative distribution of mature-object hotness and their total volume (as a percentage of total mature allocation) per allocation site. Allocation sites are sorted on the horizontal axis by their hotness. We observe that a large fraction of mature-object accesses are captured by a relatively small fraction of the mature heap. For example, for Fop, 90% of the mature-object accesses are concentrated to only 32% of the mature heap. This result suggests an opportunity to allocate the relatively small fraction of hot objects in HBM to improve performance while placing the bulk of the mature heap in DRAM to exploit its capacity. Unfortunately, using hotness as the sole criterion to place objects in HBM versus DRAM severely compromises a program's vulnerability to soft errors. The graphs in the right column of Figure 1 report AVF-X for the objects allocated from the top-100 hot allocation sites. To provide a point of reference, we also report mean AVF-X across all mature objects. We observe a remarkable variation in AVF-X across allocation sites from well below to well above the mean. It is clear from these graphs that hotness does not imply low risk, i.e., a hot object may be high-risk or low-risk. In other words, hotness is not predictive for risk. This result implies that using hotness alone as a criterion to classify allocation sites as lowversus high-risk severely compromises soft error vulnerability. Instead, we need a method that classifies allocation sites for both hotness and risk combined, which is what we describe next.

Allocation-site Homogeneity
The key insight that underpins RiskRelief is that allocation site is a good predictor for both hotness and risk. To demonstrate this is indeed the case, we first compute the hotness and risk for all objects and we determine which objects are among the top 10% (cutoff-threshold) for either criterion. More specifically, we label an object as hot if it is among the 10% hottest objects; if not, the object is classified as cold. Similarly for risk, we label an object as low-risk if it is among the 10% lowest-risk objects; otherwise, the object is classified as high-risk. We then compute for each allocation site, the fraction hot versus cold objects, the fraction low-risk versus high-risk objects, and the fraction of objects that are both hot and low-risk (i.e., combined). We define homogeneity of an allocation site with respect to hotness, risk or combined hotness/risk, as the fraction of objects that are classified in the same category. For example for the combined metric, perfect (100%) homogeneity means that all objects allocated from this site are both hot and low-risk, or they are not, i.e., they are either cold or high-risk. However, a value of 50% means no homogeneity, i.e., 50% of objects are hot and low-risk, whereas the remaining 50% is either cold or high-risk. Figure 2 reports the percentage heap volume as a function of allocation site homogeneity for hotness, risk, and the combined metric; we report average results across all benchmarks. This graph shows the fraction of heap volume allocated by sites that have a homogeneity of at least N %, with N varying from 100 to 50%. The higher the fraction heap volume covered, the better. As expected, heap volume increases with decreasing allocation site homogeneity. At 100% Fig. 3. Overview of RiskRelief. Offline analysis records the number of reads and writes to all objects. Then, per-object hotness and risk metrics are used to generate an allocation site classification advice, which serves as input to a bytecode rewriter. The rewriter annotates hot and low-risk sites as HBM, steering the garbage collector to place objects in HBM.
homogeneity, a relatively small fraction of the total heap volume is covered. However, reducing homogeneity quickly increases the heap volume covered. At 50% homogeneity, the entire heap is covered. The most important, and perhaps surprising, insight from this graph is that the combined metric outperforms the isolated hotness and risk metrics. For example, for 90% homogeneity, more than 97% of the heap is correctly classified for the combined metric, versus 79% and 72% for hotness and risk, respectively. This implies that allocation site is a more accurate predictor for hotness and risk combined, than for hotness and risk in isolation. The intuition is that fewer objects satisfy both the hotness and risk thresholds. We thus conclude that allocation site is a very accurate predictor to predict whether objects are hot and low-risk for placement in HBM.
Note that high allocation site homogeneity does not imply that the majority of objects are both hot and low-risk. In fact, an allocation site can have high homogeneity but produce predominantly cold objects, or produce predominantly high-risk objects, or produce predominantly hot and lowrisk objects. We only want allocation sites that allocate hot and low-risk objects to be classified as HBM. We find that RiskRelief is sensitive to the object hotness and risk cutoff threshold, but is rather insensitive to the allocation site homogeneity threshold. We use a default object hotness and risk cutoff threshold of 20% and explore its sensitivity in the evaluation section. We use an aggressive allocation site homogeneity threshold of 1% to classify allocation sites as HBM that produce even a small fraction of hot and low-risk objects, i.e., at least 1% of the objects allocated from this site are both hot and low-risk. (Note that because of high allocation site homogeneity, this implies that most objects are hot and low-risk.) We choose this aggressive threshold to make sure that hot and low-risk objects are allocated in HBM to the extent possible.

RELIABILITY-AWARE GARBAGE COLLECTION
Reliability-aware garbage collection places hot and low-risk objects in HBM and the rest in DRAM. We first provide a general overview of RiskRelief after which we describe the different components in more detail. Figure 3 shows the workflow of RiskRelief. We first profile the Java application to collect perobject read-and write-intensity traces. We then group objects in traces by their allocation site. We use per-object hotness and risk to classify allocation sites as HBM to DRAM, based on heuristics. This classification constitutes advice that we use to annotate Java bytecodes as HBM. All other allocation sites default to DRAM. During production, RiskRelief uses a unique allocation sequence for HBM-marked allocation sites. This sequence places hot and low-risk objects in HBM.

Profiling
RiskRelief relies on offline profiling of Java programs to discover hot and low-risk objects. The outcome of profiling is an access trace of per-object reads and writes, see Figure 4 for an example (we will discuss the example in more detail later). We track reads and writes in an architectureindependent manner, i.e., we count all load/store accesses to an object's fields. We count accesses to an object's primitive and reference fields, and to its meta-data header, which contains information such as the class type information, synchronization bits, and garbage collector bits.
Profiling per-object accesses can be done in two ways: (1) using read and write barriers in the managed runtime, or (2) using dynamic instrumentation. All generational garbage collectors use reference write barriers for correctness. Write barriers record all mature-to-nursery pointers in a remembered set, which are processed during a minor collection to precisely identify all live nursery survivors. Primitive write barriers are a straightforward extension of reference write barriers. Unlike write barriers, read barriers incur prohibitive overheads [46]. Most production JVMs include collectors that do not require read barriers. Jikes RVM provides both primitive and reference write barriers [4] but does not implement read barriers.
We therefore rely on dynamic binary instrumentation instead using Pin [45]. Because Pin has no notion of an object's boundary in memory, we deploy a cooperative scheme in which Jikes RVM records each object's starting address, its size in bytes, and its allocation site identifier; in turn, Pin records the number of read and write accesses to each memory location. At the end of the program execution, we gather logs from Jikes RVM and Pin, and we aggregate the two logs to create the access trace, which contains all the objects instantiated by each allocation site and the total number of accesses to each object on a per allocation-site basis.
To give each object a unique address in the access trace, we size the mature heap during profiling to preclude full-heap collections. We further set the nursery size to 4 MB. Using this nursery size is a good balance between the size of the access trace and the coverage of mature object behaviors. We label allocation sites with unique identifiers, as in Reference [31].

Allocation Site Classification
After profiling, we analyze the access trace to generate allocation advice, classifying allocation sites as HBM versus DRAM. Figure 4(a) shows an example access trace. Two allocation sites contained in methods A() and B() allocate a total of six objects. The trace also shows the different number of reads and writes to objects. We analyze the trace to compute per-object hotness and risk using the definitions described in the previous sections, see Figure 4(b). Next, we use two criteria to label allocation sites: (1) the fraction of total objects allocated from a site that are hot and low-risk, and (2) heuristics to decide which objects are hot and low-risk. If the fraction of hot and low-risk objects allocated from a site is larger than the homogeneity threshold (θ h ), then the site is labeled as HBM; otherwise, the site is a DRAM site. Next, we use two heuristics to qualify objects as hot versus cold, and low-versus high-risk.
Fixed-midpoint (FMID) is inspired by Gupta et al. [27] and uses the average hotness (or AVF-X) across all mature space objects as the cut-off to quantify the hotness (or risk) of objects from an allocation site. Specifically, with FMID, we qualify an object as hot if the sum of reads and writes to that object are above the cut-off (average). FMID has the advantage that hotness and AVF-X are straightforward to compute. The disadvantage is that it uses a single cut-off value, which leads to a specific design point in terms of SER, performance and HBM usage. In practice, a heuristic that exposes a trade-off is more desirable, which we advocate in this article.
Moving-ratio (MRAT) uses a ratio, namely, θ t (e.g., top-10%) to divide objects into two quadrants, e.g., hot and cold. The hotness cut-off (θ hot ) places an object allocated from a site in the top-10% of hot objects. Similarly for identifying low-risk objects, the risk cut-off (θ av f −x ) places an object within the top-10% low-risk objects. The user or system administrator specifies the ratio based on environmental constraints. Varying the ratio opens up a trade-off between HBM capacity, performance, and overall SER.
Example. Figure 4(a) shows an example access trace consisting of six objects from two allocation sites in methods A() and B(), respectively. Per-object hotness and risk is shown in Figure 4(b). We analyze the trace using the FMID and MRAT heuristics, and identify which of the two sites are classified as HBM in Figure 4(c). We fix θ h at 1%, and vary θ t from 20% to 100% for MRAT. The average hotness and risk is 14 and 4, respectively. Therefore, with FMID, the allocation site in method A() has one object (O1) with hotness larger than the average value, and risk larger than or equal to the average risk. Since 1 out of 3 objects from this site are hot and low-risk, which is higher than the homogeneity-threshold of 1%, this site is classified as HBM. Next, we set θ t to 20% for MRAT and compute the HBM sites. Since θ t is 20%, we only consider the hottest object (1 out of 6), and the lowest risk object to compute θ hot and θ av f −x . O1 is the hottest leading to θ hot of 24. Similarly, O4 has the lowest risk, leading to a θ av f −x of 16. Neither allocation site in Figure 4(a) has an object with both hotness larger than or equal to 24, and risk larger than or equal to 16. Thus, using MRAT with θ t at 20% leads to all allocations in DRAM. However, setting θ t to 40% or 60% results in allocation in HBM for A(). Finally, setting θ t to 80% and 100% results in all allocations in HBM. This example demonstrates the flexibility exposed by MRAT in exploiting the rich trade-offs that exist between SER, performance, and HBM capacity.

Bytecode Generation
The previous step generates allocation site advice as a file of <site-string, advice> pairs. The advice file only includes the HBM-labeled allocation sites. Unlabeled allocation sites default to DRAM. Since a minority of allocation sites are labeled HBM, the size of the advice file is minimized. We use bytecode rewriting to communicate allocation site labels to the managed runtime. The bytecode rewriter first identifies the allocation site and then queries the advice file to check whether the site is present. If it is not, then the rewriter leaves the new bytecode unchanged. If it is, then the rewriter overwrites the new bytecode with a newly introduced new_hbm bytecode. The runtime, when interpreting or compiling the new bytecode, uses the default allocator, called ALLOC_DEFAULT. The runtime then copies all objects allocated by such sites to DRAM if they survive a nursery collection. For the new_hbm bytecode, the runtime uses the newly added ALLOC_HBM allocator. This allocator sets a bit in the object header, which notifies the garbage collector to copy these objects to HBM if they survive a nursery collection. Note that because RiskRelief is a profile-based approach, there might exist allocation sites that were not seen during profiling, i.e., an allocation site was not executed in the profile run while it gets executed in a production run. These unprofiled sites will be unlabeled, and default to DRAM, following the above procedure. Future work may explore whether labeling unprofiled sites as HBM might be desirable, or whether dynamically profiling just these objects might be tractable and beneficial.

Heap Organization
We now describe RiskRelief's heap organizations. The heap organization for a conventional homogeneous DRAM-only system is shown in Figure 5(a). The RiskRelief collectors place the nursery in HBM, because the nursery is highly mutated, and hence contains objects that are both hot and low-risk. RR-N places only the nursery in HBM and the rest, i.e., the mature space and large object space, in DRAM, see Figure 5(b). RR-M further partitions the mature and large object spaces into DRAM and HBM regions, see Figure 5(c).
RR-N operates as follows. Nursery objects are allocated in the HBM nursery. Objects that survive a nursery collection are copied to the mature space in DRAM. Large objects (larger than 8 KB as in our baseline configuration) are allocated directly in the Large Object Space (LOS), which is mapped in DRAM.
RR-M is more complicated as it requires adjusting the allocation process. In general, new allocation is a two-step process: (1) reserving space and (2) initializing the object header, called post-allocation. For RR-M, post-allocation sets a bit in the object's header if its allocation site is labeled HBM, as shown in Figure 6. We steal a bit, not in use from the object header in Jikes RVM, and call it the HBM_BIT. Objects with the HBM_BIT set are predicted to be hot and low-risk. During nursery collection, the garbage collector checks the HBM_BIT of each object. If the bit is set, then it promotes the object to the mature space in HBM. Otherwise, it promotes the object to the DRAM mature space.
RR-M also involves changes to how large objects are treated. For these objects, RR-M's AL-LOC_DEFAULT allocates the object directly in the LOS DRAM space, whereas ALLOC_HBM places the object directly in the LOS HBM space.

EXPERIMENTAL SETUP
The main results presented in Section 7 are obtained through detailed architectural simulation to accurately assess performance and reliability. This section elaborates on this methodology. We complement these simulation results with emulation results on real hardware in Section 8.

Java Virtual Machine and workloads.
We use Jikes RVM 3.1.2 [5,6] and nine applications from the DaCapo suite [13] that work with our simulation and VM infrastructure. We use four benchmarks from the DaCapo-9.12-bach benchmark suite (sunflow, lusearch, pmd, and xalan). We use an updated version of lusearch, called lu.Fix [70], that eliminates useless allocation, and an updated version of pmd, called pmd.S [23], that eliminates a scaling bottleneck due to a large input file. We use three benchmarks from DaCapo 2006: fop, antlr and bloat. As in established methodology, we use 2× the minimum heap size for our benchmarks, and we use different inputs for profiling (default) versus measurement (large). We consider 32-instance workloads of our benchmarks to generate realistic memory traffic. Java performance evaluation. We follow best practices in Java performance evaluation [15,29,32]. We use replay compilation to eliminate non-determinism introduced by just-in-time compilation. During a profiling run, the VM records a plan with the optimization level for each method for the run with the shortest execution time. We then run each benchmark for two iterations. In the first unmeasured iteration, the JIT compiler applies the optimization plan to each method. We then measure the second iteration, which excludes compilation overhead and which represents application steady-state behavior. We report the average across four simulation runs.
Simulator. We use Sniper [16] v6.0, a parallel and high-speed cycle-level x86 simulator for multicore systems, using its most detailed cycle-level hardware-validated core model. Prior work extended Sniper for managed language runtimes, including dynamic compilation, and emulation of frequently-used system calls [58].

Simulated architectures.
We consider a 32-core processor with three memory systems: DRAMonly, HBM-only (both with 32 GB of main memory) and a hybrid HBM-DRAM system with 2 GB HBM and 32 GB DRAM, see also Table 1. We emphasize that the 32 GB HBM-only system is an idealized but unrealistic point of comparison. We further assume a shared 32 MB L3 cache,  [20,52]. In line with production systems, we assume single-Chipkill ECC for DRAM. Simulating multi-programmed Java workloads is time-consuming. Specifically, simulating a 32-core system executing 32 instances of the same Java benchmark in rate mode takes up to one month of simulation time for several benchmarks. Moreover, we ran into simulator infrastructure issues when simulating that many cores. We therefore report results for a single-core system with all shared hardware structures scaled down proportionally. We simulate an L3 cache of 1 MB/core, DRAM bandwidth of 0.8 GB/s for each core, and HBM bandwidth of 4 GB/s per core. Our analysis (not shown due to space constraints) confirms that the reported experimental results are conservative-in reality, the improvements in performance and SER through RiskRelief are higher-we confirmed this for up to 8 cores for two benchmarks (pmd and pmd.S) and up to 16 cores for the remaining benchmarks.

Page migration overhead.
We compare RiskRelief to the state-of-art reliability-aware OS approach for hybrid memories proposed by Gupta et al. [27]. Page migration overhead is critical to such OS approaches and includes (1) the latency for moving pages between HBM and DRAM, and vice versa, and (2) TLB shootdown overhead. 1 We assume the latency of copying pages across DRAM and HBM to be 5,000 CPU cycles [9,24]. The total overhead of a TLB shootdown is independent of the application and depends on the number of cores in the system. The OS keeps track of the "slave" cores that requested a modified virtual to physical page mapping in the past. During a TLB shootdown, the "initiator" core requests all slave cores to invalidate the modified TLB entries, flushes its own TLB and waits for the responses from all the slave cores. Following prior work by Villavieja et al. [67], we model the overhead of a TLB shootdown in a system with N cores as follows: with T slave and T initiator the time overheads incurred by each slave and initiator cores, respectively. We use published overhead numbers [24] scaled to our 4 GHz processor.
SER calculation. SER, as mentioned before, is computed as the device's raw FIT-Rate times its AVF. We use the default configuration of FaultSim for evaluating our hybrid HBM-DRAM architecture [49]. FaultSim's default transient FIT rate values for DRAM and HBM are based on a field study conducted on the Oak Ridge "Jaguar" supercomputer [64]. We further assume SEC-DED and single-Chipkill ECC for HBM and DRAM, respectively. Using this methodology, we find that the FIT-Rates equal 0.1140 and 0.0005 for HBM and DRAM, respectively. Our simulation platform precisely computes AVF by counting the number of reads and writes per cache line, which is not possible on real hardware. More specifically, we logically divide memory into 64-byte cache lines and measure the number of reads and writes per cache line, which we then use to compute AVF per cache line. For a hybrid HBM-DRAM system, we first compute the SER for DRAM and HBM as the product of their respective FIT-Rate and AVF. We then scale the individual SER numbers by the percentage of program heap that is placed in DRAM and HBM.

Key Trade-Offs
Using HBM to store a portion of the program heap provides a reliability/performance trade-off, see Figure 7. An HBM-only system delivers the best performance, but the heap is highly susceptible to soft errors, i.e., the overall normalized SER equals 1. However, a DRAM-only system is 34% slower than HBM-only, but SER is close to 0 (0.003 to be precise). RiskRelief-Nursery places the nursery in HBM and achieves 20% higher performance than a DRAM-only system. It also reduces SER by 18× compared to an HBM-only system. HBM capacity for the 32-core system equals 128 MB, which is moderate relative to the total 2 GB HBM capacity.
The state-of-the-art OS approach achieves roughly similar SER as RiskRelief-Nursery, while also requiring 128 MB HBM capacity. Our analysis shows that the OS approach correctly predicts that the nursery is hot and low risk. It thus migrates the nursery pages to HBM. Unfortunately, on x86 multi-core platforms, page migrations incur a substantial performance penalty. The large number of page migrations results in high overhead, and the OS approach performs 24% worse than RiskRelief-Nursery. The significant performance penalty of the OS approach makes it unsuitable for Java applications, because the benefits of high HBM bandwidth to access highly mutated and frequently read data is offset by the high cost of page migrations. Our analysis further shows that the overhead of TLB shootdowns is the major contributor to the high cost of page migrations.
Both the state-of-the-art OS approach and RiskRelief-Nursery place the nursery in HBM, using only a modest fraction of the available HBM capacity. Figure 7 shows that the RiskRelief-Mature collector uses a larger fraction of the available HBM capacity by placing part of the mature heap space in HBM as well. RiskRelief-Mature is highly effective at improving performance beyond RiskRelief-Nursery. On average, the execution time reduces by an additional 9% compared to RiskRelief-Nursery, and by 29% compared to a DRAM-only system, while still improving SER by a factor 9× compared to an HBM-only system.

Soft Error Rate
We now discuss soft error rates for the different systems we evaluate in this work. Figure 8 shows SER of DRAM-only, RR-N, RR-M, and the OS approach, normalized to HBM-only for the individual workloads. We observe that a DRAM-only memory system is highly reliable with negligible SER compared to HBM-only. This observation is consistent with prior work, which reports that in DRAM-only systems, non-DRAM failures, such as those in memory controllers and memory channels, dominate the majority of errors [47]. Whereas HBM-only is highly unreliable with a normalized SER of 1, RR-N reduces the SER by 18× on average. All benchmarks observe a reduction in SER and the reduction in SER varies from 9× (Fop) to 48× (Xalan). The differences in per-benchmark SER reduction are due to access patterns in the nursery, more specifically, the ratio of nursery writes to reads. RR-N is the most reliable system of all systems we evaluate in this work, but it does not fully utilize the available HBM capacity. We can utilize the available HBM capacity to gain more performance. As mentioned before, RR-M is the best performing system, however, it sacrifices reliability over RR-N. Still, RR-M reduces SER by 9× over HBM-only. Some benchmarks, such as Bloat, experience no change in SER reduction with RR-M compared to RR-N. This phenomenon occurs because SER depends on several factors including the ratio of object writes to reads, the rate of memory allocation, and how often the objects in the program heap are accessed after the first allocation. Per-benchmark SER reduction with RR-M compared to HBM-only varies from 5× to 30×. For completeness, the OS approach achieves a normalized SER that is comparable to RR-N.

Performance
We show per-benchmark performance results in Figure 9, normalized to a DRAM-only system. Execution time with an HBM-only system reduces by 34% on average. Individual benchmarks show a variety of trends. For example, the execution time of Xalan, Pmd, Pmd.S, and Lusearch reduces by more than 40%. As reported in Table 3, these benchmarks are characterized by either large heaps, high allocation rates, or high nursery survival rates. The compute-bound Sunflow benefits the least from HBM bandwidth. Our analysis indicates that memory read operations in Sunflow exhibit very high on-chip cache hit rates, thus leading to limited traffic to main memory.
The RiskRelief collectors deliver performance in-between DRAM-only and HBM-only. Placing the nursery in HBM with RiskRelief-Nursery (RR-N) reduces execution time by 20% on average compared to a DRAM-only system. Benchmarks that allocate rapidly benefit more from HBM bandwidth. For example, Lusearch allocates the largest volume of objects across our benchmarks, and RR-N reduces its execution time by 34%. The reasons for this large reduction in execution time include: (1) faster read and write operations to memory, (2) higher throughput of memory zeroing to provide security as guaranteed by Java semantics [1,70], and (3) faster nursery collections. Surprisingly, Sunflow allocates young objects rapidly in the nursery and has the largest number of nursery collections of all of our benchmarks, yet its execution time reduction with RR-N is only 15%. This small reduction is because Sunflow has a small nursery survival rate (only 2%) and copying nursery survivors to the mature space does not require high bandwidth. RiskRelief-Mature (RR-M) reduces the execution time on average by an additional 9% over RR-N, and by 29% over a DRAM-only system. RR-M splits the mature and large object spaces across DRAM and HBM. The benchmark that benefits the most from RR-M is Lusearch. The execution time of Lusearch reduces by 43%. The performance of Lusearch with RR-M is only 5% less compared to HBM-only, showing The OS approach leads to a significant performance degradation compared to RR-N. Performance degrades for most benchmarks and we note a significant performance degradation for Fop (42%) and Xalan (77%). The reason is the high number of page migrations per unit of time, see also Table 2, which reports the number of page migrations from DRAM to HBM and vice versa, the number of 100 ms migration epochs, and the number of page migrations per epoch. We note that Fop and Xalan are the benchmarks with the highest number of page migrations per unit of time: 347.3 and 463.5 migrations per epoch. Each page migration incurs the overhead of copying the pages and TLB shootdowns. Our measurements indicate that TLB shootdowns account for 41% and 45% of the total execution time for Fop and Xalan, respectively. In other words, TLB shootdowns lead to significant performance degradations for workloads that incur a large number of page migrations per unit of time. The OS approach delivers performance that is better than RR-N for Sunflow, and only slightly worse than RR-N for Bloat, Lu.fix, and Lusearch. This is due to the relatively small number of page migrations per 100 ms epoch for these benchmarks; see Table 2. The number of page migrations per epoch is substantially smaller for these benchmarks-Sunflow (32.3), Bloat (73.2), Lu.fix (51.8), and Lusearch (67.2)-compared to the other benchmarks with more than one hundred and up to several hundreds of page migrations per epoch; note that Sunflow has the lowest number of page migrations, which leads to a small performance overhead (6%) and a net performance improvement over RR-N.

RR-M versus Performance-Focused GC
Utilizing HBM capacity is a trade-off between performance and reliability. RR-M can be configured in a variety of ways to exploit this trade-off space. Performance improves when RR-M is configured to place more mature-space objects in HBM, but this compromises reliability. We show this tradeoff in Figure 10. We vary θ t from 10% to 40%. Execution time reduces by 3%, but the SER increases by 5.4×. The reason for the SER increase is that, as RR-M tries to achieve higher performance by placing an increasingly larger fraction of the mature space in HBM, it copies objects with low AVF to HBM. In other words, as θ t increases, allocation sites with a larger number of high-risk objects are classified as HBM, which results in higher performance, but lower reliability.   Figure 10 plots a similar performance versus reliability trade-off curve for a performancefocused variant of RR-M. This performance-focused variant labels allocation sites as HBM based only on the percentage of hot objects allocated from the site. Similar to RR-M, it uses the θ t threshold to classify objects as hot versus cold. The resulting trade-off curve with this performancefocused collector clearly shows the benefits of RiskRelief in mitigating HBM's high susceptibility to soft errors. Specifically, for the same performance, RR-M exhibits 4.8× lower SER than the performance-focused variant. RR-M takes into account both how often an object is accessed and its AVF before placing it in HBM. Table 3 summarizes total allocation, nursery survival rates, and percentage of mature heap in HBM for RR-M for the different 32-instance workloads. Our applications allocate frequently ranging from 1.8 GB (Fop) up to 137 GB (Lusearch). Our nursery survival rates vary from 2% to 27%. Copying objects to HBM is faster than DRAM, and hence benchmarks that copy a larger fraction of objects to HBM on a nursery collection benefit more from HBM's high bandwidth. Examples include Xalan and Pmd. The next columns show the average and maximum HBM (in MB) for different configurations of RR-M. Specifically, we show the HBM capacity in MB for different θ t thresholds. HBM capacity with the most reliable RR-M configuration (θ t of 10%) equals 294 MB on average, and up to 972 MB. Lusearch consumes the largest HBM capacity with close to 1.5 GB. RR-M places only 12% of the total heap volume in HBM with a 10% θ t threshold. The percentage of heap volume in HBM increases to 23% of the total heap volume in HBM with a θ t of 40%. However, the OS approach places only 5% of the total heap in HBM.

EVALUATION ON REAL HARDWARE
Accurately assessing SER for a hybrid memory systems requires per-cacheline read/write statistics, which we can only obtain through simulation. We now complement our simulation results with experimentation on commercial hardware, for three reasons: (1) to demonstrate that we can deploy RiskRelief on real systems, (2) to show that RiskRelief directs the vast majority of writes to HBM, and (3) to report the runtime overhead of RR-M relative to RR-N.
Emulation platform. Since we lack access to a commercial machine with HBM, we emulate hybrid HBM-DRAM memory on an existing multi-socket NUMA platform, as in Reference [3]. Commercial HBM systems present HBM as an additional NUMA node to the OS [53], which is exactly what we emulate. In other words, by running Java workloads on the emulation platform with RiskRelief collectors, we incorporate OS and runtime effects as expected on commercial HBM systems. We isolate the Java workload on one socket and disable the other socket. We populate both sockets with commodity DRAM chips. Local memory emulates HBM, and remote memory emulates DRAM. We modify Jikes' MMTk to split the virtual heap into HBM and DRAM. Our twosocket Intel Sandy Bridge E5-2650L processor has 8 physical cores per socket and two hyperthreads per core. We use Ubuntu 12.04.2 with a 3.16.0 kernel. We run 8-instance workloads to utilize all the available cores.
Number of writes to HBM. We now quantify the number of writes to HBM versus DRAM on the emulation platform, which features 132 GB of main memory, evenly distributed between the two sockets. We use all DRAM channels on both sockets. All cores share the 20 MB LLC on each processor. The available bandwidth to memory is 51.2 GB/s, more than the maximum bandwidth consumed by any of our workloads. A QPI link that supports up to 8 GB/s connects the two sockets. We use Intel's pcm-memory utility to measure the number of writes to HBM and DRAM.
RiskRelief allocates the frequently accessed low-risk objects in HBM and the rest in DRAM. We thus expect that most writes happen to HBM. We observe that in simulation, on average, 90% and 87% of writes happen to HBM for RR-M and RR-N, respectively. On the emulation platform, we find that 87% and 83% of writes happen to HBM, respectively. Simulation and emulation thus confirm that RiskRelief captures the vast majority of writes to HBM-this indicates that the frequently accessed low-risk objects are indeed allocated in HBM. The small discrepancy between emulation and simulation is a result of differences in the OS, hardware prefetcher, memory controller, among other things.
RR-M runtime overhead. RR-M incurs runtime overhead because of the extra steps involved during post-allocation and nursery evacuation. To quantify these overheads as accurately as possible, we compare the performance of RR-M versus RR-N on the emulation platform while placing the entire heap on one socket of our NUMA platform. On average, the overhead incurred by RR-M is less than 1%, with a maximum of 1.3% for lusearch.

OTHER RELATED WORK
Beyond the related work already discussed in this article, some prior work focuses on automated memory management for hybrid DRAM-PCM memories. However, to the best of our knowledge,