{'fraction_non_alphanumeric': 0.035359200530260544, 'fraction_numerical': 0.012160301840615917, 'mean_word_length': 4.710729363808414, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
CoPA: Cold Page Awakening to Overcome Retention Failures in STT-MRAM Based I/O Buffers

Performance and reliability are two prominent factors in the design of data storage systems. To achieve higher performance, recently storage system designers use DRAM-based buffers. The volatility of DRAM brings up the possibility of data loss, so a part of the main storage is conventionally used as the journal area to be able of recovering unflushed data pages in the case of power failure. Moreover, periodically flushing buffered data pages to the main storage is a common mechanism to preserve a high level of reliability, which leads to an increase in storage write traffic. To address this shortcoming, recent studies offer a small NVM as the Persistent Journal Area (PJA) along with DRAM as an efficient approach, named NVM-Backed Buffer (NVB-Buffer). This approach aims to address DRAM vulnerability against power failure while reducing storage write traffic. In this paper, we use the most promising technologies for PJA among the emerging technologies, which is STT-MRAM to meet the requirements of PJA (high endurance, non-volatility, and DRAM-like latency). However, STT-MRAM faces major reliability challenges, i.e. Retention Failure, Read Disturbance, and Write Failure. In this paper, we first show that retention failure is the dominant source of errors in NVB-Buffers as it suffers from long and unpredictable page idle intervals. Then, we propose a novel NVB-Buffer management scheme, named, Cold Page Awakening (CoPA), which predictably reduces the idle time of PJA pages. To this aim, CoPA employs Distant Refreshing to periodically overwrite the vulnerable PJA page contents by using their replica in DRAM-based buffer. We compare CoPA with the state-of-the-art schemes over several workloads based on physical journaling. Our evaluations show that employing CoPA leads to three orders of magnitude lower failure rate with negligible performance degradation (1.1%) and memory overhead (1.2%).


INTRODUCTION
Storage systems use Dynamic Random Access Memory (DRAM)-based buffers to mitigate the considerable latency gap between storage and main memory [1][2][3][4]. DRAM benefits from low access time, high density, and unlimited lifetime, which makes it a suitable candidate for these applications. However, data loss due to power failure is the major drawback of DRAM-based buffers because of its volatility. Thus, DRAM-based buffers exploit mechanisms such as periodic flush [2,5] and/or partial use of main storage as the journal area [6][7][8][9][10] to recover data in the case of power failure. Nevertheless, these techniques significantly increase storage write traffic.
Thanks to emerging Non-Volatile Memories (NVMs) [11][12][13][14][15][16][17][18][19], a recent approach to design an efficient buffer is to use a relatively small NVM as the journal area [2,8]  DRAM-based buffer. This approach is referred to as NVM-Backed Buffer (NVB-Buffer) in this study. In this approach, the buffer takes advantage of NVM persistency to reliably store dirty data pages, beside low latency of DRAM for fast accesses. To this end, NVM is recognized as P ersistent Journal Area (PJA) where a copy of all dirty data pages is stored in NVM. In this case, there always exists a persistent valid copy of the DRAM data page. Thus, NVB-Buffer can use PJA data pages for data recovery in the case of power failure while considerably reducing storage write traffic [2].
Despite STT-MRAM advantages, it faces three reliability challenges such as Retention Failure, Write Failure, and Read Disturbance. Bit errors in a PJA page due to these challenges lead to data loss in the case of power failure, as there is no valid copy of the page in the main storage. Moreover, previous studies mainly aim to enhance performance or reduce journaling overhead [2,7,9,41]. To our knowledge, none of the prior studies considers the reliability of STT-MRAM based NVB-Buffers, while the previous schemes to overcome STT-MRAM retention failure are either expensive or not applicable as they increase the probability of read disturbance or cause performance degradation [44][45][46][47].
In this paper, we first investigate existing NVB-Buffer schemes using several well-known storage workloads to explore their vulnerability to STT-MRAM error sources. Our extensive set of experiments on Microsoft Research Cambridge (MSRC) traces [48] show that PJA pages in NVB-Buffer suffer from long idle time, varying from 34.5 to 100.8 minutes, which makes them highly vulnerable to retention failure. Moreover, technology downscaling makes retention failure more severe [32,42]. Based on our experiments, retention failure is the dominant source of PJA failures in recent technology nodes, as the probability of data loss due to retention failure is five orders of magnitude higher than write failure, on average [32,42]. However, read disturbance is not a serious challenge in PJA as the PJA pages are inherently used for data residency (pages are just read in the case of data recovery due to power failure).
We then propose a novel management scheme for NVB-Buffers, named, Cold Page Awakening (CoPA) to efficiently reduce the idle time of PJA pages. CoPA reduces idle intervals using Distant Refreshing, where it overwrites PJA pages using error-free replicas in DRAM-based buffers. CoPA also differentiates PJA pages based on their vulnerabilities to retention failure and prevents recently written pages from refreshing. To show the effectiveness of Distant Refreshing, we examine and compare the proposed scheme against the conventional refreshing. Our experiments show that Distant Refreshing reduces the failure rate by 89.9%, on average, compared to conventional refreshing. Conventional refreshing employs read-correct-write approach, which increases the probability of read disturbance and, consequently, leads to increase in total failure rate.
CoPA aims at providing different levels of reliability and can be tuned depending on the application requirements. Our experimental evaluations based on the physical journaling [5,6] are performed across twelve storage workloads from MSRC [48]. CoPA is tuned to guarantee an upper bound for maximum idle time of PJA pages in such a way that reduces the maximum idle time of PJA pages by an average of 53.5× (up to 66.9×) compared to the state-of-theart NVB-Buffer management schemes [2,9,10]. CoPA results in three orders of magnitude failure rate reduction, with negligible performance (an average of 1.1%) and memory (1.2%) overhead. We also compare CoPA with the conventional periodic flush-enabled scheme, which provides high reliability at the cost of considerable storage write traffic. Tuning CoPA in favor of reliability leads to an average of 43% (up to 81.9%) response time reduction.
The main contributions of this paper are as follows: • This is the first study that investigates the reliability of STT-MRAM based NVB-Buffers. We examine the stateof-the-art schemes and show that retention failure is the main contributor to PJA errors. PJA pages suffer from long page idle times, which significantly degrades NVB-Buffer reliability due to high probability of retention failure. •

NVM-Backed Buffer
Buffering I/O requests is one of the conventional solutions for performance improvement due to reducing storage write traffic and buffer hits in storage systems [3]. In addition to performance improvement, in high-performance Solid-State Drive (SSD)-equipped storage systems, reducing storage write traffic leads to the lifetime improvement, as Flashbased SSDs suffer from limited endurance [49][50][51]. DRAMbased buffers, however, face with unrecoverable data loss in case of power failure due to volatile characteristic of DRAM.  Fig. 2: Hit ratio and storage traffic of NVB-Buffer normalized to Hyb-Buffer and b) partial use of the main storage as the journal area [6][7][8][9][10] are exploited for data recovery in the case of power failure. Nevertheless, these techniques lead to a considerable increase in storage write traffic [2]. Taking advantage of NVMs persistency along with DRAM performance is one of the promising schemes for designing efficient buffers [1,2]. NVM-Backed Buffer (NVB-Buffer) is one of the recent schemes that provides efficient I/O buffers. In this scheme, DRAM-based buffer is equipped with a relatively small NVM as a Persistent Journal Area (PJA), which only tracks the dirty data pages of DRAM. In NVB-Buffers, the NVM pages are only read in the case of data recovery while in other scenarios, the accesses to NVM are only writing dirty pages. NVM pages are flushed whether NVM becomes full or their corresponding pages in volatile buffer are evicted. Thus, DRAM pages always have a persistent copy that prevents data loss in case of power failure.
We analyze the efficiency of NVB-Buffer compared to the conventional One-tier Hybrid Buffer (Hyb-Buffer) to understand the impact of adding NVM space to operational buffer capacity on system performance. In Hyb-Buffers, DRAM and NVM are managed as a single tier, where the dirty data pages are redirected to both memories to prevent data loss in the case of power failure. Also, evicted data pages from DRAM are admitted to the NVM while NVM hit leads to migration of the page to the DRAM (if the page is dirty, NVM still buffers its copy). Fig. 2 shows the hit ratio and storage traffic of NVB-Buffer normalized to the Hyb-Buffer. The evaluations are performed based on a buffer consists of 8GB DRAM and 512MB NVM. Although both schemes provide the same performance behavior in workloads with working set smaller than the buffer capacity, NVB-Buffer significantly reduces storage traffic in workloads with working set greater than the buffer capacity. NVB-Buffer alleviates storage traffic by 47.4%, 43.9%, and 49.4% in Stg 1, Src2 1, and Usr 2, respectively. Hyb-Buffer, however, provides slightly higher hit ratio in the same set of workloads (up to 1.4%).
The traffic reduction is achieved by providing more presence time for dirty data pages in the buffer. Inserting the anti-parallel state [17,52] evicted data pages from DRAM to the NVM increases the number of NVM evictions. If the evicted page from NVM is dirty, as a DRAM dirty page should have a persistent copy, the page is written to the main storage and the status of its corresponding page in DRAM is changed to clean. Thus, even if the page still resides in the buffer, it is already flushed to the main storage. We also evaluate the scenario where NVM tracks dirty pages even after the eviction of their corresponding copy from DRAM to provide less storage traffic and more chance for buffer hit. The improvement of this scenario, however, is less than 0.01%, at the best case.

STT-MRAM Basics
The most promising technology among emerging NVMs is STT-MRAM, which can be used in NVB-Buffers. An STT-MRAM cell consists of an access transistor and a Magnetic Tunnel Junction (MTJ), which is responsible for data storage, shown in Fig. 3-a. MTJ is made of an insulating layer (Magnesium Oxide), and two magnetic layers named Free Layer and Reference Layer. The magnetic field direction of the reference layer is fixed while the direction of the free layer can be changed and determine the constructed resistance value. If the free layer is paralleled with the reference layer in the case of flowing write current (I write ) from the free layer to the reference layer, MTJ will be in low resistance state, which is the logical '0' value ( Fig. 3-b). An anti-parallel pattern between the spin directions of the free layer and the reference layer is due to I write flowing from the reference layer to the free layer, which results in high MTJ resistance and logical '1' value ( Fig. 3-c) [17,52]. STT-MRAM provides promising features in terms of endurance, performance, power consumption, and scalability. It is expected that STT-MRAM partly replaces technologies such as NOR Flash, Static RAM (SRAM), and DRAM, while it is estimated that standalone Magnetic RAM (MRAM) and STT-MRAM baseline revenues reach $3.8B in 2029 [53]. However, STT-MRAM is threatened by three types of failures. To read the stored data in an STT-MRAM cell, a small current is applied to the free layer to recognize high or low resistance level. It is probable that this small current leads to a bit flip in the cell, which is called Read Disturbance. To write data, a write current is applied to change the magnetic field  [42,43] direction of the free layer. Nevertheless, the cell content may not change during the write pulse, which leads to a Write Failure. Moreover, stochastic characteristics of STT-MRAM leads to inadvertent bit flip, as the stored data would be changed, without applying any current to the MTJ, and leads to a Retention Failure. To make STT-MRAM a widely commercialized replacement for conventional technologies to be used in NVB-Buffers, it is a must to address its reliability challenges.

Reliability Challenges of STT-MRAM Based PJA
Any candidate technology to be used in PJA needs to provide three features, a) non-volatility: PJA must be capable of retaining data even in the case of power failure to be used for data recovery, b) high endurance: due to extreme write pressure on PJA and its relatively small size (compared to DRAM), it should be able to endure a high number of writes without lifetime degradation, and c) DRAM-comparable write latency: to prevent inconsistency in reading data from PJA and the buffer (because of large difference between their write performance), PJA delay should be less or equal to DRAM buffers. Among emerging technologies, STT-MRAM is the most suitable for PJA. PCM, 3D-Xpoint, and Flash neither can provide PJA endurance nor its performance requirements [27][28][29][30]. Although FeRAM offers high endurance, its write latency is considerably higher than STT-MRAM [17,28,29]. ReRAM is another possible PJA candidate, which provides promising performance feature. ReRAM-based PJA suffers from limited lifetime [17,28,29,31]. Therefore, due to STT-MRAM high endurance and low write latency (compared to FeRAM, PCM, Flash, and 3D-Xpoint), it is the most promising PJA candidate. On the other hand, STT-MRAM provides lower write energy compared to the mentioned technologies [29]. However, to provide an efficient NVB-Buffer, the aforementioned reliability challenges of STT-MRAM should be carefully addressed. Fig. 4 shows the footprint of each error type on the total failure rate of a 32MB STT-MRAM cache [42]. The probability of write failure is considerably reduced by technology downsizing, while the probability of read disturbance is increased. Although read disturbance is still a serious concern in processor cache levels and main memory [ Fig. 5: An example of NVB-Buffer access pattern impact on PJA page idleness from the PJA perspective, read disturbance is not the main challenge as PJA is inherently exploited for data residency. Thus, the stored data in PJA is only read for the sake of data recovery due to power failure or system crash (once during its lifetime). By technology node downsizing, however, retention failure becomes the dominant source of STT-MRAM errors [13,42,43,57] and leads to a significant reduction in STT-MRAM Mean Time To Failure (MTTF), as shown in Fig. 4 [32]. Therefore, PJA data pages could only be valid for a very short time intervals. Increasing the probability of retention failure is one of the main challenges of employing STT-MRAM as PJA, since it threatens data retaining, which is essential for PJA.

Impact of NVB-Buffer Accesses on PJA Page Idle Time
To examine the idle time of PJA pages, which is the main contributor of retention failure rate, a motivational example of how NVB-Buffer access pattern affecting the PJA page idleness is provided as shown in Fig. 5. It is assumed that the buffer and PJA capacity are four and two pages (for the sake of visibility), respectively, while both use Least Recently Used (LRU) replacement policy and physical journaling [6] is employed. 1 st and 2 nd accesses lead to the insertion of A and B in the buffer and PJA (as both requests are write-requests). The 3 rd access leads to reading A, which calls to update both queues. However, A is read from the buffer and the page A in PJA is not either read or refreshed, so it is still idle in PJA. At 4 th access, C is written, which leads to eviction of B from PJA. Thus, B is written to the main storage and interpreted as a clean page. From the 5 th access to the 7 th access, the buffer is filled, A is redirected to the Most Recently Used (MRU) position, and B is evicted from NVB-Buffer. In the 8 th access, F is inserted into the buffer leading to the eviction of C from the buffer. As C is a dirty page, it is redirected to the main storage and its corresponding page is also evicted from PJA. A is admitted to the NVB-Buffer at the 1 st access and then is read multiple times, which prevents A from redirecting to the main storage. In the case of system crash or power failure, the only dirty page that should be recovered is A, which is exposed to retention failure due to its long idle time.
This instance demonstrates the access sequences on a small scale. As it shows, a long idle time can be caused for a page after a while, during the access sequences. In this case, long durations should be shortened to decrease the retention failure rate. Therefore, a reliable NVB-Buffer management scheme should seriously consider mechanisms to alleviate the vulnerability of pages such as A.

Impact of Various Error Types on PJA Failures
We provide an illustration about the contribution of retention failure in PJA failures compared to other reliability challenges. We set up an experiment to investigate the impact of each error type on the failure of PJA pages. NVB-Buffer consists of 8GB DRAM and 512MB STT-MRAM based PJA, while employing physical journaling with 4KB page size.

Formulation
Retention failure depends on the idle intervals and the probability of its occurrence for a cell is according to (1) [42]: where t and ∆ represent idle time and thermal stability factor, respectively. Suppose that each STT-MRAM page consists of 512 64-bit word, and each word is protected by a Single Error Correction-Double Error Detection (SEC-DED) code. The probability of data loss due to retention failure of a page for idle time equal to t is according to (2): where k is the number of bits in a word and W is the number of words in a page, which are 64 and 512, respectively. The probability of data loss of a page for all idle intervals is according to (3): where n is the number of intervals while t i represents the page idle time during interval i.
Write failure is another error type that is likely to happen in PJA pages. The probability of write failure for an STT-MRAM cell is according to (4) : where t write represents the width of write pulse, µ β is Bohr magneton, p is the tunneling spin polarization, and I write represents write pulse width, while c and m are Euler constant and magnetic momentum of the free layer, respectively.
The probability of data loss due to write failure for a SEC-DED protected page is according to (5): where k is the number of bits in a word, W is the number of words in a cell, and N represents the number of writes committed to a page. In NVB-Buffer, since PJA content is just read once for data recovery upon power failure, the impact of read disturbance on PJA failure is highly negligible.

PJA Failure Characterization
To investigate the probability of PJA data loss due to each STT-MRAM error type in recent technology nodes, we assume the probability of an erroneous write is 10 −8 for an STT-MRAM cell [32,42,56]. The probability of data loss for each error type is calculated based on equations (1)- (5). Note that read disturbance is not a major concern in PJA as it is only probable in case of data recovery where each PJA page is read once. Fig. 6 shows the probability of PJA data loss, normalized to data loss due to write failure. Retention failure is the dominant source of failure as its footprint is drastically higher than write failure (an average of five orders of magnitude). Besides technology node characteristics, long and undetermined idle time of PJA pages is one of the main contributors to the data failure due to retention failure. Fig. 7 shows the maximum page idle time of PJA pages. Neglecting page idleness as one of the design parameters of NVB-Buffer management scheme leads to pages with maximum idle time for an average of 75 minutes (up to 100.8 minutes), which makes them more vulnerable to retention failure. One of the conventional approaches that can reduce page idle time in PJA is periodic flush, the same approach as Linux pdflush function. In this approach, the dirty pages of NVB-Buffer are flushed to the main storage, based on a predetermined time period. Using periodic flush, an upper bound is set for maximum idle time of PJA pages as data is written to the main storage and PJA data pages can be discarded. Despite reducing page idle time, employing periodic flush considerably increases the storage write traffic [2]. Fig. 8 shows the impact of using (Baseline) and eliminating periodic flush (No-pdflush) on PJA reliability and storage write traffic. In the baseline, dirty pages of NVB-Buffer are monitored and the flushing procedure is invoked every five seconds to flush the pages with 30-seconds idle time. Enabling periodic flush in the baseline reduces the failure rate by three orders of magnitude (940×, on average) compared to No-pdflush by preventing long page idle time and guaranteeing an upper bound for data pages (depicted in Fig. 8-a). Nevertheless, employing the baseline significantly increases storage write traffic (as shown in Fig. 8-b). Although using No-pdflush increases the probability of data loss, it reduces storage write traffic by an average of 66.7% (up to 93.9% in Rsrch 0), as it provides more chance of reaccessing for dirty data pages in NVB-Buffer.
Designing an efficient NVB-Buffer needs approaches that address long idle time of PJA pages with respect to the storage write traffic. However, none of the prior studies address the reliability challenges of STT-MRAM based PJA. Moreover, existing schemes for mitigating retention failure at circuit-and/or architecture-level are either expensive or inefficient and optimized for processor cache or main memory [44,59]. Based on these observations, we investigate that there is a need for a system-level scheme to enhance the reliability of STT-MRAM based NVB-Buffers, which is the motivation of this work.

PROPOSED SCHEME
A commercialized STT-MRAM based NVB-Buffer needs to reconcile its retention failure challenge. To this end, we propose a novel NVB-Buffer management scheme named Cold Page Awakening (CoPA) to reduce the idle time of PJA pages. Fig. 9 shows the overall structure of CoPA. While the buffer regularly performs its conventional functions such as a) admission of read/write requests to NVB-Buffer, b) handling buffer hits, and c) metadata management, it also aims at preventing retention failure using NVB-Buffer characteristics. CoPA employs Distant Refreshing instead of conventional refreshing [45] mechanism to overcome the large idle time of PJA pages. Regarding the features of recent technologies and NVB-Buffer structure, Distant Refreshing aims at refreshing PJA pages based on their corresponding replica in buffer space without increasing the probability of read disturbance.
CoPA controls the refreshing procedure by tracking the metadata of PJA pages using two queues and a simple 2bits counter (bits are indicated as QI and DC in Fig. 9) to differentiate between recently-written and idle pages. Consequently, CoPA reduces the probability of retention failure by time-based refreshing of idle pages, without intensifying the probability of read disturbance. The rest of this section details Distant Refreshing mechanism and its benefits over conventional refreshing (Section 4.1), then provides a stepby-step depiction of page awakening procedure (Section 4.2), and finally presents the flow of CoPA for each request (Section 4.3). as the length of refreshing intervals (Refresh Period) or the way to select some pages for refreshing. In an aggressive approach, all PJA pages are refreshed with a short refresh period. This approach suffers from performance overhead and higher probability of write failure (because of successive write operations). Short refresh period along with refreshing all of PJA pages lead to reading a considerable amount of data from DRAM and writing them to STT-MRAM. Moreover, re-writing pages every few seconds in the aggressive refreshing method can increase the total data failure probability, as it imposes more write failure. Aggressive refreshing also increases the temperature due to extensively reading from DRAM and writing to STT-MRAM, which increases the total probability of failure [32]. Therefore, aggressive refreshing is not an efficient approach for NVB-Buffers due to not only its reliability side effects, but also imposing significant memory overhead. In a conservative refresh approach, the refresh period takes longer, while specific pages are refreshed. However, a long refresh period increases the probability of retention failure. Moreover, detecting proper pages for refreshing is another challenge of the conservative approach. A simple   Fig. 11: Impact of distant refreshing vs. conventional refreshing on failure rate way to detect the best candidate is continuously checking the access time of each PJA page, which imposes memory overhead. Another solution is to employ detection mechanism such as a) reading the page content and detecting failures by ECC checking, b) comparing the page content with its corresponding copy in DRAM, or c) prediction [44,45]. However, these solutions either cause extra read operations from PJA or are not accurate. Thus, conservative refreshing is inefficient due to high memory overhead and still probable retention failure. The ideal scheme should balance between these two extreme approaches.

CoPA Queue Management
CoPA a) increases the length of refresh intervals, b) guarantees an upper bound for idle time of each page, and c) prevents the recently-written pages from refreshing to reduce the overheads. To this end, CoPA takes advantage of two queues, named Q1 and Q2, and interprets them as Sleepy Queue and Awake Queue to differentiate and manage idle and recent pages. Sleepy queue tracks pages with high idle times while Awake queue tracks the metadata of pages with low idle times. CoPA also uses a 2-bit counter called State Counter to manage both queues, as shown in Fig. 12. State Counter is employed for differentiating and managing Q1 and Q2. CoPA divides each refresh period into two time-steps where at the end of each time-step, the value of the State Counter is increased by one. The Most Significant Bit (MSB) and Least Significant Bit (LSB) of the State Counter are recognized as QI (Queue Identifier) and DC (Drowsiness Categorizer). Fig. 12 shows how CoPA manages these queues, based on the value of State Counter. QI is responsible for identifying the Sleepy and Awake queues. When QI is '0', Q1 is recognized as the Sleepy queue, otherwise, Q2 is interpreted as the Sleepy queue. DC manages the insertion of each page metadata. If DC is '0', the metadata of incoming page is inserted into the Sleepy queue. On the other hand, the metadata of incoming page is inserted into the Awake queue. The refresh operation is performed at the end of the refresh period (after two time-steps), which leads to QI transition (from '0' to '1' or from '1' to '0') and refreshing PJA pages based on the Sleepy queue. Thus, CoPA gets rid of the high overhead of tracking and checking the idle time of each page by employing a simple State Counter. Moreover, using DC, CoPA is able to prevent refreshing of recently written pages and using QI, CoPA just changes the label of its queues (Q1 and Q2), instead of copying their contents. Fig. 12 shows an example of how CoPA manages the refreshing procedure. It is assumed that the initial value of the State Counter is 0. In the first time-step, DC is '0' so A and B as incoming requests are inserted into the Sleepy queue. As QI is '0', Q1 and Q2 are interpreted as the Sleepy and Awake queues, respectively. At the end of each timestep, the value of State Counter is increased by one. In the second time-step, as DC is '1', the metadata of requests are redirected to the Awake queue. Since QI value is not changed, Q1 and Q2 are still the Sleepy and Awake queues, respectively. The metadata of C is inserted into Q2 and for B, the old version in Q1 is invalidated and then the metadata of B is inserted into Q2. At the end of the second time-step, the refresh procedure is performed based on entries of the Sleepy queue, which is Q1. Thus, the corresponding page of A in PJA is refreshed. By increasing State Counter value, the QI value is changed from '0' to '1'. Hence, the labels of Q1 and Q2 are interchanged, i.e., Q1 becomes Awake queue and Q2 is now the Sleepy queue. As DC is '0', the metadata of D is inserted into the Sleepy queue (Q2). At the fourth time-step, based on DC value ('1'), E's metadata is inserted into the Awake queue (Q1). At the end of the fourth time-step, the refreshing of C, B, and D is performed as their addresses reside in the Sleepy queue (Q2). At the worst case, each page is refreshed after three time-steps. Therefore, if a page is not re-written by the application, the maximum idle time of each page is according to (6): where T timestep is the time-step period (the half of each refresh period), T idleness is the page idleness time, and is the minimum time unit. We evaluate CoPA to compare with aggressive refreshing (Conv Scheme) as a conventional scheme that aims to refresh PJA pages by a predetermined period equal to 60 seconds. Fig. 13 shows the number of refreshed pages normalized to Conv Scheme. CoPA reduces the number of refreshed pages up to 19.5% compared to Conv Scheme, as it prevents the refreshing of recently written pages. By reducing the refreshing period of Conv Scheme, the gap between CoPA and Conv Scheme is increased. CoPA also updates its queues by the incoming new write requests. If the page already exists in either of the CoPA queues, the corresponding page is invalidated. Based on the value of QI and DC, CoPA inserts the metadata of this page into the proper queue (into the Sleepy queue if DC is '0' and into the Awake queue, otherwise). Moreover, if a dirty page is evicted from NVB-Buffer, CoPA invalidates its metadata in CoPA queues.

Evaluation Flow
To evaluate CoPA, we take advantage of our in-house buffer cache simulator, which is developed based on Linux kernel 4.4 and EXT2 filesystem, validated using Flexible I/O (fio) [60], Filebench [61] and Postmark [62] over different types of workloads. As shown in Fig. 14, there is a negligible difference (up to 2.3%) between the simulator and Linux hit ratio under different patterns of accesses (random or sequential) and various distributions (e.g., zipf 0.9 and zipf 1.3), which is due to the difference between employed LRU and the semi-LRU used by Linux. We extend the simulator for: 1) analyzing timing parameters of the application block requests and 2) investigating the impact of each scheme on the response time. The simulator is also equipped with DRAMSim2 [64], configured for simulating two DDR3 modules based on DRAM and STT-MRAM characteristics for the buffer and PJA, respectively [63]. We post-process the simulator outputs based on (1)-(5) to evaluate each scheme in term of retention failure probability.
Our evaluations are performed on twelve workloads from Microsoft Research Cambridge (MSRC) traces [48], based on an NVB-Buffer consists of 8GB DRAM as buffer space, 512MB STT-MRAM as PJA, and the page size of 4KB. Each of these pages are consisted of 512 64-bit datawords, each CoPA is compared with the the state-of-the-art approaches [2,9,10], where the periodic flushing is eliminated, referred to as Nopdflush (the physical journaling is considered as the journaling mechanism). To provide an overview of CoPA efficiency in term of performance, we also compare CoPA with a conventional reliable scheme as a Baseline. The Baseline uses periodic flush based on 5-seconds intervals, where pages with 30 seconds idle time are flushed to the main storage and discarded from PJA, which is the default configuration in operating systems such as Linux [2]. Thus, this scheme reduces the probability of retention failures by providing a small idle time for PJA pages, but imposes significant storage write traffic compared to No-pdflush (as shown in Section 3.3.2).

Failure Rate Analysis
To investigate CoPA in term of reliability, we evaluate it based on different refresh periods and set the time-steps equal to 30 seconds (CoPA-T30), 90 seconds (CoPA-T90), 150 seconds (CoPA-T150), and 300 seconds (CoPA-T300). Fig. 15 shows the failure rate of CoPA and No-pdflush normalized to CoPA-T30. On average, CoPA reduces the probability of failure by three orders of magnitude compared to Nopdflush.
CoPA provides a significant idle time reduction for PJA pages (up to 66.9×) by guaranteeing to refresh each PJA page at least every three time-steps, as shown in Fig. 16. It also provides a predictable PJA idle time to fill the gap between No-pdflush and the baseline approach in term of reliability, as there is no upper bound for PJA page idle time in No-pdflush scheme. There is a considerable difference between the maximum page idle time of different workloads (more than 66 minutes) in No-pdflush, while this value in CoPA differs in few seconds. Therefore, CoPA alleviates the variation of page idleness compared to the schemes that have eliminated periodic flush, and accordingly, provides a reliable scheme without increasing main storage traffic. CoPA can be tuned to provide the same reliability as the Baseline. However, in this case, the trade-off between the number of refreshed pages and failure rate period should be considered, as lower refreshing period leads to higher number of refresh operations.

Response Time
One of the key aspects of an efficient NVB-Buffer management scheme is its impact on the response time of storage subsystem. Although the baseline considerably reduces the probability of retention failure, it significantly increases storage write traffic compared to No-pdflush. However, CoPA not only significantly reduces the probability of retention failures compared to No-pdflush, but also avoids the baseline high storage write traffic. CoPA, as well as improving reliability compared to No-pdflush, aims at preventing the baseline poor write traffic. Fig. 17 shows the response time of each scheme normalized to No-pdflush. The time-step of CoPA is set to 30 seconds, which brings the highest response time (due to higher refreshing operations). CoPA increases response time by an average of 1.1% compared to No pdflush (up to 4.2%),

RELATED WORK & DISCUSSION
Employing NVMs in buffering schemes has already been addressed in prior studies based on two main approaches: 1) storage-aware approach [4,65,66] and 2) storage-unaware approach [2,7,9,16,41,[67][68][69]. The first approach aims at improving the lifetime and performance of SSD-based storage systems using different mechanisms such as dirty page favoring [4] (the mechanism of using policies to prioritize dirty pages in the buffer), overcoming small write problem, and reducing garbage collection frequency [51]. These schemes, however, increase the idle time of dirty pages in the NVM, which increase the probability of retention failure.
Schemes in the second approach employ NVMs as part of the main memory [16,67,68] or log/journal area [2,7,9,41]. As conventional schemes exploit a part of the main storage (HDD or SSD) as the journal area for crash recovery, this approach takes advantage of NVM persistency to reduce storage traffic by committing journaled data to the NVM. Nevertheless, these schemes are optimized toward performance improvement by decreasing storage write traffic, which is achieved by increasing the residency time of dirty pages in the NVM. Therefore, the schemes in the second approach increase the probability of retention failure compared to the conventional schemes, in favor of reducing storage write traffic.
In [70], mwJFS is proposed to address the retention time of Multi-Level Cell (MLC) PCM-based jouranling file systems by employing different write pulse widths (and accordingly retention time). Although expanding the write pulse in order to increase the retention time is a conventional solution in NVMs, and it is also applicable to STT-MRAM, it leads to considerable increase in write latency and power consumption [71]. Moreover, employing the retention monitoring mechanism of mwJFS for STT-MRAM based PJAs increases the probability of read disturbance. CoPA, however, significantly decreases the error rate by reducing the page idle time compared to these schemes without increasing read disturbance probability, while providing a noticeable reduction of storage write traffic compared to the conventional schemes.
CoPA is also applicable to the main memory buffer cache layer. To this end, Operating System (OS) needs to assign a small portion of the main memory to the Awake and Sleepy queues. For example, for an 8GB main memory and 512MB PJA, OS should assign 1.2% of the main memory space to CoPA queues. To employ CoPA for systems with logical journaling, a portion of DRAM equal to the size of NVM should be considered to store the required valid copy of journaled data for Distant Refreshing. Logical journaling needs much less space since the updates to the data pages are buffered. Thus, the required DRAM space for Distant Refreshing compared to the total DRAM space is negligible.

CONCLUSION
Employing STT-MRAM as PJA provides the opportunity for designing efficient NVB-Buffer schemes. The technology downscaling increases the probability of retention failure in STT-MRAM, especially in recent technologies where retention failure becomes the main source of errors. Although existing NVB-Buffer management schemes provide considerable storage write traffic reduction compared to the conventional schemes, they suffer from long and undetermined PJA page idle time. The longer the page idle time, the more the error rate compared to the conventional schemes. This paper proposed an efficient scheme named CoPA, for reducing the probability of retention failure. CoPA utilized an NVB-Buffer-friendly approach named Distant Refreshing, which re-writes the idle PJA pages based on their replica in DRAM no matter it is corrupted or not. CoPA monitors PJA pages using two queues (Awake and Sleepy queues) to distinguish pages with long idle time. CoPA also guarantees a tunable upper bound for maximum idle time of PJA pages. Our evaluations illustrated that CoPA reduces the probability of failure by three orders of magnitude with negligible performance overhead.