{'fraction_non_alphanumeric': 0.028771126442322176, 'fraction_numerical': 0.006055493628174615, 'mean_word_length': 5.213777611381505, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 1, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
PROMPT: Learning Dynamic Resource Allocation Policies for Edge-Network Applications

A growing number of service providers are exploring methods to improve server utilization, reduce power consumption, and reduce total cost of ownership by co-scheduling high-priority latency-critical workloads with best-effort workloads. This practice requires strict resource allocation between workloads to reduce resource contention and maintain Quality of Service (QoS) guarantees. Prior resource allocation works have been shown to improve server utilization under ideal circumstances, yet often compromise QoS guarantees or fail to find valid resource allocations in more dynamic operating environments. Further, prior works are fundamentally reliant upon QoS measurements that can, in practice, exhibit significant transient fluctuations, thus stable control behavior cannot be reliably achieved. In this paper, we propose a novel framework for dynamic resource allocation based on proactive QoS prediction. These predictions help guide a reinforcement-learning-based resource controller towards optimal resource allocations while avoiding transient QoS violations due to fluctuating workload demands. Evaluation shows that the proposed method incurs 4.3x fewer QoS violations, reduces severity of QoS violations by 3.7x, improves best-effort workload performance, and improves overall power efficiency compared with prior work.


Introduction
Performance of user-facing workloads, such as packet processing and machine learning inference, is critical for service providers since even small degradations in quality-of-service (QoS) can lead to significant penalties. As such, these high-priority (HP) workloads are typically allocated sufficient resources to handle the peak load that may be encountered at any point during operation. This practice ensures satisfactory performance at all times, but leads to over-provisioning during periods of lower load. Service providers have sought to ameliorate these adverse effects by opportunistically co-scheduling best effort (BE) workloads, thereby enabling higher average resource utilization. Naive co-scheduling can, however, introduce resource contention between HP and BE workloads that may compromise key QoS metrics (e.g., drop rate or latency) that are integral to the user expectations.
Prior work has demonstrated practical co-scheduling by strictly managing resource allocation for each workload, thus limiting contention. One approach involves static allocation, in which case the HP workload is provided sufficient resources to handle peak load, while the BE workload is given remaining resources. In practice, this approach proves highly inefficient since average HP load can fall significantly below peak load. Incremental improvements are possible by adjusting resources on a time-of-day basis, although deviations from historic averages may still compromise QoS guarantees. For these reasons, recent work has explored dynamic resource allocation as a more adaptable solution. These approaches vary; some exploit application-specific knowledge [1,2], many apply general search-based methods [3][4][5][6][7], and two recent works consider deep reinforcement learning [8,9]. In general, prior works have assumed specific operating environments under which reliable feedback can be obtained with relatively small sampling intervals. When these conditions do not apply, these methods may require unacceptably long sampling intervals and/or provide limited guarantees regarding transient QoS violations. Addressing these fundamental assumptions requires a more sophisticated approach that can predict the effects of changing workload demands and dynamically adjust resources without compromising QoS.
In this paper, we propose Predictive Resource Optimization for Multiple Prioritized Tasks (PROMPT), a novel machine learning framework for dynamic resource allocation. Proactive QoS predictions are provided by several machine learning predictors, each of which is trained on a variety of BE workloads to accurately model the effects of diverse resource contention behaviors. These QoS predictions, along with additional workload performance indicators, are passed to a reinforcement learning model [10] that dynamically allocates resources to meet QoS goals. Together, these components enable rapid resource adjustments that safely exploit periods of low workload demands while minimizing QoS violations. Additionally, while prior works have strictly assumed a fixed set of workloads, PROMPT is designed to support a more general operating environment in which co-scheduled BE workloads may change frequently. Evaluations on a real-world networking platform show that PROMPT incurs 4.3x fewer QoS violations, reduces the severity of QoS violations by 3.7x, improves BE performance, and improves overall power efficiency compared with prior work.

Preliminaries and Related Work
Workload co-scheduling enables optimization of resource utilization, but can also introduce several behaviors that may compromise QoS guarantees. We demonstrate these behaviors by characterizing a critical and prominent network-edge workload -the virtual Broadband Network Gateway (vBNG). This workload is essential to internet service providers (ISPs), as the BNG is the aggregation and access point at the network edge for business, residential and wholesale connectivity. Strict servicelevel agreements (SLAs) and QoS guarantees must be maintained to ensure continuity of service.

QoS Cliffs
Many workloads, even in isolation, exhibit dramatic performance degradation when resources are lowered beyond a critical threshold (i.e., QoS cliff). In a workload co-scheduling environment, these effects become more pronounced, especially since it becomes desirable to allocate as few resources as possible to the HP workload without causing QoS violations. For example, Figure 1(a) presents packet drop rate when co-scheduling a high-demand vBNG workload with a neural network (ResNet) training workload. Here, the vBNG workload is allocated all remaining cache ways and does not have a memory bandwidth restriction. Note that some dropped packets are inevitable due to high demand. Moving from 90% to 80% memory bandwidth restriction for the BE workload has relatively little impact on packet drop rate. A further reduction to 70% memory bandwidth restriction can, however, incur a 10-100x increase in drop rate (for higher BE cache allocations). Similar trends are observed across a wide variety of co-scheduled workloads.

Transient QoS Violations
Prior work has generally focused on HP workloads with latency-based QoS goals, most commonly using the TailBench benchmark suite [11]. When workload demand is fixed, latency is observed to deviate by a relatively small amount, thus reliable QoS measurements can be obtained every  second [9]. Alternative HP workloads, such as those with packet-based QoS goals can experience significant QoS fluctuations, even when both workload demand and resource allocation are held constant. These fluctuations can cause QoS to temporarily degrade below the QoS target, which we refer to as transient QoS violations. Specifically, as shown in Figure 1(b), we observe that the distribution of QoS measurements can exhibit a long upper tail, often over an order of magnitude higher than the average reading. This behavior is exacerbated when targeting strict QoS guarantees since the average reading may approach zero, thus offering minimal information about the worst-case behaviors that may result from a particular resource allocation.

Control Challenges
Dynamic workload demand can significantly impact both the locations of QoS cliffs and the severity of transient QoS violations. As such, the range of acceptable resource allocations can vary significantly over time. Prior work has sought to address this dynamic behavior with two distinct approaches: search-based methods and reinforcement-learning-based methods.
Search-based methods (e.g., CLITE [7]) use QoS measurements as their sole information source to guide both resource allocation sampling and control decisions. This online search can cause QoS violations, so these methods typically sample just 10-20 possible options, each for a short period (typically one second). We find, however, that reliable estimates of worst-case behaviors require far longer sampling periods. In fact, based on drop-rate data in Figure 1(b), a sampling period on the order of 100 seconds may be required for strict QoS guarantees. Further, the entire search must be repeated every time workload demand changes significantly since we do not know if previously acceptable resource allocations might now cause QoS violations.
Reinforcement-learning-based methods [8,9], in contrast, require a longer initial training period, but can use additional information sources to learn more general control strategies that remain applicable when workload demands change. Regardless, only one prior work has considered HP workloads with packet-based QoS goals [8] and no prior work has addressed the challenges created by transient QoS violations. We discuss these challenges further in the following section.

Predictive Resource Optimization for Multiple Prioritized Tasks
In this work, we propose a reinforcement-learning-based dynamic resource allocation framework -Predictive Resource Optimization for Multiple Prioritized Tasks (PROMPT). Given the state of a system (e.g., several workloads being co-scheduled), PROMPT is trained to predict the resource allocation that minimizes QoS violations for the HP workload(s) while optimizing BE workload(s) performance and improving overall power efficiency. Whereas prior works have strictly assumed a fixed set of workloads, PROMPT is designed for a more general operating environment and supports co-scheduling of new BE workloads not encountered during policy training. In doing so, we address a limitation in prior work and enable stronger QoS guarantees in more realistic operating environments.
The problem specification described above suggests a straightforward application of reinforcement learning with an intuitive reward structure; positive rewards for higher BE workload performance with lower power consumption and negative penalties when the measured QoS exceeds the target QoS (i.e., a violation). In practice, however, this approach can be problematic. Transient QoS violations are caused by deviations in workload execution that can, in part, be attributed to stochastic events involving branch mispredictions, cache misses, system interrupts, etc. This stochastic behavior persists even when resource allocation and workload demand are held constant. As such, it is possible that we may not observe transient QoS violations even when exploring risky resource allocation policies. A dynamic control policy trained with finite knowledge of these transient behaviors may mislearn that risky resource allocations are reliable when, in fact, these policies achieve poor worstcase QoS. We address this issue by training a model to predict the worst-case QoS given the system state and current resource allocation policy. This worst-case QoS estimate is then used to compute the reward.
Our overall pipeline is illustrated in Figure 2. During each sampling interval, the telemetry module collects runtime information for all workloads. This information is then passed to the QoS prediction model, which predicts the worst-case QoS value (e.g., packet drop rate) for each HP workload, assuming no change in resource allocation. Finally, these QoS predictions, along with the runtime information, are provided to a reinforcement-learning-based controller, which determines an appropriate resource allocation for the next sampling interval. This framework supports resource allocation between any number of HP and BE workloads provided that there are sufficient resources to meet QoS goals for all HP workloads.

Telemetry: Measuring System State
Modern CPUs implement dedicated counters for architectural events such as cache misses, memory requests, etc. Intuitively, this information can be used to identify deviations from ideal execution behaviors and thereby predict transient QoS behaviors and guide resource allocations. Unfortunately, due to hardware limitations, it is only possible to measure a small subset of these counters without degrading accuracy and increasing overhead, so feature selection is required.
Prior works often select features manually based on domain knowledge. Naturally, this approach introduces substantial human effort while potentially missing useful counters. As an alternative, we apply a hierarchical method that sequentially trims the counter list and uses domain knowledge near the end, thus greatly simplifying human effort. These steps are as follows: 1. We sample each performance counter for one second while running an arbitrary co-scheduling setup. Events that are rare (and therefore unreliable) or exhibit low variance are eliminated. 2. The reduced list of performance counters is sampled for a longer period across a large range of co-scheduling configurations. We then apply a method similar to Boruta [12], which eliminates all counters that are equally (or less) informative than randomly permuted readings. 3. Some performance events exhibit orders-of-magnitude shift in values when co-scheduling different BE workloads. Including these events would likely lead to poor performance when we consider generalization to BE workloads not observed during policy training. We therefore drop counters with this behavior. 4. We use domain knowledge to eliminate counters that measure exceptionally specific scenarios or counters that are not easily understood by human experts. 5. Finally, we apply step-wise selection methods [13] to obtain the desired number of counters.

Predicting Worst-Case QoS
Given the system state at time t represented as a vector s t of counter values, we would like a model f QoS (s t ) to predict the worst-case HP QoS q t that could be expected in the next time step. We define q t as the worst QoS value observed within a 100 second window after time t while keeping the resource allocation constant. As shown in Figure 1(b), this is typically enough time to observe transient QoS violations that may occur. Transient QoS violations are by definition rare and often extreme events, so directly fitting a regression model can lead to poor worst-case QoS predictions. We mitigate these issues using a two-level prediction setup as depicted in Figure 3(a). Hardware performance counter values are first passed to a binary classifier that predicts whether the worst-case QoS is above or below a specified threshold. This prediction is then used to select between two regressors. The fine-grained regressor is trained on the critical range of QoS values in which the system will ideally operate (e.g., zero up to the threshold value). The coarse-grained regressor is, on the other hand, trained on the full range of QoS values. By separating these predictors, the fine-grained regressor can focus on contention behaviors that are most likely to occur during typical framework execution, while the coarse-grained regressor can predict more extreme behaviors that may occur due to rapid changes in workload demands.
Both the classifier and regressors are implemented as boosted decision trees. These models are intended to be robust against changes in resource contention behaviors, so training data is gathered across a wide variety of co-scheduling configuration (i.e., HP workload demands, BE workloads, and resource allocations). Performance counter measurements are aggregated across all cores assigned to  each HP workload. The overall QoS prediction model (f QoS (s t )) is trained offline and, by necessity, prior to policy training.

Dynamic Resource Allocation Controller
We adopt a resource controller based on deep reinforcement learning. This controller learns a policy π : S → A that maps system states to resource allocation actions.

Model Architecture
The resource controller uses a deep Q-network with an action-branching architecture [14] as depicted in Figure 3(b). This architecture features a shared representation layer, followed by distinct action branches, one for each resource control knob. Splitting these action dimensions allows the number of network outputs to grow linearly, rather than combinatorially, with respect to the action space, thus simplifying control complexity significantly when allocating more than a few distinct resources.

State, Action, and Reward Specification
Representation for state, action, and reward is built around a generalized operating environment in which BE workloads may change at any time, thus avoiding frequent and impractical re-training. This generalization introduces a variety of challenges that have not been addressed by prior work.
State Representation: State for the reinforcement learning inherently requires similar information to the QoS predictors, thus we re-use the performance counters discussed earlier. This re-use is made possible by eliminating performance counters with high distributional shift (as described in Section 3.1). We further augment the state vector with the predicted QoS. The performance counters act as low-level indicators for changes in resource changes and potential deviations in workload execution that may negatively impact QoS. Conversely, the predicted QoS offers a high-level perspective and informs the controller about its margin from the QoS target.
Action Space: Action space specification diverges substantially from prior work. Specifically, prior work [9] assumed a fixed set of workloads, so the model directly selects the resource allocation for all workloads (HP and BE). With this setup, the model becomes dependent upon the particular BE workload(s) used during training since there is no guarantee that the state (i.e., performance event values) for a new BE workload will remain similar to the state values observed during training. We could try to mitigate this issue by training with many different co-scheduled workloads, but this solution would quickly become impractical when we consider mixes of BE workloads. Consequently, frequent retraining could still be required to maintain strict QoS guarantees. PROMPT addresses these issues by indirectly specifying BE resource allocation based on HP resource allocation.
We consider resource allocations defined by four resources: last-level cache (LLC), memory bandwidth (MBW), core frequency (CF), and uncore frequency (UCF). Given that the number of BE workloads may change dynamically, we predict allocations only for the HP workload, then determine BE allocations based on HP allocations. We illustrate an example configuration in Figure 4. Specifically, • LLC can be strictly divided such that no contention is possible, thus all LLC resources not selected for HP workload(s) are given to BE workload(s). We specify a minimum of two LLC units per workload and are free to allocate the remaining units. • MBW is not strictly divided and is instead restricted on a per-workload basis. As a result, contention can be reduced, but not eliminated. Regardless, when co-scheduling, we typically want to restrict either the HP workload (at low HP demand) or the BE workload (at high HP demand  the lower end of the range restrict HP memory bandwidth while leaving BE memory bandwidth nearly unrestricted. Conversely, values in the upper end of the range heavily restrict BE memory bandwidth while leaving HP memory bandwidth unrestricted. • CF is similar to MBW in that a different setting is possible for all HP cores and BE cores, but also introduces power considerations. It may be possible to reduce core frequency for both the HP and BE workload(s) in order to improve power efficiency. As such, we separate these dimensions and select a separate value for HP core frequency (HPCF) and BE core frequency (BECF). • UCF must take the same value for all CPU cores, thus we apply the HP workload setting to all HP/BE cores.
Allocation options for each resource depend on the target hardware. In our case, we have 8 possible LLC, 11 MBW, 5 HPCF, 5 BECF, and 7 UCF settings, so resource allocation actions are given as We avoid combinatorial explosion in the action space by independently predicting each resource, such that a t = {a LLC , a M BW , a HP CF , a BECF , a U CF }, where each sub-action is chosen by a separate prediction head in our network.
We have, thus far, discussed a single HP workload for simplicity. Nevertheless, PROMPT is readily extensible to multiple HP workloads by replicating the action space for each HP workload and adding constraints. Specifically, each HP workload could select a number of LLC units, leaving remaining units for the BE workloads. HP MBW would also be separately selected by all HP workloads, while BE MBW would use the lowest setting chosen by HP workloads, essentially restricting the BE to the lowest contention level specified by any HP workloads. The same strategy applies for HPCF and BECF. Finally, UCF would be set to the maximum selected by any HP workload.
Reward Structure: We specify rewards based on the predicted QoS as opposed to the measured QoS used in prior work. In almost all cases, the predicted maximum QoS is closer to the true maximum QoS than estimates based on a small number of QoS measurements in the sample period. As such, rewards based on current samples are far more likely to grant inappropriate rewards for resource allocations that would allow QoS violations. Further, rewards given by current samples may be inconsistent due to natural fluctuations, thus hindering learning.
Negative rewards are given as penalties for undesirable actions that cause the predicted QoS (drop rate) to degrade past the target QoS. Specifically, as shown in Equation 2, these penalties are based on the ratio of the predicted QoS divided by the target QoS. In some cases, poor action selection can lead to the predicted QoS being several orders-of-magnitude greater than the target QoS. As such, we log transform this ratio to obtain a relative indicator of violation severity. We further clip this log transformed ratio at β = 3 to improve learning stability in cases where the predicted QoS is more than 1000x the target QoS.
r − = − min log predicted_qos target_qos , β (predicted_qos > target_qos) (2) Positive rewards are given when the predicted QoS is less than the target QoS. In this case, we consider our secondary goals of optimizing BE performance while reducing power consumption. These goals are specified in Equation 3. BE performance, denoted as f perf can be increased by increasing LLC, MBW, BE core frequency, or uncore frequency. Power consumption, denoted as f power , can be reduced by decreasing HP core frequency, BE core frequency, or uncore frequency. We balance these goals with the parameter α, which we set to be 0.8, thus favoring BE performance.
Both f perf and f power are normalized [0,1].

Evaluation
Baseline Comparisons: Comparison against prior work is problematic for several reasons. In particular, prior work has focused on operating environments with static workload demands. Searchbased methods (e.g., CLITE [7]) can incur unacceptably large QoS violations during their search process, which may take at least 1-2 minutes due to transient QoS behaviors. Prior works based on deep reinforcement learning are limited in their design; one work assumed a single HP/BE workload so did not include performance event information in the state [8] while the other work does not even support BE workloads [9]. We therefore evaluate our proposed framework, PROMPT, against a heavily-modified version of the closest prior work, Twig [9]. Our version is referred to as Twig+ and should be regarded as an altogether new work due to the significant modifications that were required. CLITE supports multiple HP/BE workloads, although modifications were required to support additional resources, latency-based QoS, and power goals.
Workloads: We select the vBNG as our HP workload for training/testing due to its prominent role in networking applications [15]. BE workloads (22 total) are selected from industry-standard benchmark suites (SPEC CPU2006, SPEC CPU2017, and PARSEC) as well as several machine learning workloads. Selection criteria for BE workloads is based on sensitivity to resource allocation while maintaining a diverse set of workloads from many domains.
Train/Test Setup: Training data for the QoS predictor in PROMPT consists of roughly 500 labeled examples for each of nine co-scheduled workload mixes (vBNG + one BE). QoS prediction accuracy is tested using k-fold cross validation in which each fold comprises the 500 examples from a particular co-scheduled workload mix. Similarly, when QoS prediction is used in PROMPT during evaluation, we use the QoS predictor model that is trained without any data from the currently co-scheduled BE workload(s). The reinforcement learning model in both Twig+ and PROMPT is trained with STREAM [16] as the only BE workload 2 , meaning that neither model has prior knowledge about the 22 BE workloads used in evaluation.

Results
QoS Prediction Accuracy: We evaluate our QoS predictor against several baseline prediction configurations, as shown in Table 1. These baseline configurations remove either the general counters (so use IPC only), use a single-level QoS prediction setup with the coarse-grained regressor only (see Figure 3(a)), or both. For this comparison, we set the binary classifier threshold to be 250 packets dropped per second. The simplest setup (configuration 1) achieves a mean-average-error (MAE) of 75.15 in the critical region, which provides limited improvement over individual QoS measurements (given a scenario similar to that in Figure 1(b)). Adding two-level prediction (configuration 2) significantly reduces regression error in the critical range, even when using only IPC; resource contention behaviors tend to be more similar in this critical range, thus simplifying the prediction task for the fine-grained regressor. Alternatively, adding all features to the single-level predictor (configuration 3) further reduces regression error to 26.52 packets dropped per second, indicating that IPC alone cannot accurately predict QoS. Finally, adding all features to the two-level setup (configuration 4) provides the best performance in all metrics except for MAE of the full-range regressor, which is not affected. Practical benefits of two-level prediction can be quantified by the number of large mispredictions, where regression error was greater than 50 packets per second. The single-level predictor (configuration 3) makes 254 large mispredictions while our two-level predictor (configuration 4) makes just 136 large mispredictions.  Practical Control Capabilities: We began by testing whether CLITE [7], which represents the prior state-of-the-art in search-based methods, would be suitable for dynamic operating environments with strict QoS guarantees. For this test, we co-scheduled the HP vBNG workload alongside a machine learning (ResNet) training workload which is known to cause modest resource contention. HP workload demand was based on a traffic trace gathered from a real-world operating environment. The total test duration (four hours) was chosen such that CLITE could always successfully finish sampling (i.e., changes in workload demand during sampling were negligible). Even with this relatively simple operating environment, we observed dramatic differences in QoS violations between CLITE and PROMPT. Specifically, as shown in Table 2, the online search performed by CLITE caused 14 occurrences (roughly 42 seconds) during which over one million packets were dropped per second, roughly 10% of the total injected traffic. These periods could cause substantial interruptions in real-time services and are therefore unacceptable in many operating environments. The worst QoS violations incurred by PROMPT, on the order of 20,000 packets dropped per second, were caused by a temporary drop in HP MBW or HP core frequency during a period of high HP workload demand.
Notably, the actions leading to these violations are observed to be extreme outliers. In other words, practically all nearby resource allocation decisions were appropriate (and tended to be identical). Consequently, these violations could likely be eliminated by a time-window average over recent resource allocation actions. In contrast, the systematic violations incurred by CLITE during sampling cannot be easily mitigated. This issue would be exacerbated by quicker changes in workload demands since CLITE would need to re-sample more often and could even fail to determine an appropriate resource allocation if re-sampling takes too long. Given these limitations, we do not include CLITE in further testing.

Resource Allocation Behaviors:
We observe that QoS predictions (PROMPT), rather than QoS measurements (Twig+), improve the consistency of resource control decisions. As shown in Figure  5, PROMPT correctly identifies the initial period of low HP load, during which few resource are allocated to the HP workload. As HP load rises, PROMPT briefly fluctuates between several RDT allocation options and then settles on a relatively stable configuration. PROMPT then makes minor BE core frequency adjustments, when necessary, to mitigate transient QoS transients. Finally, when  load drops, resources are again taken from the HP workload (and given to the BE workload). In contrast, Twig+ exhibits relatively inconsistent control behavior and tends to make more drastic resource allocation changes. We also note that most QoS violations incurred by Twig+ (e.g., readings 83-84) are caused by complex action combinations in which Twig+ simultaneously adjusts every resource without appropriately considering the resulting instability. The worst violations incurred by PROMPT across all workloads is caused by the behavior described in the previous subsection. Again, since these violations tend to be caused by actions that are clear outliers, they can likely be mitigated using a time-window average while the more complex violations caused by Twig+ likely could not.
Resource Allocation Efficacy: We evaluate the control strategies learned by PROMPT and Twig+ when co-scheduling the vBNG workload with each of 22 BE workloads not seen during reinforcement learning model training. Select results are given in Table 3 (see appendix for full results). First, we compare the average number of QoS violations incurred by each control framework. Only 0.7% of actions made by PROMPT result in QoS violations, compared to 2.97% for Twig+ (a 4.3x increase). These percentages may seem small, but are crucial in industry applications with strict service level agreements. In practice, the higher proportion of QoS violations incurred by Twig+ could lead to substantial penalties, potentially up to millions of dollars. Next, we compare the average QoS tardiness 3 , which quantifies the severity of QoS violations. The QoS violations incurred by Twig+ are 9.64x the target (864% above) whereas the QoS violations incurred by PROMPT are just 3.34x the target (234% above). In other words, the QoS violations incurred by Twig+ are 3.7x more severe. We also observe that PROMPT improves BE performance (measured as instructions per second) by an average of 6% (and up to 11.4%) compared with Twig+. Finally, PROMPT improves overall power efficiency by an average of 4% compared with Twig+. PROMPT therefore offers better results in all metrics by incurring fewer and less severe QoS violations while still improving BE performance.
Controller Overhead: Twig+ and PROMPT use an identical reinforcement learning model, thus execution time is identical except for overhead due to QoS prediction. Without QoS prediction, each control interval requires approximately 12ms when running on one core. QoS prediction incurs an additional 1ms overhead. This additional overhead is easily outweighed by the gains in BE performance discussed above. Note that both Twig+ and PROMPT are executed every three seconds during evaluation, thus total execution overhead is around 0.5%.

Conclusion
Co-scheduling of high-priority and best-effort workloads, enabled by dynamic resource allocation, provides a promising option to improve server utilization and reduce total cost of ownership. Prior works, however, have assumed largely static operating conditions in which workloads do not change and workload demands only rarely change. Further, prior works are fundamentally reliant upon QoS measurements so provide limited protection against transient QoS violations. PROMPT addresses these issues with a generalized framework that predicts, rather than measures, the worst-case QoS and supports co-scheduling with changing BE workloads. Evaluation shows that the proposed framework incurs 4.3x fewer QoS violations, reduces the severity of QoS violations by 3.7x, improves BE performance, and improves power efficiency compared with prior work.