{'fraction_non_alphanumeric': 0.03403759467664285, 'fraction_numerical': 0.0063782159394069485, 'mean_word_length': 4.739528335093277, 'pattern_counts': {'":': 0, '<': 3, '<?xml version=': 0, '>': 2, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 5, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Efficient Utility Function Learning for Multi-Objective Parameter Optimization with Prior Knowledge

The current state-of-the-art in multi-objective optimization assumes either a given utility function, learns a utility function interactively or tries to determine the complete Pareto front, requiring a post elicitation of the preferred result. However, result elicitation in real world problems is often based on implicit and explicit expert knowledge, making it difficult to define a utility function, whereas interactive learning or post elicitation requires repeated and expensive expert involvement. To mitigate this, we learn a utility function offline, using expert knowledge by means of preference learning. In contrast to other works, we do not only use (pairwise) result preferences, but also coarse information about the utility function space. This enables us to improve the utility function estimate, especially when using very few results. Additionally, we model the occurring uncertainties in the utility function learning task and propagate them through the whole optimization chain. Our method to learn a utility function eliminates the need of repeated expert involvement while still leading to high-quality results. We show the sample efficiency and quality gains of the proposed method in 4 domains, especially in cases where the surrogate utility function is not able to exactly capture the true expert utility function. We also show that to obtain good results, it is important to consider the induced uncertainties and analyze the effect of biased samples, which is a common problem in real world domains.


Introduction
In the real world, we often encounter parameter optimization problems when designing new products. This is still in many cases a manual effort, but the efficiency of Bayesian Optimization [19] makes it interesting to try to replace these processes with autonomous approaches. These parameter optimization problems are usually subject to multiple output dimensions. Often, these dimensions are assumed to be multiple objectives, enabling the computation of a Pareto front and performing post elicitation of a best result. However, the post elicitation requires expert knowledge, preventing a fully autonomous application. Furthermore, Pareto front optimization is computationally very costly, as it is required to optimize for all objectives. An alternative is to learn a utility function, defining a scalar optimization target, based on the multi-dimensional output. With the availability of a scalar target, fully autonomous optimization is possible, but defining such a utility function is difficult. Usually, domain experts are not able to define such a function in advance, as they are not able to map their implicit knowledge to an explicit, precise and complete quantitative utility function. Therefore, comparative approaches are better suited for capturing expert knowledge. [16] Preferential Bayesian Optimization (PBO) [11] can be used to iteratively learn such a utility function, based on pairwise comparisons, assuming (nearly) unlimited access to experts, which is a valuable resource. Therefore, the expert knowledge should be captured beforehand and generalized, as far as possible. To this end, we use additional sources of information that are usually not considered: Experts can give general statements, like "a lower value is better", which is not only a preference indication, but also limits the space of viable utility functions. Therefore, we use parametric function spaces that are partially predefined, based on expert knowledge. Differences between the real (unknown) expert utility function and the restricted, parametric function space make it relevant to not only learn a best estimate of the utility function, but to model potentially complex uncertainty structures. Therefore, we employ an empirical approximation of the full utility function posterior, wrt. the expert preferences. Additionally, we ensure that the utility function is still interpretable and can be optimized efficiently. We also consider that historical examples from real world processes are usually not unbiased and analyze the effect. Our main contributions are: -A high-efficiency method for learning utility functions for Bayesian optimization, based on prior knowledge. -A method for integrating prior knowledge into parametric function spaces.
-Showing the relevance of considering uncertainty, when dealing with mismatched utility function spaces. -Analysing the effect of using biased samples for utility function learning.
In the following Section 2, we introduce our problem setting and discuss related work in Sec. 3. Our approach is explained in Section 4, followed by the empirical evaluation (Sec. 5) and the conclusion in Sec. 6.

Problem Setting
We assume a black box system f : X → Y , with inputs x ∈ X ⊂ R Xn and outputs in Y ⊂ R Yn . The optimization target is an unknown expert evaluation system 3 : e : Y → R, meaning we try to find the solution to the maximization problem arg max x e(f (x)). Additionally, we assume examples H ∈ X × Y and a set of preferences ζ : H × H ∈ [≻, ≺, =], indicating a preference wrt. the examples. It should be noted, preferences are a generalization of several different forms of feedback, as it is possible to convert ranking rules as well as ordinal feedback to preferences. In practise, obtaining expert rules is more sensible than pairwise comparisons, as this requires less effort.
In our setting, H may be obtained from different subspaces of X × Y . This means, each h ∈ H has some x i ∈ x already fixed to a specific value. With x i as the i-th value in x. This relates to the fact that real world results are usually obtained by applying a process to variants of the same problem. E.g. different applications behaving differently, but still are subject to the same evaluation criteria. The fixed values are implicitly defined by the product design or other predetermined factors. However, it is important that all historical examples and the current optimization target are subject to the same evaluation function e, to be able to generalize over these.

Related Work
Our work is embedded in the area of Preferential Bayesian Optimization (PBO) [11], capturing expert knowledge with means of preference learning, for approximating the expert evaluation function. Commonly, PBO is applied in a setting where preferences are obtained wrt. to online obtained examples, allowing to generate and select samples that are most informative for the learning process. This means, historical examples are not considered and it is assumed that a domain expert is available during the runtime of the optimizer. Therefore, most work focus on reducing the required expert involvement, by maximizing the information contained in each preference feedback or by enabling batch-acquisition of preference feedback. Information contained in each preference can be maximized by using suitable acquisition functions for Bayesian Optimization [2]. We do not assume continued availability of the expert, but PBO-like approaches can be used in combination with our method, for online refinement of the utility function.
Batch-acquisition of preferences allows the expert to only give feedback a few times during the optimization process [20]. Our setting is comparable, in the sense that we only assume a single batch of preference obtained at the beginning. Inline with fully online settings of PBO, we also assume that the samples of this batch are obtained from the same problem distribution as the optimization target.
Capturing the uncertainty of the utility function and considering it for the BO procedure was also considered in [2], but with a focus on improving the efficiency of preference queries. Additionally, we use a Markov Chain Monte Carlo sampling method, instead of plain Monte Carlo. We show that this is also relevant for obtaining better estimates in the setting of misspecified utility functions.
Multi-objective Bayesian Optimization [9] (MO-BO) is also related to our problem setup, in case the output dimensions can be interpreted as objective and also may use preferences [1] However, the result of these approaches is a (Pareto) set of solutions, requiring post elicitation a solution, by a human expert. An alternative is to learn the trade-off between the different objectives [18], but this disregards dependencies between the objectives, because the utility function is considered to be a weighted sum of the output dimensions. It is also only applicable if it is possible to directly interpret output dimensions as objectives.
In general, to the authors knowledge, this is the first method using domain expert information for restricting the utility function space, as well as combining all other relevant elements (preference learning, uncertainty estimation, batch learning), for obtaining a MO-BO that can be applied to improve, existing, manual parameter optimization processes.
Please also note that Preference-based Reinforcement Learning (PbRL) may also use preference-based utility function learning [22]. This setting is more common than PBO, as RL greatly suffers from the problems of manually defining a utility (or reward) function. Therefore, several advances from PbRL can be applied to PBO, as long as they disregard the sequential property of PbRL. However, most of the approaches to PbRL do not consider utility function uncertainty or approximate it with computationally cheap methods.

Approach
Our approach captures e by a parametric utility function G : P × Y → R, with P ⊂ R Pn as the space of free parameters. The basic principle is to use preferencebased utility function learning [8] for learning p opt ∈ P , defining g opt ∈ G as an approximation for e. For G, we use an aggregation of monotonic objective functions, enabling the incorporation of prior knowledge while maintaining interpretability (c.f. Sec. 4.1). In contrast to other approaches, we do not assume that it is possible to directly interpret the outputs Y as linear objective functions. This is implicitly assumed when reducing the problem to finding the weighing factors for the Pareto front induced by Y .
The parameters of G are learned via offline preference-learning, based on historical examples and expert knowledge. As we need to assume e to be more complex than our surrogate space G allows, we do not learn a single utility function g opt (y) = g(p opt , y), but recover the full posterior distribution p opt ∼ Pr(P |ζ, H) over utility function parameters, as in [2]. This enables us to capture dependencies between parameters, as well as uncertainties. Therefore, e may contain interactions between elements of Y that can not be directly modeled by G.

Parametric Utility Functions
Most modern approaches focus on utility functions with a high expressiveness, like Gaussian Processes [5] or neural networks [14]. Due to their high flexibility, these function spaces usually include a function that can act as an reliable proxy for the unknown expert evaluation systems. However, the drawback is that a high amount of samples is required for learning such a function. Additionally, it disregards that it is often possible to restrict the utility function space, based on expert knowledge. We capture each element of the output space y i ∈ y with a scalar utility function u i (y i ) and perform a linear aggregation over these possibly nonlinear functions. This allows the practitioner to define restrictions for each output dimension independently. In our setting, we assume monotony of these elements, but other functions can be learned and optimized with the same approach.
Our monotonic utility function is defined as (index i is disregarded): ensuring u(y) ∈ [0, 1] This is relevant for interpretability, as it allows to define minimal and maximal values. For details, see Section 4.3. In this term, the free parameters are: does define these points only implicitly by u(y) = 0 and u(y) =. These points depend on a, b and pw ( Fig. 4.1). This has several disadvantages: -The value range for u(y) can be smaller 1, violating our interpretability assumption (Sec. 4.3). -Separating a defines if the function is in-or decreasing as well as the steepnes, preventing to determine this properties indepdently. Therefore, we can not fix the known factors (usually the direction), based on prior knowledge. -Sensible value ranges for b depend on the range of y.
-Reasonable priors for a and b are difficult to define, which is important because the priors can have substantial effect in a small data regime.
Please note that one should not optimize y min and y max , but predefined it -either by expert knowledge or by observed min/max values. An optimization problem considering these parameters is computational difficult, because it requires an additional constraint y min < y max which invalidates 50% of the search space. Furthermore, setting the priors for both parameters to the same value induces a bias y min ≈ y max , leading to unreasonable steep functions in the absence of sufficient data. Of course, different formulations that do not suffer from these problems are also possible. However, it is also relevant to consider that a given formulation should not define an optimization space with invalid parameter combinations or different parameters defining in the same function. In both cases, the parameters are very hard to learn due to constrained search spaces or non unique-solutions. E.g., therefore it is not sensible to enforce max u(y) = 1 via constraints.
Our complete utility function space is with p = (p T 0 , . . . , p T N ) T and w i ∈ R + . p i are function parameters b, d, p, m, independently for each function and w i are the weights for each function. As u i is already encoding the expert preferences, a linear form is sufficient for only capturing the tradeoff between different encodings and therefore preferences. Note, that we are considering a maximization problem, meaning a utility function enconding a preference for lower values, should be decreasing (m = −1). As we are not assuming any form (linear, polynomial, ...) or properties (convex, monotonic, . . . ), for the utility function term(s), other definitions are also viable and can be combined. As example, for elements in Y , that can be considered a classification output, we can use an F β score [3] with β as a learnable parameter.

Preference Learning
Preferences describe the relation (better, worse, equal) between elements of H, based on e. E.g. e(h 0 ) > e(h 1 ) ≈ h 0 ≻ h 1 . This preference r 0 ∈ ζ can be turned into a probabilistic fulfillment function Pr(p|r 0 ) = sigmoid(g(p, y 0 ) − g(p, y 1 )) → [0, 1] with y i as the output observed by example h i [4]. Hence, we can find the optima utility function parameters by the negative log likelihood: with y i0 and y i1 as the preferred (and dominated) examples of preference r i ∈ ζ. However, we do not solve this minimization problem, as this will usually overfit, given the limited amount of training data available in real world scenarios. Furthermore, usually e ∈ G, because experts are usually only able to describe simplified versions of their assumptions, while still considering all dependencies and exceptions. Hence, we need to assume that utility function parameters p ∈ P are interdependent, making it relevant to capture uncertainty and co-variances. We define a full Bayesian posterior distribution over utility functions instead of the commonly used, scalar g(p opt , f (x)) with p opt = arg max p Pr(p|ζ, H) and compute an approximation for Pr(p|ζ, H): by No-U-Turn sampling [12]. p opt is then a set of samples of p ∈ P , approximating the full posterior distribution. We use potential scale reduction [10](PSR) as criteria for terminating the sampling procedure.

Interpretability
As the utility function is intended to be a replacement for an expert evaluation system, it is also important that experts are able to understand the utility function. Hence, a linear aggregation of independent utility terms is a reasonable approach, because the experts can now analyze each term independently. Furthermore, this enables us to fulfill another condition: Usually, humans assume to obtain a utility value from a bounded space, e.g. G : P × Y ∈ [0, 1], as this allows to derive notations of (relatively) good or bad. The single terms already fulfill this assumption (see Eq. 1), hence we only need to ensure that w 1 = 1 and w i ∈ R + . Note, this not restricting the function space, because the w i < 0 is equivalent to setting the direction parameter m = −1 and ranking functions are scale invariant. However, adding the normalizing constraint to the utility function parameter space G makes optimization very difficult, as most parameter combinations are now invalid (c.f. Sec. 4.1). Additionally, the optimal values also depend on the preference fulfillment function (Eq. 4), because the sigmoid term Eq. 4 is not invariant to scaling. Hence, we make use of the linear form of Eq. 3. We learn w without normalizing constraints and scale to w 1 = 1 afterwards.

Optimization
Usually, a scalarized multi-object optimization has the following target: As we assume that e is not known, a straightforward target would be withp opt as the posterior sample mean. However, this results in similar problems as with the maximum likelihood estimate Eq.4, as we only exchanged the point estimator maximum likelihood estimator p opt with posterior meanp opt . Therefore, we optimize withḡ(p opt , f (x)) as the mean of the utility. This turns our optimization target into the full Bayesian mean estimate Pr(x|ζ). Please noteḡ(p opt , f (x)) = g(p opt , f (x)) is only true in the case of linear utility functions, and not for more complex utility functions (c.f. Sec. 4.1)

Ethical Implications
In general, our approach is intended to work in cooperation with humans. Theoretically, the system can work fully autonomously, but this is not advisable, because of two factors: 1) Usually, it is assumed that a fully autonomous system always produces optimal results, but this can not be guaranteed; 2) the learned utility function will not be optimal for any new product variant, because of different customer expectations or special variants (outliers). Consequently, a human is still required for validating the computed solution. Hence, we only automatize the repetitive part of finding a good solution. However, in the cooperative as well as the fully autonomous setting, the required amount of manual labor is substantially reduced, which may lead to redundancies or the requirement to re-skill the existing labor force.
As any system that learns from human behavior, the learning of an expert utility function may reflect and reinforce human biases and discriminatory practices. Although, we here consider engineering systems only, it is possible to apply our methods in settings where humans and their behavior are evaluated. The strictest care must then be taken to ensure that no conscious or unconscious biases on part of the expert(s) are learned by the system. This will require explicit evaluation of the learned utility function to evaluate it with respect to variables such as gender, race, age, disability, and others.

Evaluation
We evaluate our approach on the multi-objective problems ZDT3 [23], DTLZ2 [7], car side impact [13] (CAR) and water resource planing [21] (WATER), with ZDT3 and DTLZ2 as classic benchmark problems and CAR and WATER as real-world surrogate problems. It should also be noted, CAR and WATER are constraint MO problems. See Table 1 for an overview of the according problem complexities. For evaluating our system, we assume a known expert utility function e and create artificial, historical results H. For a realistic setting, we assume that multiple variants of the same problem have been encountered in the past, as described in Sec. 2, and use a single sample from each of the encountered variants. Therefore, we generate N examples, by fixing x 0 to N different values and randomly sampling the remaining x i ∈ x. Our pairwise preferences are obtained by ζ = ∀h 0 , h 1 ∈ H × H : ½[e(h 0 ) > e(h 1 )], excluding symmetry. This mean, we compare all samples h ∈ H once, and obtain N · (N − 1)/2 preferences. Table 2 defines the expert functions e. We induce dependencies between output dimensions by adding a multiplicative term. This can not be exactly captured by G. Additionally, some of the output dimensions are truncated, meaning the expert is indifferent if the output value reaches at least some minimal level. This can not be captured by a linear utility function. None of the function spaces can correctly capture the quadratic term, as we set pw = 1. All values f i are normalized using the given bounds, that are determined by the problem definition or a large result sample. Terms including 2 · min 0.5 relate to the truncation (at 50%).
In general, we differentiate 4 utility function spaces: Linear, Adaptable, Informed and RBF Kernel. Linear is a baseline setting, where G is defined as space of linear functions G(y) = yi∈y w i u s (y i ) on the normalized output values u s (y i ) (c.f. Eq. 1). This setting assumes that an imprecise approximation of e is preferred, because the lower number of free parameters (only w) can be inferred easily, even given very few examples. The RBF Kernel settings relates to the idea, that it is most important to use expressive utility function spaces and defines G as the space of radial basis functions [15]. Informed use the utility function space as defined in Sec. 4.1 with y min , y max and m already set, based on  the information from the expert (c.f. Tbl. 2), e.g. for ZDT3 f 2 : m = 1, y min = 0, y max = 4.0. Adaptable uses the same function space, but m is learned, y min , y max set to the general bounds (e.g. for ZDT3 f 2 : y min = 0, y max = 8.0) and the interpretability normalization (Sec. 4.3) is not applied. Therefore, Adaptable can learn the same utility function as Informed, but does not have access to the explicit expert information (m, y min , y max ) and is not interpretable. Note, the difference in y min , y max can be resolved by adapting b and d. The Informed and Adaptable settings include dependencies between the function parameters. Furthermore, we compare 3 different learning settings: Max, Mean and Dist. Max and Mean are the utility functions derived from the maximum likelihood estimate p opt (Eq. 4) and the mean estimatep opt (Eq. 8) for the utility function parameters. Dist containes empirical samples from the full posterior distribution p opt (Eq. 6). The RBF Kernel utility function is learned using an SVM [15] 4 and a Gaussian Process (GP) [6] 5 . For a fair comparison, the RBF Kernel methods also use normalized values, according to the ranges from Tbl.2.

Surrogate Utility Function
First, we will show, that our system is able to recover the (unknown) expert function e with few examples. We compare the different utility function spaces G and learning settings for obtaining g and evaluate the difference between the expert function e and the learned utility function. We always run the MCMC procedure introduced in Section 4.2 till PSR < 1.1 or 1M samples are generated, using 4 parallel sampling chains. We average the obtained samples for the Mean estimate or randomly select 20k elements as Dist estimate. Max is obtained by selecting a sample according to Eq. 4 and apply a few steps of gradient decent optimization till convergence. Directly comparing the utility functions is difficult, as they are subject to different parameter spaces and the optimization target arg min x (c.f. 4.4) is invariant to several transformations, like scaling or translation. Therefore, we compute the ranking error of g, when applying the function to 10k random samples. The samples are then ranked by e(f (x)). Our ranking error measure is Kendalls tau [17] and all results are averages over 5 trials.  Table 3. Induced ranking similarity between e and surrogate g for ZDT3 and DTLZ2 (Kendalls Tau, higher is better) As can be seen in Tables 3 and 4, incorporating prior knowledge into the utility function definition is beneficial for most domains, as demonstrated by our Informed method, in conjunction with the Dist estimate. This is especially visible for trials with very low sample counts (N = 10) and the advantage reduces with additional samples. This can be explained by the fact that additional samples may contain information implicitly, that is explicitly available to the Informed method. The Linear method is mostly not able to capture a sufficiently good approximation of the true utility function, as it is too restricted. RBF Kernel performance differs substantially, but is consistently better than our method only for WATER. However, the RBF Kernel solutions are not deemed interpretable, as they do not conform to Sec. 4.3. When restricting results to function spaces that can be adapted according to interpretability requirements (Linear, Adaptable, Informed ), the Informed outperforms other methods also in the WATER domain.
It should be noted, the Bayesian approach Dist is highly relevant for capturing function parameter dependencies. The utility function definitions with such dependencies (Adaptable and Informed ) are usually not able to compete, without the obtained distribution information. Even the mean estimate Mean, subject to a point-estimate approximation of the posterior, is already better when compared to fully disregarding the induced parameter distribution (c.f. Max setting), but is still substantially outperformed by the fully Bayesian approach Dist. As expected, the Linear utility function does not benefit from the Bayesian method, as it does not contain any interdependent function parameters.

Ablation Study: Biased Samples
For real world applications, historical results used as H are usually biased towards good results because only final parameters and results have been stored. This means, each h ∈ H is the result of a human expert approximately maximizing e(f (x)). To simulate this setting, we do not randomly sample the free parameters x i ∈ x, but create N sets with 1k random examples. From each set, we chose the best result by arg max e(y). Resultingly, H contains N examples biased towards arg max x e(f (x)). Tables 5 and 6 show the corresponding results. In most cases, the learned utility function is worse when compared to learning with random samples. This is to be expected, because the biased samples only cover parts of the space of y. Therefore, it is difficult to derive a function that generalizes over all possible values. This is especially prominent for the CAR problem, where only Informed and RBF Kernel are able to learn a utility function that is positively correlated with the ranking induced by applying e. Additionally, this explains why more samples are not beneficial in several cases: The utility function becomes more strongly biased towards sufficiently approximating the covered part of the state space, which can contradict generalization to the whole space Y . Besides these bias-induced artifacts, it is still visible that the Informed method is able to achieve better approximations when learning with few samples, even when considering not interpretable alternatives (exception: CAR). Most test cases also show the advantage of using multiple posterior samples (Dist ). Therefore, we conclude that it is important to obtain unbiased samples for learning utility functions. For real world applications, this mean that one should log all trials, not only the obtained results. Independent of the used sample source, incorporating prior knowledge into the utility function space definition and parameterizing it by a distribution of function parameters still leads to improved utility functions, especially in the small sample regime.

Black-Box Multi-Objective Optimization
Recovering the expert utility function is not sufficient for achieving good blackbox optimization results. This requires a utility function, that not only ranks solutions comparable, but also induces the same maximum. Therefore, we use our learned utility functions as optimization target for a Bayesian optimization procedure [19], with a common Gaussian Process model. We use the implementation and default hyperparameters from scikit-optimize 6 and run it for 800 iterations. x 0 is set to a value not seen before, to mimic the application to a new product variant.  Table 7. Score of the best optimization results, re-evaluated by e Tables 7 shows the best utility scores obtained after 800 runs, re-evaluated by e, averaged over 5 repetitions. This means, we selected the best result according to the surrogate g and evaluated it in terms of the expert. The trials use the utility functions from Sec. 5.1 (Random) and Sec. 5.2 (Biased ), with N = 10 and Dist approximation. Manual is an approximate upper bound, obtained by drawing 10M random samples and selecting max e(f (x)). The results show that learning a utility function with prior knowledge is also helpful for black-box optimization. Interestingly, learning based on biased sample is not an issue in the Informed setting. This can be explained, because the additional information is likely sufficient to guide the optimizer into an area with high utility solutions, where the biased samples enable a sufficiently good approximation. For noninformed settings, the effect of biased samples can be substantial (CAR). In 3 out of 4 domains, our approach was able to recover a solution close to the approximate upper bound, with the exception of the WATER problem. However, in this domain, no approach was able to obtain good results. Therefore, we argue, that additional samples for learning the utility function are likely needed.

In the Wild
The system presented here is part of a real world engineering process, which aims at determining control parameters for a pinch detection system in the automotive domain. The problem space is subject to 15 parameter dimensions and 9 output dimensions and we trained the system on N = 30 historical examples. We obtained 8 expert rules, inducing 8 different rankings of the historical examples, to determine the pairwise preferences (c.f. Sec. 4.2). The utility function space G is defined as described in Sec. 4.1, but including an additional F β score term with β as free parameter. As evaluation, we applied the system to 7 different product variants and evaluated on 4 different quality metrics. The system result matched or surpassed the quality of the expert solution in all metrics for 6 of the 7 variants. A qualitative evaluation of 3 experts concluded that the system results are competitive to the manual results.

Conclusion
We show a novel method for obtaining a utility function as surrogate for an unknown expert evaluation function. In contrast to existing approaches, we do not only use qualitative preference signals, but also prior knowledge concerning the expected shape of the function. This allows us to obtain good approximations with very few samples, also leading to improvements for multi-objective optimization. Additionally, the resulting utility function can be better validated and analyzed by human experts as it ensures some interpretability constraints. Both issues are substantial for applying such a system to existing real world process. In the real world, historical data is usually scarce, but expert knowledge is available. Therefore, it is important to learn with this knowledge as main information source and obtain outcomes that are usable for human experts.
We also show the importance of correctly handling uncertainty, by means of Bayesian posterior sampling, and analyze the effect of using biased samples, that are usually available in real world domains. We conclude that uncertainty handling is of great importance, in cases where the surrogate function space is subject to function parameter dependencies. Additionally, we show the need for unbiased samples, for learning good utility functions. However, in incorporating prior knowledge into the utility function can remedy this issue, with regard to black-box multi-objective optimization.
We see the proposed method and obtained insights as another step towards reducing the reliance on large amounts of data and improving human-AI cooperation. In the future, more generic function spaces should be adapted in a way that allow the explicit incorporation of prior knowledge, while still enabling interpretability.