{'fraction_non_alphanumeric': 0.027493222999001286, 'fraction_numerical': 0.007276358967042374, 'mean_word_length': 5.027259437612864, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 1, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 4, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Modeling Human Behavior Part II -- Cognitive approaches and Uncertainty

As we discussed in Part I of this topic, there is a clear desire to model and comprehend human behavior. Given the popular presupposition of human reasoning as the standard for learning and decision-making, there have been significant efforts and a growing trend in research to replicate these innate human abilities in artificial systems. In Part I, we discussed learning methods which generate a model of behavior from exploration of the system and feedback based on the exhibited behavior as well as topics relating to the use of or accounting for beliefs with respect to applicable skills or mental states of others. In this work, we will continue the discussion from the perspective of methods which focus on the assumed cognitive abilities, limitations, and biases demonstrated in human reasoning. We will arrange these topics as follows (i) methods such as cognitive architectures, cognitive heuristics, and related which demonstrate assumptions of limitations on cognitive resources and how that impacts decisions and (ii) methods which generate and utilize representations of bias or uncertainty to model human decision-making or the future outcomes of decisions.


INTRODUCTION
Future autonomous and adaptive systems are expected to further exploit the concept of cyber-physical convergence [12], and realize an environment where autonomous agents and humans work together as teams, understanding each other and anticipating each other's behavior and intentions. This is propelled, on the one hand, by the pervasive diffusion of connected devices in the physical environment, which are directly owned (e.g. personal devices) or in tight interaction (e.g. IoT devices) with the human user. On the other hand, the vast diffusion of AI can bring autonomy of agents to a new level, making their behavior much more refined, and adaptive to the varying conditions of the environment and users. Human-Centric AI (HCAI) is expected to be a fundamental element in this vision. Not only because users need to trust AI agents, e.g., thanks to explainable algorithm [38]. Quite interestingly, AI agents will need to interpret human behavior in the context, so as to better interact with users, understand their actions, predict their choices, and ultimately orchestrate between actions performed directly by humans and those delegated to AI agents in autonomy.
To this end, it is fundamental to equip AI agents with practical models of the human behavior. Potential application areas are numerous, spanning from robotics, medicine, e-health, autonomous driving, just to mention a few. A key distinction between the goals and approaches is often the fidelity of the replication and the expected deployment case.
For instance, researchers may try to replicate the neurological pathways in an attempt to replicate the neuro-physical process underpinning reasoning [5], or they may instead attempt to generate a computational model which is meant to mimic heuristically biased behavior [71]. In any case, a common aspect is the desire to use humans as the template for desirable patterns of reasoning.
Though emerging, the literature on HCAI and related human behavioral models is quite vast already. Various types of HCAI exist. A first kind of HCAI systems focus primarily on explainability, making sure users can understand the process leading to a certain outcome by the AI agent [25,136]. A more advanced form of HCAI (typically referred to as hybrid intelligence) consists in AI agents and humans interacting directly, and impacting each other's operations [39,49]. Examples of such cases are AI agents learning in presence of human teachers [43,73,88,89,100,103], or humans exploiting the outcome of AI agents to acquire better comprehension of a phenomenon [115]. Moreover, in other cases humans and AI agents perform a common operation as a team [85,106]. In this context, designing approaches to orchestrate delegation of tasks and decisions among the team's members is also fundamental [92,102]. Figure 1 provides a taxonomy of the approaches existing in the literature to address these aspects. Work falling in the areas of direct learning of human behavior and modeling beliefs and reasoning are covered extensively in [30].
Specifically, they deal with approaches to learn human behavior by trial-and-error (such as Reinforcement Learning or Instance-based Learning) and approaches to prescriptively describe key features of human behavior such as reasoning processes and beliefs into a model (such as Theory of Mind). In this paper (Part II) we complete the survey by presenting in detail approaches falling in the other two classes depicted in Figure 1.
In this paper, we will discuss some of the popular topics and applications relating to Human-Centric AI, Human-AI Interaction, and Hybrid Intelligence. These topics include: bounded rationality and heuristics, cognitively plausible representations, bias, and more. These topics represent methods which attempt a model mimicking or inspired by differing biological/neurological, cognitive, and social levels of reasoning. Additionally, we will discuss how these topics align with application areas of interest. These application areas cover a wide assortment of both scenario as well as level of autonomy expected. For instance, this can include topics such as demographic preferences [45] to something as safety-critical as fully autonomous driving [28,135]. The specific scenario can rely significantly on the level of autonomy expected and the level of risk or control humans are willing to allow. In Section 3, we discuss methods which attempt to replicate the cognitive abilities and sub-optimality in humans. These allow for models which replicate biased or bounded rational behavior inspired by human cognition. Key to this Part Is modeling human cognitive resources, and ways adopted by humans to efficiently use them, possibly at the cost of obtaining imprecise understanding of the learned process or make mistakes. In Section 4 we focus on approaches that model uncertainty in the human reasoning process, to the point of leading to choices that do not appear to be the outcome of a rational process. Finally, Section 5 provides a critical discussion on the approaches presented in the paper. Before presenting them, we briefly mention key application areas in Section 2, highlighting how the considered approaches can be applied there. Note that, as in [30], we present each specific approach according to a common scheme. First, we point to specific surveys and related dealing in greater detail with that topic. Then, we discuss the general principles. Next, we discuss one concrete example where those principles are made practical. Finally, we briefly mention additional examples where the same principles have been applied.
Manuscript submitted to ACM Section-4. 2 Bias and Fairness Finally, also in this paper, as in [30], we limit the analysis to (a representative subset of) works providing quantitative models (e.g. math models or algorithms), as these are the approaches that allow to "code" human behavior in autonomous systems.

SAMPLES OF APPLICATION AREAS AND RELATED
In this section, we will briefly discuss popular application areas demonstrating uses of the techniques discussed in this paper. This list is not comprehensive, but serves to demonstrate topics which are likely more familiar and of immediate interest. The approaches used demonstrate methods which serve to replicate, model, or learn from human behavior and capabilities.

Driver Prediction and Autonomous Driving
There have been numerous examples of research performed to model and predict behavior in a driving scenario [60].
In the case of autonomous vehicles, there is a need to model and predict the outcome of control delegation between the autonomous system and the human driver. To do so, researchers have investigated cognitive models to predict the time to take over control given the type and difficulty of actions the human driver is performing when the control is switched [74,113,114]. This allows for a model which can simulate the cognitive and bio-mechanical responses when changing tasks for the human drivers.

AI in Games and Teaching
In the area of video and serious games, AI is being considered with respect to multiple aspects. In Part I of this topic, we discussed approaches integrating AI methods for learning models of behavior. In the case Serious Games, systems can be implemented to train human users and learn models of their behavior [56]. This allows for systems which can teach and allow for improvement with a more thorough understanding of the user's behavior. For instance, cognitive architectures (see Section 3.2) can be used to define intelligent tutoring systems [122].

Agent-Based Modeling
To achieve a model of human behavior and decision-making, numerous topics have been investigated. Some examples include agent-based modeling [22,37,45,54,118], which allows for models of groups of people or populations. In the case of agent-based modeling, models of human behavior are often defined and then studied in an environment over a simulated timeline. The agents follow the defined patterns of behavior and the resulting global patterns can be analyzed. For instance, population segregation based on demographic preferences regarding neighbors can be modeled by defining a diversity preference and modeling the movement of agents in an environment [45]. Populations of humans can also be modeled with other techniques designed to model the interdependence of the agents. Interactions can be modeled mathematically with humans represented as nodes in a network, particles in an environment, or more [23]. Additionally, the model of human behavior in an ABM can also be supported by models of Uncertainty, Bounded Rationality, or Cognitive Architectures, which we discuss in Section 3 and Section 4 with topics such as Section 3.2, Section 3.4, etc. Moreover, As we will focus on more direct and individual-level models of behavior, we will not present this topic in greater detail in this paper.

BOUNDED RATIONALITY AND COGNITIVE LIMITATIONS
In the following sections, we will discuss methods inspired by the cognitive limitations and characteristics demonstrated in human reasoning. The approaches discussed in Section 3.1 will demonstrate how humans utilize heuristics to enable fast and frugal reasoning as well as more deliberative systems. This combination of systems enables more efficient use of cognitive resources and has been demonstrated in numerous studies of human reasoning. Related to these concepts, we see the use of systems designed to replicate the cognitive and neurological performance (not necessarily the physical structure) seen in humans (see Section 3.2). These systems are designed to mimic human performance on tasks by replicating how humans use knowledge and memories to make decisions and perform tasks. Next, in Section 3.3, we discuss techniques which are inspired by how humans attend to stimuli and make associations between observed values. This can relate to human vision (i.e. foveation) or how humans identify correlations between different items in the same context (e.g. the word 'book' in a sentence would increase the relevance of the word 'library' in the same sentence). Next, in Section 4.1, we will discuss the use of quantum representations to accommodate for uncertainty and provide methods which can support a quantum representation of states. Last, we will outline concepts relating to bounded rationality in the context of Game Theory in Section 3.4. Similar to previous sections, Section 4.2) will discuss topics which focus less on a direct replication of cognitive functions; instead, these topics focus on the resulting biases that come from the use of heuristics and similar shortcuts in reasoning. Further, these limitations in reasoning generate immediate and long-term effects motivating studies on fairness.

Relevant survey(s).
For relevant survey papers and related, please refer to [9] 3.1.2 Principles and Definitions. Bounded rationality describes the notion of humans making rational choices under the constraints ascribed to cognitive limitations of the decision-maker [6,109,117]. These constraints are a reflection of the assumed limitations or deficiencies in a human's computational abilities/capacity and knowledge. Similarly, these constraints can be viewed as respecting a notion of cognitive or computational cost. As a reflection of these limitations, there is an assumption that humans perform decision-making in a manner which allows them to find a reasonable approximation of the optimal solution while reducing overall cost or time. Reasonable could be viewed as a "good enough" or satisficing solution. The process of finding such an acceptable result is supported by shortcut techniques (i.e. heuristics) which allow a person to approximate the collection of alternative solutions by finding a set of satisfactory alternatives.
An additional aspect of bounded rationality is the assumption that humans will sacrifice exactness or optimality of a solution for the sake of efficiency. In this context, efficiency can refer to time required to a solution, cognitive resources needed, etc. In general, this alludes to a sense of frugality when it comes to the cognitive resources a human is willing to dedicate to a decision process. In most circumstances, this frugality does not cause issues and allows for sufficient stimulus processing. On the other hand, there are examples of where this may cause a significant omission of perception.
For example, humans can be tasked with observing a scene and then asked questions at the end of the observation pertaining to specific content. Often, there can be items or people hidden in plain sight due to the observer being distracted by more attention-grabbing stimuli, which demonstrates possible errors in the use of cognitive heuristics by the human brain [132].
Dual-System Reasoning. Relating to bounded rationality, there has been extensive interest in what is referred to as dual-system reasoning or dual process theory, now extending to topics in AI [9]. The argument is that humans utilize two systems of reasoning based on the context of the problem and the limitations of their cognitive systems. The assumption is the two levels handle the problems at different speeds, fidelity, cognitive cost, etc. These distinctions are based on the belief that humans tend to utilize a lower-cost reasoning system when the penalty for a sub-optimal solution is minor or when the time or cognitive burden of reaching a higher accuracy solution is too great (see [77,97] for more on cognitive cost). For simplicity, we follow a common convention and refer to them as System-1 and System-2. For an example of System-1 reasoning, catching a falling object typically doesn't leave sufficient time for deeper reasoning, so we rely on instinctive movements made quickly by System-1. On the other hand, System-2 can support deeper reasoning, longer time to a decision, etc.
System-1 is commonly assumed to be based on approximations generated via heuristics. These heuristics provide shortcuts to reasonably accurate solutions. For example, it has been argued that humans utilize what is known as the availability heuristic, which selects a solution based on the strongest association between the current situation and memories of potentially similar instances. This means humans will tend to place higher weight on memories more closely aligned with the current observations. As such, they will be biased toward solutions with higher likelihood of recall. This allows humans to use similar past experience to simplify the decision-making process when using System-1.
If the problem is too complex for System-1, then System-2 needs to be utilized. System-2 allows for deeper inspection and the possibility to use or combine multiple underlying processes.
The use of these systems can lead to biases in reasoning and potential incorrect assessments. A simple example is the Gambler's Fallacy, which demonstrates how humans tend to believe that a sequence of flips from a fair coin should be self-correcting [48,128]. In other words, when the coin is flipped multiple times, a sequence of identical outcomes is considered less and less likely as the length of the sequence grows. This misconception leads the person to feeling that the alternate outcome should be more likely in the next instance. As is apparent from the independence of the samples, this is in fact incorrect reasoning. Probability theory dictates that the outcomes of each toss should have no effect on the next toss. However, this is a common bias observed in human behavior resulting from this form of reasoning.
Humans demonstrate additional forms of heuristic-based and biased reasoning. The following are some additional examples [27]: • Satisficing: Use a sufficient option rather than the optimal one • Affect: Make decision based on intuition or "gut feeling" • Simulation: Estimate likelihood of an outcome based on the how easy it is to imagine the outcome • Availability: Estimate the likelihood of a future event based on the strength of recall for similar past occurrences • Representation: Assume X is the same as Y when you notice X is similar to Y in some way(s) • Association: Connect ideas based on the word association and the memories, meanings, or impressions they trigger • Optimistic Bias: The tendency to underestimate our own risks and overestimate our own control in dangerous situations • Hindsight Bias: The tendency to remember successful events as resulting from your own decisions and failures as resulting from bad luck or decision from others • Loss and Risk Aversion: Avoid risk and loss by maintaining the status quo • All or Nothing: Simplify decisions by treating remote probabilities as if they were not even possibilities These heuristics/biases demonstrate systems which can generate correct solutions in many cases, but can also lead to misconceptions or ignored information. As a result, it stands to reason that methods which wish to model the behavior and reasoning of humans will need to take these potential inaccuracies into account.

Applications and Recent
Results. In [71], the authors demonstrate an application of the Dual-Process concept of human reasoning and use of heuristics in a RL paradigm demonstrating multi-alternative risky choice in the Mouselab scenario (widely used to study decision strategies). In this scenario, their approach demonstrates the emergence of Manuscript submitted to ACM two known heuristics: take-the-best (TTB) (chooses alternative favored by the most predictive attribute and ignores others) and random choice. They note these are resource-rational strategies for low-stakes decisions with high and low dispersion of their outcome probabilities, respectively. They further note how the TTB heuristic is commonly used by humans when under time pressure and one outcome is much more likely than others. Similarly, the authors demonstrate how humans tend to accept random selection when the stakes are low in low-dispersion cases. The authors represent the bounded optimal decision process as a meta-level Markov Decision Process by considering the cost of computing a solution which impacts the utility of a decision or action. The actions are treated as costly computations, necessitating the ability to make decisions with efficiency in mind. This need for efficiency follows those seen in the justification of the representation and use of heuristics.
An augmentation to the MDP considered in this research is the meta-level MDP. In this case, actions for the metalevel MDP are cognitive operations C performed in belief states ∈ B. Additionally, the meta-level MDP has a transition function and reward function ∈ . The operations in C include an operator ⊥ which terminates deliberation and subsequently translates the current belief into an action. The determination to end deliberation and select an action can be seen as a representation of how humans select System-1 or System-2 reasoning, which then results in an outcome from the selected system. The reward combines the cognitive cost ∈ C with the expected immediate reward the agent expects to receive once deliberation terminates and an action is taken. In the case of a computation, the reward is defined as Additionally, researchers have used heuristics to represent how information diffusion occurs in a network of individuals [4,11,[78][79][80].

Relevant survey(s).
For relevant survey papers and related, please refer to [61] 3.2.2 Principles and Definitions. With the goal of achieving a general AI (i.e. reaching human-level intelligence [72]), there have been numerous approaches inspired by the cognitive mechanisms enabling the intelligence observed in humans. In [110], the authors note several ways how reaching human-level general intelligence might be possible. One noted method relates to the design and justification of cognitive architectures. In the case of cognitive architectures, the goal isn't always to achieve a perfect analog of the human brain and its neurological function; instead, a common goal is to generate a system capable of demonstrating the same kinds of abilities and deficiencies seen in human cognition, reasoning, intuition, etc. (e.g. perception, memory, attention) [21,32,53,61,126]. Under these circumstances, the goal is often the creation of a model of behavior which fits the cognitive/neurological dynamics of the human brain [2,5,61,108,119,123,130,134].
As noted in Section 3.1.2, it is generally accepted that humans reason with systems operating at different levels of fidelity. Humans can make faster and cognitively frugal decisions or utilize slower and more cognitively burdensome resources. As such, research has been dedicated to the creation of systems demonstrating these characteristics (and beyond) [61]. These systems demonstrate an ability to learn behavior as we've seen in previous sections in Part I (e.g. Reinforcement Learning), but the distinction in this case is the emphasis on replicating the cognitive performance of humans. This distinction motivated us to place a higher emphasis on the cognitive and biologically plausible mechanisms of this portion of the paper. Further, there have been studies which show these representations can provide the best performing (and likely best fitting) approximations to human cognitive performance [121]. For example, cognitive architectures utilize memory systems which can replicate how humans retain information and utilize that information when making decisions. As a result, we see this section as more suitable in a cognitive limitations and biases context.
World Representation -Symbolic, Emergent, and Hybrid. To support reasoning and behavior, the system needs a method for representing the world. For cognitive architectures, there are three main categories for the underlying representations: symbolic, emergent, and hybrid. As the name would suggest, symbolic systems use symbols to represent concepts or knowledge. Given the symbols, the system can manipulate them by using a given set of instructions. The instructions can be provided through if-then rules or similar means. As can be expected, a symbolic representation allows for accurate planning and reasoning., but the potential downside being that this approach is brittle and does not adapt to changes in the environment. Emergent systems operate similar to what is seen in Artificial Neural Network (ANN) systems. Information is processed by the system and associations are made through a learning process. This of course increases the system's responsiveness to changes in an environment, but can reduce the transparency or the easiness of interpreting the system's behavior. To utilize the advantages of both systems, with the hope of overcoming the shortcomings, there are hybrid systems which combine the symbolic and emergent approaches.
Learning Methods. Learning in a cognitive architecture can be performed in several ways including: Declarative, Procedural, Associative, etc. [61]. In the case of Declarative learning, the system is provided a collection of facts about the world as well as relationships between them. For instance, many systems such as ACT-R, SAL, CHREST, or CLAR-ION utilize chunking mechanisms to declare new knowledge items. For Procedural learning, the system learns skills gradually through repetition, which can be accomplished through the accumulation of examples of successful execution of a task or problem. More closely aligned to RL, Associative learning is based on observations of rewards or punishments.
Memory. Architectures can be supported by different memory mechanisms depending on the type of capabilities being replicated. When performing a task, the memory utilized to temporarily store information related to the task at hand is referred to as working memory [3]. This memory is updated rapidly as the state of the world changes and actions are taken. Further, there is commonly an assumption regarding the capacity limitations of working memory for humans. In addition to working memory, other systems provided a means to accomplish long-term memory storage.
This can support storage of procedural memory to define basic skills or behavior or declarative memory for knowledge.
This allows for the storage of innate skills as well as accumulated knowledge. Additionally, some systems are defined with a hybridization of long and short-term memory, referred to as global memory. This results in all knowledge and memories being represented by the same system.

Applications and Recent
Results. The above characteristics are broad aspects covering different approaches for cognitive architectures. For a specific example, we present a recent result based on the ACT-R architecture. The authors present a cyber security game designed to demonstrate cognitive biases of cyber attackers [16]. This displays how humans are susceptible to fallacies in reasoning which result in sub-optimal and biased behavior. The authors demonstrate how their models replicate the biases motivating human behavior patterns in system selection and the choice to abandon a system and forfeit the previous effort on the current system.
The authors used an Instance-based Learning (IBL) model using the Adaptive Character of Thought -Rational (ACT-R) architecture. ACT-R is a theory of cognition which models how humans recall "chunks" of information from memory and how humans solve problems by splitting them into sub-goals [134]. Knowledge is applied from working memory as needed to find a pattern of behavior meeting the goal. This model utilizes techniques designed to mimic human memory retrieval, pattern matching, and decision making. IBL uses ACT-R's blending mechanism, which interpolates across past experiences to estimate an outcome. The interpolation is weighted by the contextual similarity between the present observation or instance and the past experiences. This provides an estimate expected outcome based on the consensus value which minimizes the dissimilarity (measured by ) from the values contained in instance defined as: where refers to an instance stored as a memory chunk representing a past state-action-outcome observation and refers to the retrieval probability (based on IBL-based measures). In the case where is interpreted as the error, then Equation 1 generates a least-squared error method [65]. In other words, this finds an estimated value which best fits the past observations that are weighted by their strength of recall. These estimated values are used to make a determination regarding which action/production should be executed. The measure or threshold which determines whether an action is available for execution limits the set of possible actions further. This means that the value is based on a representation which considers how strongly a memory is remembered, how similar the memory is to the current context, and the value observed by the choice made in that past observation. The strength of a memory represented by the retrieval probability utilizes a Boltzmann softmax equation where defines the temperature parameter, which scales probabilities defined by the activation function. The activation function provides a measure of how strongly a memory is remembered and associated with the current context. This strength is based on elapsed time since the observation was made. The activation for a chunk or instance is defined where refers to the elapsed time since the ℎ occurrence of instance , is the decay rate (commonly set to 0.5), refer to the context elements, refer to the instance in memory, and is the mismatch penalty (in this case, set to the default of 1.0). The first term in Equation 3 provides the measure of strength based on the time elapsed and the second term is another similarity term similar to what is seen in Equation 1. is a weight term parameter which scales the similarity scores in the sum, and the last term, , is a variance parameter providing stochasticity in the activation function. Similar to Equation 1, the measure ensures the memories considered are a suitable match to the current context in order to prevent consideration of too dissimilar of instances. In more general terms, the above equations define a method for determining which memories are considered, how strongly they impact the estimate based on past observations, and how the resulting behavior occurs based on this historically weighted knowledge.
The agents are trained to perform the cyber attacker role. As such, the agents are provided observation instances which include the probability of a system being monitored, the reward for successful infiltration of a system, the penalty for detection, and a warning signal denoting whether a system is being monitored. The model is then primed with seven instances: five simulating a practice round, two representing knowledge of occurrences (absent and success, absent and uncertainty). This provides the system with an initial set of experience to allow for initialization of learning behavior without relying on random decisions. The model then is trained for four rounds of 25 trials.
Results. The model was tested in comparison to human performance. Human players were studied to generate a baseline of behavior and identify any demonstrated biases in outcomes. Based on the experiments, the authors show the human players demonstrating preferences or likelihoods of attack for different systems. They also demonstrated the cognitive systems performing equivalent preferences/probabilities. Additional Relevant Results. As mentioned, an underlying justification for using cognitively plausible approaches is the use of algorithms which mimic the performance of humans under the same task. Through the use of cognitive architectures, the cognitive and bio-mechanical processes for a human attempting to accomplish a goal can be replicated. For instance, models of user swipe behavior, mental folding, driver takeover of an autonomous vehicle, simulated cyber attackers, pilots, etc. can be created [15-17, 58, 59, 74, 99, 111, 113, 114]. The structure of the artificial systems architecture can have a significant impact on performance [116] and there are numerous examples of architectures and their underlying structures [61].
Recent research has investigated closer ties between cognitively inspired representations and topics in RL. For example, there have been attempts to utilize these representations to observe and predict behavior of agents in an environment [90,91]. These demonstrate the use of an agent's ability to learn a model of likely behavior in an attempt to anticipate likely next steps. The observer sees actions selected by RL agents and builds a model of likely behavior, be it next actions or likely goal states. Another aspect of mimicking processes exhibited by humans is the ability to imagine. This is demonstrated in [138] where they create an agent which can perform conceptual blending and daydreaming in order to generate dialogues inspired by the user behavior. Similarly, [24] demonstrates a similar use of related past instances and object relations in an RL context. For a simulation environment supporting these sorts of representations, [120] demonstrates an integration of the ACT-R architecture and Unity 2D/3D modeling system. This allows for control and learning for artificial agents in a Unity environment with the behavior being managed by ACT-R representations. This enables testing of cognitively inspired representations of behavior in a simulated world, which can enable the use of advanced game engine features for fidelity of the world model. In a similar topic, [96] demonstrates integrating a physics engine with a cognitive architecture to aid an agent's decision making. This is shown to aid the agent in playing racquetball in a simulated environment.
Cognitive representations can also be utilized to define an agent intended to interact or coordinate with a human.
For example, [84] utilizes a cognitive architecture and RL concepts to predict and model behaviors in a HAI context.
The scenario involves control of a moving circle on a screen which needs to follow a path which is scrolling down the screen. The human can either choose to take control or pass off control to the agent.
In another context, [63] demonstrates training of an RL agent using a sort of layered approach which they refer to as a two-learner system. The layering in their approach refers to the use of two agents to learn a policy. The first agent is used to learn an approximation of feedback signal which would result from backpropagation and the second learns the behavior policy based on the approximated feedback. The argument is the two-learner system could simulate cortical neuron physiology.

Principles and Definitions.
Humans and other living beings do not process all the available perceptual information available to them. Instead, they utilize a cognitive and behavioral system which allows them to reduce the complexity of perception through an objective or subjective selectiveness with respect to information. This selectivity or bias is referred to as attention [14]. A basic interpretation justifying the biological need for attention would be the fact that our environment provides more stimuli than we can reasonably process fast enough for our environment. In this case, fast enough is with respect to the actions or behaviors necessary for survival.
When facing a critical situation, timeliness can be crucial; otherwise, an overload of stimuli could cause a costly delay (e.g. moving out of the path of an oncoming vehicle). As such, our brains allow us to reduce, or even ignore, information perceived in order to reduce the cognitive burden. Further, attention allows us to prioritize the information and assign more or less significance based on learned or perceived importance. In the oncoming vehicle example, it is likely not important to note the color of a building in the distance while estimating the speed and trajectory of the vehicle.
From a computational perspective, attention initially was studied primarily from the context of vision [14] where images were studied with under the task of identifying salient regions of the image. Artificial systems were developed to generate maps which would filter the input for processing. With the growing popularity of Deep Learning (DL), attention techniques were transitioned to neural network paradigms. Attention is used to modify the flow or processing of information in the network(s). The task of learning attention allows the systems to learn how to ignore stimuli, similar to the natural analogs mentioned. This allows systems to contextually alter the significance of information in order to better suit the underlying task. The attention mechanisms utilized in DL can be categorized as follows: Attention. Soft attention uses softmax functions to weight the input elements with a weight value in (0, 1). This allows the system to learn and utilize an interdependence between different input parameters. Being based on softmax functions, soft attention provides a differentiable mechanism for attention. The soft attention scales the relative intensity of the input parameters.
Hard Attention. Hard attention, as the name suggests, is the complement of soft attention. It utilizes weights in {0, 1} to generate a mask to signify whether information is used or entirely ignored. As a result, the hard attention mechanism is non-differentiable. This necessitates a learning process for determining where to assign the weight values. In this case, there is a distinct exclusion of regions of the input domain while the remainder is observed at normal scale.

Self-Attention.
In self-attention, the system is learning an interdependence between sequential input elements. This allows a system to identify and utilize a notion of relation between items in the same input sequence. As a result, selfattention can be useful in understanding deeper relationships between items in the input rather than a holistic view of the input. For example, [131] introduces the transformer network which performs self-attention using representations The weights are used to compute a representation of each phase , as = ∈L × ( ). The representations of the possible phases ∈ P are then fed into an LSTM, to capture the sequential dependence between phases. The output of the LSTM ( ) is used, together with the representation of the possible phases , ∈ P for the second attention Manuscript submitted to ACM mechanism, modelled as in Equation 6. Specifically, this provides the probability ≔ { , ∈ P} of switching to any of the phases of the next time step.
The reward is based on the negative of the intersection pressure defined in [133].  [29], which attempts to address self-attention in RL to help solve credit assignment. Similarly, [52] utilizes attention to address the temporal credit assignment problem. Further, [131,137] are in reference to the widely popular transformer network. The work in [131] provides an introduction of the approach while [137] extends the approach to introduce sparsity to allow the most contributive components for attention to be reserved while the other irrelevant information are removed. This is effective in preserving important information and removing noise, so the attention can be much more concentrated.
Further, [55] demonstrate the use of in an LSTM/RNN context for copying, transfer, and denoising tasks.

Relevant survey(s).
For relevant survey papers and related, please refer to [86,112]. Note that, since game theory is a widely investigated and well-known topic, in the following we only sketch very briefly the key ideas behind this theory. The main reason for mentioning it in this survey is to place it in the overall context of human behavioral models. Also, we don't mention explicitly applications and results, as the literature is huge and pinpointing only a (few) specific example(s) would be not that useful for the readers.

Principles and Definitions.
Extending and utilizing the notion of rational behavior, game theory focuses on the interdependence of choices when the circumstances involve a collection of individuals [86]. In game theory, the decisionmaker, often referred to as the player, is operating with an assumed feedback signal. The feedback (payoff) provides a value associated with how desirable or costly an outcome might be for the player. Based on the payoff and anticipated behavior of other players, a player can attempt to optimize their choice (action) in order to best ensure an acceptable outcome. In this case, rather than attempting to learn a policy of behavior, the models of utility define the constraints of an optimization problem. Based on these constraints, the players of a game can find a suitable solution which follows the assumptions of rationality.
More formally, a player will have a set of available actions from which they can select an action . The optimality of an action is dependent on multiple factors. First, the player has a payoff function : → R that will map an action to a value. This mapping depends on the definition of the problem and the interdependence of actions to values between the players. The values ( ) given by allow the player to generate a preferential ordering of actions, which indicates a player's need to identify desirable actions. Therefore, if the preferences of the player satisfy the following, then that player is considered rational under certainty [6]: (1) (Completeness) 1 2 or 2 1 (2) (Transitivity) If 1 2 and 2 3 , then 1 3 The payoff values and the assumptions relating to rationality can then be utilized to identify an appropriate action or behavioral policy. In game theory, the behavioral policy of a player is referred to as the player's strategy or strategy profile.
The strategy of a player is a distribution over actions indicating the likelihood of selecting an action. A strategy which places the mass on more than one action would be considered a mixed strategy, which indicates a player does not select a particular action 100% of the time in a given scenario (i.e. a pure strategy).
The identification of a strategy is the process of finding an equilibrium. In an Nash equilibrium, the player strategies are such that deviation from the current strategy for any player would be undesirable as it would lead to another player having the means to take advantage of the change in order to achieve a better result. Depending on the game, it is possible to find zero, one, or even multiple equilibria. In the case of multiple equilibria, it is possible for the payoff for a particular player to vary between the equilibria, but it remains true that a deviation from the equilibrium from a single player would be undesirable [86].

∈ X
The payoffs and assumption of rationality provide the constraints for an optimization-based solution method. However, the interdependency of the behaviors, outcomes, and payoffs can increase the difficulty of finding a suitable solution.

Relevant survey(s).
For relevant survey papers and related, please refer to [1,20,47,76,83] 4.1.2 Principles and Definitions. It has been argued that traditional probabilistic representations do not fully represent the reasoning of humans [81] or can require exponentially more complex representations [76,83]. Instead, researchers have suggested the use of quantum-based methods for representing the statistical/probabilistic relationships between knowledge [47]. The argument is that the superposition-like representation better demonstrates how humans can have varying beliefs, which might not directly match the assumptions or requirements of probability (e.g. summing to one).
This allows for a representation which can perform in cases where the reasoning currently operates in a state where multiple outcomes are possible or represent the same indefinite state. This is also a proposed method to account for potentially irrational or probability-violating reasoning of humans [20,44]. The use of quantum representations also allows for replicating or modeling fallacies in human reasoning [1]. A common method for this representation is the use of a quantum-based Bayesian Network. In this case, a similar representation of the Bayesian network is utilized, but the dynamics are represented using quantum representations of probabilities.
Quantum Dynamics for Decision Models. As defined in the Reinforcement Learning section of Part I, sequential decision making can be modeled using a MDP. This representation provides a mechanism for representing the transition between world states based on a decision or action. Such a representation enables learning associations between actions and outcomes to build a model of behavior. In [10], the authors suggest the use of a quantum dynamics model to represent brain processes as a replacement for the classic MDP model. The quantum representation allows for transitioning from a 'single path' assumption to one which represents unknown previous states as a 'superposition of states'. The superposition model allows for modeling interference effects for unobserved paths, which violates the Markov representation.
As noted by [10], a key distinction in the representations is as follows: According to the Markov model, for any given realization, the unobserved system occupies exactly one basis state | at each moment in time. A sample path of the Markov process is a series of jumps from one basis state to another, which moves like a bouncing particle across time. A different path is randomly sampled for each realization of the Markov process. According to the quantum model, for any given realization, the unobserved system does not occupy any particular basis state at each moment in time. A realization of the quantum process is a fuzzy spread of membership across the basis states, which moves like a traveling wave across time. Each realization of the quantum process is identical, producing the same series of states across time. All of the randomness in the quantum model results from taking a measurement.
Hence, the quantum representation supports the notion of wave interference in the dynamics representation.
From a modeling standpoint, the main difference between a Markov and a Quantum representation is therefore the following. In a specific realization of a Markov process, the state at any point in time is deterministic and can be represented with | ( ) as follows where denotes the possible states, Ω is the set of possible states, and ( ) ∈ {0, 1} is an indicator variable. Essentially, | ( ) provides the (unique) state of the process at time for the specific realization.
On the contrary, in a quantum representation, the specific realization, at any point in time is not deterministically in one and only one state, but is in a superposition of all possible states, each with a given weight. In other words, even for a specific realization the process can be in any of the possible states, while the uncertainty can be only removed by "measuring" the specific state of the process. Specifically, the state, at time of the realization, denoted as | ( ) is modeled as follows where where ( ) is a complex number representing the probability amplitude that the process is in the specific state | at time . As in any quantum superposition case, note that the squared magnitude of must be unity (i.e. | | 2 = 1) in order to ensure the squared amplitudes produces a probability distribution over the possible states.
Finally, transitions between states become, in the case of quantum representation, specific quantum operations over the representation of the states provided by | ( ) , which translate, in the quantum domain, the traditional concept of transition probabilities between states of deterministic Markov Processes.

Applications and Recent
Results. In [40], the authors investigate the use of quantum representations in the context of categorization. The use of a quantum model is intended as a means to represent interference effects observed in categorization and the resulting impact of decision making. The inclusion of a quantum system allows the modeling of a state which represents uncertainty in the reasoning process. This models how humans demonstrate hesitance when facing a decision. For their experiments, the authors investigate two paradigm conditions: categorization decisionmaking (C-D) and decision alone (D alone). In the C-D condition, in each trial, participants were shown pictures of faces, which vary along two dimensions: face width and lip thickness. Participants were asked to categorize the face as a "good" ( ) guy or a "bad" ( ) guy and then make a decision to "attack" ( ) or to "withdraw" ( ). In the D alone condition, the participants were asked to make a decision directly without categorizing, but the faces shown were the same as in the C-D condition.
Proposed Method. Although categorization happens in the belief representation, it can influence the action part by producing the interference effect, which can also model the disjuction fallacy (i.e. the false judgement that the probability ( ∪ ) is less than either ( ) or ( )). Consequently, the authors utilized a method to predict it. For the proposed problem, the initial state involves six combination of beliefs and actions

{|
, where, for example, | symbolizes a participant categorizing the face as good while still intending to attack.
Since participants are assumed to have some potential to be in any of the six quantum states, the person's state is a superposition of the six basis states: with initial state corresponding to an amplitude distribution: where | | 2 is the probability of observing state | initially. As an assumption, the the initial state is treated as equally distributed. In the process of decision making, updated information regarding a player's beliefs causes a Manuscript submitted to ACM transition in belief states. The decision-maker must convert this transition in reasoning states into a decision. To convert the observations and measurements into actions, participants must convert the uncertain state to either or . This represents when the decision maker transitions from uncertainty due to hesitation to a decision state. In this case, the authors propose the use of Pignistic Probability Transformation (PPT). This provides the following total probability of attacking given that the face is categorized as or respectively: where Ψ( | ) is the conditional amplitude (i.e. quantum equivalent of conditional operator) of attacking given the face is categorized as ∈ { , }. The above denotes an even attribution of the probability of transitioning from the uncertain state to decision. This then provides the means to calculate the total probability of attacking: A similar approach is utilized for ( ) and in the D alone condition. This allows a model of representing the state of uncertainty encountered before a decision is made and the process of converting updated beliefs into an action. Based on this model, the action likelihoods can be represented and estimated using a quantum modeling approach.
Results. For experiment conditions, the proposed method was tested against prior observations of human-generated data as well as prior prediction model results. In the data generation process, 26 participants were asked to categorize the face as good guy ( ) or bad guy ( ) and then make a decision to attack ( ) or to withdraw ( ). The faces roughly fall into two categories: "narrow" faces (narrow width and thick lips) or "wide" faces (wide width and thin lips). Participates were informed "narrow" faces had a 0.6 probability of belonging to the "bad guy" population. Similarly, participants were informed "wide" faces had a 0.60 probability of belonging to the "good guy" population. Rewards were were given for choosing attack for and withdraw for . In the D alone condition, the participants were asked to make a decision directly without categorizing, but the faces shown to participants were the same as those in the C-D condition.
Each participant provided 51 observations for the C-D condition and 17 observations for the D alone condition. The proposed approach showed a strong alignment with the pattern of behavior observed in the human participant results.
The authors also demonstrate the performance of the proposed approach with respect to sensitivity analysis. The authors show how sensitive the result of the method are to a ±5% change in the original human results. In this test, the method again shows strong performance, but the authors note a stronger sensitivity to changes in the case versus the case. For a plot of these results, please refer to the original article.
Additional Relevant Results. In addition to the prior examples, quantum representations can also support fuzzier associations between knowledge items. The use of quantum representations allows for deviations from normative probabilistic methods and can support a notion of interference between variables and paradoxical behavior [44,82].
When considering multiple individuals, quantum-based methods have also been investigated to model the diffusion of information [87]. The flexibility of representation from quantum methods also enables handling of the disjunction fallacy [i.e. the false judgment that ( ∪ ) is less than either ( ) or ( )], which violates the law of total probability [40]. Quantum representations can also be used to represent states in game theory settings where there is uncertainty about the potential behavior of others or the current state [83,98]. Further, as noted above, the concept of Bayesian Networks and causality can be extended to utilize representations supported by quantum dynamics [20,44,82].

Principles and Definitions.
Bias. Humans demonstrate multiple forms and sources of biases. In one case, humans can demonstrate skewed interpretations of data, probabilities, confidence, etc. [69]. This kind of irrationality and bias is often attributed to the use of cognitive heuristics. It is argued that the use of heuristics causes humans to often accept coarse analysis and sub-optimal solutions. This demonstrates how humans can violate the traditional notions of rational behavior. Further, the use of heuristics and other shortcuts can lead to skewed interpretations of information. These biases and coarse representations can lead to sensitivity regarding the interpretation of rate events, risk, probability, and more. Related, it has also been shown how humans tend to under-react to probabilistic information and also fail to follow belief updates modeled by Bayes' rule [18].
Another aspect of bias can come in the form of inattentiveness, which could be linked to the bounded computational power of human cognition. Humans also demonstrate a resistance to changes in views or opinions when facing contradictory information [75]. In fact, it is common for people to become more convicted in their views rather than convinced they might have been wrong. Bias can also come in the form of inductive bias introduced in architecture design and algorithmic choices made by the humans generating algorithms. Both implicitly and explicitly, humans introduce inductive biases into the artificial systems they are developing. In fact, [33] notes how much of the success of Deep Learning models could be attributed to the inclusion of inductive bias in these systems. Further, the authors note how the use of inductive biases might be a requirement for the creation of generalized artificial intelligence. They note how the biases allow for assumptions regarding the problem being solved or the interpretation of information, which makes the system better suited for adaptation to broader datasets.
Probability-Based Models of Behavior and Reasoning. To model deficiencies or biases in understanding and reasoning in humans, researchers have investigated Bayesian representations and similar. For instance, (Hierarchical) Bayesian models can demonstrate how humans reason about likelihoods of outcomes and likelihoods of scenarios [18,19,105,125,129,139], and how human reasoning shows flawed interpretations or skewed scales of importance (high or low). This illustrates how humans can under-react to probabilities [18,19] or estimate likelihoods based on observed frequencies. Studies have shown how people tend to select options at a similar frequency to their probability of good outcome rather than developing a bias to the choice with highest likelihood of good outcome [139]. Similarly, Bayesian models can replicate the performance of human study participants when tasked with selecting in a multiarm bandit problem [107]. This type of representation of understanding can also be extended to a causal model, which models the cause and effect relationship between different environment aspects [36]. Causal models are an important aspect of human reasoning as they allow for predictions and retrospective reasoning.
Fairness. With the use of algorithms, a consideration needs to be made with respect to the fairness of outcomes. As noted in [67], fairness requires equal or equitable treatment of everyone based on their performance or needs. As a concept, this is straightforward. There should be as little (preferably none) bias in the treatment of individuals. We would prefer the outcome of a human's decision be fair, so it is natural to desire the same behavior from an algorithm.
Fairness can both be a quantity measured with respect to a metric [42], or fairness can represent a qualitative impression people have about a feature or outcome [35,67]. These measures provide an indication of how people perceive the features of an algorithm or its behavior. The notion of fairness can also be considered with respect to longer timescales and deeper consequences. In [42], the longer effects of fairness measures and decisions are weighed to indicate how strongly an approach may impact the behavior of the humans affected.

Applications and Recent
Results. In [42] the authors model population-level changes at the macro scale that are caused by algorithmic decisions and how this relates to fairness. Using measures of segregation from sociology and economics, they quantified these changes. This allowed them to measure different directions of the shift in the group-conditional distribution based on the different models demonstrated. In this context, the authors note how most notions or measures of fairness assume a static population. They argue that this approach fails to account for longterm welfare and prosperity. Therefore, they propose a measure which considers a notion of effort based on economic literature on Equality of Opportunity. Their effort function highlights idea that the necessary changes for a desirable outcome often are more difficult for a member of a disadvantaged group compared to an advantage counterpart. Based on this concept, they formulate effort unfairness as the discrepancy in effort required for members in different groups to obtain desired outcomes.
In their approach, they assume individuals imitate an exemplar individual who demonstrates a more desirable algorithm outcome, which is related to social learning. The assumption being that the observer would believe this imitation offers a higher likelihood of a better outcome, which suggests an individual would exert effort to attain a replication of the exemplar's social model if doing so could result in higher overall utility. To model this dynamic, the authors propose a group-dependent, data-driven measure of effort, which is inspired by literature on Equality of Opportunity (EOP). This is noted as the effort it takes individual to improve their feature value from to ′ . This effort is assumed proportional to the difference between the rank/quantile of ′ and in the distribution of feature in 's social group.
This means an individual which successfully replicates the role model obtains a positive utility (reward minus effort).
In this context, the authors consider a standard supervised learning setting with training data set = {( , )} =1 of instances = ( , ), where ∈ X specifies the feature vector for individual and ∈ Y, the ground truth label for him/her. Further, let refer to the sensitive feature value (e.g. race, gender, or their intersection) for individual .
To measure the impact of a change in label, the authors define benefit function : X, Y × Y → R that quantifies the benefit of a change in label from toˆ and is assumed a linear function.
To formally define measures of effort, reward, and utility, the authors provide the following. For deployed model ℎ and individual characterized by = ( , ), let R ℎ ( , ′ ) specify the reward/benefit as a result of characteristic change from to ′ : where > 0 is a constant specifying the individual's degree of risk aversion. Again, this demonstrates a measure which considers the impact on an individuals outcome when they change a characteristic. For instance, a person could attain a degree or find a new job. Let E ℎ ( , ′ ) specify the effort required for a qualification change from to ′ . The overall utility of the individual is then U ℎ ( , ′ ) = R ℎ ( , ′ ) − E ℎ ( , ′ ) (i.e. utility equals reward minus effort).
Based on the above assumptions and definitions, the authors define several effort-based measures of unfairness, including: Definition 4.1 (Effort-reward Unfairness). For a predictive model h, the effort-reward unfairness is the inequality of the following metric across different groups: This provides a measure of the performance of the highest utility members of a group achieved by a higher level of effort. The authors note the impact of both threshold and the models. It is demonstrated that and the model can cause a significant change in outcome and effort level for the same data.
To measure performance, the authors utilize several techniques from sociology measuring segregation. First, evenness measures how unevenly the minority group is distributed over areal units. The evenness value is maximized when all units have equivalent relative numbers of minority and majority members as the whole population. For a formal definition, the authors use the Atkinson Index (AI), which measures inequality. The second measure, centralization is the degree to which a group is spatially located near the center of an urban area. This can be measured by comparing the percentage of minority residents living in the central areas of the city. The authors utilize the Centralization Index (CI), which is defined as: where is the total minority population. The third measure of segregation, the Absolute Clustering Index (ACI), Additional Relevant Results. As noted above, biases can be represented in both interpretations of information as well as in the exhibited behavior. This can be demonstrated by Bayesian models of cognitive biases [105], biased probability judgments [139], sensitivity to risk/uncertainty [26], and similar topics relating to representation and interpretation of data/statistics [19,70]. Biases can also be exhibited in how people respond to others or digital avatars (e.g. racial preferences in representation or interpretation of faces [46,68]). These biases can also influence behavior due to or via heuristics [51,64] both in terms of reasoning as well as decision-making (e.g. experience level impacting performance of heuristic outcome). Another factor relating to bias is its impact on model designs and inductive biases introduced into systems [33,62,101]. These biases can impact how a system is formed, how it utilizes contextual information, or how the system might need to compensate for unobserved factors available at decision time when learning from past behaviors of humans. Alternatively, it can also be important to avoid introducing biases caused by algorithms making determinations or learning from data which might be corrupted. The approaches in [127] and [104] demonstrate an effort to enable systems with the ability signal when they are uncertain whether a label for an input item is accurate via an abstention action. This can make systems more accurate as well as less susceptible to perturbation-based attacks or label noise. An additional context for bias is the aspect of fairness [7,8,13,50]. This can be in relation to both the representation provided to the algorithm as well the decision or outcome. Systems which exhibit unfair biases can generate disadvantages when the outcome has impacts on decisions relating to humans. For instance, non-uniformity or under-representation of samples in data can bias the performance of the trained model. If the lack of representation is in relation to race or gender, this can cause sociological consequences.

CONCLUSION
This paper completes the survey on approaches to model the human behavior in Human-Centric AI systems, complementing areas covered in [30]. In general, modeling the human behavior is seen as a cornerstone of future autonomous and adaptive systems, which can unleash full integration between humans and AI agents working in tandem. It will allow AI agents to make their choices more comprehensible by users, on the one hand, and AI agents to interpret, predict and take account of human actions and choices, on the other hand. While in [30] we have covered approaches where AI agents learn human behavior by trial-and-error, as well as approaches "coding" directly models of human reasoning, in this paper we have focused on approaches taking into account limitation of cognitive resources and how to optimize the, as well as approaches modeling uncertainty in reasoning and irrational behaviors.
In Section 3 (Bounded Rationality and Cognitive Limitations), we outlined and discussed approaches which can or attempt to replicate how humans gain/utilize knowledge to perform tasks, taking in particular account an efficient use of (limited) cognitive resources. In these cases it is often necessary to provide models of world dynamics and actions available so the model can replicate human performance, which clearly can offer effective performance. However, the models required to generate solutions typically require domain knowledge and so can be difficult or costly to generate.
For instance, architectures like those described in Section 3.2, typically rely on models of world and cognitive dynamics to enable reasoning over available capabilities given the bounds on time and cognitive cost. Still, these approaches can be effective when trying to better align with how humans perform tasks and allocate cognitive resources. On the other hand, these approaches can also be sample inefficient or require detailed (possibly expensive) models of knowledge. Related, behavior and reasoning can be represented through the use of heuristics mimicking the two systems humans are believed to utilize as seen in Section 3.1. The use of these heuristics has similar benefits and pitfalls. Again, these models often prove effective and are efficient to utilize, but often rely on domain knowledge or potentially coarse representations of human reasoning. The representation of reasoning with heuristics demonstrates an approach to model the assumed cognitive bounds on human reasoning. Such boundedness is also related to the assumption of bounded rationality found in Game Theory (Section 3.4). This assumption allows for constraints on the representation which enables generation of solutions. These solutions optimize behavior based on these assumptions as they enable reasoning over the behavior of all players in a system based on their expected payoffs for possible outcomes.
This representation enables strong guarantees and methods, but can often be conservative and a less than accurate portrayal of actual human behavior (e.g. players may not behave rationally). With another focus, in order to integrate capabilities related to reasoning and how humans utilize stimuli given their available cognitive resources, researchers have investigated methods for mimicking how humans attend to information. In these approaches, as discussed in Section 3.3, systems learn a model for masking and associating input values to learn a deeper understanding of the stimulus. These models have shown strong performance in language and behavior scenarios and can allow for more abstract representations supporting more variation in the scenario composition. On the other hand, as with many Deep Learning scenarios, these methods can be difficult to train or require a large amount of samples, so careful consideration is needed when defining an approach.
For the remainder of the paper, in Section 4 (Uncertainty and Irrationality), we discuss approaches which represent methods reasoning or considering irrationality, biases, or beliefs of humans. In this context, the presented approaches learn or use models of belief. This can be the effect of beliefs with respect to how it introduces biases in behavior and reasoning (see Section 4.2). Similar to the previous topic, this also requires development of models, which can be costly and challenging. On the other hand, these methods can offer effective means for measuring the impact of biases in behavior or reasoning with respect to future outcomes or fairness. In addition to biases, the process of human reasoning involves moments of uncertainty. In this case, the decision-maker exists in a state of uncertainty as they transition from an initial state to a decision. These transitions represent new knowledge or changes in belief, which impact the final outcome. We discussed one approach for representing uncertainty in reasoning in Section 4.1. Such a representation utilizes the flexibility of quantum mechanisms and the aspect of the interference effect for unobserved paths rather than the assumption of a single path seen in Markovian models. This improves the flexibility of representation and allows for representations which model fallacies in reasoning (e.g. disjunction fallacy), but also increases the complexity of the resulting models.
All in all, we can conclude there is not a one-size-fits-all approach to best model the human behavior in HCAI. Each approach has its onw pros and flip side, and the best choice largely depends on the specific problem at hand. However, the literature on this topic is very vast and articulated, and this allows designers of HCAI systems to leverage an extensive toolbox to equip AI agents with practical approaches to model human behavior. Therefore, we believe this area is going to emerge as one of the most active in the coming years, in the field cutting across autonomous and adaptive systems, pervasive environments, and advanced AI agents.