{'fraction_non_alphanumeric': 0.027625056331680938, 'fraction_numerical': 0.008066696710229833, 'mean_word_length': 4.955716586151369, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Knowledge Engineering Within A Generalized Bayesian Framework

During the ongoing debate over the representation of uncertainty in Artificial Intelligence, Cheeseman, Lemmer, Pearl, and others have argued that probability theory, and in particular the Bayesian theory, should be used as the basis for the inference mechanisms of Expert Systems dealing with uncertainty. In order to pursue the issue in a practical setting, sophisticated tools for knowledge engineering are needed that allow flexible and understandable interaction with the underlying knowledge representation schemes. This paper describes a Generalized Bayesian framework for building expert systems which function in uncertain domains, using algorithms proposed by Lemmer. It is neither rule-based nor frame-based, and requires a new system of knowledge engineering tools. The framework we describe provides a knowledge-based system architecture with an inference engine, explanation capability, and a unique aid for building consistent knowledge bases.


Introduction
Recent arguments have been made for the use of Bayesian lnferencing techniques for expert systems dealing with uncertainty [Chamiak, 1983;Cheeseman, 1986;Pearl, 1982;Yeh, 1983]. Some expert system (ES) applications involving reasoning under uncertainty may best be handled by mechanisms that are as purely Bayesian as possible. This paper describes a Generalized Bayesian framework for constructing expert systems using networks of probability distributions, based on Lemmer's approach to Generalized Bayesian Inference (GBI) [Lemmer, 1976;Lemmer, 1983;Lemmer and Barth, 1982).
The knowledge representation scheme for our GBI framework consists of a network of intersecting sets of events, and their associated probability distributions. The GBI Framework contains the key ingredients of a classic knowledge-based system architecture: an inference engine with an'explanation capability, and a tool for creating, editing, and maintaining consistency in the knowledge base. These individual components, however, are very different from those of rule-based probabilistic systems such as PROSPECTOR [Duda, Hart, et al., 1979). The purpose of this paper is to describe the GBI framework for building expert systems, its tools for estimating consistent prior probability distributions (the CMD Estimation Aid), and its facilities supporting the utilization and incremental development of knowledge bases, the Updating Mechanism and Explanation Facility. The description of the Explanation Facility will be ommitted here, since it is described In detail in [Norton, 1986] in this volume.
The system described in this paper was developed for application to a problem of real-time discrimination of object types from sensor information. The original approach to this problem involved using a Markov model with a sequence of discriminant tests to classify objects. This approach had several difficulties. Order dependancies among the tests made It hard to tune the system for accurate classifications. Heuristic estimates of probabilities incorporated into state transition matrices caused instabilities in the decision process. In general, the original approach lacked a framework for understanding the results from system test and evaluation and using those results for Improving system performance. A knowledge·based approach to the problem was taken to alleviate these problems. The Generalized Bayesian Framework was developed to provide the tool for knowledge-based handling of the rich environment of probabilistic information available for the problem.
Unfortunately the details of the problem domain are classified. Therefore, in this paper, we will illustrate knowledge representation in the system with a small knowledge base for predicting the weather in Upstate New York; In particular, providing expert estimates of the probability of precipitation. This toy problem example has a similar diagnostic character to the real problem: the goal is to infer the correct Identification or prediction from observed characteristics or measurable quantities. The advice provided by the system, or an expert, for both problem domains, always takes the form of a probabilistic recommendation, because of the uncertainty of the data and Its interpretation.

Knowledge Engineering within the Generalized Bayesian Framework
GBI is a tool for building expert systems to solve Bayesian hierarchical inferencing problems. The Initial phases of knowledge engineering for an expert system using this framework proceed as in any other hierarchical inferenclng problem [Hayes-Roth, Waterman, and Lenat 1983]. Together with one or more domain experts, the knowledge engineer considers the expert thought process used in solving the problem at hand. A series of stages is sketched out which takes observable evidence, relates it to intermediate hypotheses, and uses these Intermediate hypotheses to determine the validity of one or more goal variables. The problem variables (evidence, hypothesis, and goal variables) represent assertions of Interest. Sets of these variables form the nodes in the network structure of a GBI knowledge base.
In a typical rule-based expert system framework, the knowledge engineer, In cooperation with one or more domain experts, specifies the relationships between the problem variables using rules. In GBI, the knowledge engineer groups together those variables for which some probabilistic constraints (correlation Information) are known .. These structures are called Local Event Groups by Konolige in an appendix to [Duda, Hart, et al, 1979]. A LEG Network (LEG Net) is a collection of LEGs, some of which share common variables. The LEG Net structure arises naturally from the hierarchical problem organization.
The aim of our sample knowledge base is to combine information from many sources to match the weather predicting capability of local experts in Upstate New York. An Initial LEG Net for this knowledge base Is shown in Figure 1. Each LEG Is Indicated by a round edged box labeled with a name In bold type.The names of the variables in the LEG are listed inside the box. Square shaded boxes show the variables in LEG Intersections, and are attached to lines connecting the intersecting LEGs.
In our example the goal variables are Snow-Tomorrow, Rain-Tomorrow, and No-Precip-Tomorrow; In the Kind-of-Precip LEG, representing the events or assertions "it will rain tomorrow," "It will snow tomorrow: and -.here will be no precipitation tomorrow.• The advice to the user will be whatever values are attained by their posterior probabilities. The hypothesis variables Folk-Precip and Others-Precip represent the assertions that folk signs and other-agencies should be considered in predicting precipitation for tomorrow. Examples of evidence variables are Moon-Haze and Bunions-Ache In the Folk-Precip LEG. Bunions-Ache will be true if Grandma has complained about her feet lately, and Moon-Haze will be true if haze appears around the moon. Other evidence variables in the example are mutually exclusive. In Figure 1 mutually exclusive events are shown boxed within their LEG. The evidence variables In this example represent events that either occur or do not occur and hence have a posterior probability of either 0.0 or 1.0 . Such variables are called Binary Evidence Variables__ or BEVs.
If all the problem variables were grouped together, we could, in principle, establish a single underlying probability distribution. In practice that Is quite infeasible since the storage requirements for probability distributions are exponential (2°) in tenns of the number of variables covered. Even for the simple weather example of Figure 1, over 8 miUion probabilities may be required for the underlying distribution.
Lemmer's GBI approach avoids this problem by using only Component Marginal Distributions (CMDs) for each LEG. [Lemmer, 1983]. Each LEG in a LEG Net is associated with a CMD which covers just those variables in the LEG. Taken together, the CMOs for a LEG Net are an approximation to the underlyin g distribution. It is an approximation in the sense that no n-variate constraints are specified. If the largest LEG contains only m variables then the LEG Net can co. ntain no constraints of order greater than m. This approximation is reasonable since by breaking up the event space into LEGs we are specifying which constraints are important, and which are not. The size requirement for the network is the total of the sizes of the CMDs. The CMOs which are to be specified for the network In Figure 1 will have a total of 368 elements versus more than 8 miUion In the underlying distribution.
The Lemmer approach requires 2 m -1 constraints to fully specify an m-variate CMD. However, It allows the system to make use of information about mutually exclusive or implied events which may cause many of the constraints to be set to zero. The CMD Estimation Aid exploits this capability, so that in practice the knowledge engineer has many fewer probabilities to determine or estimate than the product of the number of LEGs and (2 m -1). The number of constraints to be specified can be reduced still further by accepting minimum Information default values for the higher · order constraints, which are automatically generated. The examples In the next section illustrate these features of the system.

Specifying LEGs and CMDs
The CMD Estimation Aid of the GBI framework provides an interactive, menu-driven Interface for defining events, LEGs, LEG Nets, and estimating CMDs. LEG Intersections, through which probability Is propagated, are managed automatically. In the following paragraphs we describe knowledge engineering within the GBI framework by walking through the features of the CMD Estimation Aid and illustrating the thought processes used during estimation In terms of the example in Figure 1.
One body of evidence weighed by our local expert Is that of precipitation predictions from other sources. Our example considers two: the probability of precipitation predicted by the National Weather Service, NWS--Precip, and that predicted by the Farmers Almanac, FA--Precip. The heuristic that our local expert uses Is that when either one of these sources predicts precipitation It increases the chances of rain or snow tomorrow. The hypothesis variable Others-Precip Is Introduced to Indicate the probability of precipitation predicted from either of these two sources, and we form the LEG Other-Predictions to handle It and the two evidence variables. The constraints on the distribution for Other-Predictions are that If either or both of the variables NWS--Precip and FA--Preclp are certain, then Others-Precip Is certain. In addition, let us say we know, from statistical Information, that the National Weather Service predicts precipitation 55% of the time, that the Farmers Almanac predicts precipitation 45% of the time, and that the two sources agree on the prediction 35% of the time.
Using the CMD Estimation aid requires the knowledge engineer to specify the constraints on a LEG In a canonical order, to support efficiency In the algorithm for constructing the distribution; however, the knowledge engineer has the freedom to select the style of constraints by choosing the order of variables for consideration and the form of the constraint that is most easily understood or calculated, either a joint or conditional probability. (The details of the CMD estimation algorithms are described in [Lemmer, 1976] and [Lemmer and Norton, 1986).) In the LEG Other-Predictions we chose the constraints with a diagnostic •style· In mind; since we wanted to specify the conditional probabilities of the hypothesis variable given the evidence variables as symptoms. To accomplish this we proceed by specifying the marginal probabilities for FA-Precip and NWS-Precip first, 0.45 and 0.55, respectively. The system prompts next for either the conditional of FA-Precip given NWS-Preclp, or the marginal joint probability of FA-Precip and NWS-Precip.
The latter we know to be 0.35. Now, we must specify the marginal probability for the hypothesis variable Others-Precip. A simple calculation with a Venn diagram shows that It must be 0.65. From here on out the system prompts us for constraints of either Others-Precip conditioned on the evidence variables, or the related joint probabilities. The former is easiest to specify, since P(Others-Precip 1 FA-Precip), P(Others I estimate to occ ur about 45% of the time, the latter about 65%, and that both events happen together about . 30% of the time. The hypothesis variable Folk-Precip represents the belief we have that the folk signs indicate rain, which we estimate at 0.55; I.e. they are slightly predisposed to predict rain. If there's haze around the 1 moon, however, we're 60% sure that a folksy prediction 1s being made. If Grandma is complaining about her feet, we're 80% sure. And If both of these events occur then it's 99% certain that folk signs are telling us that precipitation will occur. The constraints just described are summarized In Figure 3. The effect of knowing that the folk signs indicate rain on our actual prediction is taken account of in the constraints on the goal leg

Expediting CMD Estimation
The examples above show how a simple probabilistic OR relationship and weighing of beliefs can be specified using the CMD Estimation Aid. Generally, however, the knowledge engineer needs to make use of shortcut facilities that the CMD estimation aid provides. These are: allowing minimum information values as defaults for constraint probabilities, and taking advantage of "forbidden" and "cutoff" relationships among events. Whenever the CMD Estimation Aid prompts the user to specify a constraint it offers a range of values possible for the constraint that is consistent with the previously specified constraints. Within this range the value offered as a default for the constraint is the probability that satisfies a minirrum information condition on the distribution under construction (Lemmer and Norton 1986]. In the example above the default value for the constraint P(FA-Precip & NWS-Precip) was 0.2475, which represents independence of the two variables. For small LEGs of only three or four variables, or larger ones that are highly constrained by cutoff or forbidden relationships, such default values are not so important except as a convenient function of the user interface. In other cases the ability to take default values for constraints that are needed to determine the distribution, but for which values are not known or cannot be estimated easily, is a critical feature. For example in the goal LEG Kind-Of-Precip, the user may not feel comfortable estimating constraints on more than four variables, such as P(Rain-Temp, Snow-Temp, Others-Precip, Folk-Precip), or any conditional form of these four variables, though It is not desirable to decompose Klnd-Of-Precip into smaller LEGs. The mechanism for default values allows the user to ignore constraints above a certain order, or specify only those for which some justification is available, taking defaults for the rest.
The CMD Estimation Aid also allows the user to specify "forbidden" and "cutoff" relationships between variables that automatically limit the number of constraints required to specify the distribution. Forbidden relationships indicate that the occurrence of two variables in a joint event is impossible, as is the case for mutually exclusive events. In the LEG Expected-Temperature the three evidence variables Temp-L T-28F, Temp-BT-28�36F, and Temp-GT-36F are mutually exclusive (and collectively exhaustive) and so are given the forbidden relationship. As a result, the knowledge engineer needs to specify only 12 constraints on joint events, Instead of 26, and, if defaults are taken for constraints on more than two variables, then only six constraint probabilities need to be specified.
Cutoff relationships between variables indicate that one variable cannot occur in a joint event unless another does, in an implication relation. For example, in the Other-Predictions LEG, we might have specified that NWS--Precip only occ urs if FA-Precip occurs. This means that joint events in which NWS--Precip occur, but FA--Precip does not occur, have probability 0.0. Thus the number of constraints required to specify the distribution is reduced.
The knowledge engineer can make use of all of these mechanisms to reflect the expertise being modeled, and to reduce the amount of information needed to define the knowledge base. As might be expected, our experience has been that cutoff and forbidden relationships nearly always exist among evidence variables but are harder to find in LEGs where hypothesis variables have been introduced to represent a concept or summarize an effect. In either case, but especially the latter, the default values mechanism makes It easy to specify what low order constraints are known for a distribution, while retting the system make reasonable assumptions for the higher order constraints that the expert or knowledge engineer may be unable to specify.

Generalized Bayesian Inference
Once prior CMDs for each LEG have been estimated by the knowledge engineer using the CMD Estimation Aid, posterior CMDs are calculated by the GBI Updating mechanism as evidence is gathered; I.e. as the posterior probabilities of the evidence variables are determined from observable data. Summing out the marginal probability of a LEG variable from a posterior CMO provides the advice about the occ urrence of the event represented by that variable, based on the evidence gathered so far. As more evidence is gathered, a new posterior CMD may be calculated for a LEG, and new marginal probabilities determined for the LEG variables. When the posterior probabilities for all evidence variables have been determined the marginal probabilities of the goal variables represent the final advice of the system.
The GBI Updating rule has been formally described in [Lemmer, 1983] and [Lemmer and Barth, 1982). Inferences are propagated through the CMOs over LEG Intersections. If A and a are two Intersecting LEGs. the formula for updating the CMD of A after a new posterior CMD for a has been deteiJll ined is: where a is a subset of the variables In A. and P(A,a) represents the prior probability for the joint event in which the variables of a occur and the other variables of A do not. (As a ranges over the subsets of�. P(A,a) ranges over the prior probabilities of the joint events In A's CMD). Similarly, P'(A,a) represents the posterior probability of the joint event in which the variables of a occur and the other variables of A do not.
It is important to realize that P(A,a) represents probabilities in the most recent posterior that was determined for A. Initially P(A,a) has the value that was assigned during the knowledge engineering phase with the CMO Estimation Aid, but as evidence is collected and udating proceeds, several new CMDs may be calculated for A. Each time a new CMD needs to be calculated the P(A,a) values in the formula are the ones from the last update to the CMD, not the a priori distrbution.
Viewed as a sequential process, the updating formula means that for each joint event .a in A (P(e) = P{A,a) for some .a) we 1. Determine the joint event i in the LEG intersection of � and 6 which is a component of ,e.
2. Sum out from the previous CMD of B. to obtain the prior probability of i,. This is the denominator of the fraction given in the formula.
3. Sum out from the new CMD of a to obtain the new posterior marginal probability of i This is the numerator of the fraction in the formula. 4. Multiply P(e) by the quotient of the values obtained in 2 and 3 above to obtain its posterior P'(e).
As an example of the Updating mechanism consider an update of the CMO of the goal LEG Kind-of-Precip from the evidence LEG Folk-Predictions in Figure 1. Figure   After it is determined (through querying the user or accessing a data base) that the event Bunions-Ache occurs and that Moon-Haze does not occur, the posterior distribution of Folk-Predictions is calculated by applying the GBI Updating rule with the marginal distribution of the two evidence events as the updating distribution. In this instance, the GBI Updating rule is equivalent to Bayes' Rule applied to the CMD of Folk-Predictions, since the conditioning events are certain.{Lemmer 1983]. In the next step, the new CMD for the intersection of Folk-Predictions and Kind-of-Precip Is calculated by summing out over the CMD of Folk-Predictions. Now, the CMD of Kind-of-Precip is calculated using the GBI Updating rule. For example, the prior probability of the joint event that Snow-Tomorrow, Folk-Precip, Others-Precip, and Snow-Temp occurred, but No-Precip-Tomorrow, Rain-Tomorrow, and Rain-Temp did not occur, in the Kind-of-Precip LEG, is multiplied by the quotient of the posterior and prior probabilities for the event Folk-Precip in the LEG Intersection. (0.1599 • 0.1543 • 0.5700 I 0.5500}. Next the CMD of Other-Predictions is updated using the CMD for the LEG intersection between Kind-of-Precip and Others-Predictions. The posterior CMD for the LEG intersection is calculated by summing out from the new posterior CMD for Kind-of-Precip to obtain the probabilities of Others-Precip and not Others-Precip. The GBI Updating rule is then used to calculate the new posterior CMD for the Other-Predictions LEG. Posterior CMDs for the other LEGs would also be calculated from the effects of the update to Folk-Predictions so that consistent CMDs are maintained throughout the network after the completion of each update.
The fact that propagation of probability occ urs throughout the LEG Net is an important difference between the the GBI Updating rule and the mechanisms used In systems like MYCIN [Buchanan and Shortliffe, 1984] and PROSPECTOR. In PROSPECTOR for example, probability is only propagated from evidence to goal variables, and the prior probabilities for unexamined evidence remain unmodified. In the GBI mechanism consistent probability distributions are maintained: ·summing out the marginal probability for a