|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
PMML 4.4.1 - Naïve BayesNaïve Bayes uses Bayes' Theorem, combined with a ("naive") presumption of conditional independence, to predict the value of a target (output), from evidence given by one or more predictor (input) fields. Given a categorical target field T with possible values
T1,...Tm, and predictor fields
I1,...In, with values (in the current record) of
I1*,...In*, the probability that the target T has value
Ti, given the values of the predictors, is derived as
follows:
Naïve Bayes models require the target field to be discretized so that a finite number of values are considered by the model. On the other hand, predictor fields may be either discrete or continuous. For a discrete field, the probability can be calculated given the number of occurrences of that field. For example, the probability of the target T having value Ti is:
For a continuous field, on the other hand, we assume it has a probability distribution representable by a Gaussian distribution with mean μ and variance σ2 or a Poisson distribution with mean μ. A gaussian distribution would imply that the probability of the continuous variable having value Ij* given a target value Ti is:
while a poisson distribution with mean μ implies:
In this way, for example, a gaussian distribution will give
Where μij is the mean of the joint distribution of variable i with target j. Similarly, σij2 is the variance of the joint distribution of variable i with target j. A count of zero requires special attention. Without adjustment, a count of zero would exercise an absolute veto over a likelihood in which that count appears as a factor. Therefore, the Bayes model incorporates a threshold parameter that specifies a default (usually very small) probability to use in lieu of P(Ij* | Tk) when count[Ij*Ti] is zero. Similarly, since the probabilily of a continuous distribution can reach the value of 0 as the lower limit, the same threshold parameter is used as the probability of the continuous variable when the calculated probability of the distribution falls below that value. A second adaptation to missing values in the training data, involves the denominator count[Ti] in the conditional-probability terms. Accuracy improves if the denominator for P(Ij* | Ti) is replaced by the sum Sumk count[Ij*Ti], that is, the sum of the counts of co-occurrences of target value Ti with any (non-missing) value of item Ij. In sum, a Naïve Bayes model requires the following attributes and elements:
A NaiveBayesModel essentially defines a set of matrices. For each input field there is a matrix which contains the frequency counts of a discrete input value or in case of a Gaussian distribution, the mean and the standard deviation of the joint distribution of the continuous input with a target value.
XSD<xs:element name="NaiveBayesModel"> <xs:complexType> <xs:sequence> <xs:element ref="Extension" minOccurs="0" maxOccurs="unbounded"/> <xs:element ref="MiningSchema"/> <xs:element ref="Output" minOccurs="0"/> <xs:element ref="ModelStats" minOccurs="0"/> <xs:element ref="ModelExplanation" minOccurs="0"/> <xs:element ref="Targets" minOccurs="0"/> <xs:element ref="LocalTransformations" minOccurs="0"/> <xs:element ref="BayesInputs"/> <xs:element ref="BayesOutput"/> <xs:element ref="ModelVerification" minOccurs="0"/> <xs:element ref="Extension" minOccurs="0" maxOccurs="unbounded"/> </xs:sequence> <xs:attribute name="modelName" type="xs:string"/> <xs:attribute name="threshold" type="REAL-NUMBER" use="required"/> <xs:attribute name="functionName" type="MINING-FUNCTION" use="required"/> <xs:attribute name="algorithmName" type="xs:string"/> <xs:attribute name="isScorable" type="xs:boolean" default="true"/> </xs:complexType> </xs:element> The isScorable attribute indicates whether the model is valid for scoring. If this attribute is true or if it is missing, then the model should be processed normally. However, if the attribute is false, then the model producer has indicated that this model is intended for information purposes only and should not be used to generate results. In order to be valid PMML, all required elements and attributes must be present, even for non-scoring models. For more details, see General Structure. Bayes InputsThe BayesInputs element contains several BayesInput elements. <xs:element name="BayesInputs"> <xs:complexType> <xs:sequence> <xs:element ref="Extension" minOccurs="0" maxOccurs="unbounded"/> <xs:element ref="BayesInput" maxOccurs="unbounded"/> </xs:sequence> </xs:complexType> </xs:element> Bayes InputFor a discrete field, each BayesInput contains the counts pairing the discrete values of that field with those of the target field. For a continuous field, the BayesInput element lists the distributions obtained for that field with each value of the target field. BayesInput may also be used to define how continuous values are encoded as discrete bins. (Discretization is achieved using DerivedField; only the Discretize mapping for DerivedField may be invoked here). <xs:element name="BayesInput"> <xs:complexType> <xs:sequence> <xs:element ref="Extension" minOccurs="0" maxOccurs="unbounded"/> <xs:choice> <xs:element ref="TargetValueStats" minOccurs="1" maxOccurs="1"/> <xs:sequence> <xs:element ref="DerivedField" minOccurs="0" maxOccurs="1"/> <xs:element ref="PairCounts" minOccurs="1" maxOccurs="unbounded"/> </xs:sequence> </xs:choice> </xs:sequence> <xs:attribute name="fieldName" type="FIELD-NAME" use="required"/> </xs:complexType> </xs:element> Note that a BayesInput element encompasses either one TargetValueStats element or one or more PairCounts elements. Element DerivedField can only be used in conjunction with PairCounts. Bayes OutputBayesOutput contains the counts associated with the values of the target field. <xs:element name="BayesOutput"> <xs:complexType> <xs:sequence> <xs:element ref="Extension" minOccurs="0" maxOccurs="unbounded"/> <xs:element ref="TargetValueCounts"/> </xs:sequence> <xs:attribute name="fieldName" type="FIELD-NAME" use="required"/> </xs:complexType> </xs:element> Target Value StatsTargetValueStats serves as the envelope for element TargetValueStat. It is used for a continuous input field Ii to define statistical measures associated with each value of the target field. As defined in CONTINUOUS-DISTRIBUTION-TYPES, different distribution types can be used to represent such measures. For Bayes models, these are restricted to Gaussian and Poisson distributions. For more details on these continuous distribution types, please refer to BaselineModel. <xs:element name="TargetValueStats"> <xs:complexType> <xs:sequence> <xs:element ref="Extension" minOccurs="0" maxOccurs="unbounded"/> <xs:element ref="TargetValueStat" maxOccurs="unbounded"/> </xs:sequence> </xs:complexType> </xs:element> <xs:element name="TargetValueStat"> <xs:complexType> <xs:sequence> <xs:element ref="Extension" minOccurs="0" maxOccurs="unbounded"/> <xs:group ref="CONTINUOUS-DISTRIBUTION-TYPES" minOccurs="1"/> </xs:sequence> <xs:attribute name="value" type="xs:string" use="required"/> </xs:complexType> </xs:element> Pair CountsPairCounts lists, for a field Ii's discrete value Iij, the TargetValueCounts that pair the value Iij with each value of the target field. <xs:element name="PairCounts"> <xs:complexType> <xs:sequence> <xs:element ref="Extension" minOccurs="0" maxOccurs="unbounded"/> <xs:element ref="TargetValueCounts"/> </xs:sequence> <xs:attribute name="value" type="xs:string" use="required"/> </xs:complexType> </xs:element> Target Value CountsTargetValueCounts lists the counts associated with each value of the target field. However, a TargetValueCount whose count is zero may be omitted. Within BayesOutput, TargetValueCounts lists the total count of occurrences of each target value. Within PairCounts, TargetValueCounts lists, for each target value, the count of the joint occurrences of that target value with a particular discrete input value. <xs:element name="TargetValueCounts"> <xs:complexType> <xs:sequence> <xs:element ref="Extension" minOccurs="0" maxOccurs="unbounded"/> <xs:element ref="TargetValueCount" maxOccurs="unbounded"/> </xs:sequence> </xs:complexType> </xs:element> <xs:element name="TargetValueCount"> <xs:complexType> <xs:sequence> <xs:element ref="Extension" minOccurs="0" maxOccurs="unbounded"/> </xs:sequence> <xs:attribute name="value" type="xs:string" use="required"/> <xs:attribute name="count" type="REAL-NUMBER" use="required"/> </xs:complexType> </xs:element> Scoring procedureGiven an input vector like (i12,i23,i3), where i12,i23 are discrete variables and i3 is a continuous variable, the probability for class t1 is computed as
Note that we used a Gaussian distribution for scoring continuous variables. The scoring procedure would need to adjusted, if the Poisson distribution were to be used. When scoring, missing values are simply ignored. That is, the
conditional-probability factor associated with a missing predictor field is
omitted. For example, given an input vector with missing values
(-,i23,-) the probability for class t1 is computed
as
Scoring procedure, example<PMML xmlns="https://www.dmg.org/PMML-4_4" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" version="4.4"> <Header copyright="Copyright (c) 2013, DMG.org"/> <DataDictionary numberOfFields="6"> <DataField name="age of individual" optype="continuous" dataType="double"/> <DataField name="gender" optype="categorical" dataType="string"> <Value value="female"/> <Value value="male"/> </DataField> <DataField name="no of claims" optype="categorical" dataType="string"> <Value value="0"/> <Value value="1"/> <Value value="2"/> <Value value=">2"/> </DataField> <DataField name="domicile" optype="categorical" dataType="string"> <Value value="suburban"/> <Value value="urban"/> <Value value="rural"/> </DataField> <DataField name="age of car" optype="continuous" dataType="double"/> <DataField name="amount of claims" optype="categorical" dataType="integer"> <Value value="100"/> <Value value="500"/> <Value value="1000"/> <Value value="5000"/> <Value value="10000"/> </DataField> </DataDictionary> <NaiveBayesModel modelName="NaiveBayes Insurance" functionName="classification" threshold="0.001"> <MiningSchema> <MiningField name="age of individual"/> <MiningField name="gender"/> <MiningField name="no of claims"/> <MiningField name="domicile"/> <MiningField name="age of car"/> <MiningField name="amount of claims" usageType="target"/> </MiningSchema> <BayesInputs> <BayesInput fieldName="age of individual"> <TargetValueStats> <TargetValueStat value=" 100"> <GaussianDistribution mean="32.006" variance="0.352"/> </TargetValueStat> <TargetValueStat value=" 500"> <GaussianDistribution mean="24.936" variance="0.516"/> </TargetValueStat> <TargetValueStat value=" 1000"> <GaussianDistribution mean="24.588" variance="0.635"/> </TargetValueStat> <TargetValueStat value=" 5000"> <GaussianDistribution mean="24.428" variance="0.379"/> </TargetValueStat> <TargetValueStat value="10000"> <GaussianDistribution mean="24.770" variance="0.314"/> </TargetValueStat> </TargetValueStats> </BayesInput> <BayesInput fieldName="gender"> <PairCounts value="male"> <TargetValueCounts> <TargetValueCount value="100" count="4273"/> <TargetValueCount value="500" count="1321"/> <TargetValueCount value="1000" count="780"/> <TargetValueCount value="5000" count="405"/> <TargetValueCount value="10000" count="42"/> </TargetValueCounts> </PairCounts> <PairCounts value="female"> <TargetValueCounts> <TargetValueCount value="100" count="4325"/> <TargetValueCount value="500" count="1212"/> <TargetValueCount value="1000" count="742"/> <TargetValueCount value="5000" count="292"/> <TargetValueCount value="10000" count="48"/> </TargetValueCounts> </PairCounts> </BayesInput> <BayesInput fieldName="no of claims"> <PairCounts value="0"> <TargetValueCounts> <TargetValueCount value="100" count="4698"/> <TargetValueCount value="500" count="623"/> <TargetValueCount value="1000" count="1259"/> <TargetValueCount value="5000" count="550"/> <TargetValueCount value="10000" count="40"/> </TargetValueCounts> </PairCounts> <PairCounts value="1"> <TargetValueCounts> <TargetValueCount value="100" count="3526"/> <TargetValueCount value="500" count="1798"/> <TargetValueCount value="1000" count="227"/> <TargetValueCount value="5000" count="152"/> <TargetValueCount value="10000" count="40"/> </TargetValueCounts> </PairCounts> <PairCounts value="2"> <TargetValueCounts> <TargetValueCount value="100" count="225"/> <TargetValueCount value="500" count="10"/> <TargetValueCount value="1000" count="9"/> <TargetValueCount value="5000" count="0"/> <TargetValueCount value="10000" count="10"/> </TargetValueCounts> </PairCounts> <PairCounts value=">2"> <TargetValueCounts> <TargetValueCount value="100" count="112"/> <TargetValueCount value="500" count="5"/> <TargetValueCount value="1000" count="1"/> <TargetValueCount value="5000" count="1"/> <TargetValueCount value="10000" count="8"/> </TargetValueCounts> </PairCounts> </BayesInput> <BayesInput fieldName="domicile"> <PairCounts value="suburban"> <TargetValueCounts> <TargetValueCount value="100" count="2536"/> <TargetValueCount value="500" count="165"/> <TargetValueCount value="1000" count="516"/> <TargetValueCount value="5000" count="290"/> <TargetValueCount value="10000" count="42"/> </TargetValueCounts> </PairCounts> <PairCounts value="urban"> <TargetValueCounts> <TargetValueCount value="100" count="1679"/> <TargetValueCount value="500" count="792"/> <TargetValueCount value="1000" count="511"/> <TargetValueCount value="5000" count="259"/> <TargetValueCount value="10000" count="30"/> </TargetValueCounts> </PairCounts> <PairCounts value="rural"> <TargetValueCounts> <TargetValueCount value="100" count="2512"/> <TargetValueCount value="500" count="1013"/> <TargetValueCount value="1000" count="442"/> <TargetValueCount value="5000" count="137"/> <TargetValueCount value="10000" count="21"/> </TargetValueCounts> </PairCounts> </BayesInput> <BayesInput fieldName="age of car"> <DerivedField optype="categorical" dataType="string"> <Discretize field="age of car"> <DiscretizeBin binValue="0"> <Interval closure="closedOpen" leftMargin="0" rightMargin="1"/> </DiscretizeBin> <DiscretizeBin binValue="1"> <Interval closure="closedOpen" leftMargin="1" rightMargin="5"/> </DiscretizeBin> <DiscretizeBin binValue="2"> <Interval closure="closedOpen" leftMargin="5"/> </DiscretizeBin> </Discretize> </DerivedField> <PairCounts value="0"> <TargetValueCounts> <TargetValueCount value="100" count="927"/> <TargetValueCount value="500" count="183"/> <TargetValueCount value="1000" count="221"/> <TargetValueCount value="5000" count="50"/> <TargetValueCount value="10000" count="10"/> </TargetValueCounts> </PairCounts> <PairCounts value="1"> <TargetValueCounts> <TargetValueCount value="100" count="830"/> <TargetValueCount value="500" count="182"/> <TargetValueCount value="1000" count="51"/> <TargetValueCount value="5000" count="26"/> <TargetValueCount value="10000" count="6"/> </TargetValueCounts> </PairCounts> <PairCounts value="2"> <TargetValueCounts> <TargetValueCount value="100" count="6251"/> <TargetValueCount value="500" count="1901"/> <TargetValueCount value="1000" count="919"/> <TargetValueCount value="5000" count="623"/> <TargetValueCount value="10000" count="71"/> </TargetValueCounts> </PairCounts> </BayesInput> </BayesInputs> <BayesOutput fieldName="amount of claims"> <TargetValueCounts> <TargetValueCount value="100" count="8723"/> <TargetValueCount value="500" count="2557"/> <TargetValueCount value="1000" count="1530"/> <TargetValueCount value="5000" count="709"/> <TargetValueCount value="10000" count="100"/> </TargetValueCounts> </BayesOutput> </NaiveBayesModel> </PMML> Given an input vector (age of individual = "24", gender = "male", no of claims = "2", domicile = (missing), age of car = "1") the probability for class "1000" is computed as
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|