Statistics
PMML2.0 Menu

Home


PMML Notice and License

General Structure

Header

Data
Dictionary


Mining
Schema


Data Flow

Transformations

Statistics

Conformance

Taxomony

Trees

Regression

General
Regression


Cluster
Models


Association Rules

Neural
Network


Naive
Bayes


Sequences

PMML 2.0 -- Statistics

This DTD subset for statistics provides a basic framework for representing univariate statistics. It is used by DTDs for specific models as ModelStats.


<!ELEMENT ModelStats ( Extension*, UnivariateStats+ ) >

The statistics for a model is made of the collection of the statistics for single fields.

Univariate Statistics


<!ENTITY  % AGGREGATE "(Counts?, NumericInfo?)" >

<!ELEMENT UnivariateStats ( Extension*, (%AGGREGATE;)?, 
                            DiscrStats?, ContStats? ) >
<!ATTLIST UnivariateStats 
     field              %FIELD-NAME;               #IMPLIED 
>

An UnivariateStats element contains statistical information about a single mining field. UnivariateStats elements in ModelStats must have a field name. In other cases the attribute 'field' may be missing. Discrete and continuous statistics are possible simultaneously for numeric fields. This may be important if a numeric field has too many discrete values. The statistics can include the most frequent values and also a complete histogram distribution.

Statistics for ordinal fields are contained in DiscrStats.


<!ELEMENT Counts EMPTY>
<!ATTLIST Counts
     totalFreq           %NUMBER;                  #REQUIRED
     missingFreq         %NUMBER;                  #IMPLIED
     invalidFreq         %NUMBER;                  #IMPLIED
>

The element Counts carries counters for frequency of values with respect to their state of being missing, invalid, or valid.

totalFreq counts all records, same as for statistics of all mining fields, missingFreq counts the number of records where value is missing, invalidFreq counts the number of records with values other than valid. The total frequency includes the missing values and invalid values.


<!ELEMENT NumericInfo (Quantile*) >
<!ATTLIST NumericInfo
     minimum              %NUMBER;                  #IMPLIED
     maximum              %NUMBER;                  #IMPLIED
     mean                 %NUMBER;                  #IMPLIED
     standardDeviation    %NUMBER;                  #IMPLIED
     median               %NUMBER;                  #IMPLIED
     interQuartileRange   %NUMBER;                  #IMPLIED
>

The values for mean, minimum, maximum and standardDeviation are defined as usual. median is calculated as 50% quantile; interQuartileRange is calculated as (75% quantile - 25% quantile).


<!ELEMENT Quantile EMPTY>
<!ATTLIST Quantile
     quantileLimit        %PERCENTAGE-NUMBER;       #REQUIRED
     quantileValue        %NUMBER;                  #REQUIRED
>

quantileLimit is a percentage number between 0 and 100. quantileValue is the corresponding value in the domain of field values.


<!ELEMENT DiscrStats ( Extension*, (%STRING-ARRAY;)?, (%INT-ARRAY;)? ) >
<!ATTLIST DiscrStats
     modalValue            CDATA                    #IMPLIED
>

modalValue is the most frequent discrete value. The INT-ARRAY contains a compact representation of all frequency numbers. If there is an array of string values then the frequency numbers in the INT-ARRAY correspond to the string values one by one. Otherwise the frequency numbers correspond to the list of (valid) values as given in the DataDictionary. If the statistics refer to a DerivedField, then all values are listed in the STRING-ARRAY because a DerivedField has no means to specify a list of valid values. If DiscrStats contains an array of string values and the corresponding DataField defines a list of valid values, then both sets must be the same.


<!ELEMENT ContStats (Extension*, Interval*,
                    (%INT-ARRAY;)?, 
                    (%NUM-ARRAY;)?, (%NUM-ARRAY;)? )>
<!ATTLIST ContStats
     totalValuesSum         %NUMBER;                #IMPLIED
     totalSquaresSum        %NUMBER;                #IMPLIED
>

The three ARRAY's contain the frequencies, sum of values, and sum of squared values for each interval.

Note: Interval is defined in the DTD for DataDictionary.

A Partition contains statistics for a subset of records, for example it can describe the population in a cluster.



<!ELEMENT Partition (PartitionFieldStats+) >
<!ATTLIST Partition
     name                   CDATA                   #REQUIRED
     size                   %NUMBER;                #IMPLIED
>

name identifies the partition. size is the number of records, all AGGREGATEs in PartitionFieldStats must have total - frequency = size.

 

<!ELEMENT PartitionFieldStats ( (%AGGREGATE;), (%NUM-ARRAY;)* ) >
<!ATTLIST PartitionFieldStats
     field                     %FIELD-NAME;           #REQUIRED
>

field references to (the name of) a MiningField for background statistics. The sequence of NUM-ARRAYs is the same as for ContStats. For categorical fields there is only one array containing the frequencies; for numeric fields, the second and third array contain the sums of values and the sums of squared values, respectively. The number of values in each array must match the number of categories or intervals in UnivariateStats of the field.

e-mail info at dmg.org