Statistics
PMML3.1 Menu

Home


PMML Notice and License

Changes


Conformance

General Structure

Header

Data
Dictionary


Mining
Schema


Transformations

Statistics

Taxomony

Targets

Output

Functions

Built-in Functions

Model Composition

Model Verification


Association Rules

Cluster
Models


General
Regression


Naive
Bayes


Neural
Network


Regression

Ruleset

Sequences

Text Models

Trees

Vector Machine

PMML 3.1 - Statistics

This schema for statistics provides a basic framework for representing univariate statistics. It is used by schemas for specific models as ModelStats.


  <xs:element name="ModelStats">
    <xs:complexType>
      <xs:sequence>
        <xs:element ref="Extension" minOccurs="0" maxOccurs="unbounded" />
        <xs:element maxOccurs="unbounded" ref="UnivariateStats" />
      </xs:sequence>
    </xs:complexType>
  </xs:element>

The statistics for a model is made of the collection of the statistics for single fields.

Univariate Statistics


  <xs:element name="UnivariateStats">
    <xs:complexType>
      <xs:sequence>
        <xs:element ref="Extension" minOccurs="0" maxOccurs="unbounded" />
        <xs:element minOccurs="0" ref="Counts" />
        <xs:element minOccurs="0" ref="NumericInfo" />
        <xs:element minOccurs="0" ref="DiscrStats" />
        <xs:element minOccurs="0" ref="ContStats" />
      </xs:sequence>
      <xs:attribute name="field" type="FIELD-NAME" />
    </xs:complexType>
  </xs:element>

An UnivariateStats element contains statistical information about a single mining field. UnivariateStats elements in ModelStats must have a field name. In other cases the attribute 'field' may be missing. Discrete and continuous statistics are possible simultaneously for numeric fields. This may be important if a numeric field has too many discrete values. The statistics can include the most frequent values and also a complete histogram distribution.

Statistics for ordinal fields are contained in DiscrStats.


  <xs:element name="Counts">
    <xs:complexType>
      <xs:sequence>
        <xs:element ref="Extension" minOccurs="0" maxOccurs="unbounded" />
      </xs:sequence>
      <xs:attribute name="totalFreq" type="xs:nonNegativeInteger" use="required" />
      <xs:attribute name="missingFreq" type="xs:nonNegativeInteger" />
      <xs:attribute name="invalidFreq" type="xs:nonNegativeInteger" />
    </xs:complexType>
  </xs:element>

The element Counts carries counters for frequency of values with respect to their state of being missing, invalid, or valid.

totalFreq counts all records, same as for statistics of all mining fields, missingFreq counts the number of records where value is missing, invalidFreq counts the number of records with values other than valid. The total frequency includes the missing values and invalid values.


  <xs:element name="NumericInfo">
    <xs:complexType>
      <xs:sequence>
        <xs:element ref="Extension" minOccurs="0" maxOccurs="unbounded" />
        <xs:element minOccurs="0" maxOccurs="unbounded" ref="Quantile" />
      </xs:sequence>
      <xs:attribute name="minimum" type="NUMBER" />
      <xs:attribute name="maximum" type="NUMBER" />
      <xs:attribute name="mean" type="NUMBER" />
      <xs:attribute name="standardDeviation" type="NUMBER" />
      <xs:attribute name="median" type="NUMBER" />
      <xs:attribute name="interQuartileRange" type="NUMBER" />
    </xs:complexType>
  </xs:element>

The values for mean, minimum, maximum and standardDeviation are defined as usual. median is calculated as 50% quantile; interQuartileRange is calculated as (75% quantile - 25% quantile).


  <xs:element name="Quantile">
    <xs:complexType>
      <xs:sequence>
        <xs:element ref="Extension" minOccurs="0" maxOccurs="unbounded" />
      </xs:sequence>
      <xs:attribute name="quantileLimit" type="PERCENTAGE-NUMBER" use="required" />
      <xs:attribute name="quantileValue" type="NUMBER" use="required" />
    </xs:complexType>
  </xs:element>

quantileLimit is a percentage number between 0 and 100. quantileValue is the corresponding value in the domain of field values.

Discrete Statistics


  <xs:element name="DiscrStats">
    <xs:complexType>
      <xs:sequence>
        <xs:element ref="Extension" minOccurs="0" maxOccurs="unbounded" />
        <xs:element minOccurs="0" maxOccurs="2" ref="Array" />
      </xs:sequence>
      <xs:attribute name="modalValue" type="xs:string" />
    </xs:complexType>
  </xs:element>

modalValue is the most frequent discrete value. If only a single array is present, it must be of type INT-ARRAY and contain a compact representation of all frequency numbers. If there is an additional STRING-ARRAY then the frequency numbers in the INT-ARRAY correspond to the string values one by one. Otherwise the frequency numbers correspond to the list of (valid) values as given in the DataDictionary. If the statistics refer to a DerivedField, then all values are listed in the STRING-ARRAY because a DerivedField has no means to specify a list of valid values. If DiscrStats contains an array of string values and the corresponding DataField defines a list of valid values, then both sets must be the same.

Continuous Statistics


  <xs:element name="ContStats">
    <xs:complexType>
      <xs:sequence>
        <xs:element ref="Extension" minOccurs="0" maxOccurs="unbounded" />
        <xs:element minOccurs="0" maxOccurs="unbounded" ref="Interval" />
        <xs:group minOccurs="0" ref="FrequenciesType" />
      </xs:sequence>
      <xs:attribute name="totalValuesSum" type="NUMBER" />
      <xs:attribute name="totalSquaresSum" type="NUMBER" />
    </xs:complexType>
  </xs:element>

  <xs:group name="FrequenciesType">
    <xs:sequence>
      <xs:group ref="INT-ARRAY" />
      <xs:group minOccurs="0" maxOccurs="2" ref="NUM-ARRAY" />
    </xs:sequence>
  </xs:group>

The three ARRAYs contain the frequencies, sum of values, and sum of squared values for each interval.

Note: Interval is defined in the schema for DataDictionary.

A Partition contains statistics for a subset of records, for example it can describe the population in a cluster. The content of a partition mirrors the definition of the general univariate statistics. That is, each partition describes the distribution per field. For each field there can be information about frequencies, numeric moments, etc.


  <xs:element name="Partition">
    <xs:complexType>
      <xs:sequence>
        <xs:element ref="Extension" minOccurs="0" maxOccurs="unbounded" />
        <xs:element minOccurs="0" maxOccurs="unbounded" ref="PartitionFieldStats" />
      </xs:sequence>
      <xs:attribute name="name" type="xs:string" use="required" />
      <xs:attribute name="size" type="xs:nonNegativeInteger" />
    </xs:complexType>
  </xs:element>

The attribute name identifies the partition. The attribute size is the number of records. All aggregates in PartitionFieldStats must have size = totalFrequency in Counts if specified.


  <xs:element name="PartitionFieldStats">
    <xs:complexType>
      <xs:sequence>
        <xs:element ref="Extension" minOccurs="0" maxOccurs="unbounded" />
        <xs:element minOccurs="0" ref="Counts" />
        <xs:element minOccurs="0" ref="NumericInfo" />
        <xs:group minOccurs="0" ref="FrequenciesType" />
      </xs:sequence>
      <xs:attribute name="field" type="FIELD-NAME" use="required" />
    </xs:complexType>
  </xs:element>

field references to (the name of) a MiningField for background statistics. The sequence of NUM-ARRAYs is the same as for ContStats. For categorical fields there is only one array containing the frequencies; for numeric fields, the second and third array contain the sums of values and the sums of squared values, respectively. The number of values in each array must match the number of categories or intervals in UnivariateStats of the field.

e-mail info at dmg.org