com.tdunning.math.stats

Class Comparison

java.lang.Object

com.tdunning.math.stats.Comparison

public class Comparison
extends Object

Static class with methods for comparing distributions.

Constructor Summary

Constructors

Constructor and Description
Comparison()

Method Summary

Modifier and Type	Method and Description
static double	compareChi2(Histogram dist1, Histogram dist2) Use a log-likelihood ratio test to compare two distributions.
static double	compareChi2(TDigest dist1, TDigest dist2, double[] qCuts) Use a log-likelihood ratio test to compare two distributions.
static double	ks(TDigest d1, TDigest d2) Returns the observed value of the Kolmogorov-Smirnov statistic normalized by sample counts so that the score should be roughly distributed as sqrt(-log(u)/2).
static double	llr(double[][] count)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

Comparison

public Comparison()

Method Detail

compareChi2

public static double compareChi2(TDigest dist1,
                                 TDigest dist2,
                                 double[] qCuts)

Use a log-likelihood ratio test to compare two distributions. This is done by estimating counts in quantile ranges from each distribution and then comparing those counts using a multinomial test. The result should be asymptotically chi^2 distributed if the data comes from the same distribution, but this isn't so much useful as a traditional test of a null hypothesis as it is just a reasonably well-behaved score that is bigger when the distributions are more different, subject to having enough data to tell.

Parameters:

dist1 - First distribution (usually the reference)

dist2 - Second distribution to compare (usually the test case)

qCuts - The quantiles that define the bin boundaries. Values ≤0 or ≥1 may result in zero counts. Note that the actual cuts are defined loosely as

dist1.quantile(qCuts[i])

.

Returns:

A score that is big when dist1 and dist2 are discernibly different. A small score does not mean similarity. Instead, it could just mean insufficient data.

compareChi2

public static double compareChi2(Histogram dist1,
                                 Histogram dist2)

Use a log-likelihood ratio test to compare two distributions. With non-linear histograms that have compatible bin boundaries, all that we have to do is compare two count vectors using a chi^2 test (actually a log-likelihood ratio version called a G-test).

Parameters:

dist1 - First distribution (usually the reference)

dist2 - Second distribution to compare (usually the test case)

Returns:

A score that is big when dist1 and dist2 are discernibly different. A small score does not mean similarity. Instead, it could just mean insufficient data.

llr

public static double llr(double[][] count)

ks

public static double ks(TDigest d1,
                        TDigest d2)

Returns the observed value of the Kolmogorov-Smirnov statistic normalized by sample counts so that the score should be roughly distributed as sqrt(-log(u)/2). This is equal to the normal KS statistic multiplied by sqrt(m*n/(m+n)) where m and n are the number of samples in d1 and d2 respectively.

Parameters:

d1 - A digest of the first set of samples.

d2 - A digest of the second set of samples.

Returns:

A statistic which is bigger when d1 and d2 seem to represent different distributions.