BIJstats

Overview

Package

Class

Tree

Deprecated

Index

Help

PREV CLASS NEXT CLASS

FRAMES NO FRAMES

SUMMARY: NESTED | FIELD | CONSTR | METHOD

DETAIL: FIELD | CONSTR | METHOD

bijnum
Class BIJstats

java.lang.Object
  bijnum.BIJstats

public class BIJstats
extends java.lang.Object
extends java.lang.Object

Field Summary
`static float`	`CI95` Conversion from standard error to confidence intervals at different significance levels.
`static float`	`CI99`
`static float`	`CI99_9`

Constructor Summary
`BIJstats()`

Method Summary
`static double`	`accuracy(float[] table2x2)` Compute accuracy or observed proportion of overall agreement of a test for which the result is in exp and the ground truth in truth accuracy = true pos + true neg / (true neg + false pos + true pos + false neg) accuracy = (a+d) / (a+b+c+d)
`static double`	`accuracyMultipleClasses(float[] exp, float[] truth, int n)` Compute the multiclass accuracy of a test of which the result is in exp and the ground truth in truth, where test results in multiple classifications.
`static double`	`accuracyMultipleClasses(int[][] tablenxn)` Compute the accuracy of a test of which the confusion matrix is in tablenxn.
`static double`	`aroc(int[][] frequencies, int bins)` Determine area under the roc curve for an experiment that gave the frequencies given in the table.
`static double[]`	`autocorr(double[] data, int maxk)` Computes autocorrelation up to maxk for the time series data.
`static double[]`	`autocov(double[] data, int maxk)` Computes autocovariance up to maxk for the time series data.
`static double`	`avg(double[] v)` Compute average of all values in vector v.
`static double`	`avg(float[] v)` Compute average of all values in vector v.
`static double`	`avg(float[][] m)` Compute average of all values in vector v.
`static float`	`avg(float[] v, float[] mask)`
`static float`	`avgNoExtremes(float[] v, float nrstddev)`
`static int`	`binIndex(int[] histogram, float fraction)` Compute the lowest bin into which the highest p percent of occurrences falls.
`static float`	`correl(float[] a, float[] b)` Compute the correlation of a vector with another vector b.
`static float[][]`	`covariance(float[][] m, boolean doShowProgress)` Compute the covariance matrix for a matrix of size NxM.
`static double`	`erf(double x)` Compute the erf of x.
`static double`	`getSignificanceLevel(double significance)`
`static java.lang.String`	`getSignificanceString(double sign)`
`static int[]`	`histogram(float[] v, float min, float d, int n)` Compute a histogram with n bins for the vector v, each bin separated by d.
`static double`	`kappa(float[] table2x2)` Compute kappa for a 2x2 table.
`static float[]`	`meanColumn(float[][] m)`
`static float[]`	`meanColumnT(float[][] m)`
`static double`	`means(float[][] m)`
`static float`	`mse(float[] a, float[] b)` Compute Mean Square Error (or residual) of vectors a and b.
`static int`	`n(float[] v)` Determine length of a vector.
`static int`	`n(float[] v, float i)` Number of occurrences of i in v.
`static int`	`n(float[] v, float[] mask)` Length of masked(v), i.e.
`static int[][]`	`probfrequencies(float[] truth, float[] exp, int bins)` For a 2 class experiment make a table[2] for class class0 and class1 with for each class the frequencies of each probability.
`static float[]`	`randomFraction(float[] v, double fraction)` Randomly sample a fraction of the elements of vector v.
`static float[]`	`residuals(float[] a, float[] b)` Compute residuals of b versus a.
`static float`	`rmse(float[] a, float[] b)` Compute Root Mean Square Error of vectors a and b.
`static double`	`sem(double estimateAB, double estimateA, double semA, double estimateB, double semB)` Compute the combined standard error of the mean of the two standard errors stderr0 and stderr1 and corresponding averages avg0 and avg1.
`static double`	`sem(float[] v)` sem(1) computes the standard error of the mean of the elements of v.
`static double`	`sem(float[] v, float[] mask)` sem(2) computes the standard error of the mean of the elements of v only where the mask value is not 0.
`static float[]`	`sensitivities(float[] tablenxn)` Compute sensitivities of multiclass test from a confusion matrix matrixnxn.
`static float[]`	`sensitivities(float[] truth, float[] exp, int n)` Compute the sensitivities of a test of which the result is in exp and the ground truth in truth, for all classes n that occur in truth.
`static double`	`sensitivity(float[] table2x2)` Compute sensitivity of a test of which the result is in exp and the ground truth in truth, for class c.
`static float[]`	`specificities(float[] exp, float[] truth, int n)` Compute the specificities of a test of which the result is in exp and the ground truth in truth, for all classes n that occur in truth.
`static double`	`specificity(float[] table2x2)` Compute specificity of a test of which the result is in exp and the ground truth in truth, for class c.
`static float[]`	`spectrum(float[] v)` Compute the spectrum of v.
`static float`	`stdev(float[] v)` Compute stdev (SQRT(var)) of all values in vector v.
`static float`	`stdev(float[] v, float[] mask)` stdev(2) computes the stddev of the elements of only where the mask value is not 0.
`static double`	`sum(double[] v)` Compute average of all values in vector v.
`static double`	`sum(float[] v)` sum(1) computes summation of all values in vector v.
`static double`	`sum(float[][] m)` sum(1) computes summation of all values in matrix m.
`static double`	`sum(float[] v, float[] mask)` sum(2) computes the sum of the elements of vector v only where the mask value is not 0.
`static float[]`	`table2x2(float[] obs0, float[] obs1, int c)` Make a 2x2 table for the observations obs0 and obs1, where an observation c is taken as correct.
`static int[][]`	`tablenxn(float[] obs0, float[] obs1, int n)` Make a nxn confusion matrix for the observations obs0 and obs1, for n classes.
`static float`	`thresholdFraction(float[] v, double fraction)`
`static float`	`thresholdFraction(float[] v, float fraction)` Find the value in vector over which fraction of all values in vector lie.
`static double`	`tptest(double[] data1, double[] data2)` Perform paired Student's t-test on two data sets data1 and data2, with the the t values into t and the probabilities into prob.
`static double`	`ttest(double[] data1, double[] data2)` Perform Student's t-test on two data sets data1 and data2, with the the t values into t and the probabilities into prob.
`static double`	`ttest(float[] data1, float[] data2)` Perform Student's t-test on two data sets data1 and data2, with the the t values into t and the probabilities into prob.
`static float[]`	`unitvar(float[] v)` Make a new vector of v that has zero mean and unit variance.
`static void`	`unitvar(float[][] m)` Make m unit variance and zero mean with respect to all elements of m.
`static double`	`var(double[] v)` Compute variance of all elements in vector v.
`static double`	`var(float[] v)` Compute variance of all elements in vector v.
`static double[]`	`zeromean(double[] v)` Zero the mean of all elements of v.
`static float[]`	`zeromean(float[] v)` Zero the mean of all elements of v.

Methods inherited from class java.lang.Object
`equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait`

Field Detail

CI95

public static float CI95

Conversion from standard error to confidence intervals at different significance levels. To make a confidence interval (99%, 2.576; 95%, 1.96; 99.9 3.39).

CI99

public static float CI99

CI99_9

public static float CI99_9

Constructor Detail

BIJstats

public BIJstats()

Method Detail

getSignificanceLevel

public static double getSignificanceLevel(double significance)

covariance

public static float[][] covariance(float[][] m,
                                   boolean doShowProgress)

Compute the covariance matrix for a matrix of size NxM. (= m (TRANSPOSE(m)/length(m)) The returned matrix will be size NxN.

Parameters:: m - a matrix of float[N][M]
Returns:: a matrix of covariances float[N][N]

n

public static int n(float[] v)

Determine length of a vector.

Parameters:: v - float[]
Returns:: int

n

public static int n(float[] v,
                    float i)

Number of occurrences of i in v.

Parameters:: v - float[]; i - int
Returns:: int

n

public static int n(float[] v,
                    float[] mask)

Length of masked(v), i.e. count all occurences of the elements of v where the corresponding mask is nonzero.

Parameters:: v - float[]; mask - float[]
Returns:: int

avg

public static float avg(float[] v,
                        float[] mask)

getSignificanceString

public static java.lang.String getSignificanceString(double sign)

stdev

public static float stdev(float[] v,
                          float[] mask)

stdev(2) computes the stddev of the elements of only where the mask value is not 0. See Press, Numerical recipes in C, pp 617 for the computation of variance using the two pass algorithm.

Parameters:: v - a float[] vector.; mask - a float[] with a value of ! 0 for all elements of v that are valid.
Returns:: the stddev of the masked elements in v.

sem

public static double sem(float[] v)

sem(1) computes the standard error of the mean of the elements of v. Standard error is the standard deviation divided by the square root of the number of elements. See Press, Numerical recipes in C, pp 617 for the computation of variance using the two pass algorithm.

Parameters:: v - a float[] vector.
Returns:: the standard error of the mean of all elements in v.

sem

public static double sem(float[] v,
                         float[] mask)

sem(2) computes the standard error of the mean of the elements of v only where the mask value is not 0. Standard error is the standard deviation divided by the square root of the number of elements. See Press, Numerical recipes in C, pp 617 for the computation of variance using the two pass algorithm.

Parameters:: v - a float[] vector.; mask - a float[] with a value of ! 0 for all elements of v that are valid.
Returns:: the stderr of the masked elements in v.

sem

public static double sem(double estimateAB,
                         double estimateA,
                         double semA,
                         double estimateB,
                         double semB)

Compute the combined standard error of the mean of the two standard errors stderr0 and stderr1 and corresponding averages avg0 and avg1. The combined SEM is used to compute the SEM correspnding to the multiplication or division of two estimates. The method used to calculate the SE of a calculated result involving multiplication and/or division is similar to, but slightly more complicated than that used for addition and subtraction. In this case, the relative standard deviations (or relative standard errors) of the values involved in the calculation must be used rather than the absolute standard errors. The relative SD (or SE) is obtained from the absolute SD (or SE) by dividing the (SE or SD) by the data value itself. Note that these relative quantities are always unitless.

Parameters:: estimateAB - the estimate of the combination A and B (determined by either multiplication or division, don't care here).; estimateA, - semA the estimate and standard deviation of estimate.; estimateB, - semBV the estimate and standard deviation of estimate.
Returns:: standard error of the mean for estimateAB.

avgNoExtremes

public static float avgNoExtremes(float[] v,
                                  float nrstddev)

meanColumnT

public static float[] meanColumnT(float[][] m)

meanColumn

public static float[] meanColumn(float[][] m)

means

public static double means(float[][] m)

avg

public static double avg(float[] v)

Compute average of all values in vector v.

Parameters:: v - a vector of float[]
Returns:: the average of all elements in v.

avg

public static double avg(float[][] m)

Compute average of all values in vector v.

Parameters:: v - a vector of float[]
Returns:: the average of all elements in v.

avg

public static double avg(double[] v)

Compute average of all values in vector v.

Parameters:: v - a vector of float[]
Returns:: the average of all elements in v.

sum

public static double sum(float[] v)

sum(1) computes summation of all values in vector v.

Parameters:: v - a vector of float[]
Returns:: the average of all elements in v.

sum

public static double sum(float[][] m)

sum(1) computes summation of all values in matrix m.

Parameters:: v - a vector of float[]
Returns:: the average of all elements in v.

sum

public static double sum(float[] v,
                         float[] mask)

sum(2) computes the sum of the elements of vector v only where the mask value is not 0.

Parameters:: v - a float[] vector.; mask - a float[] with a value of ! 0 for all elements of v that are valid.
Returns:: the avg of the masked elements in v.

sum

public static double sum(double[] v)

Compute average of all values in vector v.

Parameters:: v - a vector of float[]
Returns:: the average of all elements in v.

var

public static double var(float[] v)

Compute variance of all elements in vector v.

Parameters:: v - a vector of float[]

unitvar

public static void unitvar(float[][] m)

Make m unit variance and zero mean with respect to all elements of m. m is modified in place.

Parameters:: m - a vector of float[][]

var

public static double var(double[] v)

Compute variance of all elements in vector v.

Parameters:: v - a vector of float[]

unitvar

public static float[] unitvar(float[] v)

Make a new vector of v that has zero mean and unit variance.

Parameters:: v - a float[] vector
Returns:: a vector with proportionally the same elements as v but with zero mean and unit variance.

zeromean

public static double[] zeromean(double[] v)

Zero the mean of all elements of v.

Parameters:: v - a vector.
Returns:: a vector with zero mean and same variance as v.

zeromean

public static float[] zeromean(float[] v)

Zero the mean of all elements of v.

Parameters:: v - a vector.
Returns:: a vector with zero mean and same variance as v.

stdev

public static float stdev(float[] v)

Compute stdev (SQRT(var)) of all values in vector v.

Parameters:: v - a vector of float[]

thresholdFraction

public static float thresholdFraction(float[] v,
                                      double fraction)

thresholdFraction

public static float thresholdFraction(float[] v,
                                      float fraction)

Find the value in vector over which fraction of all values in vector lie.

Parameters:: vector - a vector of values; fraction - a float [0-1].

histogram

public static int[] histogram(float[] v,
                              float min,
                              float d,
                              int n)

Compute a histogram with n bins for the vector v, each bin separated by d.

Parameters:: v - a float[] vector; min - the lowest value for the first bin.; d - the difference in value between each bin; n - the number of bins.
Returns:: a float[] with the number of occurences for each bin value.

binIndex

public static int binIndex(int[] histogram,
                           float fraction)

Compute the lowest bin into which the highest p percent of occurrences falls. Used to compute things as "give me the occurence above which 10% of the histogram falls."

Parameters:: histogram - an int[] with the ouccrence counts for each bin; p - the fraction of histogram values desired.
Returns:: an int, the index into the lowest bin that still falls among p.

correl

public static float correl(float[] a,
                           float[] b)

Compute the correlation of a vector with another vector b. The mean of a and b is subtracted before further processing. a and b therefore do not have to be zero-mean.

Parameters:: a - a float[] vector.; b - a float[] vector of same length.
Returns:: r, the correlation coefficient.

spectrum

public static float[] spectrum(float[] v)

Compute the spectrum of v. The spectrum gives the ratio of each element to v to the sum of v. The variance spectrum gives the ratio of each coordinate to the sum of all coordinates.

Parameters:: v - a float[] vector
Returns:: the spectrum of v as a float[].

erf

public static double erf(double x)

Compute the erf of x.

Parameters:: x - the argument
Returns:: the erf of x.

mse

public static float mse(float[] a,
                        float[] b)
                 throws java.lang.IllegalArgumentException

Compute Mean Square Error (or residual) of vectors a and b. a and b are normalized to zero mean and unit variance.

Parameters:: a - a float[] vector; b - a float[] vector.
Returns:: a float that is the Mean Square Error of a and b.
Throws:: java.lang.IllegalArgumentException

residuals

public static float[] residuals(float[] a,
                                float[] b)
                         throws java.lang.IllegalArgumentException

Compute residuals of b versus a.

Parameters:: a - a float[] vector; b - a float[] vector.
Returns:: a float[] that contains the residuals for each b-a.
Throws:: java.lang.IllegalArgumentException

rmse

public static float rmse(float[] a,
                         float[] b)
                  throws java.lang.IllegalArgumentException

Compute Root Mean Square Error of vectors a and b. a and b are normalized to zero mean and unit variance.

Parameters:: a - a float[] vector; b - a float[] vector.
Returns:: a float that is the Root Mean Square Error of a and b.
Throws:: java.lang.IllegalArgumentException

ttest

public static double ttest(float[] data1,
                           float[] data2)

Perform Student's t-test on two data sets data1 and data2, with the the t values into t and the probabilities into prob. Variances should be the same. From Press, Numerical Recipes in C, second edition.

Parameters:: data1 - array of datapoints; data2 - array of datapoints; t -; prob -

autocov

public static double[] autocov(double[] data,
                               int maxk)

Computes autocovariance up to maxk for the time series data.

autocorr

public static double[] autocorr(double[] data,
                                int maxk)

Computes autocorrelation up to maxk for the time series data.

ttest

public static double ttest(double[] data1,
                           double[] data2)

Parameters:: data1 - array of datapoints; data2 - array of datapoints

tptest

public static double tptest(double[] data1,
                            double[] data2)

Perform paired Student's t-test on two data sets data1 and data2, with the the t values into t and the probabilities into prob. Variances should be the same. From Press, Numerical Recipes in C, second edition, pp 618

Parameters:: data1 - array of datapoints; data2 - array of datapoints

randomFraction

public static float[] randomFraction(float[] v,
                                     double fraction)

Randomly sample a fraction of the elements of vector v.

Parameters:: v - a vector; fraction - the fraction of elements from v to be included.
Returns:: a float[] vector with all the sampled elements.

probfrequencies

public static int[][] probfrequencies(float[] truth,
                                      float[] exp,
                                      int bins)

For a 2 class experiment make a table[2] for class class0 and class1 with for each class the frequencies of each probability. I.e. if ground truth was 1 and the prob (of it being 1) was 0.8 the bin for 1,0.8 is increased.

Parameters:: truth - float[] the reference standard classifications for each entry; exp - float[] the estimate of the probability of the class being 1 for each experiment; bins - the number of discrete probabilities in exp.
Returns:: int[2][] the histogram for class 0 and 1 of the frequencies of each of the probabilities in exp.

aroc

public static double aroc(int[][] frequencies,
                          int bins)

Determine area under the roc curve for an experiment that gave the frequencies given in the table.

Parameters:: frequencies - float[][] a table of probability frequencies for each of the classes 0 and 1.
Returns:: double

sensitivities

public static float[] sensitivities(float[] truth,
                                    float[] exp,
                                    int n)
                             throws java.lang.IllegalArgumentException

Compute the sensitivities of a test of which the result is in exp and the ground truth in truth, for all classes n that occur in truth.

Parameters:: exp - a float[] vector of test results, where 0 <= exp[n] < n.; truth - a float[] vector of ground truth, where 0 <= truth[n] < n.; n - the number of classes to determine the sensitivity for.
Returns:: a float[] vector with the sensitivities for all classes in truth.
Throws:: java.lang.IllegalArgumentException

specificities

public static float[] specificities(float[] exp,
                                    float[] truth,
                                    int n)
                             throws java.lang.IllegalArgumentException

Compute the specificities of a test of which the result is in exp and the ground truth in truth, for all classes n that occur in truth.

Parameters:: exp - a float[] vector of test results, where 0 <= exp[n] < n.; truth - a float[] vector of ground truth, where 0 <= truth[n] < n.; n - the number of classes to determine the sensitivity for.
Returns:: a float[] vector with the specificities for all classes in truth.
Throws:: java.lang.IllegalArgumentException

table2x2

public static float[] table2x2(float[] obs0,
                               float[] obs1,
                               int c)
                        throws java.lang.IllegalArgumentException

Make a 2x2 table for the observations obs0 and obs1, where an observation c is taken as correct. The table is Obs0==c Obs0!=c Obs1==c a b Obs1!=c c d The table will contain a,b,c,d in that order

Returns:: the 2x2 table.
Throws:: java.lang.IllegalArgumentException

tablenxn

public static int[][] tablenxn(float[] obs0,
                               float[] obs1,
                               int n)
                        throws java.lang.IllegalArgumentException

Make a nxn confusion matrix for the observations obs0 and obs1, for n classes. The table is Obs0==0 Obs0==1 Obs0==2 ... Obs0==n-1 Obs1==0 a b c ... Obs1==1 g h ... Obs1==2 l m Obs1==n-1 ..

Returns:: the nxn confusion matrix.
Throws:: java.lang.IllegalArgumentException

kappa

public static double kappa(float[] table2x2)

Compute kappa for a 2x2 table. Kappa is

Returns:: the kappa.

accuracyMultipleClasses

public static double accuracyMultipleClasses(float[] exp,
                                             float[] truth,
                                             int n)
                                      throws java.lang.IllegalArgumentException

Compute the multiclass accuracy of a test of which the result is in exp and the ground truth in truth, where test results in multiple classifications.

Parameters:: exp - an int[] vector of test results, where 0 <= exp[n] < n.; truth - an int[] vector of ground truth, where 0 <= truth[n] < n.; n - the number of classes to determine the accuracy for.
Returns:: the combined accuracy.
Throws:: java.lang.IllegalArgumentException

sensitivity

public static double sensitivity(float[] table2x2)
                          throws java.lang.IllegalArgumentException

Compute sensitivity of a test of which the result is in exp and the ground truth in truth, for class c. sensitivity = true pos / (true pos + false neg) sensitivity = a / (a+c)

Parameters:: exp - a float[] vector of test results, where 0 <= exp[n] <= c.; truth - a float[] vector of ground truth, where 0 <= truth[n] <= c.; c - the class to determine sensitivity for.
Returns:: the sensitivity.
Throws:: java.lang.IllegalArgumentException

specificity

public static double specificity(float[] table2x2)
                          throws java.lang.IllegalArgumentException

Compute specificity of a test of which the result is in exp and the ground truth in truth, for class c. specificity = true neg / (true neg + false pos) specificity = d / (b+d)

Parameters:: exp - an int[] vector of test results, where 0 <= exp[n] <= c.; truth - an int[] vector of ground truth, where 0 <= truth[n] <= c.; c - the class to determine specificity for.
Returns:: the specificity.
Throws:: java.lang.IllegalArgumentException

accuracy

public static double accuracy(float[] table2x2)
                       throws java.lang.IllegalArgumentException

Compute accuracy or observed proportion of overall agreement of a test for which the result is in exp and the ground truth in truth accuracy = true pos + true neg / (true neg + false pos + true pos + false neg) accuracy = (a+d) / (a+b+c+d)

Parameters:: table2x2 - a 2x2 table.
Returns:: the accuracy.
Throws:: java.lang.IllegalArgumentException

accuracyMultipleClasses

public static double accuracyMultipleClasses(int[][] tablenxn)
                                      throws java.lang.IllegalArgumentException

Compute the accuracy of a test of which the confusion matrix is in tablenxn.

Returns:: the combined accuracy.
Throws:: java.lang.IllegalArgumentException

sensitivities

public static float[] sensitivities(float[] tablenxn)
                             throws java.lang.IllegalArgumentException

Compute sensitivities of multiclass test from a confusion matrix matrixnxn. sensitivity = true pos / (true pos + false neg) sensitivity = diagonal / (total on column)

Returns:: the sensitivities for each class.
Throws:: java.lang.IllegalArgumentException

Overview

Package

Class

Tree

Deprecated

Index

Help

PREV CLASS NEXT CLASS

FRAMES NO FRAMES

SUMMARY: NESTED | FIELD | CONSTR | METHOD

DETAIL: FIELD | CONSTR | METHOD

bijnum Class BIJstats

CI95

CI99

CI99_9

BIJstats

getSignificanceLevel

covariance

n

n

n

avg

getSignificanceString

stdev

sem

sem

sem

avgNoExtremes

meanColumnT

meanColumn

means

avg

avg

avg

sum

sum

sum

sum

var

unitvar

var

unitvar

zeromean

zeromean

stdev

thresholdFraction

thresholdFraction

histogram

binIndex

correl

spectrum

erf

mse

residuals

rmse

ttest

autocov

autocorr

ttest

tptest

randomFraction

probfrequencies

aroc

sensitivities

specificities

table2x2

tablenxn

kappa

accuracyMultipleClasses

sensitivity

specificity

accuracy

accuracyMultipleClasses

sensitivities

bijnum
Class BIJstats