org.apache.mahout.classifier.sgd
Class AbstractOnlineLogisticRegression

java.lang.Object
  extended by org.apache.mahout.classifier.AbstractVectorClassifier
      extended by org.apache.mahout.classifier.sgd.AbstractOnlineLogisticRegression
All Implemented Interfaces:
Closeable, OnlineLearner
Direct Known Subclasses:
OnlineLogisticRegression

public abstract class AbstractOnlineLogisticRegression
extends AbstractVectorClassifier
implements OnlineLearner

Generic definition of a 1 of n logistic regression classifier that returns probabilities in response to a feature vector. This classifier uses 1 of n-1 coding where the 0-th category is not stored explicitly.

Provides the SGD based algorithm for learning a logistic regression, but omits all annealing of learning rates. Any extension of this abstract class must define the overall and per-term annealing for themselves.


Field Summary
protected  Matrix beta
           
protected  int numCategories
           
protected  PriorFunction prior
           
protected  int step
           
protected  Vector updateCounts
           
protected  Vector updateSteps
           
 
Fields inherited from class org.apache.mahout.classifier.AbstractVectorClassifier
MIN_LOG_LIKELIHOOD
 
Constructor Summary
AbstractOnlineLogisticRegression()
           
 
Method Summary
 Vector classify(Vector instance)
          Returns n-1 probabilities, one for each category but the 0-th.
 Vector classifyNoLink(Vector instance)
          Compute and return a vector of scores before applying the inverse link function.
 double classifyScalar(Vector instance)
          Returns a single scalar probability in the case where we have two categories.
 double classifyScalarNoLink(Vector instance)
           
 void close()
          Prepares the classifier for classification and deallocates any temporary data structures.
 void copyFrom(AbstractOnlineLogisticRegression other)
           
abstract  double currentLearningRate()
           
 Matrix getBeta()
           
 double getLambda()
           
 PriorFunction getPrior()
           
 int getStep()
           
 boolean isSealed()
           
 AbstractOnlineLogisticRegression lambda(double lambda)
          Chainable configuration option.
static double link(double r)
          Computes the binomial logistic inverse link function.
static Vector link(Vector v)
          Computes the inverse link function, by default the logistic link function.
protected  void nextStep()
           
 int numCategories()
          Returns the number of categories that a target variable can be assigned to.
 int numFeatures()
           
abstract  double perTermLearningRate(int j)
           
 void regularize(Vector instance)
           
 void setBeta(int i, int j, double betaIJ)
           
 void setGradient(Gradient gradient)
           
 void setPrior(PriorFunction prior)
           
 void train(int actual, Vector instance)
          Updates the model using a particular target variable value and a feature vector.
 void train(long trackingKey, int actual, Vector instance)
          Updates the model using a particular target variable value and a feature vector.
 void train(long trackingKey, String groupKey, int actual, Vector instance)
          Updates the model using a particular target variable value and a feature vector.
protected  void unseal()
           
 boolean validModel()
           
 
Methods inherited from class org.apache.mahout.classifier.AbstractVectorClassifier
classify, classifyFull, classifyFull, classifyFull, classifyScalar, logLikelihood
 
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
 

Field Detail

beta

protected Matrix beta

numCategories

protected int numCategories

step

protected int step

updateSteps

protected Vector updateSteps

updateCounts

protected Vector updateCounts

prior

protected PriorFunction prior
Constructor Detail

AbstractOnlineLogisticRegression

public AbstractOnlineLogisticRegression()
Method Detail

lambda

public AbstractOnlineLogisticRegression lambda(double lambda)
Chainable configuration option.

Parameters:
lambda - New value of lambda, the weighting factor for the prior distribution.
Returns:
This, so other configurations can be chained.

link

public static Vector link(Vector v)
Computes the inverse link function, by default the logistic link function.

Parameters:
v - The output of the linear combination in a GLM. Note that the value of v is disturbed.
Returns:
A version of v with the link function applied.

link

public static double link(double r)
Computes the binomial logistic inverse link function.

Parameters:
r - The value to transform.
Returns:
The logit of r.

classifyNoLink

public Vector classifyNoLink(Vector instance)
Description copied from class: AbstractVectorClassifier
Compute and return a vector of scores before applying the inverse link function. For logistic regression and other generalized linear models, this is just the linear part of the classification.

The implementation of this method provided by AbstractVectorClassifier throws an UnsupportedOperationException. Your subclass must explicitly override this method to support this operation.

Overrides:
classifyNoLink in class AbstractVectorClassifier
Parameters:
instance - A feature vector to be classified.
Returns:
A vector of scores. If transformed by the link function, these will become probabilities.

classifyScalarNoLink

public double classifyScalarNoLink(Vector instance)

classify

public Vector classify(Vector instance)
Returns n-1 probabilities, one for each category but the 0-th. The probability of the 0-th category is 1 - sum(this result).

Specified by:
classify in class AbstractVectorClassifier
Parameters:
instance - A vector of features to be classified.
Returns:
A vector of probabilities, one for each of the first n-1 categories.

classifyScalar

public double classifyScalar(Vector instance)
Returns a single scalar probability in the case where we have two categories. Using this method avoids an extra vector allocation as opposed to calling classify() or an extra two vector allocations relative to classifyFull().

Specified by:
classifyScalar in class AbstractVectorClassifier
Parameters:
instance - The vector of features to be classified.
Returns:
The probability of the first of two categories.
Throws:
IllegalArgumentException - If the classifier doesn't have two categories.
See Also:
AbstractVectorClassifier.classify(Vector)

train

public void train(long trackingKey,
                  String groupKey,
                  int actual,
                  Vector instance)
Description copied from interface: OnlineLearner
Updates the model using a particular target variable value and a feature vector.

There may an assumption that if multiple passes through the training data are necessary that the tracking key for a record will be the same for each pass and that there will be a relatively large number of distinct tracking keys and that the low-order bits of the tracking keys will not correlate with any of the input variables. This tracking key is used to assign training examples to different test/training splits.

Examples of useful tracking keys include id-numbers for the training records derived from a database id for the base table from the which the record is derived, or the offset of the original data record in a data file.

Specified by:
train in interface OnlineLearner
Parameters:
trackingKey - The tracking key for this training example.
groupKey - An optional value that allows examples to be grouped in the computation of the update to the model.
actual - The value of the target variable. This value should be in the half-open interval [0..n) where n is the number of target categories.
instance - The feature vector for this example.

train

public void train(long trackingKey,
                  int actual,
                  Vector instance)
Description copied from interface: OnlineLearner
Updates the model using a particular target variable value and a feature vector.

There may an assumption that if multiple passes through the training data are necessary that the tracking key for a record will be the same for each pass and that there will be a relatively large number of distinct tracking keys and that the low-order bits of the tracking keys will not correlate with any of the input variables. This tracking key is used to assign training examples to different test/training splits.

Examples of useful tracking keys include id-numbers for the training records derived from a database id for the base table from the which the record is derived, or the offset of the original data record in a data file.

Specified by:
train in interface OnlineLearner
Parameters:
trackingKey - The tracking key for this training example.
actual - The value of the target variable. This value should be in the half-open interval [0..n) where n is the number of target categories.
instance - The feature vector for this example.

train

public void train(int actual,
                  Vector instance)
Description copied from interface: OnlineLearner
Updates the model using a particular target variable value and a feature vector.

There may an assumption that if multiple passes through the training data are necessary, then the training examples will be presented in the same order. This is because the order of training examples may be used to assign records to different data splits for evaluation by cross-validation. Without the order invariance, records might be assigned to training and test splits and error estimates could be seriously affected.

If re-ordering is necessary, then using the alternative API which allows a tracking key to be added to the training example can be used.

Specified by:
train in interface OnlineLearner
Parameters:
actual - The value of the target variable. This value should be in the half-open interval [0..n) where n is the number of target categories.
instance - The feature vector for this example.

regularize

public void regularize(Vector instance)

perTermLearningRate

public abstract double perTermLearningRate(int j)

currentLearningRate

public abstract double currentLearningRate()

setPrior

public void setPrior(PriorFunction prior)

setGradient

public void setGradient(Gradient gradient)

getPrior

public PriorFunction getPrior()

getBeta

public Matrix getBeta()

setBeta

public void setBeta(int i,
                    int j,
                    double betaIJ)

numCategories

public int numCategories()
Description copied from class: AbstractVectorClassifier
Returns the number of categories that a target variable can be assigned to. A vector classifier will encode it's output as an integer from 0 to numCategories()-1 (inclusive).

Specified by:
numCategories in class AbstractVectorClassifier
Returns:
The number of categories.

numFeatures

public int numFeatures()

getLambda

public double getLambda()

getStep

public int getStep()

nextStep

protected void nextStep()

isSealed

public boolean isSealed()

unseal

protected void unseal()

close

public void close()
Description copied from interface: OnlineLearner
Prepares the classifier for classification and deallocates any temporary data structures. An online classifier should be able to accept more training after being closed, but closing the classifier may make classification more efficient.

Specified by:
close in interface Closeable
Specified by:
close in interface OnlineLearner

copyFrom

public void copyFrom(AbstractOnlineLogisticRegression other)

validModel

public boolean validModel()


Copyright © 2008–2014 The Apache Software Foundation. All rights reserved.