dev.nm.stat.regression.linear.ols

Class OLSRegression

java.lang.Object

dev.nm.stat.regression.linear.ols.OLSRegression

All Implemented Interfaces:

LinearModel

public class OLSRegression
extends Object
implements LinearModel

(Weighted) Ordinary Least Squares (OLS) is a method for fitting a linear regression model. This method minimizes the (weighted) sum of squared distances between the observed responses in the dataset, and the responses predicted by the linear approximation.

The OLS estimator is consistent when the regressors are exogenous and there is no multicollinearity, and optimal in the class of linear unbiased estimators when the errors are homoscedastic and serially uncorrelated. OLS can be derived as a maximum likelihood estimator under the assumption that the errors are normally distributed, however the method has good statistical properties for a much broader class of distributions (except for efficiency).

Fit and predict

Construction fits the model immediately. Obtain coefficient estimates using beta().betaHat(); predictor coefficients come first and the optional intercept comes last. Ey(x) takes just the predictor values in training-column order, without the appended intercept.

Example

 import dev.nm.algebra.linear.matrix.doubles.matrixtype.dense.DenseMatrix;
 import dev.nm.algebra.linear.vector.doubles.Vector;
 import dev.nm.algebra.linear.vector.doubles.dense.DenseVector;
 import dev.nm.stat.regression.linear.LMProblem;
 import dev.nm.stat.regression.linear.ols.OLSRegression;

 DenseMatrix x = new DenseMatrix(new double[][]{{0}, {1}, {2}, {3}});
 DenseVector y = new DenseVector(1.0, 3.0, 5.0, 7.0);
 OLSRegression fit = new OLSRegression(new LMProblem(y, x, true));
 Vector beta = fit.beta().betaHat(); // slope first, intercept last
 assert Math.abs(beta.get(1) - 2.0) < 1e-10;
 assert Math.abs(beta.get(2) - 1.0) < 1e-10;
 double prediction = fit.Ey(new DenseVector(new double[]{4.0}));
 assert Math.abs(prediction - 9.0) < 1e-10;
 

See Also:

• Wikipedia: Linear regression
• Wikipedia: Least squares
• Wikipedia: Ordinary least squares

Constructor Summary

Constructors

Constructor and Description
OLSRegression(LMProblem problem) Constructs an OLSRegression instance.
OLSRegression(LMProblem problem, double epsilon) Constructs an OLSRegression instance.

Method Summary

Modifier and Type	Method and Description
OLSBeta	beta() Gets \(\hat{\beta}\) and statistics.
LMDiagnostics	diagnostics() Gets the diagnostic measures of an OLS regression.
double	Ey(Vector x) Predicts the conditional response for one observation.
static double	Ey(Vector beta, Vector x, boolean intercept) Predicts a response using explicitly supplied coefficients.
LMInformationCriteria	informationCriteria() Gets the model selection criteria.
OLSResiduals	residuals() Gets the residual analysis of an OLS regression.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

OLSRegression

public OLSRegression(LMProblem problem,
                     double epsilon)

Constructs an OLSRegression instance.

Parameters:

problem - the linear regression problem to be solved

epsilon - a precision parameter: when a number |x| ≤ ε, it is considered 0

OLSRegression

public OLSRegression(LMProblem problem)

Constructs an OLSRegression instance.

Parameters:

problem - the linear regression problem to be solved

Method Detail

Ey

public double Ey(Vector x)

Predicts the conditional response for one observation.

Specified by:

Ey in interface LinearModel

Parameters:

x - predictor values in training-column order, excluding the intercept

Returns:

the fitted conditional mean

Ey

public static double Ey(Vector beta,
                        Vector x,
                        boolean intercept)

Predicts a response using explicitly supplied coefficients.

Parameters:

beta - predictor coefficients, followed by the intercept if enabled

x - predictor values, without an intercept entry

intercept - whether to append a constant one to x

Returns:

the dot product of beta and the augmented predictor vector

beta

public OLSBeta beta()

Description copied from interface: LinearModel

Gets \(\hat{\beta}\) and statistics.

Specified by:

beta in interface LinearModel

Returns:

\(\hat{\beta}\) and statistics

residuals

public OLSResiduals residuals()

Description copied from interface: LinearModel

Gets the residual analysis of an OLS regression.

Specified by:

residuals in interface LinearModel

Returns:

the residual analysis

diagnostics

public LMDiagnostics diagnostics()

Gets the diagnostic measures of an OLS regression.

Returns:

the diagnostic measures of an OLS regression

informationCriteria

public LMInformationCriteria informationCriteria()

Gets the model selection criteria.

Returns:

the model selection criteria