Click or drag to resize

Ordinary Least Squares (OLS)

This topic contains the following sections:

OLS Specification

OLS is the most elementary type of linear Least Squares model, the problem corresponds to the canonical specification described in Basics of Linear Least Squares.

OLS Implementation and Usage

OLS solver is implemented by the OrdinaryLS class.

To initialize a class instance use one of the two constructors:

The class extends the basic interface with the methods that can increase and decrease observation base, i.e. it is designed in such a way that it can recalculate regression by adding of new observations and 'forgetting' portions of previously used observations:

Operation

Description

Performance

LLS Bar 2

Increasing of observation base with new observations

Adding new observations to the model so that the resulting model coefficients are the same as they would computed with the whole (concatenated) set of observations at once:

methodUpdate(Vector, Double)

methodUpdate(Matrix, Vector)

Single observation:

Multiple observations:

Decreasing of observation base by excluding observations

Modifying the model by excluding some of the previously used observations:

methodForget(Vector, Double)

methodForget(Matrix, Vector)

Note Note

To use Decreasing methods user should somehow keep the previously used portions of input observations that he wants to exclude.

Single observation:

Multiple observations:

Increasing/Decreasing mechanism provides ability to build the model as for expanding set of observations as well as in sliding data series manner.

The class also provides the FitOLS(Matrix, Vector) static method to build regression without initializing of a class instance; this method is more resource consuming in comparison to the corresponding constructor and Update methods.

Code Sample

The code demonstrates using of the OrdinaryLS class to build regression on finite data set and to update the model with Update and Forget methods.

C#
Copy
 1using System;
 2using FinMath.LinearAlgebra;
 3using FinMath.LeastSquares;
 4using FinMath.Statistics.Distributions;
 5
 6namespace FinMath.Samples
 7{
 8    class OrdinaryLSSample
 9    {
10        static void Main()
11        {
12-            #region Generate inputs.
13 
14             // Input Parameters.
15             const Int32 inSampleObservations = 100;
16             const Int32 outOfSampleObservations = 100;
17             const Int32 regresssorsCount = 10;
18             const Double homoscedasticVariance = 0.1;
19 
20             // Here we generate synthetic data according to model: Y = X * Beta + Er.
21             // Allocate real beta vector for model.
22             Vector realBeta = Vector.Random(regresssorsCount);
23             // Allocate in-sample observation of regressors X.
24             Matrix inSampleRegressors = Matrix.Random(inSampleObservations, regresssorsCount);
25             // Allocate out-of-sample observation of regressors X.
26             Matrix outOfSamplesRegressors = Matrix.Random(outOfSampleObservations, regresssorsCount);
27 
28             // Calculate in-sample regressand values Y, without errors.
29             Vector inSampleRegressand = inSampleRegressors * realBeta;
30             // Calculate out-of-sample regressand values Y, without errors.
31             Vector outOfSamplesRegressand = outOfSamplesRegressors * realBeta;
32 
33             // Add noise. As far as our model is Y = X * Beta + Er. Where Er ~ N(0, v) and v is constant variance.
34             Normal normalDistribution = new Normal(0.0, homoscedasticVariance);
35             // Add errors to in-sample regressand values;
36             inSampleRegressand.PointwiseAddInPlace(normalDistribution.Sample(inSampleObservations));
37             // Add errors to out-of-sample regressand values;
38             outOfSamplesRegressand.PointwiseAddInPlace(normalDistribution.Sample(outOfSampleObservations));
39 
40             #endregion
41
42            // Compute least squares.
43            OrdinaryLS ls = new OrdinaryLS(inSampleRegressors, inSampleRegressand);
44            // Get least squares beta.
45            Vector lsBeta = ls.Parameters;
46
47            Console.WriteLine("Parameters:");
48            Console.WriteLine("  Real parameters (which were used for data generation): ");
49            Console.WriteLine("  " + realBeta.ToString("0.000"));
50            Console.WriteLine("  Parameters calculated by least squares: ");
51            Console.WriteLine("  " + lsBeta.ToString("0.000"));
52
53            Console.WriteLine("");
54            Console.WriteLine("Least Squares Results:");
55            Console.WriteLine("  Parameters difference norm is: " + (realBeta - lsBeta).L2Norm());
56            // Build forecast for out-of-sample regressand values and estimate residuals.
57            Console.WriteLine("  Out-of-sample error is: " + ls.EstimateResiduals(outOfSamplesRegressors, outOfSamplesRegressand).L2Norm());
58        }
59    }
60}

See Also