Click or drag to resize

SVM Vector Regression
Support Vector Regression (SVR) model

ϵ-Support Vector Regression

Consider a set of training points ML SVMRegression 1, where ML SVMRegression 2 is a feature vector and ML SVMRegression 3 is the target output. Under given parameters ML SVMClassification 8 and ML SVMRegression 4, the standard form of support vector regression (V. Vapnik. Statistical Learning Theory. Wiley, New York, NY, 1998) is

ML SVMRegression 5,

subject to ML SVMRegression 6,

ML SVMMarginML SVMMarginML SVMRegression 7,

ML SVMMarginML SVMMarginML SVMRegression 8

ν-Support Vector Regression

Similar to ν-SVC, for this regression (B. Schölkopf, A. Smola, R. C. Williamson, and P. L. Bartlett. New support vector algorithms. Neural Computation, 12:1207–1245, 2000) a parameter ν∈( 0,1] to control the number of support vectors. The parameter ϵ in ϵ-SVR becomes a parameter here. With ML SVMRegression 9 as parameters, ν-SVR solves

ML SVMRegression 10,

subject to ML SVMRegression 6,

ML SVMMarginML SVMMarginML SVMRegression 7

ML SVMMarginML SVMMarginML SVMRegression 8, ML SVMRegression 11

Examples

As you can see from the following example, the proper kernel function selection plays key role in proper regression design: in [-1.5,2] range these two functions look nearly the same, but out of training region their behavior differs significantly.

ML SVMRegression Example

Implementation
Code Sample

The example of SVM Vector Regression usage:

C#
Copy
 1public static void RegressionTest()
 2{
 3    Console.WriteLine("********************************************************************");
 4    Console.WriteLine("RegressionTest");
 5    Console.WriteLine("********************************************************************");
 6
 7    // Build observation matrices
 8    Matrix xz = new Matrix(101 * 2, 1);
 9    for (int i = 0; i < xz.Rows / 2; ++i)
10    {
11        xz[i, 0] = -1.5 + i / 100.0;
12        xz[i + 101, 0] = -(-1.5 + i / 100.0);
13    }
14
15    Vector yz = new Vector(xz.Rows);
16    for (int i = 0; i < xz.Rows; ++i)
17    {
18        Double x = xz[i, 0];
19        yz[i] = -0.5 * x * x + x + 2 + ((x * 10) % 1) / 10;
20    }
21
22    Matrix tz = new Matrix(151, 1);
23    for (int i = 0; i < tz.Rows; ++i)
24        tz[i, 0] = -3 + i / 30.0;
25
26
27    SVMRegression model = new SVMRegression();
28    model.TrainNuSVR(xz, yz, false, kernel: new FMSVM.KernelPolynomial(coef0: 10));
29    model.Save("svm_s4t1r10_cpp");
30
31    Double pred_val;
32    for (int ri = 0; ri < tz.Rows; ++ri)
33        pred_val = model.Classify(tz.GetRow(ri));
34    Console.WriteLine($"Model {model.Type} with kernel {model.Kernel}");
35
36
37    model.TrainEpsilonSVR(xz, yz);
38    model.Save("svm_s3t2_cpp");
39
40    for (int ri = 0; ri < tz.Rows; ++ri)
41        pred_val = model.Classify(tz.GetRow(ri));
42    Console.WriteLine($"Model {model.Type} with kernel {model.Kernel}");
43}

See Also