Autoencoder

This topic contains the following sections:

Implementation
Code sample
See Also

An autoencoder is a feed forward neural network which is trained in unsupervised manner to map its input to itself via the representation formed by the hidden units. The main application of such network is the initial deep neural network hidden layers pre-training.

The optimization problem for input data AE XVector is stated as:

AE Problem .

Of course, without any constraints this is a simple task as the model will just try to learn the identity. It becomes a bit more challenging when we restrict the size of the intermediate representation (i.e., the number of hidden units). An several hundred input points can not be squeezed in a representation of a few hidden neurons. Thus, it is assumed that this intermediate representation learns something meaningful about the problem. Of course, using this simple technique only works if the number of hidden neurons is smaller than the number of dimensions of the input. Also L2 regularized is used to obtain a more smooth solution.

Implementation

Next most important methods and properties are featured in the class:

Method	Description
Train(Matrix, Int32)	Train autoencoder.

Code sample

Copy

1var autoModel = new Autoencoder();
2watch.Reset();
3watch.Start();
4double autoR2 = autoModel.Train(trainObs, (trainObs.Columns + 1) / 2);
5watch.Stop();
6Console.WriteLine($"Model: Autoencoder, Coefficient of determination: {autoR2}, TrainTime: {watch.Elapsed}");
7autoModel.Save("NNSample_model_Autoencoder");
8autoModel.Evaluate(gridData).Save("NNSample_grid_Autoencoder.csv");

Other Resources