How to initialize DNN with discriminative pre-training

Oct 8, 2015 at 12:39 PM
Hi, I would like to initialize a denoising autoencoder with discriminative pre-trained output.
Using TIMIT_TrainWithPreTrain.config in ExampleSetups, discriminative pre-training output is obtained.
Then, how denoising autoencoder can be initialized with this pre-trained output in TIMIT_TrainAutoEncoder.config of ExampleSetups?
Refering to CNTK book, it seems that 'LoadModel()' and 'SetDefaultModel()' can be helpful,
but I don't know how to exactly.

Thank you
Oct 8, 2015 at 6:05 PM
if your goal is to train auto encoder, you can actually directly train it from random model.
Oct 8, 2015 at 11:40 PM
In many papers about denoising autoencoder, pretraining was applied to initialize DNN.
I already trained denoising autoencoder with random initial model,
but it was not effective on the performance improvement of noisy speech recognition(Aurora 4 task in my case).
So, I want to apply pretraining to initialize DNN and to compare the speech recognition performances
according to whether applying pretraining to DNN or not.
My first goal is to train denoising autoencoder, but the second goal is to know how to initialize DNN with pretraining
in other general DNN training cases because I don't have enough training data and computational power.

Thank you