2012_An efficient learning procedure for deep Boltzmann Machine ICML2008_Training restricted Boltzmann machines using approximations to the likelihood gradient（PCD） 1. The Boltzman Machine learning algorithm 2. More efficient ways to get the statistics(Optional) 3. Restricted Boltzmann Machine 4. An example of RBM learning 5. RBM for collabrative filtering
NIPS_2007_Sparse deep belief net model for visual area V2 这篇文章主要讲的是sparse DBN。 很多比较学习算法的结果与V1区域相似的工作，但是没有与大脑视觉体系更深层次的比较，比如V2、V4，这篇文章量化的比较了sparse DBN与V2学习的特征，V2的结果引用自这篇文章： M. Ito and H. Komatsu. Representation of angles embedded within contour stimuli in area v2 of macaque monkeys. The Journal of Neuroscience, 24(13):3313–3324, 2004. 1. Introduction J. H. van Hateren and A. van der Schaaf. Independent component filters of natural images compared with simple cells in primary visual cortex. Proc.R.Soc.Lond. B, 265:359–366, 1998. 这篇文章研究表明在自然图像中ICA学习到的filter与V1中简单细胞的局部接受野非常相似。 2. Biological comparison 2.1 Features in early visual cortex: area V1 V1中简单细胞的局部接收野是localized, oriented, bandpass filters that resemble gabor filters. 2.2 Features in visual cortex area V2 J. B. Levitt, D. C. Kiper, and J. A. Movshon. Receptive fields and functional architecture of macaque v2. Journal of Neurophysiology, 71(6):2517–2542, 1994. 这篇文章的研究暗示了area V2 may serve as a place where different channels of visual information are integrated. 接下来讲解了第一节中那篇文章对V2中细胞的选择性的分析。 3. Algorithm 3.1 Sparse RBM Gaussian RBM能量函数： compute conditional probability distribution： 是高斯密度。 加入稀疏惩罚，最终优化问题变为： 其中 是给定数据的conditional expectation， 是regularization constant，p是一个常数控制稀疏程度。 3.2 Learning deep networks using sparse RBM 跟DBN的思想一致，本文学习了含有两个隐层的网络。 4. Visualization 4.1 Learning strokes from handwritten digits 首先PCA降维到69维，然后用69-200结构学习出的结果： 4.2 Learning from natural images 用http://hlab.phys.rug.nl/imlib/index.html的自然图片学习，从2000张图片中抽取100000个14*14的patches，200个patches作为一个mini-batch，用196-400的结构学习得到的结果类似V1： 4.3 Learning a two-layer model of natural images using sparse RBMs 5. Evaluation experiments
Great work on high dimensional video sequence modelling using RBM.To read... Learning Multilevel Distributed Representations for High-Dimensional Sequences.Ilya Sutskever and Geoffrey Hinton
Useful resources on how to use RBM to handle attentional data: (1) Learning to combine foveal glimpses with a third-order Boltzmann machine. Hugo Larochelle and Geoffrey Hinton, Advances in Neural Information Processing Systems 23 , 2010 this is the first implemented system for combining glimpses that jointly trains a recognition component (the RBM) with an attentional component (the fixation controller). (2) Learning Where to Attend With Deep Architectures for Image. Misha Denil, Loris Bazzani, Hugo Larochelle and Nando de Freitas, Neural Computation , 24(8): 2151-2184, 2012 (3) Learning attentional policies for object tracking and recognition in video with deep networks. L. Bazzani, N. de Freitas, H. Larochelle, V. Murino, and J-A Ting, In International Conference on Machine Learning (ICML), 2011. Links: (1) http://www.lorisbazzani.info/ (2) http://www.dmi.usherb.ca/~larocheh/index_en.html Several very new works.
标签: RBM
