29. 深度学习的具体模型及方法σ(Wx)σ(WTz)(Binary) Input x(Binary) Features ze.g.自动编码器( AutoEncoder )Encoder filters W
Sigmoid function σ(.)Decoder filters WT
Sigmoid function σ(.)
32. 深度学习的具体模型及方法σ(Wx)DzInput Patch xSparse Features ze.g.Encoder filters W
Sigmoid function σ(.)Decoder filters D
L1 SparsityTraining稀疏自动编码器(Sparse AutoEncoder)
55. 深度学习性能比较随机文法模型
Set of production rules for objects
Zhu & Mumford, Stochastic Grammar of Images, F&T 2006
自动学习人工指定[S.C. Zhu et al.]
56. 深度学习性能比较基于文法模型的物体检测
-R. Girshick, P. Felzenszwalb, D. McAllester, NIPS 2011
-Learn local appearance& shape人工指定自动学习
57. 深度学习性能比较部件和结构模型
Defined connectivity graph
Learn appearance / relative position
[Felzenszwalb & Huttenlocher CVPR’00 ][Fischler and R. Elschlager 1973 ]人工指定自动学习
92. 参考文献Tutorials & Background Material
– Yoshua Bengio, Learning Deep Architectures for AI, Foundations and
Trends in Machine Learning, 2(1), pp.1-127, 2009.
– LeCun, Chopra, Hadsell, Ranzato, Huang: A Tutorial on Energy-Based
Learning, in Bakir, G. and Hofman, T. and Scholkopf, B. and Smola, A.
and Taskar, B. (Eds), Predicting Structured Data, MIT Press, 2006
Convolutional Nets
– LeCun, Bottou, Bengio and Haffner: Gradient-Based Learning Applied to
Document Recognition, Proceedings of the IEEE, 86(11):2278-2324,
November 1998
– Jarrett, Kavukcuoglu, Ranzato, LeCun: What is the Best Multi-Stage
Architecture for Object Recognition?, Proc. International Conference on
Computer Vision (ICCV'09), IEEE, 2009
– Kavukcuoglu, Sermanet, Boureau, Gregor, Mathieu, LeCun: Learning
Convolutional Feature Hierachies for Visual Recognition, Advances in
Neural Information Processing Systems (NIPS 2010), 23, 2010
93. 参考文献Unsupervised Learning
– ICA with Reconstruction Cost for Efficient Overcomplete Feature Learning. Le,Karpenko, Ngiam, Ng. In NIPS 2011
– Rifai, Vincent, Muller, Glorot, Bengio, Contracting Auto-Encoders: Explicit invariance during feature extraction, in: Proceedings of the Twenty-eight International Conference on Machine Learning (ICML'11), 2011
- Vincent, Larochelle, Lajoie, Bengio, Manzagol, Stacked Denoising Autoencoders: Learning Useful Representations in a Deep Network with a Local Denoising Criterion, Journal of Machine Learning Research, 11:3371--3408, 2010.
- Gregor, Szlam, LeCun: Structured Sparse Coding via Lateral Inhibition,
Advances in Neural Information Processing Systems (NIPS 2011), 24, 2011
- Kavukcuoglu, Ranzato, LeCun. "Fast Inference in Sparse Coding Algorithms with Applications to Object Recognition". ArXiv 1010.3467 2008
- Hinton, Krizhevsky, Wang, Transforming Auto-encoders, ICANN, 2011
Multi-modal Learning
– Multimodal deep learning, Ngiam, Khosla, Kim, Nam, Lee, Ng. In Proceedings of the Twenty-Eighth International Conference on Machine Learning, 2011.
94. 参考文献Locally Connected Nets
– Gregor, LeCun “Emergence of complex-like cells in a temporal product network with local receptive fields” Arxiv. 2009
– Ranzato, Mnih, Hinton “Generating more realistic images using gated MRF's”NIPS 2010
– Le, Ngiam, Chen, Chia, Koh, Ng “Tiled convolutional neural networks” NIPS 2010
Distributed Learning
– Le, Ranzato, Monga, Devin, Corrado, Chen, Dean, Ng. "Building High-Level Features Using Large Scale Unsupervised Learning". International Conference of Machine Learning (ICML 2012), Edinburgh, 2012.
Papers on Scene Parsing
– Farabet, Couprie, Najman, LeCun, “Scene Parsing with Multiscale Feature
Learning, Purity Trees, and Optimal Covers”, in Proc. of the International
Conference on Machine Learning (ICML'12), Edinburgh, Scotland, 2012.
- Socher, Lin, Ng, Manning, “Parsing Natural Scenes and Natural Language with Recursive Neural Networks”. International Conference of Machine Learning (ICML 2011) 2011.
95. 参考文献Papers on Object Recognition
- Boureau, Le Roux, Bach, Ponce, LeCun: Ask the locals: multi-way local pooling for image recognition, Proc. ICCV 2011
- Sermanet, LeCun: Traffic Sign Recognition with Multi-Scale Convolutional
Networks, Proceedings of International Joint Conference on Neural Networks (IJCNN'11)
- Ciresan, Meier, Gambardella, Schmidhuber. Convolutional Neural Network
Committees For Handwritten Character Classification. 11th International
Conference on Document Analysis and Recognition (ICDAR 2011), Beijing, China.
- Ciresan, Meier, Masci, Gambardella, Schmidhuber. Flexible, High Performance Convolutional Neural Networks for Image Classification. International Joint Conference on Artificial Intelligence IJCAI-2011.
Papers on Action Recognition
– Learning hierarchical spatio-temporal features for action recognition with
independent subspace analysis, Le, Zou, Yeung, Ng. CVPR 2011
Papers on Segmentation
– Turaga, Briggman, Helmstaedter, Denk, Seung Maximin learning of image
segmentation. NIPS, 2009.
96. 参考文献Papers on Vision for Robotics
– Hadsell, Sermanet, Scoffier, Erkan, Kavackuoglu, Muller, LeCun: Learning Long-Range Vision for Autonomous Off-Road Driving, Journal of Field Robotics,26(2):120-144, February 2009,
Deep Convex Nets & Deconv-Nets
– Deng, Yu. “Deep Convex Network: A Scalable Architecture for Speech Pattern Classification.” Interspeech, 2011.
- Zeiler, Taylor, Fergus "Adaptive Deconvolutional Networks for Mid and High Level Feature Learning." ICCV. 2011
Papers on Biological Inspired Vision
– Serre, Wolf, Bileschi, Riesenhuber, Poggio. Robust Object Recognition with Cortex-like Mechanisms, IEEE Transactions on Pattern Analysis and Machine Intelligence, 29, 3, 411-426, 2007.
- Pinto, Doukhan, DiCarlo, Cox "A high-throughput screening approach to
discovering good forms of biologically inspired visual representation." {PLoS}
Computational Biology. 2009
97. 参考文献Papers on Embedded ConvNets for Real-Time Vision Applications
– Farabet, Martini, Corda, Akselrod, Culurciello, LeCun: NeuFlow: A Runtime Reconfigurable Dataflow Processor for Vision, Workshop on Embedded Computer Vision, CVPR 2011
Papers on Image Denoising Using Neural Nets
– Burger, Schuler, Harmeling: Image Denoisng: Can Plain Neural Networks Compete with BM3D?, Computer Vision and Pattern Recognition, CVPR 2012,