Features

Pylearn2 contains at least the following features. This documentation often gets out of date, so it probably has even more than this!

If there is something missing that you would like to have, write to pylearn-dev@googlegroups.com. We’ll tell you if we’ve added it and forgotten to add it to the list here. Or if we haven’t added that feature yet, we’ll either add it or give you advice about how to do it yourself.

• Training algorithms

  • A “default training algorithm” that asks the model to train itself

  • Stochastic gradient descent, with extensions including

    • Learning rate decay
    • Momentum
    • Polyak averaging
    • Early stopping
    • A simple framework for adding your own extensions

  • Batch gradient descent with line searches

  • Nonlinear conjugate gradient descent (with line searches)

• Model Estimation Criteria

  • Score Matching
  • Denoising Score Matching
  • Noise-Contrastive Estimation
  • Cross-entropy
  • Log-likelihood

• Models

  • Autoencoders, including Contractive and Denoising Autoencoders

  • RBMs, including gaussian and ssRBM. Varying levels of integration into

    the full framework.

  • k-means

  • Local Coordinate Coding

  • Maxout networks

  • PCA

  • Spike-and-Slab Sparse coding

  • SVMs (we provide a wrapper around scikit-learn that makes it easy to

    train a multiclass svm on dense training data in a memory efficient way, which doesn’t always happen using scikit-learn directly)

  • Partial implementation of DBMs (contact Ian Goodfellow if you would like

    to complete it)

• Datasets:

  • MNIST, MNIST with background and rotations
  • STL-10
  • CIFAR-10, CIFAR-100
  • NIPS Workshops 2011 Transfer Learning Challenge
  • UTLC
  • NORB
  • Toronto Faces Dataset

• Dataset pre-processing

  • Contrast normalization
  • ZCA whitening
  • Patch extraction (for implementing convolution-like algorithms)
  • The Coates+Lee+Ng CIFAR processing pipeline

• Miscellaneous algorithms and utilities:

  • AIS

  • Weight visualization for single layer networks

  • Can plot learning curves showing how user-configured quantities

    change during learning