Download

R. Collobert. Large Scale Machine Learning. Université de Paris VI, LIP6, 2004.

Abstract

This thesis aims to address machine learning in general, with a particular focus on large models and large databases. After introducing the learning problem in a formal way, we first review several important machine learning algorithms, particularly Multi Layer Perceptrons, Mixture of Experts and Support Vector Machines. We then present a training method for Support Vector Machines, adapted to reasonably large datasets. However the training of such a model is still intractable on very large databases. We thus propose a divide and conquer approach based on a kind of Mixture of Experts in order to break up the training problem into small pieces, while keeping good generalization performance. This mixture model can be applied to any kind of existing machine learning algorithm. Even though it performs well in practice the major drawback of this algorithm is the number of hyper-parameters to tune, which makes it difficult to use. We thus prefer afterward to focus on training improvements for Multi Layer Perceptrons, which are easier to tune, and more suitable than Support Vector Machines for large databases. We finally show that the margin idea introduced with Support Vector Machines can be applied to a certain class of Multi Layer Perceptrons, which leads to a fast algorithm with powerful generalization performance.

BibTeX

@phdthesis{collobert:2004b,
    author = {Ronan Collobert},
    title = {Large Scale Machine Learning},
    school = {Universit\'e Paris {VI}},
    year = {2004},
}

Notes

This is my PhD thesis. I did my PhD both at IDIAP and Université de Montréal. I defended at Université de Paris VI, in the LIP6 lab.