SVMTorch: Support Vector Machines for Large-Scale Regression Problems
Download
R. Collobert and S. Bengio. SVMTorch: Support Vector Machines for Large-Scale Regression Problems. Journal of Machine Learning Research, 1:143-160, 2001.
Abstract
Support Vector Machines (SVMs) for regression problems are trained by solving a quadratic optimization problem which needs on the order of l square memory and time resources to solve, where l is the number of training examples. In this paper, we propose a decomposition algorithm, SVMTorch (available at http://www.idiap.ch/learning/SVMTorch.html), which is similar to SVM-Light proposed by Joachims (1999) for classification problems, but adapted to regression problems. With this algorithm, one can now efficiently solve large-scale regression problems (more than 20000 examples). Comparisons with Nodelib, another publicly available SVM algorithm for large-scale regression problems from Flake and Lawrence (2000) yielded significant time improvements. Finally, based on a recent paper from Lin (2000), we show that a convergence proof exists for our algorithm.
BibTeX
@article{collobert:2001,
author = {R. Collobert and S. Bengio},
title = {{SVMT}orch: Support Vector Machines for Large-Scale Regression Problems},
journal = {Journal of Machine Learning Research},
volume = 1,
pages = {143--160},
year = 2001
}
Notes
Our contribution extends Joachims ideas to the regression SVM problem. Though nowadays it may seems obvious, curiously it was not the technique used to train regression SVMs at the time we proposed this extension.
Links