Nonmonotonic Generalization Bias of Gaussian Mixture Models.
Saved in:
| Title: | Nonmonotonic Generalization Bias of Gaussian Mixture Models. |
|---|---|
| Authors: | Akaho, Shotaro1, Kappen, Hilbert J.2 |
| Source: | Neural Computation. Jun2000, Vol. 12 Issue 6, p1411-1427. 17p. 6 Graphs. |
| Subjects: | Artificial neural networks, Temperature measurements, Symmetry breaking, Transport theory |
| Abstract: | Theories of learning and generalization hold that the generalization bias, defined as the difference between the training error and the generalization error, increases on average with the number of adaptive parameters. This article, however, shows that this general tendency is violated for a gaussian mixture model. For temperatures just below the first symmetry breaking point, the effective number of adaptive parameters increases and the generalization bias decreases. We compute the dependence of the neural information criterion on temperature around the symmetry breaking. Our results are confirmed by numerical cross-validation experiments. [ABSTRACT FROM AUTHOR] |
| Copyright of Neural Computation is the property of MIT Press and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.) | |
| Database: | Engineering Source |
|
Full text is not displayed to guests.
Login for full access.
|
|
| Abstract: | Theories of learning and generalization hold that the generalization bias, defined as the difference between the training error and the generalization error, increases on average with the number of adaptive parameters. This article, however, shows that this general tendency is violated for a gaussian mixture model. For temperatures just below the first symmetry breaking point, the effective number of adaptive parameters increases and the generalization bias decreases. We compute the dependence of the neural information criterion on temperature around the symmetry breaking. Our results are confirmed by numerical cross-validation experiments. [ABSTRACT FROM AUTHOR] |
|---|---|
| ISSN: | 08997667 |
| DOI: | 10.1162/089976600300015439 |
