AN IMPROVED BOUND FOR LEARNING LIPSCHITZ FUNCTIONS

TitleAN IMPROVED BOUND FOR LEARNING LIPSCHITZ FUNCTIONS
Publication TypeJournal Article
Year of Publication2006
AuthorsCooper, DA
Volume10
Issue1
Start Page19
Pagination10
Date Published2006
ISSN1083-2564
AMS26A16, 68T05
Abstract

Revisited here is the problem of learning a nonlinear mapping with uncountable domain and range. The learning model used is that of piecewise linear interpolation on random samples from the domain. More specifically, a network learns a function by approximating its value, typically within some small ǫ, when presented an arbitrary element of the domain. For reliable learning, the network should accurately return the function’s value with high probability, typically higher than${1 − δ }$ for some small ${ δ }$. With new analytic results, we show that, given ǫ and δ and an arbitrary Lipschitz function ${ f : [0, 1]^k → \mathbb{R}, }$ $${ m ≥ (2M √ k)^k · \left( \frac{1}{\epsilon} \right)^k · (k \ ln 2M √ k + k \ ln \frac{1}{\epsilon} + ln \frac{1}{δ}) }$$samples from the uniform distribution on ${ [0, 1]^k }$ are sufficient to reliably learn ${ f}$. The Delaunay triangulation technique is applied in order to keep simplex sizes small. The sufficient condition is an improvement of approximately ${ \left(\frac{2}{3} \right)^k }$ over the previously known upper bound.

URLhttp://www.acadsol.eu/en/articles/10/1/4.pdf
Refereed DesignationRefereed
Full Text

REFERENCES
[1] D.A. Cooper, Probably Approximately Correct Learning on the Class of Lipschitz Functions.
PhD thesis, University of California, Berkeley, 1993.
[2] D.A. Cooper, Learning Lipschitz functions, International Journal of Computer Mathematics,
59 (1995), 15-26.
[3] F. Cucker and S. Smale, On the mathematical foundations of learning, Bulletin of the American
Mathematical Society, 39 (2002), 1-49.
[4] B.K. Natarajan, On learning sets and functions, Machine Learning, 4 (1989), 67-97.
[5] S.M. Omohundro, The Delaunay triangulation and function learning, Technical Report TR-90-
001, International Computer Science Institute (1990).
[6] S.M. Omohundro, Geometric learning algorithms, Physica D, 42 (1990), 307-321.
[7] T. Poggio and F. Girosi, Networks for approximation and learning, Proceedings of the IEEE,
78 (1990), 1481-1497.
[8] F.P. Preparata and M.I. Shamos, Computational Geometry: An Introduction, New York,
Springer-Verlag, 1985.
[9] L.G. Valiant, A theory of the learnable, Communications of the ACM, 27 (1984), 1134-1142.
[10] R.C. Williamson and A.D.B. Paice, The number of nodes required in a feed-forward neural
network for functional representation, Department of Systems Engineering, The Australian
National University, Canberra (1990).