Abstract
Selective sampling, a realistic active learning model, has received recent attention in the learning theory literature. While the analysis of selective sampling is still in its infancy, we focus here on one of the (seemingly) simplest problems that remain open. Given a pool of unlabeled examples, drawn i.i.d. from an arbitrary input distribution known to the learner, and oracle access to their labels, the objective is to achieve a target error-rate with minimum label-complexity, via an efficient algorithm. No prior distribution is assumed over the concept class, however the problem remains open even under the realizability assumption: there exists a target hypothesis in the concept class that perfectly classifies all examples, and the labeling oracle is noiseless. As a precise variant of the problem, we consider the case of learning homogeneous half-spaces in the realizable setting: unlabeled examples, x t , are drawn i.i.d. from a known distribution D over the surface of the unit ball in ℝd and labels y t are either –1 or +1. The target function is a half-space u x ≥0 represented by a unit vector u ∈ℝd such that y t (u x t ) > 0 for all t. We denote a hypothesis v’s prediction as v(x) = SGN(v ·x).
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Angluin, D.: Queries revisited. In: Abe, N., Khardon, R., Zeugmann, T. (eds.) ALT 2001. LNCS (LNAI), vol. 2225, pp. 12–31. Springer, Heidelberg (2001)
Balcan, M.-F., Beygelzimer, A., Langford, J.: Agnostic active learning. In: International Conference on Machine Learning (2006)
Cesa-Bianchi, N., Gentile, C., Zaniboni, L.: Worst-case analysis of selective sampling for linear-threshold algorithms. In: Advances in Neural Information Processing Systems 17 (2004)
Cohn, D.A., Atlas, L., Ladner, R.E.: Improving generalization with active learning. Machine Learning 15(2), 201–221 (1994)
Dasgupta, S.: Coarse sample complexity bounds for active learning. In: Advances in Neural Information Processing Systems 18 (2005)
Dasgupta, S., Kalai, A.T., Monteleoni, C.: Analysis of perceptron-based active learning. In: Auer, P., Meir, R. (eds.) COLT 2005. LNCS, vol. 3559, pp. 249–263. Springer, Heidelberg (2005)
Freund, Y., Seung, H.S., Shamir, E., Tishby, N.: Selective sampling using the query by committee algorithm. Machine Learning 28(2-3), 133–168 (1997)
Gilad-Bachrach, R., Navot, A., Tishby, N.: Query by committee made real. In: Advances in Neural Information Processing Systems 18 (2005)
Kääriäinen, M.: On active learning in the non-realizable case. In: Foundations of Active Learning Workshop at Neural Information Processing Systems Conference (2005)
Lewis, D.D., Gale, W.A.: A sequential algorithm for training text classifiers. In: Proc. of SIGIR 1994, 17th ACM International Conference on Research and Development in Information Retrieval (1994)
Long, P.M.: On the sample complexity of PAC learning halfspaces against the uniform distribution. IEEE Transactions on Neural Networks 6(6), 1556–1559 (1995)
Long, P.M.: An upper bound on the sample complexity of PAC learning halfspaces with respect to the uniform distribution. Information Processing Letters 87(5), 223–229 (2003)
Seung, H.S., Opper, M., Sompolinsky, H.: Query by committee. In: Proc. Fifth Annual ACM Conference on Computational Learning Theory (1992)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2006 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Monteleoni, C. (2006). Efficient Algorithms for General Active Learning. In: Lugosi, G., Simon, H.U. (eds) Learning Theory. COLT 2006. Lecture Notes in Computer Science(), vol 4005. Springer, Berlin, Heidelberg. https://doi.org/10.1007/11776420_47
Download citation
DOI: https://doi.org/10.1007/11776420_47
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-35294-5
Online ISBN: 978-3-540-35296-9
eBook Packages: Computer ScienceComputer Science (R0)