Abstract
Nowadays, more and more decision procedures are supported or even guided by automated processes. An important technique in this automation is data mining. In this chapter we study how such automatically generated decision support models may exhibit discriminatory behavior towards certain groups based upon, e.g., gender or ethnicity. Surprisingly, such behavior may even be observed when sensitive information is removed or suppressed and the whole procedure is guided by neutral arguments such as predictive accuracy only. The reason for this phenomenon is that most data mining methods are based upon assumptions that are not always satisfied in reality, namely, that the data is correct and represents the population well. In this chapter we discuss the implicit modeling assumptions made by most data mining algorithms and show situations in which they are not satisfied. Then we outline three realistic scenarios in which an unbiased process can lead to discriminatory models. The effects of the implicit assumptions not being fulfilled are illustrated by examples. The chapter concludes with an outline of the main challenges and problems to be solved.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Blank, R., Dabady, M., Citro, C.: Measuring Racial Discrimination. Natl Academy Press (2004)
Jonah, B.A.: Accident risk and risk-taking behavior among young drivers. Accident Analysis & Prevention 18(4), 255–271 (1986)
Calders, T., Verwer, S.: Three Naive Bayes Approaches for Discrimination-Free Classification. Data Mining and Knowledge Discovery 21(2), 277–292 (2010)
Distance Learning Center. Internet Based Benefit and Compensation Administration: Discrimination in Pay, ch. 26 (2009), http://www.eridlc.com/index.cfm?fuseaction=textbook.chpt26 (accessed: November 2011)
Duda, R.O., Hart, P.E., Stork, D.G.: Pattern Classification, 2nd edn. John Wiley & Sons (2001)
Fang, H., Moro, A.: Theories of Statistical Discrimination and Affirmative Action: A Survey. In: Benhabib, J., Bisin, A., Jackson, M. (eds.) Handbook of Social Economics, pp. 133–200 (2010)
Kamiran, F., Calders, T.: Classification with no discrimination by preferential sampling. In: Proceedings of the 19th Annual Machine Learning Conference of Belgium and the Netherlands (BENELEARN 2010), pp. 1–6 (2010)
Kamiran, F., Calders, T.: Classifying without Discrimination. In: IEEE International Conference on Computer, Control and Communication (IEEE-IC4), pp. 1–6 (2009)
Kamiran, F., Calders, T., Pechenizkiy, M.: Discrimination Aware Decision Tree Learning. In: Proceedings of IEEE ICDM International Conference on Data Mining (ICDM 2010), pp. 869–874 (2010)
Kelly, M.G., Hand, D.J., Adams, N.M.: The Impact of Changing Populations on Classifier Performance. In: Proceedings of the Fifth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD 1999), pp. 367–371 (1999)
Rice, W.: Race, Gender, “Redlining”, and the Discriminatory Access to Loans, Credit, and Insurance: An Historical and Empirical Analysis of Consumers Who Sued Lenders and Insurers in Federal and State Courts. San Diego Law Review 33, 637–646 (1996)
Turner, A., Skidmore, F.: Introduction, Summary, and Recommendations. In: Turner, A., Skidmore, F. (eds.) Mortgage Lending Discrimination: A Review of Existing Evidence (Urban Institute Monograph Series on Race and Discrimination), pp. 1–22. Urban Institute Press, Washington, DC (1999)
Widmer, G., Kubat, M.: Learning in the presence of concept drift and hidden contexts. Machine Learning 23(1), 69–101 (1996)
Zadrozny, B.: Learning and Evaluating Classifiers under Sample Selection Bias. In: Proceedings of the 21st International Conference on Machine Learning (ICML 2004), pp. 903–910 (2004)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2013 Springer-Verlag Berlin Heidelberg
About this chapter
Cite this chapter
Calders, T., Žliobaitė, I. (2013). Why Unbiased Computational Processes Can Lead to Discriminative Decision Procedures. In: Custers, B., Calders, T., Schermer, B., Zarsky, T. (eds) Discrimination and Privacy in the Information Society. Studies in Applied Philosophy, Epistemology and Rational Ethics, vol 3. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-30487-3_3
Download citation
DOI: https://doi.org/10.1007/978-3-642-30487-3_3
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-30486-6
Online ISBN: 978-3-642-30487-3
eBook Packages: EngineeringEngineering (R0)