Web Search: Challenges and Directions

Abstract

These are exciting times for the field of Web search. Search engines are used by millions of people every day, and the number is growing rapidly. This growth poses unique challenges for search engines: they need to operate at unprecedented scales while satisfying an incredible diversity of information needs. Furthermore, user expectations have expanded considerably, moving from “give me what I said” to “give me what I want”. Finally, with the lure of billions of dollars of commerce guided by search engines, we have entered a new world of “Adversarial Information Retrieval”. This talk will show that the world of algorithm and system design for commercial search engines can be described by two of Murphy’s Laws: a) If anything can go wrong, it will; and b) Even if nothing can go wrong, it will anyway.