Learning the Internet

Abstract

The Internet is arguably the most important, complex, interesting, and intellectually challenging computational artifact of our time, and it is therefore worthy of the research community’s attention. The most novel and defining characteristic of the Internet is its nature as an artifact that was not designed by a single entity, but emerged from the complex interaction of many economic agents (network operators, service providers, users, etc.), in various and varying degrees of collaboration and competition. There is a nascent research area that combines algorithmic thinking with concepts from Game Theory, as well as the realities of the Internet, in order to better understand it. This line of research may actually bring back to the fore certain foundational aspects of Game Theory (namely, the nature of equilibria as well as their relationship to repeated play and evolution) in which the methodology of Learning Theory has much to contribute.

The Internet is also an information repository of unprecedented extent, diversity, availability, and lack of structure, and it has therefore become an arena for the development of a new generation of sophisticated techniques for information retrieval and data mining (and to which Learning Theory has obviously much to offer). It has been suggested that economic considerations are crucial in formulating problems also in this environment, and can shed new light on important topics such as clustering and privacy.

Because of its spontaneous emergence, the Internet is the first computational artifact that must be studied as a mysterious object (akin to matter, market, and intelligence) whose laws we must derive by observation, experiment, and the development of falsifiable theories. Recently, a plausible explanation of the skewed degree distributions one observes in the Internet topology was based on simple models of network creation, in which arriving nodes choose connections that achieve a favorable tradeoff among criteria such as last-mile distance and communication delays. These topics will be illustrated in terms of recent models and results by the speaker and co-authors, generally available at the speaker’s web page.

Research supported by two NSF ITR grants, and an IBM award.