Word Frequency and Predictability

Janet Pierrehumbert 

Wednesdays@NICO Seminar, Noon, May 30, 2012, Chambers Hall, Lower Level

Prof. Janet Pierrehumbert, Northwestern University


Document retrieval algorithms and theories of human language processing often assume that texts are generated by a Poisson process in which each word has a stationary frequency. Word frequencies actually fluctuate greatly, at time scales from sentences through decades or centuries. In this talk, I will discuss results on modeling these fluctuations. I will also present experimental results on the role they place in human language processing.


Janet Pierrehumbert is Professor of Linguistics at Northwestern University. She received her Ph.D from MIT in 1980, and worked at AT&T Bell Labs in Linguistics and AI Research until 1989, when she joined the Northwestern faculty. Her research concerns the ways in which the dynamics of language -- in acquisition, processing, or historical change -- relate to the structure of linguistic systems. It combines experiments, statistical analyses of large corpora, and computational simulations of linguistic communities. She is a Fellow of the Guggenheim Foundation, the Linguistic Society of America, and the American Academy of Arts and Sciences. She served on the NICO Executive Committee from 2005 to 2011.