Odyssey 2014

The Speaker and Language Recognition Workshop

NIST Language Recognition Evaluation – Past and Future

Alvin F. Martin, Craig S. Greenberg, John M. Howard, George R. Doddington and John J. Godfrey
This is a review of the six NIST Language Recognition Evaluations from 1996 to 2011. The evolving nature of the task is described, including the (non-)distinction between language and dialect. The languages/dialects tested are noted, and the challenges of data collection for such evaluations and the collections actually undertaken are reviewed. The performance measures employed are defined, and the performance levels achieved in both earlier and later evaluation tasks on different tests are discussed. Plans for the next evaluation in the series are presented.