Fordham University            The Jesuit University of New York

Fordham University
Department of Computer & Information Science
The Society of Computer Science

Revising our Evaluation Practices in Machine Learning"
Professor Nathalie Japkowicz
School of Information Technology and Engineering
University of Ottawa

Date:Thursday November 1, 2007; 4:00pm
Location:  John Mulcahy Hall, Room 404

The evaluation of classifiers or learning algorithms is not a topic that has generally been given much thought in the fields of Machine Learning and Data Mining. More often than not, common off-the-shelf metrics or approaches such as accuracy, precision/recall and ROC Analysis as well as confidence estimation methods, such as t-test, are applied without much attention being paid to their meaning. Similarly, the validation of these learning algorithms is done almost exclusively on the data sets in the UCI Repository, without much thought being put into how representative these data sets are to real-world conditions.

The purpose of this talk is to underline some of the problems that can arise from our current practices. It will then describe a new framework for classifier performance evaluation that views the problem as one of visualization of high-dimensional data. It will conclude by showing how the creation of carefully constructed artificial data sets can help mitigate the shortcomings of the UCI data sets in, at least, one particular real-world setting, that of changing environments.

Dr. Nathalie Japkowicz is an Associate Professor in Computer Science in the School of Information Technology and Engineering at the University of Ottawa. She was a visiting professor at Monash University, Clayton during the 2006-2007 school year. She obtained her Ph.D. from Rutgers University in 1999. Her area of study is Machine Learning with special emphasis on the class imbalance problem, one -class learning, machine learning applied to computer and nuclear security, text mining, and more recently, performance evaluation for machine learning.

For more information, contact:
Ms. Diane Roche (718) 817-4480; (

Site  | Directories
Submit Search Request