Fordham University
Department of Computer & Information
Science
And
The Society of Computer Science
Present
Revising our Evaluation Practices in Machine Learning"
by
Professor Nathalie Japkowicz
School of Information Technology and Engineering
University of Ottawa
| Date: | Thursday November 1, 2007; 4:00pm |
| Location: | John Mulcahy Hall, Room
404 |
Abstract:
The evaluation of classifiers or learning algorithms is not a topic that
has generally been given much thought in the fields of Machine Learning and
Data Mining. More often than not, common off-the-shelf metrics or approaches
such as accuracy, precision/recall and ROC Analysis as well as confidence
estimation methods, such as t-test, are applied without much attention being
paid to their meaning. Similarly, the validation of these learning algorithms
is done almost exclusively on the data sets in the UCI Repository, without
much thought being put into how representative these data sets are to
real-world conditions.
The purpose of this talk is to underline some of the problems that can
arise from our current practices. It will then describe a new framework for
classifier performance evaluation that views the problem as one of
visualization of high-dimensional data. It will conclude by showing how the
creation of carefully constructed artificial data sets can help mitigate the
shortcomings of the UCI data sets in, at least, one particular real-world
setting, that of changing environments.
Bio:
Dr. Nathalie Japkowicz is an Associate Professor in Computer Science in the
School of Information Technology and Engineering at the University of Ottawa.
She was a visiting professor at Monash University, Clayton during the
2006-2007 school year. She obtained her Ph.D. from Rutgers University in
1999. Her area of study is Machine Learning with special emphasis on the
class imbalance problem, one -class learning, machine learning applied to
computer and nuclear security, text mining, and more recently, performance
evaluation for machine learning.
For more information, contact:
Ms. Diane Roche (718) 817-4480; (roche@cis.fordham.edu)
|
