Back to search

Cornell University

Correctly Evaluating Machine Learning Algorithms

Abstract

dc:description.abstract

Machine-learning research is progressing rapidly thanks to advances in model architectures, optimization, training algorithms, and scaling. There are many ingredients that allow this rapid progress. One is the ability to quickly build on prior papers, which depends on the clarity of the papers, and the availability of accompanying research code. Another essential ingredient is the correct evaluation of algorithms, which ensures that the papers accurately portray their algorithm's performance. This is important because researchers think of new ideas under the assumption that prior works contain accurate information. If papers do not evaluate algorithms correctly, then the inaccurate results could lead future researchers astray. Some machine learning sub-fields are studied very actively. With so many critical eyes, it is unlikely for incorrect evaluation methodologies to propagate through the literature. However, some other sub-fields attract relatively little scrutiny. As a result, various flawed evaluation methodologies can go unchallenged for years. This leads to distortions in experiment results, which misleads researchers and hinders progress. In this dissertation, we investigate two machine-learning sub-fields: metric learning and unsupervised domain adaptation. We reveal the flaws in the evaluation methodologies of both fields, and present improvements.

Degree

thesis:*
Name thesis:degree_name
Ph. D., Computer Science
Level thesis:degree_level
Doctor of Philosophy
Discipline thesis:degree_discipline
Computer Science
Grantor
Cornell University
Year dc:date.issued
2023

Author and committee

dc:creator, dc:contributor.*
Author dc:creator
  • Musgrave, Kevin
Chair dc:contributor.chair
  • Belongie, Serge
Committee members dc:contributor.committeemember
  • Alvisi, Lorenzo
  • Sine, Wesley

Rights

dc:rights
Statement dc:rights
  • Attribution 4.0 International
Language dc:language.iso
en

Identifiers

dc:identifier.*
Dc Identifier Bibid
16176572
Dc Identifier Other
ProQuest Submission ID: 13591
ProQuest Publication ID: 30426587

Chain of custody

source
Harvested from
Cornell University
Base URL
ecommons.cornell.edu/server/oai/request
Last updated
2026-11-19
Source record
OAI-PMH GetRecord
citation

Musgrave, Kevin. Correctly Evaluating Machine Learning Algorithms. Doctor of Philosophy thesis, Cornell University, 2023. https://hdl.handle.net/1813/114111