OLYMPIADS IN INFORMATICS
International Journal

	Main Page
	Editorial Board
	Contents
	Author Index
	Papers in Production

OLYMPIADS IN INFORMATICS, 2009, Vol. 3, 3-16
© Institute of Mathematics and Informatics,

ISSN 1822-7732

Using Item Response Theory to Rate (not only) Programmers

Michal FORISEK

Department of Informatics, Faculty of Mathematics, Physics and Informatics Comenius University Mlynská dolina, 842 48 Bratislava, Slovakia E-mail: forisek@dcs.fmph.uniba.sk

Abstract

We show how Item Response Theory (IRT) can be used to define a new type of rating system, one that is especially suitable for programming competitions (and other types of competitions where difficulty of competitions varies between rounds). We show some useful theoretical properties of this rating system, including the ability to argue about hardness of past competition tasks, and about the precision of contestants' skill estimates. We also define an objective method of comparing different rating systems. In the final section of the paper we apply our methods on real competition data.

Keywords:

item response theory, ranking, rating, programming competitions

To preview full article text in PDF format click here

You could obtain free Acrobat Reader from Adobe