Olympiads in Informatics Logo

OLYMPIADS IN INFORMATICS
International Journal

Main Page
Editorial Board
Instructions to Authors


Contents
Author Index
Papers in Production

OLYMPIADS IN INFORMATICS, 2009, Vol. 3, 3-16
© Institute of Mathematics and Informatics,

ISSN 1822-7732

Using Item Response Theory to Rate (not only) Programmers

Michal FORISEK

Department of Informatics, Faculty of Mathematics, Physics and Informatics Comenius University Mlynská dolina, 842 48 Bratislava, Slovakia E-mail: forisek@dcs.fmph.uniba.sk

Abstract

We show how Item Response Theory (IRT) can be used to define a new type of rating system, one that is especially suitable for programming competitions (and other types of competitions where difficulty of competitions varies between rounds). We show some useful theoretical properties of this rating system, including the ability to argue about hardness of past competition tasks, and about the precision of contestants' skill estimates. We also define an objective method of comparing different rating systems. In the final section of the paper we apply our methods on real competition data.

Keywords:

item response theory, ranking, rating, programming competitions


PDFTo preview full article text in PDF format click here

Get Free ReaderYou could obtain free Acrobat Reader from Adobe


TopTop Copyright © Olympiads in Informatics, Vilnius University Institute of Mathematics and Informatics, 2013