Submission evaluation

A submission is a collection of one or more files that should be evaluated. An evaluation consists of text and/or structured data. An evaluator is a program which evaluates a submission and produces an evaluation as output.

For algorithmic problems, a submission is usually a single file containing the source code of the solution, and possibly another file whose content specifies the programming language used. However, it is possible for a submission to contain more than one file, for example if the different files are written by different people and should be evaluated one against the other (e.g., bots in a two-player game).

TuringArena defines exactly what submission and evaluations are, and how they should be represented.