A fair comparison starts with a clear task
A result depends on more than a model name. Compare like with like where possible; record what changed, what stayed fixed, and what could not be controlled.
Record the configuration
Keep the task, inputs, harness and model versions, instructions, tools, permissions and date with each experiment.
Use a relevant rubric
Measure the outcome for its field as well as system qualities such as reliability, cost where measurable, and completion of checks.
Report limitations
Keep failures, repetitions and uncertainty visible. Do not treat a single result or another provider’s score as a universal ranking.
Learn about evaluations
