The Computer Language Benchmarks Game (formerly called The Great Computer Language Shootout) is a free software project for comparing how a given subset of simple algorithms can be implemented in various popular programming languages.
The project consists of:
Due to resource constraints, only a small subset of common programming languages are supported, up to the discretion of the game's operator.
The following aspects of each given implementation are measured:
It is common to see multiple solutions in the same programming language for the same problem. This highlights that within the constraints of a given language, a solution can be given which is either of high abstraction, is memory efficient, is fast, or can be parallelized better.
It was a design choice from the start to only include very simple toy problems, each providing a different kind of programming challenge. This provides users of the Benchmark Game the opportunity to scrutinize the various implementations.
The project was known as The Great Computer Language Shootout until 2007.
A port for Windows was maintained separately between 2002 and 2003.
The sources have been archived on GitLab.
There are also older forks on GitHub.
The project is continuously evolving. The list of supported programming languages is updated approximately once per annum, following market trends. Users can also submit improved solutions to any of the problems or suggest testing methodology refinement.
The developers themselves highlight the fact that those doing research should exercise caution when using such microbenchmarks:
This article may contain an excessive number of citations.(May 2018)
The benchmark results have uncovered various compiler issues. Sometimes a given compiler failed to process unusual, but otherwise grammatically valid constructs. At other times, runtime performance was shown to be below expectations, which prompted compiler developers to revise their optimization capabilities.
Edited: 2021-06-18 14:30:32