About this site

This site runs all benchmarks from the Benchmark Runner project, which is based on Stefan Marr's SMark. The benchmarks are taken from the computer language shootout, the Squeak mailinglist, some papers on RSqueak and SqueakJS performance, and various contributors to the Cog VM and Squeak.
The Benchmark Runner runs each benchmark in a separate process through a .st file passed at startup and then closes the image, printing results to stdout. A benchmark is first autosized, that is, it is run repeatedly until the runtime reaches at least 600ms. Once the iteration count N for that time is found, we run the benchmark 100 times in a loop using that iteration count N. So for each of the 100 runs, we get a runtime T, where the runtime for one benchmark iteration T_i=T/N. We calculate the average runtime over the 100 runs and the standard deviation without the top and bottom 2.5%. Those numbers are sent to this Codespeed site.

This site runs on top of Django and Codespeed

About the Platforms

Currently we are running benchmarks on a dedicated bare metal Ubuntu machine. We run benchmarks for every RSqueak commit and (for now) for every published Cog build. The times for the latter do not always correspond to the publication time, because we have to trigger those manually.

About Codespeed

Codespeed is a web application to monitor and analyze the performance of your code.

Code: github.com/tobami/codespeed

Wiki: wiki.github.com/tobami/codespeed/


For problems or suggestions about this website write to tim.felgentreff at hpi.de and tobias.pape at hpi.de