Skip to content

Choose an advanced experiment

Start here after you have collected a first result and can explain what it measures. These workflows add control over the experiment: which implementation runs, which Julia runtime runs it, or which machine supplies the observations. They are independent of the optional model integrations.

Your questionWhat you need firstFollow
Does this source change make the same operation faster?Two revisions and a common workloadCompare versions
How do I compare implementations while preparing and checking each sample separately?A workload with an explicit correctness checkShared workload contracts
Does a new Julia runtime change performance or compatibility?A runnable software suite and installed runtimesJulia runtime comparisons
Can measurements from other machines help predict mine?Matching measurements from donor machines and calibration runs on yoursMachine calibration

For a reproducible source comparison, use the Bibliography export experiment. It holds dependency revisions fixed while changing the export implementation. The nine-version history answers a broader question about released package stacks, whose dependencies also change. Those are different experiments.

Keep the resulting reports: they record both the measurements and the conditions under which they were collected. The measurement model explains which differences prevent a numerical comparison.

Open source · Report an issue · Contributions welcome