Aqua

Check package quality separately from an operation benchmark.

From examples/kitchen-sink in the source checkout, run julia setup.jl analyzers. Then start Julia with julia --project=.controller/analyzers for the code below.

using PerfChecker
source = Base.find_package("DataStructures")
quality = diagnose(ScenarioCatalog(dirname(dirname(source)), ScenarioSpec[]);
    project=dirname(Base.active_project()), tools=[:aqua], threads=1,
    timeout=180, reports="results/quality")
display(only(quality["records"]))

The recorded receipt contains Aqua 0.8.16, status=complete, quality=failed, with unbound type parameters in DefaultDictBase, DefaultDict and DefaultOrderedDict.

Execution completion does not mean the package passed its quality checks. Aqua analyzes the installed package, once per package, rather than only the selected operation.