ntpstats.bench

Benchmark synchronisation estimators against simulated ground truth.

rows = run_bench(load_scenarios(["internet", "examples/scenarios/route-change.toml"]),
                 estimators=["kalman-dw", "regression", "rfc5905"], seeds=range(1, 6))
table = summarize(rows)

Scenarios are preset names (:data:ntpstats.simulate.PRESETS) or TOML/JSON files (see :func:ntpstats.simulate.scenario_from_dict). Single-server estimators are scored on the first server; multi-server estimators get all.

Metrics of the estimation error e = estimate - truth: RMS, bias (mean), p95 and max of |e|, MTIE of e over 1 h windows, and wall-clock runtime. The first warmup seconds (default 30 min) are excluded, as usual when comparing synchronisation algorithms, so start-up transients do not dominate.

trace_scenario(path, mode='replay')

A scenario that replays the delays of a capture or log (trace:FILE on the command line).

The poll interval and duration are those of the trace; the clock is the internet preset's. For more control write a scenario file with a [trace] table.

summarize(rows)

Mean and standard deviation of each metric over seeds.