diff options
Diffstat (limited to 'docs/benchmarks.rst')
| -rw-r--r-- | docs/benchmarks.rst | 24 |
1 files changed, 22 insertions, 2 deletions
diff --git a/docs/benchmarks.rst b/docs/benchmarks.rst index 0439f7d4..69724aa0 100644 --- a/docs/benchmarks.rst +++ b/docs/benchmarks.rst @@ -1,11 +1,31 @@ +.. _benchmarks: + Benchmarks ========== -Soledad has a set of benchmark tests to assess the time and resources taken by -various tasks. +We currently use `pytest-benchmark <https://pytest-benchmark.readthedocs.io/>`_ +to write tests to assess the time and resources taken by various tasks. Results of benchmarking can be seen in https://benchmarks.leap.se/. +Test repetition +--------------- + +`pytest-benchmark` runs tests multiple times so it can provide meaningful +statistics for the time taken for a tipical run of a test function. The number +of times that the test is run can be manually or automatically configured. When +automatically configured, the number of runs is decided by taking into account +multiple `pytest-benchmark` configuration parameters. See the `the +corresponding documenation +<https://pytest-benchmark.readthedocs.io/en/stable/calibration.html>`_ for more +details on how automatic calibration works. + +The actual number of times a test is run depends on many parameters: the time +taken for a sample run and the configuration of the minimum number of rounds +and maximum time allowed for a benchmark. For a snapshot of the number of +rounds for each test function see `the soledad benchmarks wiki page +<https://0xacab.org/leap/soledad/wikis/benchmarks>`_. + Sync size statistics -------------------- |