diff options
| -rw-r--r-- | docs/index.rst | 1 | ||||
| -rw-r--r-- | docs/tests.rst | 52 |
2 files changed, 53 insertions, 0 deletions
diff --git a/docs/index.rst b/docs/index.rst index 3dce1b98..e2f37247 100644 --- a/docs/index.rst +++ b/docs/index.rst @@ -16,6 +16,7 @@ all user's devices that access a LEAP provider. client server attachments + tests incoming_box deprecation diff --git a/docs/tests.rst b/docs/tests.rst new file mode 100644 index 00000000..d6391dd2 --- /dev/null +++ b/docs/tests.rst @@ -0,0 +1,52 @@ +Tests +===== + +We use `pytest <https://docs.pytest.org/en/latest/>`_ as a testing framework +and `Tox <https://tox.readthedocs.io>`_ as a test environment manager. +Currently, tests reside in the `testing/` folder and some of them need a +couchdb server to be run against. + +If you do have a couchdb server running on localhost on default port, the +following command should be enough to run tests:: + + cd testing + tox + +CouchDB dependency +------------------ + +In case you want to use a couchdb on another host or port, use the +`--couch-url` parameter for `pytest`:: + + cd testing + tox -- --couch-url=http://couch_host:5984 + +If you want to exclude all tests that depend on couchdb, deselect tests marked +with `needs_couch`:: + + cd testing + tox -- -m 'not needs_couch' + +Benchmark tests +--------------- + +A set of benchmark tests is provided to measure the time and resources taken to +perform some actions. + +`pytest-benchmark` runs tests multiple times so it can provide meaningful +statistics for the time taken for a tipical run of a test function. The number +of times that the test is run can be manually or automatically configured. When +automatically configured, the number of runs is decided by taking into account +multiple `pytest-benchmark` configuration parameters. See the `the +corresponding documenation +<https://pytest-benchmark.readthedocs.io/en/stable/calibration.html>`_ for more +details on how automatic calibration works. + +The actual number of times a test is run depends on many parameters: the time +taken for a sample run and the configuration of the minimum number of rounds +and maximum time allowed for a benchmark. For a snapshot of the number of +rounds for each test function see `the soledad benchmarks wiki page +<https://0xacab.org/leap/soledad/wikis/benchmarks>`_. + +Some graphs and analysis resulting from these tests can be seen on `the +benchmarks page <https://benchmarks.leap.se/>`_. |