From 5ad13db3f3aa147efdf255d410dc8bad0b0b3113 Mon Sep 17 00:00:00 2001
From: drebs <drebs@leap.se>
Date: Fri, 30 Jun 2017 12:29:59 -0300
Subject: [doc] add tests and benchmarks documentation

---
 docs/index.rst |  1 +
 docs/tests.rst | 52 ++++++++++++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 53 insertions(+)
 create mode 100644 docs/tests.rst

(limited to 'docs')

diff --git a/docs/index.rst b/docs/index.rst
index 3dce1b98..e2f37247 100644
--- a/docs/index.rst
+++ b/docs/index.rst
@@ -16,6 +16,7 @@ all user's devices that access a LEAP provider.
    client
    server
    attachments
+   tests
    incoming_box
    deprecation
 
diff --git a/docs/tests.rst b/docs/tests.rst
new file mode 100644
index 00000000..d6391dd2
--- /dev/null
+++ b/docs/tests.rst
@@ -0,0 +1,52 @@
+Tests
+=====
+
+We use `pytest <https://docs.pytest.org/en/latest/>`_ as a testing framework
+and `Tox <https://tox.readthedocs.io>`_ as a test environment manager.
+Currently, tests reside in the  `testing/` folder and some of them need a
+couchdb server to be run against.
+
+If you do have a couchdb server running on localhost on default port, the
+following command should be enough to run tests::
+
+    cd testing
+    tox
+
+CouchDB dependency
+------------------
+
+In case you want to use a couchdb on another host or port, use the
+`--couch-url` parameter for `pytest`::
+
+    cd testing
+    tox -- --couch-url=http://couch_host:5984
+
+If you want to exclude all tests that depend on couchdb, deselect tests marked
+with `needs_couch`::
+
+    cd testing
+    tox -- -m 'not needs_couch'
+
+Benchmark tests
+---------------
+
+A set of benchmark tests is provided to measure the time and resources taken to
+perform some actions.
+
+`pytest-benchmark` runs tests multiple times so it can provide meaningful
+statistics for the time taken for a tipical run of a test function. The number
+of times that the test is run can be manually or automatically configured. When
+automatically configured, the number of runs is decided by taking into account
+multiple `pytest-benchmark` configuration parameters. See the `the
+corresponding documenation
+<https://pytest-benchmark.readthedocs.io/en/stable/calibration.html>`_ for more
+details on how automatic calibration works.
+
+The actual number of times a test is run depends on many parameters: the time
+taken for a sample run and the configuration of the minimum number of rounds
+and maximum time allowed for a benchmark. For a snapshot of the number of
+rounds for each test function see `the soledad benchmarks wiki page
+<https://0xacab.org/leap/soledad/wikis/benchmarks>`_.
+
+Some graphs and analysis resulting from these tests can be seen on `the
+benchmarks page <https://benchmarks.leap.se/>`_.
-- 
cgit v1.2.3