[doc] add info on how to update benchmarks website

3 files changed, 63 insertions, 0 deletions

diff --git a/docs/misc.rst b/docs/misc.rst
new file mode 100644
index 00000000..9f585242
--- /dev/null
+++ b/docs/misc.rst
@@ -0,0 +1,10 @@
+Miscelaneous pages
+==================
+
+The following pages are not included in the public documentation, but contain
+relevant information about the development process.
+
+.. toctree::
+   :maxdepth: 2
+
+   misc/benchmarks-website.rst
diff --git a/docs/misc/benchmarks-website.rst b/docs/misc/benchmarks-website.rst
new file mode 100644
index 00000000..14a8412b
--- /dev/null
+++ b/docs/misc/benchmarks-website.rst
@@ -0,0 +1,46 @@
+Benchmarks website
+==================
+
+Currently, showing the results of benchmarks still requires manual
+intervention. With time, we want to automatize these tasks.
+
+How to add a new test to the benchmarks website
+-----------------------------------------------
+
+Current steps are:
+
+#. Create a new test in ``soledad/testing/tests/benchmarks``, and commit.
+#. Push it to ``ssh://0xacab.org/leap/soledad``.
+#. Wait until benchmarks stage finishes (so results are posted to elasticsearch).
+#. Update kibana visualizations and dashboards::
+
+    # currently, kibana configurations are in the `scripts` repository.
+    cd scripts/elastic/
+    ./generate-config.py
+    ./load.sh -url https://moose.leap.se:9200
+
+#. Update the benchmarks website::
+
+    # currently, website lives in the `puppet` repository.
+    cd puppet/modules/site_benchmarks/
+    vim generate-config
+    ./gen-dashboard-pages.sh
+    make
+    git commit -a -m "[benchmarks] update website with new tests"
+    git push
+
+TODO
+----
+
+The following steps are needed to have the website be updated automatically
+with all existing stats:
+
+- Move kibana-related stuff to the ``puppet`` repository.
+- Push kibana-related scripts to the server.
+- Have kibana be updated periodically with new tests (either using a cron job
+  and maybe also use the CI to trigger that?)
+- Modify website generation scripts so they run in the server to generate
+  static html (also cron and CI would be great here).
+- Enable nginx ssi.
+- Modify ``is-master-benchmarked.sh`` script to generate a simple html, include
+  the generated html in benchmark website.
diff --git a/testing/tests/benchmarks/README.md b/testing/tests/benchmarks/README.md
index 06a75282..b2465a78 100644
--- a/testing/tests/benchmarks/README.md
+++ b/testing/tests/benchmarks/README.md
@@ -42,3 +42,10 @@ test process. Some notes about the current resource consumption estimation proce
   interval might have to be configured on a per-test basis, as different tests
   take different times to execute (from milliseconds to tens of seconds). For
   now, an interval of 0.1s seems to cover all tests.
+
+
+Benchmarks website
+------------------
+
+To update the benchmarks website, see the documentation in
+``../../../docs/misc/benchmarks-website.rst``.