diff options
| -rw-r--r-- | docs/misc.rst | 10 | ||||
| -rw-r--r-- | docs/misc/benchmarks-website.rst | 46 | ||||
| -rw-r--r-- | testing/tests/benchmarks/README.md | 7 |
3 files changed, 63 insertions, 0 deletions
diff --git a/docs/misc.rst b/docs/misc.rst new file mode 100644 index 00000000..9f585242 --- /dev/null +++ b/docs/misc.rst @@ -0,0 +1,10 @@ +Miscelaneous pages +================== + +The following pages are not included in the public documentation, but contain +relevant information about the development process. + +.. toctree:: + :maxdepth: 2 + + misc/benchmarks-website.rst diff --git a/docs/misc/benchmarks-website.rst b/docs/misc/benchmarks-website.rst new file mode 100644 index 00000000..14a8412b --- /dev/null +++ b/docs/misc/benchmarks-website.rst @@ -0,0 +1,46 @@ +Benchmarks website +================== + +Currently, showing the results of benchmarks still requires manual +intervention. With time, we want to automatize these tasks. + +How to add a new test to the benchmarks website +----------------------------------------------- + +Current steps are: + +#. Create a new test in ``soledad/testing/tests/benchmarks``, and commit. +#. Push it to ``ssh://0xacab.org/leap/soledad``. +#. Wait until benchmarks stage finishes (so results are posted to elasticsearch). +#. Update kibana visualizations and dashboards:: + + # currently, kibana configurations are in the `scripts` repository. + cd scripts/elastic/ + ./generate-config.py + ./load.sh -url https://moose.leap.se:9200 + +#. Update the benchmarks website:: + + # currently, website lives in the `puppet` repository. + cd puppet/modules/site_benchmarks/ + vim generate-config + ./gen-dashboard-pages.sh + make + git commit -a -m "[benchmarks] update website with new tests" + git push + +TODO +---- + +The following steps are needed to have the website be updated automatically +with all existing stats: + +- Move kibana-related stuff to the ``puppet`` repository. +- Push kibana-related scripts to the server. +- Have kibana be updated periodically with new tests (either using a cron job + and maybe also use the CI to trigger that?) +- Modify website generation scripts so they run in the server to generate + static html (also cron and CI would be great here). +- Enable nginx ssi. +- Modify ``is-master-benchmarked.sh`` script to generate a simple html, include + the generated html in benchmark website. diff --git a/testing/tests/benchmarks/README.md b/testing/tests/benchmarks/README.md index 06a75282..b2465a78 100644 --- a/testing/tests/benchmarks/README.md +++ b/testing/tests/benchmarks/README.md @@ -42,3 +42,10 @@ test process. Some notes about the current resource consumption estimation proce interval might have to be configured on a per-test basis, as different tests take different times to execute (from milliseconds to tens of seconds). For now, an interval of 0.1s seems to cover all tests. + + +Benchmarks website +------------------ + +To update the benchmarks website, see the documentation in +``../../../docs/misc/benchmarks-website.rst``. |