diff options
| author | varac <varacanero@zeromail.org> | 2017-03-22 11:24:42 +0100 |
|---|---|---|
| committer | varac <varacanero@zeromail.org> | 2017-04-03 15:17:01 +0200 |
| commit | 109608e8c1051e8569f06f02f477412d16e4ef66 (patch) | |
| tree | a94280140017e4d7e34e94c9915a54652982b4b3 | |
| parent | 6067f8c001d524d29ccc4287a5f2efec90fa5004 (diff) | |
Post benchmark results to elastticsearch
In order to do this from CI, please create a secret variable
(at i.e. https://0xacab.org/varac/soledad/settings/ci_cd)
called PYTEST_OPTS and use this example:
--benchmark-autosave --benchmark-storage="elasticsearch+https://USER:PASSWORD@moose.leap.se:9200"
| -rw-r--r-- | .gitignore | 3 | ||||
| -rw-r--r-- | .gitlab-ci.yml | 26 | ||||
| -rw-r--r-- | testing/tox.ini | 5 |
3 files changed, 32 insertions, 2 deletions
@@ -9,6 +9,7 @@ MANIFEST *.swp *.swo *.pyc +*.pyo *.log *.*~ *.csv @@ -21,4 +22,4 @@ scripts/profiling/sync/profiles testing/htmlcov testing/.coverage -testing/test-env
\ No newline at end of file +testing/test-env diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml index ee57c51c..6cafdf54 100644 --- a/.gitlab-ci.yml +++ b/.gitlab-ci.yml @@ -1,12 +1,16 @@ stages: - code-check - tests + - benchmark # Cache tox envs between builds cache: paths: - testing/.tox/ +before_script: + - echo "Running on ${HOST_HOSTNAME:=$(hostname)}" + code-check: stage: code-check image: leapcode/soledad:latest @@ -22,3 +26,25 @@ tests: script: - cd testing - tox -- --couch-url http://couchdb:5984 + +benchmark: + stage: benchmark + image: leapcode/soledad:latest + tags: + - benchmark + services: + - couchdb + allow_failure: true + script: + - cd testing + - curl -s couchdb:5984 + # You can provide a $NETRC variable containing the creds for your + # elasticsearch instance so it's protected from being leaked in the + # CI console + # We can't get it working inside docker for unknown reasons. + #- echo "$NETRC" > /root/.netrc && chmod 600 /root/.netrc + # + # Add $PYTEST_OPTS to pytest.ini to allow posting benchmark tests + # to an elasticsearch instance + - echo "addopts=$PYTEST_OPTS" >> pytest.ini && chmod 600 pytest.ini + - tox --recreate -e benchmark -- --couch-url http://couchdb:5984 diff --git a/testing/tox.ini b/testing/tox.ini index d959db0a..d800d988 100644 --- a/testing/tox.ini +++ b/testing/tox.ini @@ -62,7 +62,10 @@ install_command = pip3 install {opts} {packages} [testenv:benchmark] deps = {[testenv]deps} - pytest-benchmark + #pytest-benchmark>=3.1.0a2 + git+https://github.com/drebs/pytest-benchmark.git@use-netrc-file + elasticsearch + certifi commands = py.test --benchmark-only {posargs} [testenv:code-check] |