summaryrefslogtreecommitdiff
path: root/testing/tox.ini
AgeCommit message (Collapse)Author
2017-06-24[bug] fix testsVictor Shyba
2017-06-24[tests] install unified package with extras in toxKali Kaneko
2017-06-24[test] adapt tests to new python packagedrebs
2017-05-31[tests] use pysqlcipher workaround here tooKali Kaneko
2017-04-27[test] measure cpu percentage during benchmarkdrebs
2017-04-19[test] avoid running sqlcipher synchonous tests when benchmarkingdrebs
SQLCipher synchronous benchmark tests were introduced when we started developing benchmark tests to compare synchronous and asynchronous code. Synchronous access to sqlcipher database is not used in soledad, and those tests are much slower than asynchronous tests (more than 10 times using ssd drive), so we want to avoid running them on ci. This commit introduces a "synchronous" marker and avoid running tests markes as such in ci environment.
2017-04-19[test] use latest pysqlcipher-benchmark for benchmarksdrebs
There were some changes needed in pytest-benchmark so we could successfully use it for soledad benchmarks graphing: https://github.com/ionelmc/pytest-benchmark/pull/73 https://github.com/ionelmc/pytest-benchmark/pull/74 https://github.com/ionelmc/pytest-benchmark/pull/75 The contributions were accepted but not released yet, so this commit uses the code from upstream git repository's master branch.
2017-04-15[test] enforce bundled pysqlcipher in benchmark testsdrebs
The debian package for libsqlcipher does not enable the use of usleep(), and that might cause timeouts when doing concurrent access to sqlcipher databases. See: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=859685 In benchmark tests we use twisted's asynchronous dbapi for creating thousands of documents, and if usleep() is not enabled this operation is very likelly to fail with a "db is locked" error. This commit adds a workaround for benchmark tests by reinstalling pysqlcipher with the "--bundled" option, what causes libsqlcipher to be built and bundled in the python package, with usleep() enabled. Resolves: #8835
2017-04-04[feat] add the host hostname to benchmark machine infodrebs
2017-04-04[bug] cache pip instead of toxVictor Shyba
Caching .tox breaks when installing new dependencies. Caching pip instead and asking tox to always recreate will ensure newer dependencies are installed while getting existing ones from cache.
2017-04-03Post benchmark results to elastticsearchvarac
In order to do this from CI, please create a secret variable (at i.e. https://0xacab.org/varac/soledad/settings/ci_cd) called PYTEST_OPTS and use this example: --benchmark-autosave --benchmark-storage="elasticsearch+https://USER:PASSWORD@moose.leap.se:9200"
2017-03-17[test] Prepare new tox environmentefkin
At this point tests are not yet collectable in the new environment. But no regressions observed.
2016-12-12[tests] use options instead of marksVictor Shyba
When we use marks the new pytests from benchmarks folder are collected and ignored, but this causes trial to fail sometimes. Using --ignore avoids it from being loaded while --benchmark-only will properly select the benchmarks for tox, as intended.
2016-12-12[test] rename benchmark tests directory and tagdrebs
2016-12-12[test] use tags for selecting benchmark testsdrebs
2016-12-12[style] fix pep8 and confsVictor Shyba
Fixes setup.cfg, adding current exclude rules, simplified tox.ini to use setup.cfg and fixed all.
2016-12-12[tests] adapt testsKali Kaneko
2016-10-21[tests] specify basepython on root envVictor Shyba
code-check is running with py3 randomly on CI, this commit should pin it.
2016-09-30[test] add flake8 code check and generalize name of tox envdrebs
2016-09-09[test] add tagged perf job for gitlab-cidrebs
In order to configure performance tests to run in a specific machine we need to add a tagged job to .gitlab-ci.yml file. That job will only execute the perf tests, and then we can have runners that will only run those jobs.
2016-09-06[test] add tox env for perf testsdrebs
2016-09-02[tests] add pep8 in main tox run from within the gitlab scriptKali Kaneko
add coverage reports too. (hereby we swear not to write stupid tests just because it feels good to have an increased coverage metric). - Resolves: #8416
2016-08-22[test] Adds pytest-benchmark adapted to TwistedVictor Shyba
Adapted pytest-benchmark to Twisted as it's synchronous and added fixtures for benchmarking.
2016-08-15[test] allow for shell autocomplete for test namesdrebs
tox was configured to change to the testing/tests directory before executing pytest, by using tox's "changedir" configuration option. The reason why this was the case is that we wanted to discover tests inside the testing/tests directory only. The problem with that approach is that if we wanted to point to a specific test file, for example "tests/perf/test_sync.py", we would have to omit the "tests" part and write "tox perf/test_sync.py" because the argument would be understood as relative to the changed dir. That is not practical as doesn't allow to use the shell autocomplete, and is also not the only way to achieve what we want. Actually, pytest has a configuration option called "testpaths" where you can indicate where it should discover tests. This commit changes one approach by the other and allows to user shell autocomplete for easyness of testing during development.
2016-08-01[test] do not run pep8 tox env by defaultdrebs
2016-08-01[test] adds optional parallel env for local devVictor Shyba
2016-08-01[test] adds pep8 as a tox envVictor Shyba
"tox -e pep8" runs it standalone and "tox" includes the pep8 env.
2016-08-01[test] remove pip download cachedrebs
Recent versions of pip will ignore that option and use a cache anyway.
2016-08-01[test] use pip download cache for tests and docker imagedrebs
2016-08-01[test] remove pep8 from tox configdrebs
2016-08-01[test] add pytest initial setup for performance testsdrebs
2016-07-12add pep8/flake8 to toxKali Kaneko
2016-07-12[test] toxify testsdrebs
- move tests to root directory - split tests in different subdirectories - setup a small package with common test dependencies in /testing/test_soledad - add tox.ini that will: - install the test_soledad package and other test dependencies - install soledad common, client, server from the repository - run tests contianed in /testing/tests directory using pytest This commit also removes all oauth code from tests, as we have removed the u1db dependency (by importing it into the repo and naming it l2db) and don't neet oauth at all right now.