summaryrefslogtreecommitdiff
path: root/testing
AgeCommit message (Collapse)Author
2016-09-30[test] use pytest tmpdir fixture in all testsdrebs
Tests that were imported from u1db or created on top of that structure were leaving temporary directories behind. This could cause problems in test servers, either by filling the partition or by extrapolating the maximum amount of files in a directory. This commit replaces all usages of temporary directories in the old test structure by pytest tmpdir fixture, which properly cares for removing temporary directories.
2016-09-09[test] add tagged perf job for gitlab-cidrebs
In order to configure performance tests to run in a specific machine we need to add a tagged job to .gitlab-ci.yml file. That job will only execute the perf tests, and then we can have runners that will only run those jobs.
2016-09-06[test] add tox env for perf testsdrebs
2016-09-02[tests] add pep8 in main tox run from within the gitlab scriptKali Kaneko
add coverage reports too. (hereby we swear not to write stupid tests just because it feels good to have an increased coverage metric). - Resolves: #8416
2016-08-29[test] randomize payloadVictor Shyba
We were using 'x'*size as payload, but on real usage the payload will be random. This commit randomizes the payload using a predefined seed, so the random payload will be the same across benchmarks. Using random payloads also improves accuracy of compression or encoding impacts and we will be evaluating those changes for resouce usage issues. Also note that base64 is used on payload. That was needed for utf8 safety, but overhead was removed to leave payloads as defined by benchmarks. Base64 was chosen also due its popular usage on MIME encoding, which is used on mail attachments (our current scenario).
2016-08-25[test] stop trying to hit me and hit meVictor Shyba
TestSyncEncrypterPool.test_encrypt_doc_and_get_it_back was trying to do an operation and asserting the number of attempts. This test is about putting a doc on encrypter pool and getting it encrypted. If we dont wait for the encryption operation to succeed, then complex trial-and-error happens, but if we just ask twisted to wait for one operation before going to the other, this is not needed. -- Resolves: #8398
2016-08-23[test] test_instance -> test_initializationVictor Shyba
This isnt a test, but a benchmark. Initialization sounds more like an operation while instance is just something.
2016-08-23[test] point issue #7370 as reason for low valuesVictor Shyba
We are using lower values on test_encdecpool due high memory usage, described in #7370. Added a comment to explain it.
2016-08-22[test] refactor load_up for readabilityVictor Shyba
defer parameter wasnt clear
2016-08-22[tests] move doc creation to setupVictor Shyba
Otherwise it will add unrelated overhead to results.
2016-08-22[tests] remove unused new=False logicVictor Shyba
2016-08-22[test] make txbench ignore kwargs for readabilityVictor Shyba
They arent used so far and using empty dicts to make them work is ugly. Removing it leaves the return function on setup code clean and readable.
2016-08-22[test] calibrate encdecpool bench for memoryVictor Shyba
1000 docs at 100k~500k are exploding memory (4Gb+4Gb swap). Changed for 100 docs in order to be able to get measures on higher loads. Now its 10k, 100k and 500k
2016-08-22[test] adds tests for raw encryptionVictor Shyba
Hypothesis: raw vs doc Added the same sizes set (10k, 100k, 500k, 1M, 10M, 50M) as the document crypto test, so we can compare how close to raw the higher level operation is.
2016-08-22[test] adds test for SoledadCryptoVictor Shyba
10k, 100k, 500k, 1m, 10m and 50m for encryption and decryption of a whole document.
2016-08-22[test] adds encdecpool testsVictor Shyba
Most of them are commented as memory usage is going out of control for now.
2016-08-22[test] test soledad instatiation timeVictor Shyba
It has a heavy scrypt hashing processing with room for improvement.
2016-08-22[test] sync without changesVictor Shyba
Syncing without any changes was reported as slow. This benchmark will help measure it.
2016-08-22[test] remove fixed default uuidVictor Shyba
Use a new one to avoid reusing the same database.
2016-08-22[test] make all nested fixture function-scopedVictor Shyba
function is the default scope, so there is no need to pass this parameter. Previously, one of the scopes was 'module', but it is a nested function that fires on demand, so it should clean up itself from test to test in order to avoid conflict while putting.
2016-08-22[test] adds sqlcipher synchronous testsVictor Shyba
2016-08-22[test] adds sqlcipher create testsVictor Shyba
Creating 20/500k, 100/100k and 1000/10k.
2016-08-22[test] use a nested func to simplify scenariosVictor Shyba
If we have many scenarios (like 20/500k, 100/100k, 1000,10k) then making a nested function to generate tests based on scenario parameters simplifies the code a lot.
2016-08-22[test] Adds pytest-benchmark adapted to TwistedVictor Shyba
Adapted pytest-benchmark to Twisted as it's synchronous and added fixtures for benchmarking.
2016-08-17[tests] Adapt tests for ensure_ddocs deathVictor Shyba
2016-08-15[test] allow for shell autocomplete for test namesdrebs
tox was configured to change to the testing/tests directory before executing pytest, by using tox's "changedir" configuration option. The reason why this was the case is that we wanted to discover tests inside the testing/tests directory only. The problem with that approach is that if we wanted to point to a specific test file, for example "tests/perf/test_sync.py", we would have to omit the "tests" part and write "tox perf/test_sync.py" because the argument would be understood as relative to the changed dir. That is not practical as doesn't allow to use the shell autocomplete, and is also not the only way to achieve what we want. Actually, pytest has a configuration option called "testpaths" where you can indicate where it should discover tests. This commit changes one approach by the other and allows to user shell autocomplete for easyness of testing during development.
2016-08-02[test] avoid race condition on test_processing_orderVictor Shyba
test_processing_order aims to check that unordered docs wont be processed, but if we let the pool start and advance Twisted LoopingCall clock right before calling the processing method manually, the process method will run concurrently and cause a race condition issue.
2016-08-01[test] do not run pep8 tox env by defaultdrebs
2016-08-01[test] adds optional parallel env for local devVictor Shyba
2016-08-01[test] adds pep8 as a tox envVictor Shyba
"tox -e pep8" runs it standalone and "tox" includes the pep8 env.
2016-08-01[test] remove pip download cachedrebs
Recent versions of pip will ignore that option and use a cache anyway.
2016-08-01[test] remove duplicated function declarationVictor Shyba
`pytest_addoption` was declared twice making the second declaration replace the first, thus removing couch url parameter.
2016-08-01[test] allow passing number of docs on command line on perf testsdrebs
2016-08-01[test] use pip download cache for tests and docker imagedrebs
2016-08-01[test] remove pep8 from tox configdrebs
2016-08-01[test] avoid perf tests to be run on normal tox callsdrebs
Currently the perf tests use pytest-twisted plugin, and this has some implications in the old tests adapted from u1db that now use trial classes. Because of that, we exclude perf tests from usual tox calls, but you can still run them by explicitelly calling `tox perf`.
2016-08-01[test] allow custom couch url for perf testsdrebs
2016-08-01[test] allow custom couch url for couch testsdrebs
2016-08-01[test] add some payload to perf sync testsdrebs
2016-08-01[test] use pytest fixture scopes to provide per module soledad server for ↵drebs
perf tests
2016-08-01[test] add pytest initial setup for performance testsdrebs
2016-08-01[test] remove traces of design docs from couch testsdrebs
2016-08-01[feat] remove usage of design documents in couchdrebs
Design documents are slow and we already have alternatives to all uses we used to make of them, so this commit completelly removes all usage of design documents.
2016-07-25[test] adapt couch tests to use new generation/transaction storage schemedrebs
2016-07-12add pep8/flake8 to toxKali Kaneko
2016-07-12[test] refactor test filesdrebs
2016-07-12[refactor] make u1db connection pool args explicitdrebs
2016-07-12[bug] properly trap db errors and close resourcesdrebs
SQLCipher database access errors can raise Soledad exceptions. Database access and multithreading resources are allocated in different places, so we have to be careful to close all multithreading mechanismis in case of database access errors. If we don't, zombie threads may haunt the reactor. This commit adds SQLCipher exception trapping and Soledad exception raising for database access errors, while properly shutting down multithreading resources.
2016-07-12[test] properly close dbpool on async testdrebs
2016-07-12[test] toxify testsdrebs
- move tests to root directory - split tests in different subdirectories - setup a small package with common test dependencies in /testing/test_soledad - add tox.ini that will: - install the test_soledad package and other test dependencies - install soledad common, client, server from the repository - run tests contianed in /testing/tests directory using pytest This commit also removes all oauth code from tests, as we have removed the u1db dependency (by importing it into the repo and naming it l2db) and don't neet oauth at all right now.