Age | Commit message (Collapse) | Author |
|
Use subdir path to select/deselect benchmark tests.
Because of the way pytest and pytest benchamrk select/deselect tests, it
makes more sense for this test suite to use a --subdir option to select
only a subset of tests and a blacklist of subdirectories that should not
be run unless explicitelly asked to.
This commit adds a --subdir option, that will deselect all tests no in
./tests/<subdir> when the option is passed to pytest. Also, a blacklist
is added so, unless explicitelly added as a --subdir, tests in
blacklisted subdirectories will not be run.
The reasons for this modification are:
- pytest-benchmarks selects/deselects tests based on the type of their
fixtures. This means that fixtures have to be instantiated previous to
test selection/deselection, something that may cause side effects that
we want to avoid.
- the usual test suite is run for all benchmarks, while some tests
should only be run in specific situations (as is the case of
benchmarks and responsiveness tests). It is saner to implement subdir
selection than marking all tests with pytest marks and using them to
select/deselect.
|
|
|
|
|
|
|
|
|
|
-- Related: #8867
|
|
|
|
|
|
|
|
|
|
|
|
We noticed that instrumentation added for watching resources has an
impact in time statistics (i.e. it increases average and stddev). This
commit makes the benchmark tests run twice: once for measuring time and
a second time for measuring resources.
|
|
|
|
|
|
|
|
|
|
|
|
- use subprocess.check_call() to ensure any errors during twistd startup
will properly show up on test reports.
- use SIGTERM instead of SIGKILL to gracefully terminate twistd.
|
|
We were previously not using an empty local db for download benchmark
tests, so there was actually nothing to sync. This commit fixes that by
adding a way to force an empty local db on soledad client instantiation.
|
|
|
|
Conflicts:
server/src/leap/soledad/server/_wsgi.py
server/src/leap/soledad/server/entrypoint.py
server/src/leap/soledad/server/resource.py
testing/tests/server/test__resource.py
|
|
|
|
|
|
Tests that were imported from u1db or created on top of that structure
were leaving temporary directories behind. This could cause problems in
test servers, either by filling the partition or by extrapolating the
maximum amount of files in a directory.
This commit replaces all usages of temporary directories in the old test
structure by pytest tmpdir fixture, which properly cares for removing
temporary directories.
|
|
|