summaryrefslogtreecommitdiff
path: root/scripts/benchmark
diff options
context:
space:
mode:
authordrebs <drebs@riseup.net>2017-09-09 09:26:14 -0300
committerVictor Shyba <victor1984@riseup.net>2017-09-11 16:24:30 -0300
commit8e6cddf794c4b0e44bec6daa4363f2926e238f0a (patch)
treeab34cfd50823764e9e38a6a45be99faffef82ddb /scripts/benchmark
parentf6a409a711df032507cbe3dd1f1129c1b7e19c95 (diff)
[test] add --subdir option and blacklist
Use subdir path to select/deselect benchmark tests. Because of the way pytest and pytest benchamrk select/deselect tests, it makes more sense for this test suite to use a --subdir option to select only a subset of tests and a blacklist of subdirectories that should not be run unless explicitelly asked to. This commit adds a --subdir option, that will deselect all tests no in ./tests/<subdir> when the option is passed to pytest. Also, a blacklist is added so, unless explicitelly added as a --subdir, tests in blacklisted subdirectories will not be run. The reasons for this modification are: - pytest-benchmarks selects/deselects tests based on the type of their fixtures. This means that fixtures have to be instantiated previous to test selection/deselection, something that may cause side effects that we want to avoid. - the usual test suite is run for all benchmarks, while some tests should only be run in specific situations (as is the case of benchmarks and responsiveness tests). It is saner to implement subdir selection than marking all tests with pytest marks and using them to select/deselect.
Diffstat (limited to 'scripts/benchmark')
0 files changed, 0 insertions, 0 deletions