soledad.git - [soledad]

Age	Commit message (Collapse)	Author
2016-09-22	[style] standardize log messages	drebs

2016-09-22	[feat] centralize logging and use twisted.logger by default	drebs

2016-09-09	[test] add tagged perf job for gitlab-ci	drebs
	In order to configure performance tests to run in a specific machine we need to add a tagged job to .gitlab-ci.yml file. That job will only execute the perf tests, and then we can have runners that will only run those jobs.
2016-09-06	[test] add tox env for perf tests	drebs

2016-09-06	[style] pep8	Kali Kaneko

2016-09-02	[tests] add pep8 in main tox run from within the gitlab script	Kali Kaneko
	add coverage reports too. (hereby we swear not to write stupid tests just because it feels good to have an increased coverage metric). - Resolves: #8416
2016-09-02	[bug] fail gracefully if dbsyncer has not been initialized	Kali Kaneko
	this is needed for some mail tests.
2016-08-29	[pkg] remove deprecated requirements-testing.pip	drebs

2016-08-29	[pkg] remove leftover simplejson imports from l2db	drebs

2016-08-29	[test] randomize payload	Victor Shyba
	We were using 'x'*size as payload, but on real usage the payload will be random. This commit randomizes the payload using a predefined seed, so the random payload will be the same across benchmarks. Using random payloads also improves accuracy of compression or encoding impacts and we will be evaluating those changes for resouce usage issues. Also note that base64 is used on payload. That was needed for utf8 safety, but overhead was removed to leave payloads as defined by benchmarks. Base64 was chosen also due its popular usage on MIME encoding, which is used on mail attachments (our current scenario).
2016-08-25	[test] stop trying to hit me and hit me	Victor Shyba
	TestSyncEncrypterPool.test_encrypt_doc_and_get_it_back was trying to do an operation and asserting the number of attempts. This test is about putting a doc on encrypter pool and getting it encrypted. If we dont wait for the encryption operation to succeed, then complex trial-and-error happens, but if we just ask twisted to wait for one operation before going to the other, this is not needed. -- Resolves: #8398
2016-08-23	[test] avoid failing on interrupted couch schema migrations	drebs

2016-08-23	[pkg] improve logging of couch schema migration script	drebs

2016-08-23	[pkg] log errors and continue with next db in couch schema migration script	drebs

2016-08-23	[pkg] log any errors in couch schema migration script	drebs

2016-08-23	[pkg] ignore existing correct gen docs in couch schema migrate script	drebs

2016-08-23	[pkg] improve log message for skipped dbs on couch schema migration script	drebs

2016-08-23	[pkg] fail gracefully for missing design doc on couch schema migration script	drebs

2016-08-23	[pkg] move config doc as last action of couch schema migration script	drebs
	If the moving of the config document is the last action of the couch schema migration script, then we can test for successful migration of a certain db by checking if the config document was already moved. This commit just changes the order of migration actions to enforce this situation.
2016-08-23	[pkg] add leftovers deletion to couch scehma migration script	drebs
	Previous versions of the couchdb schema used documents "u1db_sync_log" and "u1db_sync_state" to store sync metadata. At some point this was changed, but the documents might have stayed as leftovers. This commit adds the deletion of such documents to the migration script.
2016-08-23	[pkg] add --pdb option to migration script	drebs

2016-08-23	[test] test_instance -> test_initialization	Victor Shyba
	This isnt a test, but a benchmark. Initialization sounds more like an operation while instance is just something.
2016-08-23	[doc] improve SQLITE_MAX_VARIABLE_NUMBER comments	Victor Shyba

2016-08-23	[test] point issue #7370 as reason for low values	Victor Shyba
	We are using lower values on test_encdecpool due high memory usage, described in #7370. Added a comment to explain it.
2016-08-22	[test] refactor load_up for readability	Victor Shyba
	defer parameter wasnt clear
2016-08-22	[tests] move doc creation to setup	Victor Shyba
	Otherwise it will add unrelated overhead to results.
2016-08-22	[tests] remove unused new=False logic	Victor Shyba

2016-08-22	[test] make txbench ignore kwargs for readability	Victor Shyba
	They arent used so far and using empty dicts to make them work is ugly. Removing it leaves the return function on setup code clean and readable.
2016-08-22	[test] calibrate encdecpool bench for memory	Victor Shyba
	1000 docs at 100k~500k are exploding memory (4Gb+4Gb swap). Changed for 100 docs in order to be able to get measures on higher loads. Now its 10k, 100k and 500k
2016-08-22	[test] adds tests for raw encryption	Victor Shyba
	Hypothesis: raw vs doc Added the same sizes set (10k, 100k, 500k, 1M, 10M, 50M) as the document crypto test, so we can compare how close to raw the higher level operation is.
2016-08-22	[test] adds test for SoledadCrypto	Victor Shyba
	10k, 100k, 500k, 1m, 10m and 50m for encryption and decryption of a whole document.
2016-08-22	[bug] limit pool comnsumption to 900 docs	Victor Shyba
	This was discovered during load tests: Trying to process more than 999 docs triggers an error on SQLite due a select query not supporting 999 values to query.
2016-08-22	[test] adds encdecpool tests	Victor Shyba
	Most of them are commented as memory usage is going out of control for now.
2016-08-22	[test] test soledad instatiation time	Victor Shyba
	It has a heavy scrypt hashing processing with room for improvement.
2016-08-22	[test] sync without changes	Victor Shyba
	Syncing without any changes was reported as slow. This benchmark will help measure it.
2016-08-22	[test] remove fixed default uuid	Victor Shyba
	Use a new one to avoid reusing the same database.
2016-08-22	[test] make all nested fixture function-scoped	Victor Shyba
	function is the default scope, so there is no need to pass this parameter. Previously, one of the scopes was 'module', but it is a nested function that fires on demand, so it should clean up itself from test to test in order to avoid conflict while putting.
2016-08-22	[test] adds sqlcipher synchronous tests	Victor Shyba

2016-08-22	[test] adds sqlcipher create tests	Victor Shyba
	Creating 20/500k, 100/100k and 1000/10k.
2016-08-22	[test] use a nested func to simplify scenarios	Victor Shyba
	If we have many scenarios (like 20/500k, 100/100k, 1000,10k) then making a nested function to generate tests based on scenario parameters simplifies the code a lot.
2016-08-22	[test] Adds pytest-benchmark adapted to Twisted	Victor Shyba
	Adapted pytest-benchmark to Twisted as it's synchronous and added fixtures for benchmarking.
2016-08-17	[tests] Adapt tests for ensure_ddocs death	Victor Shyba

2016-08-17	[bug] remove misleading ensure_ddoc	Victor Shyba
	ensure_ddoc doesnt make sense anymore as we dont have any ddoc other than _security, which has its own method for setting. 'ensure_security' is explicit and is set internally when user is creating a database, otherwise it will be False as it's only used during creation. This isn't exposed externally (of couch module) to avoid confusion. This confusion was making create-user-db fail to create a security ddoc as it wasn't passing ensure_ddocs=True. -- Resolves: #8388
2016-08-15	[test] allow for shell autocomplete for test names	drebs
	tox was configured to change to the testing/tests directory before executing pytest, by using tox's "changedir" configuration option. The reason why this was the case is that we wanted to discover tests inside the testing/tests directory only. The problem with that approach is that if we wanted to point to a specific test file, for example "tests/perf/test_sync.py", we would have to omit the "tests" part and write "tox perf/test_sync.py" because the argument would be understood as relative to the changed dir. That is not practical as doesn't allow to use the shell autocomplete, and is also not the only way to achieve what we want. Actually, pytest has a configuration option called "testpaths" where you can indicate where it should discover tests. This commit changes one approach by the other and allows to user shell autocomplete for easyness of testing during development.
2016-08-09	[test] use "leapcode" in docker image name	drebs
	"leapcode" is the LEAP docker hub organisation varac could squat (https://hub.docker.com/r/leap/ was already taken).
2016-08-09	[feat] log to syslog (#8286)	drebs

2016-08-08	[pkg] remove version pinning for couchdb and beaker	drebs
	We will not maintain support for older versions of debian as that introduces some unneeded complexity for now. Also, the version pinned for couchdb python library has a bug that makes some requests slow. Because of those, we remove the pinning for now.
2016-08-05	[bug] create gen document after saving the actual document in couch	drebs
	If we create the gen document before saving the actual document in couch, we may run into problems if more than one client is syncing and trying to save documents with the same id at the same time. By moving the gen document creation to after the actual document save in couch, we rely on couch/u1db resolution of conflicts before actually allocating a new generation, and the problem above doesn't occur.
2016-08-03	[pkg] support netrc couch access in migrate script	drebs

2016-08-02	[test] avoid race condition on test_processing_order	Victor Shyba
	test_processing_order aims to check that unordered docs wont be processed, but if we let the pool start and advance Twisted LoopingCall clock right before calling the processing method manually, the process method will run concurrently and cause a race condition issue.