Age | Commit message (Collapse) | Author | |
---|---|---|---|
2016-12-12 | [bug] fix upload progress | Victor Shyba | |
We need to emit zmq status during doc prepare, which is called during upload. | |||
2016-12-12 | [tests] migrate pytest to trial | Victor Shyba | |
test_deprecated_crypto was using pytest, which unfortunately doesnt work when mixed with trial. Migrated back. Also added norecursedirs option back, as it is necessary for parallel testing mode. | |||
2016-12-12 | [test] rename benchmark tests directory and tag | drebs | |
2016-12-12 | [test] add test for deprecated crypto format update | drebs | |
2016-12-12 | [test] move fixtures one level up | drebs | |
2016-12-12 | [test] use tags for selecting benchmark tests | drebs | |
2016-12-12 | [test] remove unneeded setting of environment variables | drebs | |
2016-12-12 | [style] fix pep8 and confs | Victor Shyba | |
Fixes setup.cfg, adding current exclude rules, simplified tox.ini to use setup.cfg and fixed all. | |||
2016-12-12 | [bug] include_deleted=True on sync | Victor Shyba | |
Also refactored tests and code to stop relying on old parameters which included docs instead of get_doc calls. | |||
2016-12-12 | [refactor] better naming for producer calling | Victor Shyba | |
Giving the proper name to the function and arguments helps to make the producer wizardry less magic. | |||
2016-12-12 | [pkg] add six and cryptography | Victor Shyba | |
Those are already present, but we are using the ones coming from our dependencies. Explicit is better than implicit. | |||
2016-12-12 | [tests] remove test_sync_deferred | Victor Shyba | |
Deferred encryption option is gone. | |||
2016-12-12 | [style] explicit unlimited request size | Victor Shyba | |
Request size on a stream can't be measured upfront and a limit doesn't make much sense. The real limit is user's Quota, to be implemented. | |||
2016-12-12 | [style] improve comments for sync.py | Victor Shyba | |
2016-12-12 | [refactor] simplify content as a new line | Victor Shyba | |
Code was complex and raised a flag during review. | |||
2016-12-12 | [refactor] improve readability of stream producer | Victor Shyba | |
2016-12-12 | [style] add license header | Victor Shyba | |
2016-12-12 | [refactor] remove assert logic from fetch_protocol | Victor Shyba | |
Asserts aren't a good solution for stream parsing, its cleaner to check and raise in place. Also, asserts can be ignored. | |||
2016-12-12 | [feature] Adds back support to deprecated crypto | Victor Shyba | |
Will be removed when we have the proper tool to migrate data. | |||
2016-12-12 | [feature] Adds deprecated crypto module | Victor Shyba | |
This is supposed to be used only for temporary backwards compatibility, while we develop a proper migration tool. | |||
2016-12-12 | [refactor] simplify server insert | Victor Shyba | |
Moved out magic numbers into a constant and simplified logic during doc upload. | |||
2016-12-12 | [tests] remove test_sync_very_large_files | Victor Shyba | |
We have benchmarks now to test sync limits and 100mb is too far from current needs. | |||
2016-12-12 | [tests] improve doc creation on benchmarks | Victor Shyba | |
If we create all at once we cant test higher loads because it will try to hold all in memory at the same time. Also, this code is smaller and more readable. | |||
2016-12-12 | [refactor] stop using a dict for syncers | Victor Shyba | |
A dict was used to store references for the synchronizers based on a URL. This commit removes it as it doesnt make sense with current code. | |||
2016-12-12 | [tests] fall back to TestCase classes | Victor Shyba | |
This test was using pytest, but it was making other trial based tests fail. I couldn't figure out why, but falling back to TestCase solved it. | |||
2016-12-12 | [refactor] stop using leap.common.http | Victor Shyba | |
We aren't using leap.common.http implementation and we need specific features from original Twisted Web Agent. This commit implements it on HTTP Targer. | |||
2016-12-12 | [bug] total number of documents | Victor Shyba | |
Parsing from metadata we can store the total of docs and handle it for the doc parser in order to be able to keep consistent events info. | |||
2016-12-12 | [refactor] Issues from code review | Victor Shyba | |
2016-12-12 | [refactor] more comments, less code | Victor Shyba | |
Some code were duplicated, got removed. Additional comments added for documenting such a critical and complex part as a protocol. | |||
2016-12-12 | [refactor] DocStreamReceiver <- ReadBodyProtocol | Victor Shyba | |
Both classes holds u1db error handling. Making DocStreamReceiver a subclass reduces the error handling to a single place thus removing duplicated code. | |||
2016-12-12 | [bug] defer insertion to threads during download | Victor Shyba | |
Insertion is synchronous and blocks the reactor. That's a temporary solution as we used to have on decpool. | |||
2016-12-12 | [style] remove unused imports (pep8) | Victor Shyba | |
2016-12-12 | [test] fix test and remove leftovers defer_encryption | Victor Shyba | |
2016-12-12 | [bug] fix import | Kali Kaneko | |
2016-12-12 | [bug] remove print debug statements | Kali Kaneko | |
2016-12-12 | [style] pep8 | Kali Kaneko | |
2016-12-12 | [bug] fix bad merge in imports block | Kali Kaneko | |
2016-12-12 | [refactor] remove lingering refs to syncdb | Kali Kaneko | |
2016-12-12 | [refactor] remove legacy crypto implementation | Kali Kaneko | |
2016-12-12 | [tests] adapt tests | Kali Kaneko | |
2016-12-12 | [refactor] remove encdecpool, finally | Kali Kaneko | |
2016-12-12 | [bug] fix wrong content parsing | Kali Kaneko | |
2016-12-12 | [refactor] add SoledadCrypto interface | Kali Kaneko | |
2016-12-12 | [refactor] adapt fetcher to decryptor | Kali Kaneko | |
2016-12-12 | [feature] blob encryptor / decryptor | Kali Kaneko | |
2016-12-12 | [refactor] remove encryption pool | Kali Kaneko | |
2016-12-12 | [feature] streaming crypto implementation | Kali Kaneko | |
2016-12-12 | [feature] batch based on payload size | Victor Shyba | |
batch is slower than usual insert for a single doc, so, if a document exceeds the buffer, commit the batch (if any) and put the huge load by traditional insert. refactor coming. | |||
2016-12-12 | [feature] fix and enable batch | Victor Shyba | |
Batching is now decided by server, this commits enables it. | |||
2016-12-12 | [feature] stream content in a separate line | Victor Shyba | |
This allow different paths for raw data and metadata, avoiding unnecessary json parsing. |