summaryrefslogtreecommitdiff
path: root/client/src
AgeCommit message (Collapse)Author
2016-12-12[refactor] simplify _cryptoVictor Shyba
After adding the streaming decrypt, some classes were doing almost the same thing. Unified them. Also fixed some module level variables to upper case and some class name to camel case.
2016-12-12[refactor] improve loggingVictor Shyba
Some exceptions were missing a proper description and client_side_db.py script wasn't capturing logs from Twisted.
2016-12-12[bug] make the semaphore cover all parsingVictor Shyba
Unfortunately, if a doc finishes decryption before the previous one we will still have an issue while inserting. This commits solves it by adding the parse and decrypt inside of the semaphore.
2016-12-12[feature] make _crypto stream on decryptionVictor Shyba
We are already doing this on encryption, now we can stream also from decryption. This unblocks the reactor and will be valuable for blobs-io.
2016-12-12[feature] delimit preamble from ciphertextVictor Shyba
We now encode preamble and ciphertext+hmac in two distinct payloads separated by a space. This allows metadata to be extracted and used before decoding the whole document. It also introduces a single packer for packing and unpacking of data instead of reads and writes. Downside: doc_id and rev are limited to 255 chars now.
2016-12-12[refactor] Hide IV, simplify some callsVictor Shyba
IV was being set during tests and this required some defensive coding to avoid IV being set in production. This commits makes the test use the generated IV and "hides" it using a read-only property to let it clear this should never happen. Also refactored out some parameters that are generated automatically to reduce some lines of code and enhance readability.
2016-12-12[docs] add docstrings for _cryptoVictor Shyba
Also explaining how we are using Twisted's consumer interfaces.
2016-12-12[refactor] remove dead parameters, improve commentsVictor Shyba
received docs makes no sense for a single request download, plus all its comments and docstrings. Also updated docstrings for other methods. The method that tests if sqlcipher is encrypted can return a db handle that can be used right away. If we ignore it and reopen we can end up with a lost open cursor.
2016-12-12[refactor] Remove dead codeVictor Shyba
Batching is now decided on server side, so the code can be simplified. Also, sync_db and other parameters were used to initialize encdecpool, which is no longer supported.
2016-12-12[bug] emit last sent doc eventVictor Shyba
Document sending happens after encryption, so the last sent document needs to be signalled after request end.
2016-12-12[bug] fix upload progressVictor Shyba
We need to emit zmq status during doc prepare, which is called during upload.
2016-12-12[style] fix pep8 and confsVictor Shyba
Fixes setup.cfg, adding current exclude rules, simplified tox.ini to use setup.cfg and fixed all.
2016-12-12[bug] include_deleted=True on syncVictor Shyba
Also refactored tests and code to stop relying on old parameters which included docs instead of get_doc calls.
2016-12-12[refactor] better naming for producer callingVictor Shyba
Giving the proper name to the function and arguments helps to make the producer wizardry less magic.
2016-12-12[style] improve comments for sync.pyVictor Shyba
2016-12-12[refactor] simplify content as a new lineVictor Shyba
Code was complex and raised a flag during review.
2016-12-12[refactor] improve readability of stream producerVictor Shyba
2016-12-12[style] add license headerVictor Shyba
2016-12-12[refactor] remove assert logic from fetch_protocolVictor Shyba
Asserts aren't a good solution for stream parsing, its cleaner to check and raise in place. Also, asserts can be ignored.
2016-12-12[feature] Adds back support to deprecated cryptoVictor Shyba
Will be removed when we have the proper tool to migrate data.
2016-12-12[feature] Adds deprecated crypto moduleVictor Shyba
This is supposed to be used only for temporary backwards compatibility, while we develop a proper migration tool.
2016-12-12[refactor] stop using a dict for syncersVictor Shyba
A dict was used to store references for the synchronizers based on a URL. This commit removes it as it doesnt make sense with current code.
2016-12-12[refactor] stop using leap.common.httpVictor Shyba
We aren't using leap.common.http implementation and we need specific features from original Twisted Web Agent. This commit implements it on HTTP Targer.
2016-12-12[bug] total number of documentsVictor Shyba
Parsing from metadata we can store the total of docs and handle it for the doc parser in order to be able to keep consistent events info.
2016-12-12[refactor] Issues from code reviewVictor Shyba
2016-12-12[refactor] more comments, less codeVictor Shyba
Some code were duplicated, got removed. Additional comments added for documenting such a critical and complex part as a protocol.
2016-12-12[refactor] DocStreamReceiver <- ReadBodyProtocolVictor Shyba
Both classes holds u1db error handling. Making DocStreamReceiver a subclass reduces the error handling to a single place thus removing duplicated code.
2016-12-12[bug] defer insertion to threads during downloadVictor Shyba
Insertion is synchronous and blocks the reactor. That's a temporary solution as we used to have on decpool.
2016-12-12[style] remove unused imports (pep8)Victor Shyba
2016-12-12[test] fix test and remove leftovers defer_encryptionVictor Shyba
2016-12-12[style] pep8Kali Kaneko
2016-12-12[refactor] remove legacy crypto implementationKali Kaneko
2016-12-12[tests] adapt testsKali Kaneko
2016-12-12[refactor] remove encdecpool, finallyKali Kaneko
2016-12-12[bug] fix wrong content parsingKali Kaneko
2016-12-12[refactor] add SoledadCrypto interfaceKali Kaneko
2016-12-12[refactor] adapt fetcher to decryptorKali Kaneko
2016-12-12[feature] blob encryptor / decryptorKali Kaneko
2016-12-12[refactor] remove encryption poolKali Kaneko
2016-12-12[feature] streaming crypto implementationKali Kaneko
2016-12-12[feature] stream content in a separate lineVictor Shyba
This allow different paths for raw data and metadata, avoiding unnecessary json parsing.
2016-12-12[bug] handle 400Victor Shyba
There was an if without an else on error handler that avoided handling errors that falled back current logic. Added a generic one to the tail so we dont miss it.
2016-12-12[feature] upload streamingVictor Shyba
1) enable HTTP 1.1 chunked upload on server 2) make the client sync.py generate a list of function calls instead of a list of full docs 3) disable encryption pool 4) make the doc encryption a list of function calls 5) create a twisted protocol for sending 6) make a producer that calls the doc generation as necessary
2016-12-12[feature] streaming download protocolVictor Shyba
This commit finishes reversion into u1db original streaming protocol for downloads.
2016-12-12[refactor] remove decpoolVictor Shyba
It's not being used
2016-12-12[bug] use an empty string to represent tumbstonesVictor Shyba
If a doc doesnt have a content it means it was deleted. Sync stream was unable to represent this state.
2016-12-12[test] commit isnt part of the backend apiVictor Shyba
Check if the backend provides a commit method before calling or we will break the tests with InMemoryDatabase
2016-12-12[feature] simple adaptation to let the client runVictor Shyba
Make the client parse a 2-line doc on sync download stream.
2016-12-12[bug] disable decpoolVictor Shyba
Temporary fix for server streaming
2016-12-12[feature] use transactions on syncVictor Shyba
We were using 1 transaction per doc, which is bad. Reference: http://stackoverflow.com/questions/1711631/improve-insert-per-second-performance-of-sqlite Code now uses 1 transaction for the whole sync.