summaryrefslogtreecommitdiff
path: root/client/src
AgeCommit message (Collapse)Author
2015-08-12[bug] changes multiprocessing.Queue to Twisted'sVictor Shyba
multiprocessing.Queue is suitable for process communication, but its not the ideal for a reactor model. This commit changes it to DeferredQueue, where consumers and producers doesnt block and Twisted can handle them better.
2015-08-12[bug] raise u1db errors based on http responsedrebs
When we started to use the twisted http agent, we forgot to intercept http response and raise the appropriate u1db errors based on the response status code and messages. This commit implements that by redefining the http body reader used by the http agent.
2015-08-12[bug] Encdecpool won't explode if stopped twiceBruno Wagner
The encryption pool could be stopped twice and would break on the second attempt because it deletes the encryption queue variable. Added a condition to make sure it only deletes the encryption queue if it exists, making it more idempotent
2015-08-12[bug] Sync Locking based on class + db file pathVictor Shyba
Change locking to be class based and each lock generated by db file path.
2015-08-12[bug] fixes concurrent sync and their testsVictor Shyba
Changes threading.lock to DeferredLock and checks syncing attribute by looking into the lock state. Also, applies more of startTwistedServer on tests that relies on HTTP/1.1. Fixes mock for events
2015-08-12[bug] close http_target when soledad closesVictor Shyba
Soledad has a close method that wasn't calling http_target close. The reference to sync exchange was being deleted without proper closing of underlying resources.
2015-08-11[feat] emit a dict instead of a strIvan Alejandro
2015-08-05[tests] Fixed remaining parse testsBruno Wagner
All the response parse tests are passing now, response with no entries was broken because it wasn't being treated and the others were broken because of calls that no longer existed
2015-08-05[tests] Created a setup for the http target testsBruno Wagner
Created a setup for the http target tests Fixed two tests relying on http target that were outdated Fixed a call for an exception that doesn't exist, it won't break anymore if it gets to that exception
2015-08-05[style] Fixed PEP8 warningBruno Wagner
Line break before binary operator breaks PEP8, fixed that in the client api.py
2015-08-04[bug] Soledad._crypto has to be initialized earlierKali Kaneko
before sqlcipher backend, or the attribute is not found. this is a leftover of the recent refactor
2015-08-04[style] pep8 cleanupKali Kaneko
2015-08-04[test] add test for many documents decryptiondrebs
2015-08-04[feat] add running state method to enc/dec poolsdrebs
2015-08-04[refactor] remove circular dependency in SoledadCryptoRuben Pollan
SoledadCrypto had Soledad as parameter to be able to use SoledadSecrets. SoledadSecrets had SoledadCrypto as parameter to use *crypt_sym. This commit removes this circular dependency passing directly the secret that SoledadCrypto cares about to the constructor and removing the *crypt_sym methods from SoledadCrypto. - Resolves: #7338
2015-08-04[refactor] standardize start/stop of enc/dec poolsdrebs
* change close method name to stop * add start/stop methods to both enc/dec clases * remove any delayed calls on pool shutdown
2015-08-04[bug] add log msg to sent docs on syncdrebs
2015-07-29[style] Fixed pep8 warning on api.pyBruno Wagner
The bolean operator must come before a line break, not after according to pep8
2015-07-27[bug] avoid double decryption of documentsdrebs
Because of how the incoming document queue is implemented, it could be the case that a document was sent to async decryption queue more than once. This commit creates a list of documents to be decrypted, so we avoid sending the same document to the queue more than once.
2015-07-27[feat] add log message for document encryptiondrebs
2015-07-27[bug] fix order of incoming document eventsdrebs
The incoming documents events are meant to be used by a progress bar for soledad sync, yet to be implemented. When deferred decryption was used, the events were sent out of order, depending on the order of arrival of the documents. This commit changes it so that the content of the emited events are in order, so it is meaningful for the implementation of a progress bar. Note that even after documents are received from the server, they will still be decrypted asynchronously, so another signal could be implemented to signal for the waiting of the decryption of incoming documents.
2015-07-27[bug] remove mac from secrets filedrebs
This is how a secret was stored in the secrets json file: * each secret is symmetrically encrypted amd MACed with keys derived from the user's passphrase. * the encrypted secrets dictionary is then MACed with another key derived * from the user's passphrase. * each key is derived using scrypt and a unique random salt. There are disadvantages to this approach: * repeating scrypt many times is a waste of time. * an attacker could crack whichever has weaker parameters, if they get out of sync. * if an attacker can modify the secret in a way it is good to decrypt the database, then she can also modify the MAC. The solution for this is: * completelly eliminate the MAC from the storage secrets file. * attempt to decrypt the database with whatever is got from the decryption of the secret. If that is wrong, report an error. Closes #6980.
2015-07-26[bug] fix syntax errorKali Kaneko
resulting from the previous pep8 cleanup
2015-07-24[style] fixed typo in custom docid fileBruno Wagner
2015-07-24[style] fixed do not assign a lambda in soledad clientBruno Wagner
2015-07-24[style] fixed soledad client pep8 warningsBruno Wagner
2015-07-23[style] pep8 cleanupKali Kaneko
to make all CIs happy :)
2015-07-23[bug] move sync db and encpool creation to apidrebs
Deferred encryption was disabled because the soledad u1db wrapper for adbapi did not correctly udated the parameter that controls it. Also, it did not contain the encrypter pool. This commit moves the sync db and encrypt pool to the main api, so they can be passed to the wrapper and deferred encryption can work.
2015-07-22[refactor] use a deferred to signal when SyncDecriptionPool has finishedRuben Pollan
It makes the code simpler and clearer to use a deferred instead of having to pull on 'has_finished'. - Related: #7234
2015-06-30[feature] add close() method to client http targetdrebs
HTTP client cached connections will hang around in the reactor if they are not properly cleaned up, and might raise a "reactor unclean" message on shutdown. This commit adds a close() method to the client http target that will cleanup those connections.
2015-06-15[style] minor style fixes for correctionKali Kaneko
after suggestions in the review
2015-06-15[refactor] use twisted logging facility to log failureKali Kaneko
2015-06-15[refactor] move collect_plugins to leap.commonKali Kaneko
2015-06-15[feature] filter documents before passing them to the pluginKali Kaneko
2015-06-15[feature] use raw operation, returns no resultKali Kaneko
2015-06-15[feature] add post-sync hooks using twisted pluginsKali Kaneko
implementing a generic plugin interface to allow other modules to react to soledad syncs, receiving a list of document ids that they've subscribed to. - Resolves: #6996 - Releases: 0.7.1
2015-06-15[bug] allow reuse of decr poolKali Kaneko
2015-06-03[feature] increase the async encryption perioddrebs
When async decrypting, we want to finish as fast as possible. When encrypting, though, we don't have such a rush. With an encryption loop period of 2 seconds, we're able to encrypt 30 documents in one minute (the current bitmask client sync period), which is meaningful: should moderatelly use the processor while not syncing and relief from some work when actually syncing.
2015-06-03[bug] do not block when getting doc for async encdrebs
Previous to this change, the actual encryption method used to run on its own thread. When the close method was called from another thread, the queue could be deleted after the encryption method loop had started, but before the queue was checked for new items. By removing that thread and moving the encryption loop to the reactor, that race condition should disappear. Closes: #7088.
2015-06-03[bug] use exception from correct moduledrebs
Queue exceptions are not in multiprocessing.Queue module, but in plain Queue instead.
2015-06-03[bug] use correct sync enc pool queue attributedrebs
2015-05-27[feat] adapt to new events api on commonIvan Alejandro
- Related: #6359
2015-05-27[refactor] move the twisted http code to leap.commonRuben Pollan
2015-05-26[bug] dictionary cant be modified during iterationVictor Shyba
I tested that code and this cant happen. We need to iterate keys and then ask 'del'. The previous method raised: RuntimeError: dictionary changed size during iteration
2015-05-26[bug] Empty comes from QueueVictor Shyba
When handling this exception Python got lost because the import was incorrect. Queue.Empty comes from Queue, not from multiprocessing.Queue
2015-05-25[bug] remove client syncer call to close methoddrebs
2015-05-25[feature] add pool of http/https connectionsdrebs
Instead of opening one TCP connection for each HTTP request, we want to reuse connections. Also, we need to be able to verify SSL certificates. This commit implements both features in the twisted http client sync.
2015-05-22[refactor] remove inline enc/dec from client pooldrebs
The whole idea of the encrypter/decrypter pool is to be able to use multiple cores to allow parallel encryption/decryption. Previous to this commit, the encryptor/decryptor pools could be configured to not use workers and instead do encryption/decryption inline. That was meant for testing purposes and defeated the purpose of the pools. This commit removes the possibility of inline encrypting/decrypting when using the pools. It also refactors the enc/dec pool code so any failures while using the pool are correctly grabbed and raised to the top of the sync deferred chain.
2015-05-21[refactor] remove unneeded proxy for insert_doc_cbdrebs
When we initialized the async decrypter pool in the target's init method we needed a proxy to ensure we could update the insert doc callback with the correct method later on. Now we initialize the decrypter only when we need it, so we don't need this proxy anymore. This commit removes the unneeded proxy.
2015-05-20[bug] ensure async decryption failures are loggeddrebs
We have to make sure any failures in asynchronous decryption code is grabbed and properly transmitted up the deferred chain so it can be logged. This commit adds errbacks in the decryption pool that grab any failure and a check on the http target the failure if that is the case.