summaryrefslogtreecommitdiff
path: root/client
AgeCommit message (Collapse)Author
2015-12-03[feat] prepare server to handle batchesVictor Shyba
Created two methods on the backend to start and finish a batch. A dict of callbacks is available to defer actions for the last document, allowing temporary (changing often) metadata to be recorded only once. Using those methods we will also be able to put all docs in one go on the CouchDatabase implementation, but that is another step.
2015-12-03[feat] Adds support to batching limited by sizeVictor Shyba
u1db provides batching by default. Current Soledad HTTPS Sync Target was stuck at 1 doc per request. This commit adds batching capability, limiting the size to a predefined value. Default limit size: 500kB
2015-12-01[style] fix pep8Victor Shyba
2015-12-01[bug] concurrency bug while querying and insertingVictor Shyba
This line was missing an yield and without it we end up inserting a document that is being retrieved and bad things happen. This is the core fix from yesterday debugging session. During sequential syncs the pool was inserting and querying at the same time and sometimes repeating or failing to delete documents.
2015-12-01[bug] fire callback after reseting instance varsVictor Shyba
If we reset the vars after firing the finish callback, other thread can pick up a dirty state on due concurrency.
2015-11-19[bug] fix indirect dependency on couchdb for clientKali Kaneko
by subclassing the MissingDesignDocError, we don't have to import the soledad.common.couch submodule into the soledad.client.sync - Resolves: #7626
2015-11-03[refactor] move couch errors and expose methodsVictor Shyba
errors.py was holding a few specific CouchDB errors, now moved into couch.errors module. Also, some of CouchDatabase methods were declared as private, but external classes needs them.
2015-10-05[bug] handle MissingDesignDocError after get_sync_infoVictor Shyba
MissingDesignDocError raised on get_sync_info due to a missing design document will be handled by the server during sync. Ensure is now False by default, and thus database creation can deliver an empty one that will be ensured during sync, following the ensure parameter.
2015-10-02[bug] increase http request timeout time to 90sKali Kaneko
this is a workaroud to reduce the chances of failed sync due to timeouts. this should be properly tackled by: 1. implementing proper cancellable for the sync operation. 2. implementing a retry count at the level of a single request, handled internally by soledad. in this way we can remove the retries logic from the soledadbootstrapper in the bitmask client. - Related: #7382
2015-10-02[bug] do not signal sync completion if failedKali Kaneko
- Related: #7503
2015-09-28[feat] handle DatabaseDoesNotExist during syncVictor Shyba
This error raises while getting info on target (or server) replica. On previous implementation there was nothing to do here, but now that we have db creation in place this error should be handled just like u1db original implementation. The reason is that db creation occurs during sync exchange, but before this we try to ask for info and the code that checks for info raises an error that will be used to signal the client that a database will be created and that it must save the replica uid returned by server, after sync exchange (where we send/fetch documents).
2015-09-21[pkg] fold in changes0.7.3Ivan Alejandro
2015-09-21[bug] set the received active secret before saving local fileKali Kaneko
- bug: we were dumping the received secrets locally to disk *before* setting the received property for the active secret, and therefore the 'active_secret' was always marked as null. - refactor common code into an utility method.
2015-09-17[refactor] decrease verbosity of sync logsKali Kaneko
2015-09-16[bug] review some of the close methodsVictor Shyba
We are getting "too many files open" while running tests with 1024 max files open. This is a leak from close methods. Some of them should be fixed on this commit, but further investigation may be necessary.
2015-09-16[pkg] bump leap versions needed for clientKali Kaneko
2015-09-16[feat] use async events clientKali Kaneko
in this way we use the reactor pattern to dispatch the events, instead of having the overhead of running a separate client thread. - Resolves: #7274
2015-09-09[bug] check threadpool state before closing itVictor Shyba
Code is trying to close a closed threadpool. This raises errors on Twisted 15.4.
2015-08-31[bug] emit dict instead of strIvan Alejandro
- Resolves: #7412
2015-08-28[bug] cleanup http_target.py file from refactordrebs
The http_target.py refactor started in 8adf2dedb74941352520d8de42326b0c59818728 forgot to remove the old file.
2015-08-28[style] pep8 fixesKali Kaneko
2015-08-27[tests] fix variable name from refactorVictor Shyba
From: [refactor] removing getters and setters from couch.py _couch_url was a private variable with getter and setter doing the same as a public variable. It is accessed all over the code, so being private with getters and setters didnt make sense. This commit fixes the tests to also follow this style from now on.
2015-08-26[bug] reduce memory usage by consuming single docVictor Shyba
Preparing many docs is useful for batching only. As we are sendind one by one I will leave prepare_one_doc method to do the encrypt as the docs goes to upload. Also fixes method name as kaliy suggested.
2015-08-26[refactor] Move constructor, use isinstanceVictor Shyba
isinstance is better, as kaliy pointed out, and the constructor is also in a safer place on __init__.py to be explicit. Also re-apply a change from last rebase;
2015-08-26[docs] Fix docstringsVictor Shyba
There were some missing or on incorrect format (sphinx) as drebs and kaliy pointed out.
2015-08-26[bug] fixes small issues pointed by drebsVictor Shyba
* file headers * variable names * missing docstrings * prune_conflicts ** extra: tests failed on a 1-based index bug
2015-08-26[refactor] splits http_target into 4 modulesVictor Shyba
SoledadHTTPSyncTarget is composed of 4 main groups of responsibility: * api.py - Public and main methods of a SyncTarget * fetch.py - Document fetching logic * send.py - Document sending logic * support.py - Support functions and patches Previous single file had ~600 lines with those 4 logic groups mixed, making it harder to read and understand.
2015-08-26[refactor] simplify send_docs operationsVictor Shyba
Just extracted some common logic to create u1db formatted requests into RequestBody class and created new methods to represent operations done during send_docs. This also removes send_one_doc, but does not add batching yet. The single send behavior still the same, represented by the parameter passed into RequestBody 'remove' method.
2015-08-26[refactor] simplify entity and content typeVictor Shyba
_prepare was being used to concatenate and prepare request body to send or receive data on the format expected by the server. This behavior wasnt clear, so I added a new class to abstract this out. Content type and auth headers was being copied around methods. Now the request method accepts a content_type parameter to remove this duplication.
2015-08-26[refactor] extract logging and emitting on targetVictor Shyba
Creating a message, emitting an event and logging afterwards is a single operation outside of of those method's responsabilities.
2015-08-26[pkg] fold in changes0.7.2Ivan Alejandro
2015-08-26[pkg] add missing changes filesIvan Alejandro
2015-08-24[refactor] add changes file about simplejson dep removaldrebs
2015-08-24[refactor] remove simplejson dep on clientdrebs
2015-08-20[bug] always emit end-of-sync eventdrebs
2015-08-20[bug] update target metadata after first receivedrebs
After we receive one document from the target database, we have to update the target metadata or else we will not be able to succesfully return the new generation and transaction id of the target when receiving exactly one document during a sync.
2015-08-20[bug] return tuple in get_sync_infodrebs
Tests actually expect a tuple instead of a list on the return value of get_sync_info().
2015-08-19[style] pep8 cleanupsKali Kaneko
2015-08-19[bug] wait for db init on sync decrypter pooldrebs
Previous to this modification, the initialization of the sync decrypter pool could happen concurrently with other database operations. That could cause the pool to hang because it could be waiting for something that was mistakenly deleted because of the wrong order of database operations. This commit implements a standard which we already use in leap.keymanager and leap.mail which makes some methods wait for the initialization operation before they are actually called. Closes: #7386
2015-08-19[refactor] remove threadpool from sqlcipher backenddrebs
The old strategy for having Soledad sync running asynchronously with other API calls was to have the sync running in its own threadpool. This is not needed now that all sync code uses deferreds and will not block. This commit removes what's left from the old threadpool.
2015-08-19[bug] hold sync lock while running post-sync hooksdrebs
Previous to this modification, the post-sync hooks were run after the sync lock was released. In that scenario, the next sync process could start before the previous sync's post-sync hooks were run. In general, we want the hooks to run while the current sync lock is still locked, even though for some plugins this might not make a difference.
2015-08-18[refactor] rename sync lock and improve docstringsdrebs
2015-08-18[refactor] clarify sync target closingdrebs
2015-08-17[style] pep8 cleanupKali Kaneko
2015-08-12[bug] wrong sqlcipher passphrase now raises correctlyBruno Wagner
When you tried to start a local sqlcipher that was created before, with the wrong passphrase, the code was raising a sqlcipher DatabaseError, there were tests covering this but they were expecting a WrongMacError that was never raised. I added code to wrap the DatabaseError and raise a new exception DatabaseAccessError that is specific to soledad and adapted the tests to expect it
2015-08-12[bug] changes multiprocessing.Queue to Twisted'sVictor Shyba
multiprocessing.Queue is suitable for process communication, but its not the ideal for a reactor model. This commit changes it to DeferredQueue, where consumers and producers doesnt block and Twisted can handle them better.
2015-08-12[bug] raise u1db errors based on http responsedrebs
When we started to use the twisted http agent, we forgot to intercept http response and raise the appropriate u1db errors based on the response status code and messages. This commit implements that by redefining the http body reader used by the http agent.
2015-08-12[bug] Encdecpool won't explode if stopped twiceBruno Wagner
The encryption pool could be stopped twice and would break on the second attempt because it deletes the encryption queue variable. Added a condition to make sure it only deletes the encryption queue if it exists, making it more idempotent
2015-08-12[bug] Sync Locking based on class + db file pathVictor Shyba
Change locking to be class based and each lock generated by db file path.
2015-08-12[bug] fixes concurrent sync and their testsVictor Shyba
Changes threading.lock to DeferredLock and checks syncing attribute by looking into the lock state. Also, applies more of startTwistedServer on tests that relies on HTTP/1.1. Fixes mock for events