summaryrefslogtreecommitdiff
path: root/client/src/leap
AgeCommit message (Collapse)Author
2016-06-06[bug] delete all docs on start and ensure isolationVictor Shyba
Docs created from one failed sync would be there for the next one, possibly causing a lot of hard to find errors. This commit adds a sync_id field to track each sync documents isolated and cleans up the pool on start instead of constructor.
2016-06-06[refactor] encdecpool queries and testingVictor Shyba
This commit adds tests for doc ordering and encdecpool control (start/stop). Also optimizes by deleting in batch and checking for a sequence in memory before asking the local staging for documents.
2016-06-06[refactor] bye multiprocessing poolVictor Shyba
This commit removes the multiprocessing pool and gives a step closer to make encdecpool simpler. Download speed is now at a constant rate, CPU usage lower and reactor responding fast when running with a HTTP server like Pixelated.
2016-06-06[feature] use deferred semaphoreKali Kaneko
2016-06-06[bug] remove doc content conversion to unicodedrebs
Theoretically (until now), Soledad inherits from U1DB the behaviour of only accepting valid JSON for documents contents. JSON documents only allow for unicode strings. Despite that, until now we had implemented lossy convertion to unicode to avoid encoding errors when dumping/loading JSON content. This allowed for API users to pass non-unicode to Soledad, but caused the application to take more time because of conversion. There were 2 problem with this: (1) conversion may take a long time and a lot of memory when convertin large payloads; and (2) conversion was being made before deferring to the adbapi, and this was blocking the reactor. This commit completelly removes the conversion to unicode, thus leaving the responsibility of unicode conversion to users of the Soledad API.
2016-05-23[refactor] remove user_id argument from Soledad initCaio Carrara
The constructor method of Soledad was receiving two arguments for user id. One of them was optional with None as default. It could cause an inconsistent state with uuid set but userid unset. This change remove the optional user_id argument from initialization method and return the uuid if anyone call Soledad.userid method.
2016-05-16[style] pep8Kali Kaneko
2016-04-26[refactor] remove old code for enlarging secretsdrebs
2016-04-26[refactor] cleanup bootstrap processdrebs
2016-04-26[refactor] remove shared db locking from clientdrebs
Shared db locking was used to avoid the case in which two different devices try to store/modify remotelly stored secrets at the same time. We want to avoid remote locks because of the problems they create, and prefer to crash locally. For the record, we are currently using the user's password to encrypt the secrets stored in the server, and while we continue to do this we will have to re-encrypt the secrets and update the remote storage whenever the user changes her password.
2016-04-01[pkg] updated to versioneer 0.16 (patched)Kali Kaneko
2016-03-09[bug] specify openssl backend explicitelyKali Kaneko
for some reason, available_backends does not work inside a frozen PyInstaller binary. - Resolves: #7952
2016-01-26[feat] use cryptography instead of pycryptoppVictor Shyba
cryptography comes from OpenSSL and Twisted dependencies, so it's already installed. This commit removes a compiled dependency, also possibly making it easier to use on Windows.
2016-01-21[Fix] slow IO-bound calls block reactorVictor Shyba
- Move them to a thread so reactor can continue processing e.g. http requests
2015-12-15[style] pep8Kali Kaneko
2015-12-15[feat] get_or_create_service_tokenKali Kaneko
2015-12-15[feat] set syncable attributeKali Kaneko
this allows to switch the online/offline mode on a running soledad instance.
2015-12-14[bug] fix failing tests after last events modificationKali Kaneko
2015-12-10[feat] use userid in soledad events tooKali Kaneko
for the moment, userid has to be passed to constructor. eventually, we might drop support for passing uuid, since it will be mapped in the service tree
2015-12-10[feat] use uuid in events, multi-user awareKali Kaneko
- Resolves: #7656 - Releases: 0.8.0
2015-12-03[feat] set default to False on batching for nowVictor Shyba
All batching code has no effect by default with this commit. Since we know that this is a dangerous new feature we will enable them only on our test servers and check them manually before setting it as default or adding more configuration features. Use SyncTarget and server conf file to enable it for testing.
2015-12-03[feat] Adds support to batching limited by sizeVictor Shyba
u1db provides batching by default. Current Soledad HTTPS Sync Target was stuck at 1 doc per request. This commit adds batching capability, limiting the size to a predefined value. Default limit size: 500kB
2015-12-01[style] fix pep8Victor Shyba
2015-12-01[bug] concurrency bug while querying and insertingVictor Shyba
This line was missing an yield and without it we end up inserting a document that is being retrieved and bad things happen. This is the core fix from yesterday debugging session. During sequential syncs the pool was inserting and querying at the same time and sometimes repeating or failing to delete documents.
2015-12-01[bug] fire callback after reseting instance varsVictor Shyba
If we reset the vars after firing the finish callback, other thread can pick up a dirty state on due concurrency.
2015-11-19[bug] fix indirect dependency on couchdb for clientKali Kaneko
by subclassing the MissingDesignDocError, we don't have to import the soledad.common.couch submodule into the soledad.client.sync - Resolves: #7626
2015-11-03[refactor] move couch errors and expose methodsVictor Shyba
errors.py was holding a few specific CouchDB errors, now moved into couch.errors module. Also, some of CouchDatabase methods were declared as private, but external classes needs them.
2015-10-05[bug] handle MissingDesignDocError after get_sync_infoVictor Shyba
MissingDesignDocError raised on get_sync_info due to a missing design document will be handled by the server during sync. Ensure is now False by default, and thus database creation can deliver an empty one that will be ensured during sync, following the ensure parameter.
2015-10-02[bug] increase http request timeout time to 90sKali Kaneko
this is a workaroud to reduce the chances of failed sync due to timeouts. this should be properly tackled by: 1. implementing proper cancellable for the sync operation. 2. implementing a retry count at the level of a single request, handled internally by soledad. in this way we can remove the retries logic from the soledadbootstrapper in the bitmask client. - Related: #7382
2015-10-02[bug] do not signal sync completion if failedKali Kaneko
- Related: #7503
2015-09-28[feat] handle DatabaseDoesNotExist during syncVictor Shyba
This error raises while getting info on target (or server) replica. On previous implementation there was nothing to do here, but now that we have db creation in place this error should be handled just like u1db original implementation. The reason is that db creation occurs during sync exchange, but before this we try to ask for info and the code that checks for info raises an error that will be used to signal the client that a database will be created and that it must save the replica uid returned by server, after sync exchange (where we send/fetch documents).
2015-09-21[bug] set the received active secret before saving local fileKali Kaneko
- bug: we were dumping the received secrets locally to disk *before* setting the received property for the active secret, and therefore the 'active_secret' was always marked as null. - refactor common code into an utility method.
2015-09-17[refactor] decrease verbosity of sync logsKali Kaneko
2015-09-16[bug] review some of the close methodsVictor Shyba
We are getting "too many files open" while running tests with 1024 max files open. This is a leak from close methods. Some of them should be fixed on this commit, but further investigation may be necessary.
2015-09-16[feat] use async events clientKali Kaneko
in this way we use the reactor pattern to dispatch the events, instead of having the overhead of running a separate client thread. - Resolves: #7274
2015-09-09[bug] check threadpool state before closing itVictor Shyba
Code is trying to close a closed threadpool. This raises errors on Twisted 15.4.
2015-08-31[bug] emit dict instead of strIvan Alejandro
- Resolves: #7412
2015-08-28[bug] cleanup http_target.py file from refactordrebs
The http_target.py refactor started in 8adf2dedb74941352520d8de42326b0c59818728 forgot to remove the old file.
2015-08-28[style] pep8 fixesKali Kaneko
2015-08-26[bug] reduce memory usage by consuming single docVictor Shyba
Preparing many docs is useful for batching only. As we are sendind one by one I will leave prepare_one_doc method to do the encrypt as the docs goes to upload. Also fixes method name as kaliy suggested.
2015-08-26[refactor] Move constructor, use isinstanceVictor Shyba
isinstance is better, as kaliy pointed out, and the constructor is also in a safer place on __init__.py to be explicit. Also re-apply a change from last rebase;
2015-08-26[docs] Fix docstringsVictor Shyba
There were some missing or on incorrect format (sphinx) as drebs and kaliy pointed out.
2015-08-26[bug] fixes small issues pointed by drebsVictor Shyba
* file headers * variable names * missing docstrings * prune_conflicts ** extra: tests failed on a 1-based index bug
2015-08-26[refactor] splits http_target into 4 modulesVictor Shyba
SoledadHTTPSyncTarget is composed of 4 main groups of responsibility: * api.py - Public and main methods of a SyncTarget * fetch.py - Document fetching logic * send.py - Document sending logic * support.py - Support functions and patches Previous single file had ~600 lines with those 4 logic groups mixed, making it harder to read and understand.
2015-08-26[refactor] simplify send_docs operationsVictor Shyba
Just extracted some common logic to create u1db formatted requests into RequestBody class and created new methods to represent operations done during send_docs. This also removes send_one_doc, but does not add batching yet. The single send behavior still the same, represented by the parameter passed into RequestBody 'remove' method.
2015-08-26[refactor] simplify entity and content typeVictor Shyba
_prepare was being used to concatenate and prepare request body to send or receive data on the format expected by the server. This behavior wasnt clear, so I added a new class to abstract this out. Content type and auth headers was being copied around methods. Now the request method accepts a content_type parameter to remove this duplication.
2015-08-26[refactor] extract logging and emitting on targetVictor Shyba
Creating a message, emitting an event and logging afterwards is a single operation outside of of those method's responsabilities.
2015-08-24[refactor] remove simplejson dep on clientdrebs
2015-08-20[bug] always emit end-of-sync eventdrebs
2015-08-20[bug] update target metadata after first receivedrebs
After we receive one document from the target database, we have to update the target metadata or else we will not be able to succesfully return the new generation and transaction id of the target when receiving exactly one document during a sync.