summaryrefslogtreecommitdiff
path: root/client
AgeCommit message (Collapse)Author
2015-05-20[bug] ensure async decryption failures are loggeddrebs
We have to make sure any failures in asynchronous decryption code is grabbed and properly transmitted up the deferred chain so it can be logged. This commit adds errbacks in the decryption pool that grab any failure and a check on the http target the failure if that is the case.
2015-05-20[feature] ensure reactor stops on client db scriptdrebs
2015-05-20[bug] wrap unauth errors as invalid token errorsdrebs
2015-05-20[bug] ensure sync failures are not ignoreddrebs
2015-05-20[refactor] cleanup sync, remove unused stuffdrebs
This commit does the following: * Remove the autocreate parameter from the sync() method. * Remove the syncing lock from the sync module because it did the same job as the lock in the sqlcipher module. * Remove the close/stop methods from sync module as they don't make sense after we started to use twisted in client-side sync.
2015-05-20[feature] use twisted.web.client in client syncdrebs
This change uses twisted deferreds for the whole syncing process and paves the way to implementing other transport schemes. It removes a lot of threaded code that used locks and was very difficult to maintain, and lets twisted to the dirty work. Furthermore, all blocking network i/o is now handled asynchronously by the twisted. This commit removes the possibility of interrupting a sync, and we should reimplement it using cancellable deferreds if we need it.
2015-05-20[feature] use twisted adbapi for async encryptiondrebs
The access to the sync db was modified to use twisted.enterprise.adbapi, but only the asynchronous decryption of incoming documents during sync was adapted. This commit modifies the asynchornous encryption of documents to also use the adbapi for accessing the sync db.
2015-05-20[bug] remove unused pending documents exceptiondrebs
When we started implementing the sync db, one of the ideas was to reuse the data in the database in the case of a sync interruption. We don't do that now and thus the pending documents exception is unneeded. This commit removes that exception from the code.
2015-05-20[bug] remove illegal CR from auth headerKali Kaneko
The b64 encoding of the auth token was introducing an illegal character (\n), which was breaking the authentication step since an exception was being raised - when that multi-line header was attempted to be built. this commit fixes that bug. - Resolves: #6959
2015-05-20[bug] fix order of insertion of decrypted docsdrebs
This commit actually does some different things: * When doing asynchronous decryption of incoming documents in soledad client during a sync, there was the possibility that a document corresponding to a newer generation would be decrypted and inserted in the local database before a document corresponding to an older generation. When this happened, the metadata about the target database (i.e. its locally-known generation) would be first updated to the newer generation, and then an attempt to insert a document corresponding to an older generation would cause the infamous InvalidGeneration error. To fix that we use the sync-index information that is contained in the sync stream to correctly find the insertable docs to be inserted in the local database, thus avoiding the problem described above. * Refactor the sync encrypt/decrypt pool to its own file. * Fix the use of twisted adbapi with multiprocessing. Closes: #6757.
2015-05-20[feature] use async adbapi for async decryptiondrebs
Since we started implementing twisted api in soledad, some pieces are missing. Accessing the sqlcipher database directly with the twisted adbapi facilities is one of them. The async encryption/decryption was touching the database directly, and this was causing some difficulties like having different threads accessing the same database. This commit implements the twisted adbapi stuff for the asynchronous encryption/decryption facilities. Next steps would be use async adbapi for async encryption and use async adbapi for all sqlcipher access.
2015-05-20[bug] fix log messages for secrets in storagedrebs
2015-05-20[bug] fix log messages when fetching documentsdrebs
We always got a log message saying "canceling sync threads" in the end of the sync process, even when there was no error during the sync. This commit changes that in a way that we only have that log when the sync was actually cancelled because of an error.
2015-05-20[bug] always initialize sync dbdrebs
Both deferred encryption and decryption rely on a special sync db. Previous to this fix, the sync db was only initialized if a syncer was configured with deferred encryption capabilities. This was a problem when the syncer was not configured like so, but the actual sync method was initiated configured to do deferred decryption. This commit fixes this by always initializing the sync db, so we have the option of doing all combinations of deferred encryption and decryption.
2015-04-28[doc] fix doc on why to re-raise sync exceptionsdrebs
2015-04-28[bug] remove old code for sync deferred db initdrebs
The database initialization on the client sync module is deferred to another thread. As there is only one thread in the thread pool, this should not be a problem for now, as operations will actually be queued in that thread. There was some old code left from when we had to explicitelly wait for the db to be initialize before using it. This commit removes that old code and introduces some documentation so we remember to deal with deferred db init if we ever change the number of threads in the thread pool.
2015-04-28[refactor] rename sync callbacks on client apidrebs
2015-04-28[bug] log traceback on sync failures on clientdrebs
Conversion of Twisted failures to string that rely on __str__ or __repr__ might not return all the information we would like to have, especially on sync failures. This commit asks for a detailed traceback of such failures and logs them both in Twisted and client logs.
2015-04-23[feat] ensure sync fails will raise an exceptiondrebs
This commit makes 2 changes that allow sync failures to raise exceptions that can be caught by the api: 1. Remove try/except statements in sync.py level that would prevent an exception to be caught by the soledad client api. 2. Ensure that if an asynchronous decrypting process fails the exception will be re-raised to eventually reach the api. Related: #6757.
2015-04-23[bug] fail gracefully when sync failsdrebs
With new soledad async api, we need to catch errors using errbacks instead of catching exceptions explicitelly. This commit fixed the api sync() call to intercept sync failures, log them, and do not propagate them down the callback chain.
2015-03-19[fix] add explicit dependency on leap.commondrebs
In the past, we wanted dependency on leap.common to be optional, but now because of the explicit use of the config path prefix and signaling, we want to enforce dependency on leap.common.
2015-03-19[fix] add/fix dependency on twisteddrebs
Add dependency on twisted for Soledad Client. Also remove minimum twisted version for Soledad Server because debian stable currently distributes 12.0.0 and pypi currently distributes 15.0.0. Closes: #6797
2015-02-20cutoff for encoding detectionKali Kaneko
2015-02-11Retry on SQLCipher timeout (#6625).drebs
2015-02-11add some benchmarking skeletonKali Kaneko
2015-02-11add raw sqlcipher query methodKali Kaneko
2015-02-11Do not try to unlock shared db if db is not syncable.drebs
2015-02-11Do not try to close db syncer if db is not syncable.drebs
2015-02-11Add local replica info to sync debug output.drebs
2015-02-11Several fixes in SoledadSyncTarget:drebs
* Fix arg passing to syncing failure method. * Do not try to start sync loop which should be already running. * Adapt to removal of old multiprocessing safe db, now accesses the sqlcipher database directly.
2015-02-11Several fixes in SQLCipherDatabase:drebs
* Add copy of SQLCipherOptions object to avoid modifying the options object in place when instantiating the sync db. * Add string representation of SQLCipherOptions for easiness of debugging. * Make sync db always "ready". * Fix passing options for sync db initialization. * Fix typ0 that made SQLCipherU1DBSync._sync_loop be a tuple. * Do not defer requests for stopping sync to a thread pool. * Do not make pysqlcipher check if object is using in distinct threads. * Reset the sync loop when stopping the syncer. * Fix docstrings. * Check for _db_handle attribute when closing the database.
2015-02-11Several fixes in SoledadSharedDB:drebs
* Remove check for HTTPS address. * Remove creation of shared database. * Fix docstrings.
2015-02-11Several fixes in Soledad crypto:drebs
* Adapt to removal of the old multiprocessing safe database, by directly querying the sync database. * Fix docstrings.
2015-02-11Several fixes in adbapi interface:drebs
* Get replica uid upon U1DBConnectionPool initialization. * Fix docstrings.
2015-02-11Several fixes in soledad api.drebs
* Allow passing shared_db to Soledad constructor. * Close syncers on Soledad close. * Fix docstrings.
2015-02-11Fix interruptable sync.drebs
2015-02-11Standardize export of secrets to avoid miscalculation of MAC.drebs
2015-02-11Save active secret on recovery document.drebs
2015-02-11minor naming/documentation fixesKali Kaneko
after drebs review
2015-02-11add syncable property to shared dbKali Kaneko
2015-02-11add soledad sync exampleKali Kaneko
2015-02-11working sync-threadpoolKali Kaneko
* Completed mapping of async dbpool * Fixed shared db initialization. Stuff To Be Fixed yet: [ ] All inserts have to be done from the sync threadpool. Right now we're reusing the connection from multiple threads in the syncer. I'm assuming the writes are automatically locking the file at the sqlite level, so this shouldn't pose a problem. [ ] Correctly handle the multiprocessing pool, and the callback execution.
2015-02-11Adapt sqlcipher tests to new api.drebs
2015-02-11Separate soledad interfacesKali Kaneko
* Separate local storage, syncers and shared_db * Comment out unused need_sync method * Use twisted LoopingCall * Create a threadpool for syncs * Return deferred from sync method * Do not pass crypto to SQLCipherDatabase * Pass replica_uid to u1db_syncer * Rename / reorganize some initialization methods
2015-02-11Refactor soledad api to use async dbKali Kaneko
* add examples and benchmarks * remove autocommit mode, allow wal disabling * lock initialization * make api use async calls
2015-02-11move public api to its own fileKali Kaneko
2015-02-11reorganize pragmas, stub SQLCipherOptions objectKali Kaneko
2015-02-11use max cpu_count workers on poolKali Kaneko
2015-02-11remove taskthread dependencyKali Kaneko
2015-02-11remove unused importsKali Kaneko