summaryrefslogtreecommitdiff
AgeCommit message (Collapse)Author
2015-05-20[bug] remove unused pending documents exceptiondrebs
When we started implementing the sync db, one of the ideas was to reuse the data in the database in the case of a sync interruption. We don't do that now and thus the pending documents exception is unneeded. This commit removes that exception from the code.
2015-05-20[bug] remove illegal CR from auth headerKali Kaneko
The b64 encoding of the auth token was introducing an illegal character (\n), which was breaking the authentication step since an exception was being raised - when that multi-line header was attempted to be built. this commit fixes that bug. - Resolves: #6959
2015-05-20[bug] fix order of insertion of decrypted docsdrebs
This commit actually does some different things: * When doing asynchronous decryption of incoming documents in soledad client during a sync, there was the possibility that a document corresponding to a newer generation would be decrypted and inserted in the local database before a document corresponding to an older generation. When this happened, the metadata about the target database (i.e. its locally-known generation) would be first updated to the newer generation, and then an attempt to insert a document corresponding to an older generation would cause the infamous InvalidGeneration error. To fix that we use the sync-index information that is contained in the sync stream to correctly find the insertable docs to be inserted in the local database, thus avoiding the problem described above. * Refactor the sync encrypt/decrypt pool to its own file. * Fix the use of twisted adbapi with multiprocessing. Closes: #6757.
2015-05-20[feature] use async adbapi for async decryptiondrebs
Since we started implementing twisted api in soledad, some pieces are missing. Accessing the sqlcipher database directly with the twisted adbapi facilities is one of them. The async encryption/decryption was touching the database directly, and this was causing some difficulties like having different threads accessing the same database. This commit implements the twisted adbapi stuff for the asynchronous encryption/decryption facilities. Next steps would be use async adbapi for async encryption and use async adbapi for all sqlcipher access.
2015-05-20[bug] fix log messages for secrets in storagedrebs
2015-05-20[bug] fix log messages when fetching documentsdrebs
We always got a log message saying "canceling sync threads" in the end of the sync process, even when there was no error during the sync. This commit changes that in a way that we only have that log when the sync was actually cancelled because of an error.
2015-05-20[bug] always initialize sync dbdrebs
Both deferred encryption and decryption rely on a special sync db. Previous to this fix, the sync db was only initialized if a syncer was configured with deferred encryption capabilities. This was a problem when the syncer was not configured like so, but the actual sync method was initiated configured to do deferred decryption. This commit fixes this by always initializing the sync db, so we have the option of doing all combinations of deferred encryption and decryption.
2015-05-14[pkg] bump version of server dependency on commondrebs
soledad-common versions before 0.6.5 do not contain the fix for #6833 and thus will not work with most recent server. That is why we have to bump this soledad-server dependency on soledad.common.
2015-04-28[doc] fix doc on why to re-raise sync exceptionsdrebs
2015-04-28[bug] remove old code for sync deferred db initdrebs
The database initialization on the client sync module is deferred to another thread. As there is only one thread in the thread pool, this should not be a problem for now, as operations will actually be queued in that thread. There was some old code left from when we had to explicitelly wait for the db to be initialize before using it. This commit removes that old code and introduces some documentation so we remember to deal with deferred db init if we ever change the number of threads in the thread pool.
2015-04-28[refactor] rename sync callbacks on client apidrebs
2015-04-28[bug] log traceback on sync failures on clientdrebs
Conversion of Twisted failures to string that rely on __str__ or __repr__ might not return all the information we would like to have, especially on sync failures. This commit asks for a detailed traceback of such failures and logs them both in Twisted and client logs.
2015-04-23[feat] ensure sync fails will raise an exceptiondrebs
This commit makes 2 changes that allow sync failures to raise exceptions that can be caught by the api: 1. Remove try/except statements in sync.py level that would prevent an exception to be caught by the soledad client api. 2. Ensure that if an asynchronous decrypting process fails the exception will be re-raised to eventually reach the api. Related: #6757.
2015-04-23[bug] fail gracefully when sync failsdrebs
With new soledad async api, we need to catch errors using errbacks instead of catching exceptions explicitelly. This commit fixed the api sync() call to intercept sync failures, log them, and do not propagate them down the callback chain.
2015-04-23[bug] create directory in client db access scriptdrebs
2015-04-09[fix] remove unneded params to CouchServerStatedrebs
This commit removes some leftover code from a time when Soledad Server used to check for permissions on certain databases when starting (i.e. shared and tokens databases). This was later removed as correct permissions enforcement was relayed to tapicero. Closes: #6833.
2015-03-24[fix] turn client side db access script into asyncdrebs
With changes in soledad client api to make it async, the supporting scripts also have to be updated. This commit also adds more functionalities as exporting incoming mail and public or private keys.
2015-03-19[docs] add git commit template to repoKali Kaneko
because in OCD we trust.
2015-03-19[fix] add explicit dependency on leap.commondrebs
In the past, we wanted dependency on leap.common to be optional, but now because of the explicit use of the config path prefix and signaling, we want to enforce dependency on leap.common.
2015-03-19[fix] exclude all tests from packagedrebs
Previous to this modification, leap.soledad.common.tests.u1db_tests was being installed and its files were being included in the debian package. By excluding *.tests and *.tests.* from find_packages() in setup.py, we make sure that no test file will be installed not included in the final debian package.
2015-03-19[fix] add/fix dependency on twisteddrebs
Add dependency on twisted for Soledad Client. Also remove minimum twisted version for Soledad Server because debian stable currently distributes 12.0.0 and pypi currently distributes 15.0.0. Closes: #6797
2015-03-19[feat] use monthly tokens databasedrebs
Any solead release that includes this commit will be incompatible with LEAP Platform < 0.6.1 because only from that version on the platform implements the ephemeral monthly tokens databases. Closes: #6785.
2015-03-09Fix soledad initscript uid and gid.drebs
2015-02-20cutoff for encoding detectionKali Kaneko
2015-02-11Merge branch 'feature/async-api' into developKali Kaneko
2015-02-11Retry on SQLCipher timeout (#6625).drebs
2015-02-11add some benchmarking skeletonKali Kaneko
2015-02-11Bail out ddocs installation if the path doesn't exist. Fix: #6671Kali Kaneko
2015-02-11add raw sqlcipher query methodKali Kaneko
2015-02-11Do not try to unlock shared db if db is not syncable.drebs
2015-02-11Do not try to close db syncer if db is not syncable.drebs
2015-02-11Use Twisted trial for running tests.drebs
2015-02-11Add local replica info to sync debug output.drebs
2015-02-11Several fixes in SoledadSyncTarget:drebs
* Fix arg passing to syncing failure method. * Do not try to start sync loop which should be already running. * Adapt to removal of old multiprocessing safe db, now accesses the sqlcipher database directly.
2015-02-11Several fixes in SQLCipherDatabase:drebs
* Add copy of SQLCipherOptions object to avoid modifying the options object in place when instantiating the sync db. * Add string representation of SQLCipherOptions for easiness of debugging. * Make sync db always "ready". * Fix passing options for sync db initialization. * Fix typ0 that made SQLCipherU1DBSync._sync_loop be a tuple. * Do not defer requests for stopping sync to a thread pool. * Do not make pysqlcipher check if object is using in distinct threads. * Reset the sync loop when stopping the syncer. * Fix docstrings. * Check for _db_handle attribute when closing the database.
2015-02-11Several fixes in SoledadSharedDB:drebs
* Remove check for HTTPS address. * Remove creation of shared database. * Fix docstrings.
2015-02-11Several fixes in Soledad crypto:drebs
* Adapt to removal of the old multiprocessing safe database, by directly querying the sync database. * Fix docstrings.
2015-02-11Several fixes in adbapi interface:drebs
* Get replica uid upon U1DBConnectionPool initialization. * Fix docstrings.
2015-02-11Several fixes in soledad api.drebs
* Allow passing shared_db to Soledad constructor. * Close syncers on Soledad close. * Fix docstrings.
2015-02-11Cleanup unused import.drebs
2015-02-11Adapt tests for new api.drebs
2015-02-11Fix interruptable sync.drebs
2015-02-11Standardize export of secrets to avoid miscalculation of MAC.drebs
2015-02-11Save active secret on recovery document.drebs
2015-02-11minor naming/documentation fixesKali Kaneko
after drebs review
2015-02-11add syncable property to shared dbKali Kaneko
2015-02-11add soledad sync exampleKali Kaneko
2015-02-11working sync-threadpoolKali Kaneko
* Completed mapping of async dbpool * Fixed shared db initialization. Stuff To Be Fixed yet: [ ] All inserts have to be done from the sync threadpool. Right now we're reusing the connection from multiple threads in the syncer. I'm assuming the writes are automatically locking the file at the sqlite level, so this shouldn't pose a problem. [ ] Correctly handle the multiprocessing pool, and the callback execution.
2015-02-11Adapt sqlcipher tests to new api.drebs
2015-02-11Separate soledad interfacesKali Kaneko
* Separate local storage, syncers and shared_db * Comment out unused need_sync method * Use twisted LoopingCall * Create a threadpool for syncs * Return deferred from sync method * Do not pass crypto to SQLCipherDatabase * Pass replica_uid to u1db_syncer * Rename / reorganize some initialization methods