From 3a3f2d8ca5b0ae2adb5007577f2d828677ff64e0 Mon Sep 17 00:00:00 2001 From: drebs Date: Thu, 30 Apr 2015 11:24:21 -0300 Subject: [bug] always initialize sync db Both deferred encryption and decryption rely on a special sync db. Previous to this fix, the sync db was only initialized if a syncer was configured with deferred encryption capabilities. This was a problem when the syncer was not configured like so, but the actual sync method was initiated configured to do deferred decryption. This commit fixes this by always initializing the sync db, so we have the option of doing all combinations of deferred encryption and decryption. --- client/changes/bug_always-initialize-the-sync-db | 2 ++ 1 file changed, 2 insertions(+) create mode 100644 client/changes/bug_always-initialize-the-sync-db (limited to 'client/changes') diff --git a/client/changes/bug_always-initialize-the-sync-db b/client/changes/bug_always-initialize-the-sync-db new file mode 100644 index 00000000..2b12989a --- /dev/null +++ b/client/changes/bug_always-initialize-the-sync-db @@ -0,0 +1,2 @@ + o Always initialize the sync db to allow for both asynchronous encryption + and asynchronous decryption when syncing. -- cgit v1.2.3 From 6ab3fe57764c2e5f2a5688d377fe46a51437f0be Mon Sep 17 00:00:00 2001 From: drebs Date: Thu, 30 Apr 2015 11:47:10 -0300 Subject: [bug] fix log messages when fetching documents We always got a log message saying "canceling sync threads" in the end of the sync process, even when there was no error during the sync. This commit changes that in a way that we only have that log when the sync was actually cancelled because of an error. --- client/changes/bug_improve-log-when-fetching-documents | 1 + 1 file changed, 1 insertion(+) create mode 100644 client/changes/bug_improve-log-when-fetching-documents (limited to 'client/changes') diff --git a/client/changes/bug_improve-log-when-fetching-documents b/client/changes/bug_improve-log-when-fetching-documents new file mode 100644 index 00000000..a67ce028 --- /dev/null +++ b/client/changes/bug_improve-log-when-fetching-documents @@ -0,0 +1 @@ + o Improve log messages when concurrently fetching documents from the server. -- cgit v1.2.3 From b75bedb065cfbbb2993659d867ef554ff70596ae Mon Sep 17 00:00:00 2001 From: drebs Date: Mon, 4 May 2015 13:26:57 -0300 Subject: [bug] fix log messages for secrets in storage --- client/changes/bug_6892_fix-log-message-for-local-secret | 2 ++ client/changes/bug_fix-async-decrypt | 2 ++ 2 files changed, 4 insertions(+) create mode 100644 client/changes/bug_6892_fix-log-message-for-local-secret create mode 100644 client/changes/bug_fix-async-decrypt (limited to 'client/changes') diff --git a/client/changes/bug_6892_fix-log-message-for-local-secret b/client/changes/bug_6892_fix-log-message-for-local-secret new file mode 100644 index 00000000..39c13257 --- /dev/null +++ b/client/changes/bug_6892_fix-log-message-for-local-secret @@ -0,0 +1,2 @@ + o Fix the log message when a local secret is not found so it's less + confusing. Closes #6892. diff --git a/client/changes/bug_fix-async-decrypt b/client/changes/bug_fix-async-decrypt new file mode 100644 index 00000000..eb0ce7b5 --- /dev/null +++ b/client/changes/bug_fix-async-decrypt @@ -0,0 +1,2 @@ + o Refactor asynchronous encryption/decryption code to its own file. + o Fix logging and graceful failing when exceptions are raised during sync. -- cgit v1.2.3 From 3a7ddacd06fd57afb10cc3d7083c2aa196c9328f Mon Sep 17 00:00:00 2001 From: drebs Date: Mon, 4 May 2015 13:04:56 -0300 Subject: [feature] use async adbapi for async decryption Since we started implementing twisted api in soledad, some pieces are missing. Accessing the sqlcipher database directly with the twisted adbapi facilities is one of them. The async encryption/decryption was touching the database directly, and this was causing some difficulties like having different threads accessing the same database. This commit implements the twisted adbapi stuff for the asynchronous encryption/decryption facilities. Next steps would be use async adbapi for async encryption and use async adbapi for all sqlcipher access. --- client/changes/feature_use-twisted-adbapi-for-sync-db | 1 + 1 file changed, 1 insertion(+) create mode 100644 client/changes/feature_use-twisted-adbapi-for-sync-db (limited to 'client/changes') diff --git a/client/changes/feature_use-twisted-adbapi-for-sync-db b/client/changes/feature_use-twisted-adbapi-for-sync-db new file mode 100644 index 00000000..41e5e6e3 --- /dev/null +++ b/client/changes/feature_use-twisted-adbapi-for-sync-db @@ -0,0 +1 @@ + o Use twisted.enterprise.adbapi for access to the sync database. -- cgit v1.2.3 From eae4468d99029006cc36a021e82350a0f62f7006 Mon Sep 17 00:00:00 2001 From: drebs Date: Thu, 7 May 2015 14:49:40 -0300 Subject: [bug] fix order of insertion of decrypted docs This commit actually does some different things: * When doing asynchronous decryption of incoming documents in soledad client during a sync, there was the possibility that a document corresponding to a newer generation would be decrypted and inserted in the local database before a document corresponding to an older generation. When this happened, the metadata about the target database (i.e. its locally-known generation) would be first updated to the newer generation, and then an attempt to insert a document corresponding to an older generation would cause the infamous InvalidGeneration error. To fix that we use the sync-index information that is contained in the sync stream to correctly find the insertable docs to be inserted in the local database, thus avoiding the problem described above. * Refactor the sync encrypt/decrypt pool to its own file. * Fix the use of twisted adbapi with multiprocessing. Closes: #6757. --- client/changes/bug_6757_fix-order-of-insertion-when-syncing | 2 ++ 1 file changed, 2 insertions(+) create mode 100644 client/changes/bug_6757_fix-order-of-insertion-when-syncing (limited to 'client/changes') diff --git a/client/changes/bug_6757_fix-order-of-insertion-when-syncing b/client/changes/bug_6757_fix-order-of-insertion-when-syncing new file mode 100644 index 00000000..c0470f5a --- /dev/null +++ b/client/changes/bug_6757_fix-order-of-insertion-when-syncing @@ -0,0 +1,2 @@ + o Fix the order of insertion of documents when using workers for decrypting + incoming documents during a sync. Closes #6757. -- cgit v1.2.3 From e62dafeba8f08c1f7588e37cf9cd3fb28e79a020 Mon Sep 17 00:00:00 2001 From: drebs Date: Tue, 19 May 2015 18:46:53 -0300 Subject: [feature] use twisted.web.client in client sync This change uses twisted deferreds for the whole syncing process and paves the way to implementing other transport schemes. It removes a lot of threaded code that used locks and was very difficult to maintain, and lets twisted to the dirty work. Furthermore, all blocking network i/o is now handled asynchronously by the twisted. This commit removes the possibility of interrupting a sync, and we should reimplement it using cancellable deferreds if we need it. --- client/changes/feature_use-twisted-web-for-client-sync | 1 + 1 file changed, 1 insertion(+) create mode 100644 client/changes/feature_use-twisted-web-for-client-sync (limited to 'client/changes') diff --git a/client/changes/feature_use-twisted-web-for-client-sync b/client/changes/feature_use-twisted-web-for-client-sync new file mode 100644 index 00000000..b4d1d4a4 --- /dev/null +++ b/client/changes/feature_use-twisted-web-for-client-sync @@ -0,0 +1 @@ + o Use twisted.web.client for client sync. -- cgit v1.2.3 From 31757168f6ad243ec83ba52b2e022298ba08f8d1 Mon Sep 17 00:00:00 2001 From: drebs Date: Mon, 25 May 2015 11:46:24 -0300 Subject: [feature] add pool of http/https connections Instead of opening one TCP connection for each HTTP request, we want to reuse connections. Also, we need to be able to verify SSL certificates. This commit implements both features in the twisted http client sync. --- client/changes/feature_add-pool-of-http-https-connections | 2 ++ 1 file changed, 2 insertions(+) create mode 100644 client/changes/feature_add-pool-of-http-https-connections (limited to 'client/changes') diff --git a/client/changes/feature_add-pool-of-http-https-connections b/client/changes/feature_add-pool-of-http-https-connections new file mode 100644 index 00000000..7ff2a4ee --- /dev/null +++ b/client/changes/feature_add-pool-of-http-https-connections @@ -0,0 +1,2 @@ + o Add a pool of HTTP/HTTPS connections that is able to verify the server + certificate against a given CA certificate. -- cgit v1.2.3