Tag version 0.7.0. Conflicts: client/pkg/requirements.pip common/pkg/requirements.pip
Previous to this change, the actual encryption method used to run on its own thread. When the close method was called from another thread, the queue could be deleted after the encryption method loop had started, but before the queue was checked for new items. By removing that thread and moving the encryption loop to the reactor, that race condition should disappear. Closes: #7088.
- Related: #6359
Instead of opening one TCP connection for each HTTP request, we want to reuse connections. Also, we need to be able to verify SSL certificates. This commit implements both features in the twisted http client sync.
This change uses twisted deferreds for the whole syncing process and paves the way to implementing other transport schemes. It removes a lot of threaded code that used locks and was very difficult to maintain, and lets twisted to the dirty work. Furthermore, all blocking network i/o is now handled asynchronously by the twisted. This commit removes the possibility of interrupting a sync, and we should reimplement it using cancellable deferreds if we need it.
This commit actually does some different things:
* When doing asynchronous decryption of incoming documents in soledad client
during a sync, there was the possibility that a document corresponding to
a newer generation would be decrypted and inserted in the local database
before a document corresponding to an older generation. When this
happened, the metadata about the target database (i.e. its locally-known
generation) would be first updated to the newer generation, and then an
attempt to insert a document corresponding to an older generation would
cause the infamous InvalidGeneration error.
To fix that we use the sync-index information that is contained in the
sync stream to correctly find the insertable docs to be inserted in the
local database, thus avoiding the problem described above.
* Refactor the sync encrypt/decrypt pool to its own file.
* Fix the use of twisted adbapi with multiprocessing.
Closes: #6757.
Since we started implementing twisted api in soledad, some pieces are missing. Accessing the sqlcipher database directly with the twisted adbapi facilities is one of them. The async encryption/decryption was touching the database directly, and this was causing some difficulties like having different threads accessing the same database. This commit implements the twisted adbapi stuff for the asynchronous encryption/decryption facilities. Next steps would be use async adbapi for async encryption and use async adbapi for all sqlcipher access.