summaryrefslogtreecommitdiff
AgeCommit message (Collapse)Author
2016-06-06[bug] delete all docs on start and ensure isolationVictor Shyba
Docs created from one failed sync would be there for the next one, possibly causing a lot of hard to find errors. This commit adds a sync_id field to track each sync documents isolated and cleans up the pool on start instead of constructor.
2016-06-06[refactor] encdecpool queries and testingVictor Shyba
This commit adds tests for doc ordering and encdecpool control (start/stop). Also optimizes by deleting in batch and checking for a sequence in memory before asking the local staging for documents.
2016-06-06[refactor] bye multiprocessing poolVictor Shyba
This commit removes the multiprocessing pool and gives a step closer to make encdecpool simpler. Download speed is now at a constant rate, CPU usage lower and reactor responding fast when running with a HTTP server like Pixelated.
2016-06-06[feature] use deferred semaphoreKali Kaneko
2016-06-06[bug] remove doc content conversion to unicodedrebs
Theoretically (until now), Soledad inherits from U1DB the behaviour of only accepting valid JSON for documents contents. JSON documents only allow for unicode strings. Despite that, until now we had implemented lossy convertion to unicode to avoid encoding errors when dumping/loading JSON content. This allowed for API users to pass non-unicode to Soledad, but caused the application to take more time because of conversion. There were 2 problem with this: (1) conversion may take a long time and a lot of memory when convertin large payloads; and (2) conversion was being made before deferring to the adbapi, and this was blocking the reactor. This commit completelly removes the conversion to unicode, thus leaving the responsibility of unicode conversion to users of the Soledad API.
2016-06-06[refactor] adapt profiling script to local debug serverKali Kaneko
2016-06-06[refactor] adapt create-user-db script to bypass auth for local testsKali Kaneko
2016-06-06[feature] debug-mode server with dummy authenticationKali Kaneko
to ease debugging of local servers w/o neededing the Token machinery in place. this needs still some extra changes to be fully functional: - adapt the create-userdb script to work with no auth info.
2016-05-23[refactor] remove user_id argument from Soledad initCaio Carrara
The constructor method of Soledad was receiving two arguments for user id. One of them was optional with None as default. It could cause an inconsistent state with uuid set but userid unset. This change remove the optional user_id argument from initialization method and return the uuid if anyone call Soledad.userid method.
2016-05-18[pkg] update to new versioneer json formatKali Kaneko
2016-05-18[pkg] update to new versioneer json formatKali Kaneko
2016-05-16[style] pep8Kali Kaneko
2016-04-26[refactor] add changes file for shared db lock removaldrebs
2016-04-26[refactor] remove old code for enlarging secretsdrebs
2016-04-26[refactor] cleanup bootstrap processdrebs
2016-04-26[refactor] remove shared db locking from testsdrebs
2016-04-26[refactor] remove shared db locking from serverdrebs
2016-04-26[refactor] remove shared db locking from clientdrebs
Shared db locking was used to avoid the case in which two different devices try to store/modify remotelly stored secrets at the same time. We want to avoid remote locks because of the problems they create, and prefer to crash locally. For the record, we are currently using the user's password to encrypt the secrets stored in the server, and while we continue to do this we will have to re-encrypt the secrets and update the remote storage whenever the user changes her password.
2016-04-18Merge tag '0.8.0' into developKali Kaneko
Tag soledad version 0.8.0
2016-04-18[pkg] Update changelog0.8.0Kali Kaneko
2016-04-01[pkg] updated to versioneer 0.16 (patched)Kali Kaneko
2016-04-01[pkg] updated to versioneer 0.16 (patched)Kali Kaneko
2016-04-01patch for multi-pkg repoKali Kaneko
2016-04-01[pkg] update to versioneer 0.16Kali Kaneko
2016-03-09[bug] specify openssl backend explicitelyKali Kaneko
for some reason, available_backends does not work inside a frozen PyInstaller binary. - Resolves: #7952
2016-01-28[docs] document deprecation of pycryptopp in changelogKali Kaneko
2016-01-26[feat] use cryptography instead of pycryptoppVictor Shyba
cryptography comes from OpenSSL and Twisted dependencies, so it's already installed. This commit removes a compiled dependency, also possibly making it easier to use on Windows.
2016-01-22Merge branch 'fix_sync_timeout' into developRuben Pollan
2016-01-21[Fix] fix concurrency problem in test_sync_deferredFolker Bernitt
- Use dbsyncer (SQLCipherU1DBSync) instead of SQLCipherDatabase as only the first one supports multiple threads while syncing and is actually used by Soledad.sync
2016-01-21[Fix] slow IO-bound calls block reactorVictor Shyba
- Move them to a thread so reactor can continue processing e.g. http requests
2015-12-22[docs] incomplete doc for security config parameterVictor Shyba
database_security parameter was either undocumented or incomplete. This commit adds a few more doc to make it consistent with latest changes. Closes #7689
2015-12-15[style] pep8Kali Kaneko
2015-12-15[feat] get_or_create_service_tokenKali Kaneko
2015-12-15[feat] set syncable attributeKali Kaneko
this allows to switch the online/offline mode on a running soledad instance.
2015-12-14[fix] remove trailing whitespace to please pep8Christoph Kluenter
2015-12-14[bug] fix failing tests after last events modificationKali Kaneko
2015-12-10[feat] use userid in soledad events tooKali Kaneko
for the moment, userid has to be passed to constructor. eventually, we might drop support for passing uuid, since it will be mapped in the service tree
2015-12-10[feat] use uuid in events, multi-user awareKali Kaneko
- Resolves: #7656 - Releases: 0.8.0
2015-12-04[bug] skip missing keys on existing sectionsVictor Shyba
While parsing the configuration file, if a key doesnt exist we need to skip it.
2015-12-03[feat] set default to False on batching for nowVictor Shyba
All batching code has no effect by default with this commit. Since we know that this is a dangerous new feature we will enable them only on our test servers and check them manually before setting it as default or adding more configuration features. Use SyncTarget and server conf file to enable it for testing.
2015-12-03[feat] generation caching during a batchVictor Shyba
Generation cache was removed for simple processing and it should not got back, but during a batch the server wont change its generation. So a little trick to hold that temporary information until batch finishes is needed.
2015-12-03[feat] add configuration to disable batchingVictor Shyba
Batch support is optional. This commit adds a 'batching' configuration option to disable it.
2015-12-03[feat] checks staged docs inside batchVictor Shyba
This commit adds checking for consistency on batch. When a doc is needed during a batched sync and it doesnt exists on database, current code will make a partial batch to avoid processing like it doesnt exist.
2015-12-03[feat] put all docs at onceVictor Shyba
Using _bulk_docs api from CouchDB we can put all docs at a single request. Also, prefetching all ids removes the need to HEAD requests during the batch.
2015-12-03[feat] prepare server to handle batchesVictor Shyba
Created two methods on the backend to start and finish a batch. A dict of callbacks is available to defer actions for the last document, allowing temporary (changing often) metadata to be recorded only once. Using those methods we will also be able to put all docs in one go on the CouchDatabase implementation, but that is another step.
2015-12-03[feat] Adds support to batching limited by sizeVictor Shyba
u1db provides batching by default. Current Soledad HTTPS Sync Target was stuck at 1 doc per request. This commit adds batching capability, limiting the size to a predefined value. Default limit size: 500kB
2015-12-02[docs] fix run-tests snippetKali Kaneko
2015-12-01[style] fix pep8Victor Shyba
2015-12-01[bug] concurrency bug while querying and insertingVictor Shyba
This line was missing an yield and without it we end up inserting a document that is being retrieved and bad things happen. This is the core fix from yesterday debugging session. During sequential syncs the pool was inserting and querying at the same time and sometimes repeating or failing to delete documents.
2015-12-01[test] shuffle and decrypts 5 docs 3 timesVictor Shyba
On real usage the docs will arrive shuffled and pool will be reused after many decrypts. This test asserts that everything ended up clear between execution and no inconsistency is left over for the next run.