| Age | Commit message (Collapse) | Author |
|---|
|
The couch backend makes use of attachments and multipart structure for
writing the document to the couch database. For that to work, the order
in which attachments are described must match the actual order in which
attachments are written to the couch http stream.
This was not being properly taken care of, and eventually the json
serializer was arbitrarilly ordering the attachments description in a
way that it didn't match the actual order of attachments writing.
This commit fixes that by using json.dumps() sort_keys parameter and
making sure conflicts are always written before content.
|
|
|
|
Design documents are slow and we already have alternatives to all uses
we used to make of them, so this commit completelly removes all usage of
design documents.
|
|
|
|
When compared to plain couch document get, the use of the simplest view
functions takes around double the time, while the use of the simplest
list function can take more than 8 times:
get 100 docs:
total: 0.440337 secs
mean: 0.004403
query 100 views:
total: 0.911425 secs
mean: 0.009114
query 100 lists:
total: 3.711537 secs
mean: 0.037115
Besides that, the current implementation of sync metadata storage over
couch is dependent of timestamps of document puts, what can lead to
metadata corruption if the clock of the system is changed for any
reason.
Because of these reasons, we seek to change the implementation of
database metadata. This commit implements the storage of transaction log
data on couch documents with special ids, in the form "gen-xxxxxxxxxx",
where the x's are replaced by the generation index.
Each generation document holds a dictionary containing the generation,
doc_id and transaction_id for the changed document. For each modified
document, a generation document is inserted holding the transaction
metadata.
|
|
|
|
|
|
|
|
|
|
modifying original PR [0] by cristoph to account for the recent
vendoring of l2db code, which means we no longer depend on u1db/dirspec.
I expect the whole mess about the venv setup to be further simplified
pretty soon, since we are going to merge most of the leap.* packages
into a couple of repos.
[0] https://github.com/leapcode/soledad/pull/327
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
SQLCipher database access errors can raise Soledad exceptions. Database access
and multithreading resources are allocated in different places, so we have to
be careful to close all multithreading mechanismis in case of database access
errors. If we don't, zombie threads may haunt the reactor.
This commit adds SQLCipher exception trapping and Soledad exception raising
for database access errors, while properly shutting down multithreading
resources.
|
|
|
|
|
|
|
|
|
|
|
|
- move tests to root directory
- split tests in different subdirectories
- setup a small package with common test dependencies in /testing/test_soledad
- add tox.ini that will:
- install the test_soledad package and other test dependencies
- install soledad common, client, server from the repository
- run tests contianed in /testing/tests directory using pytest
This commit also removes all oauth code from tests, as we have removed the
u1db dependency (by importing it into the repo and naming it l2db) and don't
neet oauth at all right now.
|
|
|
|
|
|
From this moment on, we embed a fork of u1db called l2db.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
This moves the reactor time to the loopingcall period.
This is necessary as the decryption is now deferred to a thread.
The test will exit before the task is executed otherwise.
|
|
It was breaking E126 and E202 before
|
|
|
|
Do not initialize the openssl context on each call to decrypt.
I'm not 100% sure of the causal chain, but it seems that the
initialization of the osrandom engine that openssl backend does might be
breaking havoc when sqlcipher is calling rand_bytes concurrently.
further testing is needed to confirm this is the ultimate cause, but in
my tests this change avoids the occurrence of the dreaded Floating Point
Exception in soledad/sqlcipher.
- Resolves: #8180
|
|
EncryptedSyncTestCase.test_sync_very_large_files is still getting an
excessive amount of memory on very slow machines (specially on old
spinning magnetic disks). This commit checks each doc at a time instead
of getting them all. More refinement is necessary for this test to pass
on any machine.
|
|
|
|
|
