| Age | Commit message (Collapse) | Author |
|---|
|
errors.py was holding a few specific CouchDB errors, now moved into
couch.errors module. Also, some of CouchDatabase methods were declared
as private, but external classes needs them.
|
|
As SoledadBackend is intended to be database agnostic, a new generic
document is now used instead of the old one made for CouchDB. The only
attribute that really relates to couch was couch_rev, removed on this
commit as it can be set on CouchDatabase implementation when needed.
|
|
First step of splitting classes across files on common.
backend.py holds SoledadBackend (generic backend logic)
couch/ is now a directory with old code inside __init__.py and
CouchServerState on state.py
Also removed mock IndexedSoledadBackend, since Soledad does not support
indexing due to encryption on server side.
Also fixed DesignDocUnknownError to show up what is the message of the
original exception. It was being lost.
|
|
CouchDatabase was renamed to SoledadBackend and a new class
CouchDatabase was created to hold all couchdb code. This should make
SoledadBackend less tied to database implementation. A few more
separations are needed to split into modules.
|
|
This monkey patch was added to separate headers from content and use
them to PUT on couchdb. The original implementation makes it possible by
using two parameters in the constructor.
|
|
CouchDocument is a subclass from SoledadDocument which holds conflicts,
making the server side aware of them. There is a module called document
on soledad.common that holds SoledadDocument. This commit moves
CouchDocument to its proper place.
Error raising logic and exceptions declarations are also moved into error
module inside common.
|
|
This parameter applies the ensure_database to all existing databases,
making all of them use the latest design documents.
This can be used to migrate security documents, update handlers or any
other kind of design document.
|
|
|
|
LEAP Platform needs to granularly allow access on user database for
other services, like mx. This is now possible by editing
soledad-server.conf file. A new section 'database-security' was added
and it is parsed during 'create-user-db' to be set on security design
document, present on every per-user database.
|
|
ensure_ddocs is a privileged operation. The code was defaulting to True,
which caused unprivileged code to fail. This commit changes it to False,
forcing you to check your privileges and declare a new argument when
calling in order to ensure that this behavior is only supposed to happen
on privileged parts.
|
|
Wheezy is still at 0.8 and it is yet supported.
This commit changes all necessary calls from python-couchdb 1.0 back to
python-couchdb 0.8. We can migrate this back to simpler implementation
with python-couchdb 1.0 when support for wheezy is dropped.
|
|
|
|
Those hardcoded mocks are leaking into other tests and are unnecessary.
|
|
README with information about latest change, missing docs and licenses,
variable naming and pep8.
|
|
As the other tests does. Make sure that a fresh database gets proper
security doc after calling ensure_security method.
|
|
Beyond ensuring ddocs, it is also necessary to ensure _security doc
presence while creating a database.
This document will tell couchdb to grant access to 'soledad' user as a
member role and no one as admin.
|
|
ensure database needs to return a db and its replica_uid. Updated tests,
doc and code to reflect that.
|
|
Tests that Unauthorized is raised in any failure scenario, leaving user
blind for tips on what happened during execution. This should lower
chances of information disclosure on execution failure.
|
|
If CouchServerState is created with a create_cmd parameter, it can now
use this parameter to invoke a command to create databases. A validator
for database name is also used to ensure that command injection is not
possible if user manages to manipulate database name argument.
|
|
Checks if arguments validation occurs properly and command execution
brings back status code and stdout or stderr on some scenarios.
|
|
This commit adds a way to validate and execute commands using an
argument validator. Commands are executed via subprocess.
|
|
|
|
As meskio found commented, setting this attribute directly is ugly,
CouchDatabase now has a init_caching method for setting up cache
instance.
|
|
We use CouchDB with single doc read/write. Following this documentation
about performance, we should get more performance by enabling couch to
delay and commit later.
See: http://guide.couchdb.org/draft/performance.html#single
|
|
Now each backend object will be retrieved from cache for sync.py and
values will live for 3600 by default. That is changed via parameter if
needed.
|
|
Before this change, we used a complicated update handler for storing the sync
state on the couchdb backend. That update handler was implemented as an
attempt to make couchdb take care of some validation for the update of the
sync log during the sync exchange, mainly to allow concurrent received
documents insertion during a sync.
Right now we rely on the remote sending one document at a time and do not
support concurrent insertions in the remote database backed by couch. Because
of that, the code removed by this commit was unneeded. And more: it was a
bottleneck of the sync process because we were writing to an unique file and
using unnecessary couch design docs processing for that. So this commit both
simplifies the storage of remote sync and removes a bottleneck of the sync
process.
Conflicts:
common/src/leap/soledad/common/couch.py
common/src/leap/soledad/common/tests/test_couch.py
|
|
The CouchDB backend implementation was accessing CouchDB too many times
for the same values. Those values are known inside the same sync_id,
which is the id of current sync session.
This commit adds caching for all redundant calls to Couch inside the
same sync_id for each replica.
Refactoring is still needed, but for now couch.py works normally as if
caching is not present, while sync.py injects the cache as a attribute
to enable it. This needs a simpler implementation.
|
|
There are two functions in couch.py used to save and retrieve the last
know gen and trans id for the syncing replica. The get function is
called very often, but is only set on one point. Added a simple caching
to avoid queying couch for a value that we already have.
If cache is empty, it just query as usual and fills it.
|
|
Python has a native ThreadPool implementation that fits our needs.
Changing it to use this instead and making some calls simpler.
|
|
_put_doc_if_newer is implemented on CommonBackend already. This was
copied over to CouchBackend just to add ensure conflicts. We can do this
before calling the super method instead.
|
|
This tests the previous fix on ensuring a db that is missing a doc other
than 'docs'.
|
|
This code only checks for 'docs' presence, while we have 3 design
documents. If one of them is missing, but 'docs' is not, then it will
not ensure the others.
This is needed to properly ensure ddocs on create command line script.
|
|
|
|
|
|
This was used during db isolation to make sure that everything created
was destroyed, but it fails with -j (multiprocess). Removing it allows
parallelism.
|
|
|
|
Removing unused code as this test case does not need a server.
|
|
Big tearDown logic can be replaced by a simple addCleanup. Also remove
unused imports and fix a small typo on a database cleanup check.
|
|
This test only defines a set of different scenarios, all other methods
are the same as this subclass.
|
|
self.db3 is closed on tearDown. This test was creating it as a local
variable, making close possibly fail.
|
|
It was hardcoded for 'test', but the database name is now random. What
is useful for test code is the replica_uid, database name for the
SyncTarget is now coming from database name.
|
|
'shared' has to be used as a DB name just because of a constant, but it
is used on only one point. This changes mock this point to have unique
names for better tests isolation. 'tokens' was removed as unnecessary.
|
|
This was a duplicate, but also was getting on the way to improve
isolation. With this small refactor it should be cleaner and have unique
names.
|
|
Test case has a addCleanup method, which provides a way to clean
resources up and express this need as soon as you create. We are now
using it to simplify some logic on database deletion during the test and
to make sure that as soon as it gets created a cleanup is there to
delete after.
|
|
We are using a single CouchDB install, which may cause tests to overlap
since many of them uses the same database name, hurting isolation. This
change tries to use uuid on most of it. Also changes for couch_url and
couch_port introduced by removal of CouchDB process.
|
|
This process per test gives a lot of headache, this is why we are
removing it. With it we would need to try to start and stop properly on
each test case. This fails badly when a test fail and, depending on how
it fails, it freezes my pc. Also, it is very heavy for a CI to run a
database process for each test case.
|
|
|
|
From:
[refactor] removing getters and setters from couch.py
_couch_url was a private variable with getter and setter doing the same
as a public variable. It is accessed all over the code, so being private
with getters and setters didnt make sense. This commit fixes the tests
to also follow this style from now on.
|
|
isinstance is better, as kaliy pointed out, and the constructor is also
in a safer place on __init__.py to be explicit.
Also re-apply a change from last rebase;
|
|
* file headers
* variable names
* missing docstrings
* prune_conflicts
** extra: tests failed on a 1-based index bug
|
