| Age | Commit message (Collapse) | Author |
|---|
|
First step of splitting classes across files on common.
backend.py holds SoledadBackend (generic backend logic)
couch/ is now a directory with old code inside __init__.py and
CouchServerState on state.py
Also removed mock IndexedSoledadBackend, since Soledad does not support
indexing due to encryption on server side.
Also fixed DesignDocUnknownError to show up what is the message of the
original exception. It was being lost.
|
|
CouchDatabase was renamed to SoledadBackend and a new class
CouchDatabase was created to hold all couchdb code. This should make
SoledadBackend less tied to database implementation. A few more
separations are needed to split into modules.
|
|
Needed after the SSL monkeypatch removal at 3b869fb7f
|
|
This was added for a Twisted 12 bug that should be gone by now.
|
|
This parameter applies the ensure_database to all existing databases,
making all of them use the latest design documents.
This can be used to migrate security documents, update handlers or any
other kind of design document.
|
|
|
|
LEAP Platform needs to granularly allow access on user database for
other services, like mx. This is now possible by editing
soledad-server.conf file. A new section 'database-security' was added
and it is parsed during 'create-user-db' to be set on security design
document, present on every per-user database.
|
|
couchdb library relies on garbage collector to close remaining
connections. Somehow, caching the instance is avoiding gc to call
__del__ on underlying couchdb sessions.
|
|
It was 3600s, but closing connections seems to yet depend on garbage
collection and now causes server to leak file handlers. 120s should be
enough to a sync session finish. Also, lowering this value will only
make very long syncs use more of couch every 2 minutes, while raising
this value will keep memory busy for useless time.
|
|
- Releas: 0.8.0
|
|
- Resolves: #7509
|
|
Wheezy has python-couchdb 0.8 and python-beaker 1.6.3.
Pinning them to avoid false positives on tests.
|
|
netrc file was hardcoded inside create-user-db. Now it reads the path
from /etc/leap/soledad-server.conf as done on server process.
The new configuration property is called 'admin_netrc'.
|
|
|
|
README with information about latest change, missing docs and licenses,
variable naming and pep8.
|
|
Added a simple script for user db creation and design docs creation.
It uses a netrc from /etc/couchdb/couchdb-admin.netrc and same validator
used on couch.py for database names.
|
|
We can now use a custom script to create databases by setting a
parameter 'create_cmd' on soledad configuration.
This will set CouchServerState to use it on ensure_database.
|
|
- Releases: 0.8.0
|
|
|
|
As meskio found commented, setting this attribute directly is ugly,
CouchDatabase now has a init_caching method for setting up cache
instance.
|
|
Now each backend object will be retrieved from cache for sync.py and
values will live for 3600 by default. That is changed via parameter if
needed.
|
|
The CouchDB backend implementation was accessing CouchDB too many times
for the same values. Those values are known inside the same sync_id,
which is the id of current sync session.
This commit adds caching for all redundant calls to Couch inside the
same sync_id for each replica.
Refactoring is still needed, but for now couch.py works normally as if
caching is not present, while sync.py injects the cache as a attribute
to enable it. This needs a simpler implementation.
|
|
There are two functions in couch.py used to save and retrieve the last
know gen and trans id for the syncing replica. The get function is
called very often, but is only set on one point. Added a simple caching
to avoid queying couch for a value that we already have.
If cache is empty, it just query as usual and fills it.
|
|
This commit changes sync_state to be in memory, with all tests passing.
The memory variable for now is a dict with each key composed by
source_replica_uid and sync_id, replicating CouchDB implementation. Next
steps includes migrating this to Beaker and refactor/clean up code.
Changed the module's INFO dict to use Beaker's caching and adapted
methods to get and save from it. Still needs refactoring, all tests
passes.
Beaker is now using memory as default; It is configurable, but we aren't
opening the possibility of config now for security. We need to check
what can be misconfigured first.
We are not sure if beaker will be the definitive solution for server
side caching. This change isolates it with more granularity.
In order to replace it, just change get_cache_for to return the proper
caching object using another implementation. This caching object is
supposed to behave as a dict.
|
|
Soledad server will use Beaker as cache provider, starting with
sync_state being in memory.
|
|
|
|
From:
[refactor] removing getters and setters from couch.py
_couch_url was a private variable with getter and setter doing the same
as a public variable. It is accessed all over the code, so being private
with getters and setters didnt make sense. This commit fixes the tests
to also follow this style from now on.
|
|
If we check for a BadRequest after calling meth_put we will end up on a
scenario where the server replies with an error, but everything got
processed.
|
|
|
|
|
|
|
|
|
|
|
|
--use-leap-wheels sets --trusted-host (remove it when we have a proper
cert) and WHEELHOUSE to https://ftp.lizard.leap.se
Until we get ftp.lizard cname, use lizard as the wheels server.
- Related: #7339
|
|
|
|
|
|
generate_wheels uses $WHEELHOUSE to generate and store the wheels for
requirements.pip and requirements-testing.pip (if it exists).
pip_install_requirements.sh installs requirements.pip from them if
possible (if not, then it fetches them from pypi) or, if passed the
--testing flag, it installs requirements-testing.pip.
Related: #7327
|
|
requirements-latest.pip will try to clone and install. Since it is meant
to be latest, I added a small change to specify the branch 'develop'.
|
|
With this, you can setup soledad for using locally
and running the tests with the latest head in a simpler
way
|
|
|
|
|
|
- update pip
- install base reqs, with insecure flags for dirspec and u1db
|
|
|
|
this is part of a process to make the setup of the development mode less
troublesome. from now on, setting up a virtualenv in pure development
mode will be as easy as telling pip to just install the external dependencies::
pip install -r pkg/requirements.pip
and traversing all the leap repos for the needed leap dependencies doing::
python setup.py develop
- Related: #7288
|
|
Once upon a time we needed to pin the PyOpenSSL version to avoid unneeded
crypto deps (see https://leap.se/code/issues/5368#note-5). Since then, jessie
was released and PyOpenSSL 0.14 is now shipped with it. We have removed that
pinning from the debian package, and it is not needed here.
|
|
|
|
soledad-common versions before 0.6.5 do not contain the fix for #6833 and thus
will not work with most recent server. That is why we have to bump this
soledad-server dependency on soledad.common.
|
|
This commit removes some leftover code from a time when Soledad Server used to
check for permissions on certain databases when starting (i.e. shared and
tokens databases). This was later removed as correct permissions enforcement
was relayed to tapicero.
Closes: #6833.
|
|
In the past, we wanted dependency on leap.common to be optional, but now
because of the explicit use of the config path prefix and signaling, we want
to enforce dependency on leap.common.
|
|
Add dependency on twisted for Soledad Client. Also remove minimum twisted
version for Soledad Server because debian stable currently distributes 12.0.0
and pypi currently distributes 15.0.0.
Closes: #6797
|
