summaryrefslogtreecommitdiff
path: root/server
AgeCommit message (Collapse)Author
2015-12-03[feat] set default to False on batching for nowVictor Shyba
All batching code has no effect by default with this commit. Since we know that this is a dangerous new feature we will enable them only on our test servers and check them manually before setting it as default or adding more configuration features. Use SyncTarget and server conf file to enable it for testing.
2015-12-03[feat] add configuration to disable batchingVictor Shyba
Batch support is optional. This commit adds a 'batching' configuration option to disable it.
2015-12-03[feat] prepare server to handle batchesVictor Shyba
Created two methods on the backend to start and finish a batch. A dict of callbacks is available to defer actions for the last document, allowing temporary (changing often) metadata to be recorded only once. Using those methods we will also be able to put all docs in one go on the CouchDatabase implementation, but that is another step.
2015-11-03[refactor] resource logic encapsulationVictor Shyba
Creating a resource from a path to use get_json causes a lot of dirty code and unexplained things like response[2]. This commit extracts that logic into a helper to let it more clear about what is happening.
2015-11-03[refactor] token verification moved to couch module + testsVictor Shyba
Added tests for this token verification as it wasn't covered. Then moved it to the new couch module that implements a couch storage. The ServerState was chosen to hold the verify_token method. CouchServerState holds the current implementation, which is called on authentication middleware as the new test shows.
2015-11-03[refactor] split out backend from couch databaseVictor Shyba
First step of splitting classes across files on common. backend.py holds SoledadBackend (generic backend logic) couch/ is now a directory with old code inside __init__.py and CouchServerState on state.py Also removed mock IndexedSoledadBackend, since Soledad does not support indexing due to encryption on server side. Also fixed DesignDocUnknownError to show up what is the message of the original exception. It was being lost.
2015-11-03[refactor] separate SoledadBackend from CouchDatabaseVictor Shyba
CouchDatabase was renamed to SoledadBackend and a new class CouchDatabase was created to hold all couchdb code. This should make SoledadBackend less tied to database implementation. A few more separations are needed to split into modules.
2015-10-30[feat] bump twisted dependency to 12.3.0Ruben Pollan
Needed after the SSL monkeypatch removal at 3b869fb7f
2015-10-29[feat] remove SSL tsafe monkeypatchVictor Shyba
This was added for a Twisted 12 bug that should be gone by now.
2015-10-29[feat] adds --migrate-all to create-user-db scriptVictor Shyba
This parameter applies the ensure_database to all existing databases, making all of them use the latest design documents. This can be used to migrate security documents, update handlers or any other kind of design document.
2015-10-28[style] fix pep8 warnigsFolker Bernitt
2015-10-28[feat] read security doc from configurationVictor Shyba
LEAP Platform needs to granularly allow access on user database for other services, like mx. This is now possible by editing soledad-server.conf file. A new section 'database-security' was added and it is parsed during 'create-user-db' to be set on security design document, present on every per-user database.
2015-10-19[bug] remove instance cachingVictor Shyba
couchdb library relies on garbage collector to close remaining connections. Somehow, caching the instance is avoiding gc to call __del__ on underlying couchdb sessions.
2015-10-14[bug] reduce overall sync session caching to 120sVictor Shyba
It was 3600s, but closing connections seems to yet depend on garbage collection and now causes server to leak file handlers. 120s should be enough to a sync session finish. Also, lowering this value will only make very long syncs use more of couch every 2 minutes, while raising this value will keep memory busy for useless time.
2015-10-05Merge branch 'pixelated/develop' into developRuben Pollan
- Releas: 0.8.0
2015-10-05[feat] Move config dir to /etc/soledadvarac
- Resolves: #7509
2015-10-02[pkg] pin beaker and couchdb for wheezyVictor Shyba
Wheezy has python-couchdb 0.8 and python-beaker 1.6.3. Pinning them to avoid false positives on tests.
2015-10-01[feat] read netrc path from configuration fileVictor Shyba
netrc file was hardcoded inside create-user-db. Now it reads the path from /etc/leap/soledad-server.conf as done on server process. The new configuration property is called 'admin_netrc'.
2015-09-28[style] pep8Victor Shyba
2015-09-28[refactor] kaliy's review and pep8 fixesVictor Shyba
README with information about latest change, missing docs and licenses, variable naming and pep8.
2015-09-28[feat] script for user db creationVictor Shyba
Added a simple script for user db creation and design docs creation. It uses a netrc from /etc/couchdb/couchdb-admin.netrc and same validator used on couch.py for database names.
2015-09-28[feat] conf for enabling db creation via custom shVictor Shyba
We can now use a custom script to create databases by setting a parameter 'create_cmd' on soledad configuration. This will set CouchServerState to use it on ensure_database.
2015-09-27Merge branch 'feature/sync_state_in_memory' into developRuben Pollan
- Releases: 0.8.0
2015-09-24[style] pep8Victor Shyba
2015-09-24[refactor] init_caching instead of setting attrVictor Shyba
As meskio found commented, setting this attribute directly is ugly, CouchDatabase now has a init_caching method for setting up cache instance.
2015-09-24[feat] adds cache expirationVictor Shyba
Now each backend object will be retrieved from cache for sync.py and values will live for 3600 by default. That is changed via parameter if needed.
2015-09-24[feat] full caching for each sync_id sessionVictor Shyba
The CouchDB backend implementation was accessing CouchDB too many times for the same values. Those values are known inside the same sync_id, which is the id of current sync session. This commit adds caching for all redundant calls to Couch inside the same sync_id for each replica. Refactoring is still needed, but for now couch.py works normally as if caching is not present, while sync.py injects the cache as a attribute to enable it. This needs a simpler implementation.
2015-09-24[feat] adds caching for other gen and trans idVictor Shyba
There are two functions in couch.py used to save and retrieve the last know gen and trans id for the syncing replica. The get function is called very often, but is only set on one point. Added a simple caching to avoid queying couch for a value that we already have. If cache is empty, it just query as usual and fills it.
2015-09-24[feat] first draft of sync_state in memoryVictor Shyba
This commit changes sync_state to be in memory, with all tests passing. The memory variable for now is a dict with each key composed by source_replica_uid and sync_id, replicating CouchDB implementation. Next steps includes migrating this to Beaker and refactor/clean up code. Changed the module's INFO dict to use Beaker's caching and adapted methods to get and save from it. Still needs refactoring, all tests passes. Beaker is now using memory as default; It is configurable, but we aren't opening the possibility of config now for security. We need to check what can be misconfigured first. We are not sure if beaker will be the definitive solution for server side caching. This change isolates it with more granularity. In order to replace it, just change get_cache_for to return the proper caching object using another implementation. This caching object is supposed to behave as a dict.
2015-09-24[pkg] add Beaker as a server dependencyVictor Shyba
Soledad server will use Beaker as cache provider, starting with sync_state being in memory.
2015-09-21[pkg] fold in changes0.7.3Ivan Alejandro
2015-08-27[tests] fix variable name from refactorVictor Shyba
From: [refactor] removing getters and setters from couch.py _couch_url was a private variable with getter and setter doing the same as a public variable. It is accessed all over the code, so being private with getters and setters didnt make sense. This commit fixes the tests to also follow this style from now on.
2015-08-26[bug] process put after last BadRequest checkVictor Shyba
If we check for a BadRequest after calling meth_put we will end up on a scenario where the server replies with an error, but everything got processed.
2015-08-26[pkg] fold in changes0.7.2Ivan Alejandro
2015-08-24[refactor] add changes file about simplejson dep removaldrebs
2015-08-24[refactor] remove simplejson dep on serverdrebs
2015-08-19[style] pep8 cleanupsKali Kaneko
2015-08-17[bug] add missing parameter to unauthorized errordrebs
2015-08-06[feat] WHEELHOUSE can be a url + --use-leap-wheelsParménides GV
--use-leap-wheels sets --trusted-host (remove it when we have a proper cert) and WHEELHOUSE to https://ftp.lizard.leap.se Until we get ftp.lizard cname, use lizard as the wheels server. - Related: #7339
2015-08-04[style] pep8 cleanupKali Kaneko
2015-08-04[style] added pep8 excludes+ignoresKali Kaneko
2015-08-03[feat] use wheels to install dependenciesParménides GV
generate_wheels uses $WHEELHOUSE to generate and store the wheels for requirements.pip and requirements-testing.pip (if it exists). pip_install_requirements.sh installs requirements.pip from them if possible (if not, then it fetches them from pypi) or, if passed the --testing flag, it installs requirements-testing.pip. Related: #7327
2015-07-29[bug] specify develop branch when using -eVictor Shyba
requirements-latest.pip will try to clone and install. Since it is meant to be latest, I added a small change to specify the branch 'develop'.
2015-07-29[feat] Added requirements-latest pip fileBruno Wagner
With this, you can setup soledad for using locally and running the tests with the latest head in a simpler way
2015-07-28[tests] add pep8 to requirements-testingKali Kaneko
2015-07-27[pkg] add AUTHORS file + one-liner to generate itKali Kaneko
2015-07-27[pkg] add script to install base requirementsKali Kaneko
- update pip - install base reqs, with insecure flags for dirspec and u1db
2015-07-24[style] fixed pep8 warnings on the soledad server codeBruno Wagner
2015-07-22[pkg] separate leap requirementsKali Kaneko
this is part of a process to make the setup of the development mode less troublesome. from now on, setting up a virtualenv in pure development mode will be as easy as telling pip to just install the external dependencies:: pip install -r pkg/requirements.pip and traversing all the leap repos for the needed leap dependencies doing:: python setup.py develop - Related: #7288
2015-06-09[pkg] remove PyOpenSSL version pinningdrebs
Once upon a time we needed to pin the PyOpenSSL version to avoid unneeded crypto deps (see https://leap.se/code/issues/5368#note-5). Since then, jessie was released and PyOpenSSL 0.14 is now shipped with it. We have removed that pinning from the debian package, and it is not needed here.