summaryrefslogtreecommitdiff
AgeCommit message (Collapse)Author
2017-04-26[feature] sanitize resource argumentsVictor Shyba
_validate will check if user_id and blob_id are letter, numbers, dashes and underscores. It is called on render_GET and render_POST, validating incoming arguments before handling on backend. - Resolves: #8832
2017-04-26[feature] use Twisted getProcessOutput on backendVictor Shyba
This makes process communication async during quota measurement, as specified on #8832 - Related: #8832
2017-04-25[test] ensure rev system works in shared dbdrebs
During a meeting with pixelated we were asked to ensure that the u1db revision system was working for the shared database. This commit adds a test to make sure that documents in the shared database can be updated successfully if the include the correct revision, but will fail if the include the incorrect revision. Resolves: #8842.
2017-04-24[bug] fix improper re-raising of exceptionKali Kaneko
it was passing way too many arguments in py2. - Resolves: #8843
2017-04-19[test] tune parameters for asynchronous sqlcipher accessdrebs
When hundreds of concurrent document creations are spawned during tests, test runs in slower machines may fail with "db is locked" error. Increasing the timeout and number of retries in the sqlcipher adbapi level may help avoiding that.
2017-04-19[test] remove docker scripts from this repodrebs
Docker scripts are only used for CI and do not need to be in this repository. Beause of that, we decided to moved the docker scripts to a private repository where dockerfiles for other parts of leap also live.
2017-04-19[test] use soledad image from 0xacab registry in cidrebs
2017-04-19[test] avoid running sqlcipher synchonous tests when benchmarkingdrebs
SQLCipher synchronous benchmark tests were introduced when we started developing benchmark tests to compare synchronous and asynchronous code. Synchronous access to sqlcipher database is not used in soledad, and those tests are much slower than asynchronous tests (more than 10 times using ssd drive), so we want to avoid running them on ci. This commit introduces a "synchronous" marker and avoid running tests markes as such in ci environment.
2017-04-19[test] use latest pysqlcipher-benchmark for benchmarksdrebs
There were some changes needed in pytest-benchmark so we could successfully use it for soledad benchmarks graphing: https://github.com/ionelmc/pytest-benchmark/pull/73 https://github.com/ionelmc/pytest-benchmark/pull/74 https://github.com/ionelmc/pytest-benchmark/pull/75 The contributions were accepted but not released yet, so this commit uses the code from upstream git repository's master branch.
2017-04-19[tests] dont mock local db on local testsVictor Shyba
2017-04-19[tests] set pool to None during testsVictor Shyba
2017-04-19[test] change assert for self.assertXVictor Shyba
2017-04-19[bug] return empty local_list instead of NoneVictor Shyba
2017-04-19[bug] local.put closes fd, return from local.get insteadVictor Shyba
2017-04-19[bug] adapt blob size to fake size from #8758Victor Shyba
Issue #8758 rounded doc size considering we would add some kind of padding to the document. Since we arent removing the pad, this commit makes the client discover the size from the decrypted amount. Proper padding and strip should be done in the future. - Related: #8758 - Related: #8814
2017-04-19[feature] fetch new blobs from serverVictor Shyba
Check what server has, what we have, compare, fetch missing. - Related: #8808
2017-04-19[test] change assert for self.assertXVictor Shyba
2017-04-19[feature] add send_missing to cliVictor Shyba
- Related: #8807
2017-04-19[feature] add send_missing to send pending uploadsVictor Shyba
This method will gather a list of local docs and a list of remote docs, compare them and send docs which server doesn't have. - Related: #8807
2017-04-15[test] enforce bundled pysqlcipher in benchmark testsdrebs
The debian package for libsqlcipher does not enable the use of usleep(), and that might cause timeouts when doing concurrent access to sqlcipher databases. See: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=859685 In benchmark tests we use twisted's asynchronous dbapi for creating thousands of documents, and if usleep() is not enabled this operation is very likelly to fail with a "db is locked" error. This commit adds a workaround for benchmark tests by reinstalling pysqlcipher with the "--bundled" option, what causes libsqlcipher to be built and bundled in the python package, with usleep() enabled. Resolves: #8835
2017-04-10[pkg] do not pass weird information to install_requiresKali Kaneko
this commit deliberately *duplicates* the dependency information between the setup.py and requirements.pip. The rationale behind it is that one of them is the generic dependency information, to be passed to system helpers in order to package it. The requirements.pip has any pinning information that we want to impose during development, and it behaves as a well-known set. Read https://caremad.io/posts/2013/07/setup-vs-requirement/ for more explanations. - Resolves: #8830
2017-04-10[refactor] use fixed rev and remove doc_id (blobs)Victor Shyba
doc_id wasn't being stored, loaded or used anywhere except for authentication. This commit removes it and uses a fixed value for rev since blobs is supposed to be immutable. - Resolves: #8813
2017-04-05Use $DOCKER_BUILD_OPTIONS for buildingvarac
2017-04-05Use soledad master branch, install curl and gitvarac
2017-04-04[feat] add the host hostname to benchmark machine infodrebs
2017-04-04[test] change assert for self.assertXVictor Shyba
2017-04-04[docs] add docstrings to _get_rounded_sizeVictor Shyba
2017-04-04[bug] enable '/blobs/{uuid}' for GET on url_mapperVictor Shyba
2017-04-04[style] fix wrong identationVictor Shyba
2017-04-04[bug] use soledad_assert instead of assertVictor Shyba
2017-04-04[docs] improve docstrings and comments from reviewVictor Shyba
2017-04-04[style] naming improvementsVictor Shyba
2017-04-04[feature] add list to _blobs cliVictor Shyba
2017-04-04[refactor] list is remote_list, avoiding confusionVictor Shyba
2017-04-04[feature] add listing to local blob dbVictor Shyba
2017-04-04[feature] add blobs listingVictor Shyba
2017-04-04[refactor] adds a PreamblePipe for preamble downloadVictor Shyba
Downloading until there is a space then splitting the content was a mess. Extracted this behaviour out of DecrypterBuffer into a new component so it eases testing by introducing a single responsibility class.
2017-04-04[refactor] extracts tail logic into TruncatedTailPipeVictor Shyba
2017-04-04[feature] use 409 status code for existing blob idVictor Shyba
Raising was generating 500, which is a generic status code for server side errors. This commit adds proper status code of 409 while handling the error on client side by translating the code into a proper exception class.
2017-04-04[refactor] simplify HTTPClientVictor Shyba
2017-04-04[test] fix treq -> _client changeVictor Shyba
2017-04-04[bug] cache pip instead of toxVictor Shyba
Caching .tox breaks when installing new dependencies. Caching pip instead and asking tox to always recreate will ensure newer dependencies are installed while getting existing ones from cache.
2017-04-04[refactor] change size into a required parameter where it is necessaryVictor Shyba
2017-04-04[bug] copy returns a closed handleVictor Shyba
copy call returns a closed file handler, instead we can query for the blob after insertion and use the returned file handle to be consumed for upload. A better solution would be to "pipe" the writes into the database into the upload, but that involves solving a larger set of issues to be done later.
2017-04-04[test] test blobs local db with file handlersVictor Shyba
2017-04-04[test] test get inexistent on local sqlcipher for blobs clientVictor Shyba
2017-04-04[feature] adds a close method for local blobs storeVictor Shyba
2017-04-04[test] upload/download integration testVictor Shyba
2017-04-04[test] use mock instead of directly patchingVictor Shyba
2017-04-04[bug] create dirs before checking disk usageVictor Shyba