summaryrefslogtreecommitdiff
path: root/src/leap/soledad/client
AgeCommit message (Collapse)Author
2017-12-12[bug] use namespace when uploading blobsdrebs
2017-12-05[bug] transfer maximum allowed number of blobsdrebs
-- Closes: #9004
2017-12-04[bug] escape tag and sizeVictor Shyba
It was generating spaces, causing split to fail sometimes.
2017-12-04[style] fixes from code reviewVictor Shyba
2017-12-01[feature] adds a stream downloaderVictor Shyba
First version, still missing consumer/producer model and some tweaks, but working. -- Related: #8809
2017-12-01[style] fix typoVictor Shyba
2017-11-30[style] remove public get/set blob priority methodsdrebs
2017-11-30[bug] avoid overwriting priorities on sync status batch updatedrebs
2017-11-30[style] some sugar for setting the priorityKali Kaneko
2017-11-30[feature] add priorities for blob transfersdrebs
Closes: #8691
2017-11-29[feature] add bandwidth throttling to the clientdrebs
Intercept the creation of the protocol factory in the HTTP connection pool to use twisted.protocols.policies.ThrottlingFactory and control the incoming and outgoing bandwidth. The factory only controls one connection, so when throttling we limit the number of connections of the pool to one per host. This way, throttling happens in a per-host basis. Closes: #8931
2017-11-13[style] fixes from code reviewVictor Shyba
2017-11-13[feature] support unsynced local_only blobsVictor Shyba
Adds the ability to have document that wont be synced. This enables applications to use soledad to store temporary blobs that should be discarded later instead of unnecessarily keeping the sync loop busy. -- Resolves: #8819
2017-11-13[feature] apply deletions from serverVictor Shyba
Sync method to propagate deletions in batch locally. -- Resolves: #8961
2017-11-13[feature] list deleted files from serverVictor Shyba
-- Related: #8961
2017-11-13[bug] update pending_upload from server listingVictor Shyba
We were doing it for downloads, but not for uploads.
2017-11-10[bug] mark deletion as pending_delete if offlineVictor Shyba
This commit creates a PENDING_DELETE sync status which can be used to keep track of whats deleted locally in order to propagate to server later. -- Related: #8961
2017-11-09[bug] retry on sqlcipher db timeouts for blobsdrebs
Closes: #8981.
2017-11-03[doc] add blobs api docdrebs
2017-10-31[style] wrong/missing docs/comments (code review)Victor Shyba
2017-10-31[feature] add sync progress attributeVictor Shyba
--Resolves: #8848
2017-10-27[bug] there is no retry limit for usual transfersVictor Shyba
Retry limit was originally specified in #8825 as a protection mechanism, but #8822 (retry) doesn't specify a retry limit. In fact, blobs is supposed to retry until transfer is complete using timed delays between attempts, but never giving up. -- Related: #8822 -- Related: #8825
2017-10-27[refactor] semaphore.run instead acquire/releaseVictor Shyba
2017-10-27[refactor] add a table for sync_statusVictor Shyba
As defined in #8970, this table and the new module will ease adding sync features such as priority queues and streaming. --Resolves: #8970
2017-10-27[style] E722 do not use bare exceptVictor Shyba
2017-10-27[refactor] split blobs into modulesVictor Shyba
So we can have manager, sync, sql and errors in its own places. --Related: #8970
2017-10-12[bug] limit number of concurrent requests to local dbdrebs
When running stress tests on blobs storage, we get weird errors when hundreds of requests are made concurrently to the sqlite backend. This commit adds a limit so only 10 requests will be delivered to the backend at a time.
2017-10-05[bug] fix retries for blobs downloaddrebs
- add a MaximumRetriesError exception to encapsulate other exceptions. - record the pending status before trying to download - modify update_sync_status to insert or update - modify retry tests to check number of retries - add a test for download retry limit
2017-10-05[bug] retry blob download for all retriable errorsdrebs
Because the exception catching was being made inside _download_and_decrypt() and only accounted for InvalidBlob exceptions, not all retriable errors would lead to an actual retry. This commit moves the exception catching to one level up and catches any kind of exception, as is done in the upload part. This allows for retrying on all retriable errors.
2017-10-05[bug] improve error message on blob download errordrebs
The previous error message had some problems: - the connection should not be a problem, as this is going over TCP. If the HTTP request was succesful, there's no reason to think its contents could have been corrupted by a connection problem. - I am not sure what's the best communication strategy here, but the real problem is either a bug or actual tampering, so i make this explicit. - A problem like this should be reported always, not only when the problem persists.
2017-10-05[bug] log the exception on blob download errordrebs
2017-10-05[bug] don't use hardcoded number of retries when downloading blobsdrebs
2017-10-05[style] use python3 compatible try/except syntaxdrebs
2017-10-05[bug] ensure maximum concurrency on blobs transferdrebs
The way in that concurrency limit was being enforced was such that transfer attempts were being spawned in groups of 3, and all of them had to finish before a new group could be spawned. This modification allows for use of maximum concurrency level at all times.
2017-10-05[style] param is not keyword argumentdrebs
2017-10-05[style] name keyword arguments in function callsdrebs
2017-10-05[style] rename exception to match standardsdrebs
We have been using "Error" instead of "Exception" in exception names, so this commit is only enforcing an unwritten policy.
2017-10-05[bug] set as PROCESSING during processing flowVictor Shyba
It was previously setting to PROCESSED. Also added some tests to check if the underlying wrapped calls matches the intent. -- Resolves: #8955
2017-10-05[feature] notify, retry and fail from invalid tagVictor Shyba
Notify, log something meaninful and retry at most 3 times before marking the download as unusable (FAILED_DOWNLOAD). -- Related: #8825
2017-10-05[feature] retry during upload + proper waitVictor Shyba
Added retry to upload and modified retry implementation to comply with discussed spec. According to it, we should wait between retries, something like 1s, 10s, .. up to 1 minute. -- Resolves: #8822
2017-10-05[feature] retry during downloadVictor Shyba
-- Related: #8822
2017-10-05[feature] send/fetch missing using local statusesVictor Shyba
Instead of querying the server, fetch_missing and send_missing now uses the PENDING_DOWNLOAD and PENDING_UPLOAD statuses to guide itself on what to do. This allows the sync mechanism to control when/how to query data from server and reuse the query data during the sync. -- Related: #8822
2017-10-05[feature] blob get/put handle unavailable statusesVictor Shyba
PENDING_DOWNLOAD is an empty blob, so during blob_manager.get we need to return empty as it's not available. This status is used during sync. During put, if we have an empty unavailable blob, then we delete and replace with is being put, marking it as SYNCED. -- Related: #8822
2017-10-05[feature] persist pending_download remote listingVictor Shyba
-- Related: #8822
2017-10-05[feature] filter out unavailable blobs on listingVictor Shyba
-- Related: #8822
2017-10-05[feature] concurrent blob download/uploadVictor Shyba
-- Related: #8932
2017-09-30[bug] fix argument passing in blobs queriesdrebs
2017-09-29[refactor] make parameters of blobmanager methods explicitdrebs
2017-09-29[bug] check all http response status codesdrebs
2017-09-29[bug] raise when trying to get flags of unexisting blobdrebs