summaryrefslogtreecommitdiff
path: root/src/leap/soledad/client/_db
AgeCommit message (Collapse)Author
2018-01-03[style] fix typos and add comments from reviewVictor Shyba
2017-12-31[bug] handle incomplete preamble as a retriable errorVictor Shyba
2017-12-26[doc] add comment to clarify order of resultsdrebs
2017-12-24[refactor] use explicit stream direction argumentVictor Shyba
download was being inferred. It's now being set on each and will raise an error if node is provided. Also removed a duplication on params variable.
2017-12-24[bug] add code to obey protocol pause and stopVictor Shyba
Stream production wasn't pausing or stopping as asked by protocol.
2017-12-22[feature] add a producer protocol for upstreamVictor Shyba
-- Resolves: #8809
2017-12-22[feature] client implementation for upstreamVictor Shyba
-- Resolves #8773
2017-12-22[feature] get_size_list for listing blobs sizesVictor Shyba
2017-12-12[bug] use namespace when uploading blobsdrebs
2017-12-05[bug] transfer maximum allowed number of blobsdrebs
-- Closes: #9004
2017-12-04[bug] escape tag and sizeVictor Shyba
It was generating spaces, causing split to fail sometimes.
2017-12-04[style] fixes from code reviewVictor Shyba
2017-12-01[feature] adds a stream downloaderVictor Shyba
First version, still missing consumer/producer model and some tweaks, but working. -- Related: #8809
2017-12-01[style] fix typoVictor Shyba
2017-11-30[style] remove public get/set blob priority methodsdrebs
2017-11-30[bug] avoid overwriting priorities on sync status batch updatedrebs
2017-11-30[style] some sugar for setting the priorityKali Kaneko
2017-11-30[feature] add priorities for blob transfersdrebs
Closes: #8691
2017-11-13[style] fixes from code reviewVictor Shyba
2017-11-13[feature] support unsynced local_only blobsVictor Shyba
Adds the ability to have document that wont be synced. This enables applications to use soledad to store temporary blobs that should be discarded later instead of unnecessarily keeping the sync loop busy. -- Resolves: #8819
2017-11-13[feature] apply deletions from serverVictor Shyba
Sync method to propagate deletions in batch locally. -- Resolves: #8961
2017-11-13[feature] list deleted files from serverVictor Shyba
-- Related: #8961
2017-11-13[bug] update pending_upload from server listingVictor Shyba
We were doing it for downloads, but not for uploads.
2017-11-10[bug] mark deletion as pending_delete if offlineVictor Shyba
This commit creates a PENDING_DELETE sync status which can be used to keep track of whats deleted locally in order to propagate to server later. -- Related: #8961
2017-11-09[bug] retry on sqlcipher db timeouts for blobsdrebs
Closes: #8981.
2017-11-03[doc] add blobs api docdrebs
2017-10-31[style] wrong/missing docs/comments (code review)Victor Shyba
2017-10-31[feature] add sync progress attributeVictor Shyba
--Resolves: #8848
2017-10-27[bug] there is no retry limit for usual transfersVictor Shyba
Retry limit was originally specified in #8825 as a protection mechanism, but #8822 (retry) doesn't specify a retry limit. In fact, blobs is supposed to retry until transfer is complete using timed delays between attempts, but never giving up. -- Related: #8822 -- Related: #8825
2017-10-27[refactor] semaphore.run instead acquire/releaseVictor Shyba
2017-10-27[refactor] add a table for sync_statusVictor Shyba
As defined in #8970, this table and the new module will ease adding sync features such as priority queues and streaming. --Resolves: #8970
2017-10-27[style] E722 do not use bare exceptVictor Shyba
2017-10-27[refactor] split blobs into modulesVictor Shyba
So we can have manager, sync, sql and errors in its own places. --Related: #8970
2017-10-12[bug] limit number of concurrent requests to local dbdrebs
When running stress tests on blobs storage, we get weird errors when hundreds of requests are made concurrently to the sqlite backend. This commit adds a limit so only 10 requests will be delivered to the backend at a time.
2017-10-05[bug] fix retries for blobs downloaddrebs
- add a MaximumRetriesError exception to encapsulate other exceptions. - record the pending status before trying to download - modify update_sync_status to insert or update - modify retry tests to check number of retries - add a test for download retry limit
2017-10-05[bug] retry blob download for all retriable errorsdrebs
Because the exception catching was being made inside _download_and_decrypt() and only accounted for InvalidBlob exceptions, not all retriable errors would lead to an actual retry. This commit moves the exception catching to one level up and catches any kind of exception, as is done in the upload part. This allows for retrying on all retriable errors.
2017-10-05[bug] improve error message on blob download errordrebs
The previous error message had some problems: - the connection should not be a problem, as this is going over TCP. If the HTTP request was succesful, there's no reason to think its contents could have been corrupted by a connection problem. - I am not sure what's the best communication strategy here, but the real problem is either a bug or actual tampering, so i make this explicit. - A problem like this should be reported always, not only when the problem persists.
2017-10-05[bug] log the exception on blob download errordrebs
2017-10-05[bug] don't use hardcoded number of retries when downloading blobsdrebs
2017-10-05[style] use python3 compatible try/except syntaxdrebs
2017-10-05[bug] ensure maximum concurrency on blobs transferdrebs
The way in that concurrency limit was being enforced was such that transfer attempts were being spawned in groups of 3, and all of them had to finish before a new group could be spawned. This modification allows for use of maximum concurrency level at all times.
2017-10-05[style] param is not keyword argumentdrebs
2017-10-05[style] name keyword arguments in function callsdrebs
2017-10-05[style] rename exception to match standardsdrebs
We have been using "Error" instead of "Exception" in exception names, so this commit is only enforcing an unwritten policy.
2017-10-05[feature] notify, retry and fail from invalid tagVictor Shyba
Notify, log something meaninful and retry at most 3 times before marking the download as unusable (FAILED_DOWNLOAD). -- Related: #8825
2017-10-05[feature] retry during upload + proper waitVictor Shyba
Added retry to upload and modified retry implementation to comply with discussed spec. According to it, we should wait between retries, something like 1s, 10s, .. up to 1 minute. -- Resolves: #8822
2017-10-05[feature] retry during downloadVictor Shyba
-- Related: #8822
2017-10-05[feature] send/fetch missing using local statusesVictor Shyba
Instead of querying the server, fetch_missing and send_missing now uses the PENDING_DOWNLOAD and PENDING_UPLOAD statuses to guide itself on what to do. This allows the sync mechanism to control when/how to query data from server and reuse the query data during the sync. -- Related: #8822
2017-10-05[feature] blob get/put handle unavailable statusesVictor Shyba
PENDING_DOWNLOAD is an empty blob, so during blob_manager.get we need to return empty as it's not available. This status is used during sync. During put, if we have an empty unavailable blob, then we delete and replace with is being put, marking it as SYNCED. -- Related: #8822
2017-10-05[feature] persist pending_download remote listingVictor Shyba
-- Related: #8822