summaryrefslogtreecommitdiff
diff options
context:
space:
mode:
authorVictor Shyba <victor1984@riseup.net>2018-01-08 16:40:17 -0300
committerdrebs <drebs@leap.se>2018-01-21 15:16:26 -0200
commite6731dea65acd3ff1f7d6c9e9187350b32f20b82 (patch)
tree8e2eb66ed6dde78b31d506177cbece30b43bd253
parente5ce16c79fa744d1821127686b4d7db9d2db98b0 (diff)
[doc] add stream format to docs
- Relates: #9003
-rw-r--r--docs/reference/blobs/sync.rst43
1 files changed, 43 insertions, 0 deletions
diff --git a/docs/reference/blobs/sync.rst b/docs/reference/blobs/sync.rst
index 6c80c8f7..d9b94d2a 100644
--- a/docs/reference/blobs/sync.rst
+++ b/docs/reference/blobs/sync.rst
@@ -78,3 +78,46 @@ increasing time interval to minimize competition for resources used by other
concurrent transfer attempts. The interaval starts at 10 seconds and increases
to 20, 30, 40, 50, and finally 60 seconds. All further retries will be
separated by a 60 seconds time interval.
+
+Streaming
+=========
+
+Streaming is a method of synchronization optimized for small payloads, it
+transfers multiple small blobs in a single stream of data. This method improves
+resource usage, specially in scenarios which blobs are so small that opening
+one connection for each would generate a noticeable overhead.
+
+Downstream
+----------
+
+During download, client provides a list of blobs by POSTing a JSON formatted
+list of Blob identifiers. Then the server starts producing a stream of data in
+which each of the requested Blobs is written using the following format:
+
+
+* Blob Size in hex (padded to 8 bytes)
+* base64 encoded AES-GCM 16 byte tag
+* Space (separator)
+* Blob content
+
+Upstream
+--------
+
+During upload, the client will produce a stream of Blobs in the following
+format:
+
+* First line: JSON encoded list of tuples with each Blob identifier and size.
+ Note that the size is the encrypted Blob size, which matches exactly what the
+ client is sending on the stream.
+* Encrypted Blob content, for each Blob in the upstream list.
+
+Server endpoint specification
+-----------------------------
+
+Endpoint: /stream/{user-uuid}/
+Method: POST
+
+Query parameters:
+* namespace: Each stream can only stream for/from a single namespace, specified
+ on this POST query parameter.
+* direction: 'upload' for upstream, 'download' for downstream.