From 0776862f274d86e2d9cdb941e74496d5f4c555c9 Mon Sep 17 00:00:00 2001 From: drebs Date: Wed, 7 Jun 2017 17:24:48 -0300 Subject: [doc] add doc on server-side blobs Closes #8850. --- docs/attachments.rst | 104 ++++++++++++++++++++++++++++++++++++++++----------- 1 file changed, 82 insertions(+), 22 deletions(-) diff --git a/docs/attachments.rst b/docs/attachments.rst index 098f634f..31b47a70 100644 --- a/docs/attachments.rst +++ b/docs/attachments.rst @@ -1,25 +1,37 @@ -Document Attachments +Document attachments ==================== .. contents:: Contents: :local: -The content of a Soledad document is assumed to be JSON. This is particularly -bad for storing larger amounts of binary data, because: +Reasoning +--------- + +The type of a Soledad document's content is `JSON `_, +which is good for efficient lookup and indexing. On the other hand, this is +particularly bad for storing larger amounts of binary data, because: * the only way to store data in JSON is as unicode string, and this uses more - space than needed for binary data storage. + space than what is actually needed for binary data storage. + +* upon synchronization, the content of a Soledad document needs to be + completelly transferred and decrypted for the document to be available for + use. + +Document attachments were introduced as a means to efficiently store large +payloads of binary data while avoiding the need to wait for their transfer to +have access to the documents' contents. -* the process of synchronization of Soledad documents depends on completing the - transfer and decryption of the content of all new/updated documents before - synchronized documents are available for use. +Client-side +----------- -Document attachments were introduced as a means to store large payloads of -binary data and have them be synchronized separate from the usual Soledad -document synchronization process. +In the client, attachments are stored as (SQLite) BLOBs in a separate SQLCipher +database. Encryption of data before it's sent to the server is the same used +for Soledad documents' content during usual synchronization process (AES-256 +GCM mode). -Example -------- +Usage example +^^^^^^^^^^^^^ The attachments API is currently available in the `Document` class, and the document needs to know about the store to be able to manage attachments. When @@ -57,16 +69,8 @@ the store that created it, and can put/get/delete an attachment: fd = yield doc.get_attachment() assert fd.read() == open('hackers.txt').read() -Implementation --------------- - -The current implementation of document attachments store data in a separate -SQLCipher database in the client (using SQLite's BLOB type) and in the -filesystem in the server. Encryption of data before it's sent to the server is -the same used by normal Soledad synchronization process (AES-256 GCM mode). - -Document attachment API ------------------------ +API +^^^ .. autoclass:: leap.soledad.client._document.AttachmentStates :members: @@ -75,3 +79,59 @@ Document attachment API .. autointerface:: leap.soledad.client._document.IDocumentWithAttachment :members: :undoc-members: + +Server-side +----------- + +In the server, a simple REST API is served by a `Twisted Resource +`_ +and attachments are stored in the filesystem as they come in without +modification. + +A token is used to allow listing, getting, putting and deleting attachments. It +has to be added as an HTTP auth header, as in:: + + Authorization: Token + +The :ref:`IBlobsBackend ` interface is provided, so in the +future there can be different ways to store attachments in the server side +(think of a third-party storage, for example). Currently, the +:ref:`FilesystemBlobsBackend ` is the only backend +that implements that interface. + +Some characteristics of the :ref:`FilesystemBlobsBackend +` are: + +* Configurable storage path. +* Quota support. +* Username, blob_id and user storage directory sanitization. + +Usage example +^^^^^^^^^^^^^ + +These are the possible ways to interact with the attachments REST API on the +server side: + +=========== ================ ======== ================== +HTTP Method URL Content Possible responses +=========== ================ ======== ================== +GET /user_id - 200 +GET /user_id/blob_id - 200, 404 +PUT /user_id/blob_id The BLOB 200, 409, 507 +DELETE /user_id/blob_id - 200 +=========== ================ ======== ================== + +API +^^^ + +.. _i-blobs-backend: + +.. autoclass:: leap.soledad.server.interfaces.IBlobsBackend + :members: + :undoc-members: + +.. _filesystem-blobs-backend: + +.. autoclass:: leap.soledad.server._blobs.FilesystemBlobsBackend + :members: + :undoc-members: -- cgit v1.2.3