| Age | Commit message (Collapse) | Author |
|---|
|
|
|
If there's no limit to the number of concurrent blob writes in the
server, the maximum limit of open files will eventually be reached, and
the processing of requests will start crashing. This commit adds
a semaphore to limit the number of concurrent writes in the server.
Related: #8973
|
|
|
|
An errback was missing in the PUT renderer method of the incoming API.
Because of that, requests to that endpoint were not being correctly
finished in case of errors when writing blobs. That was causing delivery
requests to hang until timeout.
Closes: #8977
|
|
As kali pointed out, one can disable blobs after enabling it, which
would cause data loss as blobs documents would become unreacheable. This
commit adds a warning and refuses to start the server.
-- Resolves: #8866
|
|
|
|
|
|
|
|
Introduction of local services authentication added a configuration file
containing the auth tokens for each service. There were different names
for that file, and this commit standardizes all of them to the same
value: /etc/soledad/services.tokens
|
|
Soledad Server was previously using something in /srv to store blobs in
the server side. Debian/lintian doesn't like that at all, so we are
changing to /var/lib/soledad/blobs.
Closes: #8948
|
|
It isn't closed by Twisted like the producer is.
-- Resolves: #8924
-- Related: #8932
|
|
|
|
|
|
|
|
|
|
Schema was using a default value instead of using the one passed as a
parameter. Additional test for formatter is also included.
-- Related: #8867
|
|
Incoming API is supposed to be able to, given a valid service token,
write a incoming document into any user database. Leaving the parameter
as 'uuid' triggers defensive code against unauthorized accesses between
users. This commit renames the parameter so this isn't checked.
-- Related: #8867
|
|
-- Related: #8867
|
|
-- Related: #8867
|
|
-- Related: #8867
|
|
This way the file gets copied during package install.
-- Related: #8867
|
|
-- Related: #8867
|
|
-- Related: #8867
|
|
-- Related: #8867
|
|
For tests, we may want to configure the server with non-default options,
and the easiest way to do this is by creating a configuration file in
a temporary directory and passing the file name by means of an
environment variable.
This commit changes the server config file loading scheme to account for
a variable called SOLEDAD_SERVER_CONFIG_FILE. If that variable is set,
the configuration is read from the file pointed by it. Otherwise,
/etc/soledad/soledad-server.conf is used.
|
|
We do not want to enable blobs on any server that, by mistake, deploys
from master or from a released version in the 0.10.x series.
for testing it's more sensible to allow instantiating the server with a
custom config file.
|
|
|
|
|
|
|
|
|
|
|
|
This commit is complementary to the previous one. It adds a test for the
reported bug (listing default namespace was listing others) and fixes
it.
-- Related: #8882
|
|
All blobs were being stored in a single folder when using namespaces,
this commits adds path partitioning as discussed on #8882, which should
help with a large number of files (each folder will hold a smaller
subset, allowing the use of better filesystem walk strategies).
Also, the default empty namespace is now called 'default' to prevent it
from listing other namespaces contents. So everything will always use
namespaces, with the option to use it explicitly or just fall to the
default one.
-- Related: #8882
|
|
IIncomingBoxBackend holds backend specific methods used for IncomingBox
implementation.
- Resolves: #8888
|
|
|
|
Clients will query for blobs flagged PENDING so they can start
processing. This commit adds flagging, but still in a hacky way as the
backend requires the flags as a json string inside a request. A refactor
to separate request handling from the backend itself will solve it in
the near future.
-- Related: #8874
|
|
|
|
Improves error handling, add missing header to __init__.py and remove
mkdirs from flags methods. This is a commit from code review on !117.
|
|
Improve test naming, creates a "count" method and filter flags files
properly.
|
|
filter_flag parameter can now be used to filter listing blobs by a
specific flags. Eg: I can ask for blobs on incoming namespace flagged as pending.
-- Resolves: #8913
|
|
By using `only_count=True` parameter, instead of returning a full list,
server just count the amount of blobs.
-- Resolves: #8871
|
|
IncomingBox spec has a flags feature for the processing flow of
messages. This commit adds it using a .flags file.
-- Resolves: #8869
|
|
Refactor suggested from !105 review.
|
|
|
|
Updates IBlobsBackend with list_blobs order_by parameter and add a elif
clause to order_by checking, raising if the ordering parameter isn't
supported.
|
|
'namespace' argument is supported by backend but not yet exposed on API
for clients. Since IncomingBox makes heavy usage of it, this commit
exposes the argument as a query string for clients to use it.
|
|
Listing by date is useful for listing newest/oldest documents on blobs
storage and should be used for listing new IncomingMessages as described
on specification.
-- Resolves: #8879
|
|
We started with CouchDB due legacy system relying on it. This commit
adds the possibility of adding blobs as a IncomingAPI backend if blobs
is enabled on config file.
-- Resolves: #8868
|
|
We started with CouchDB due legacy system relying on it. This commit
adds the possibility of adding blobs as a IncomingAPI backend if blobs
is enabled on config file.
-- Resolves: #8868
|
|
Adds an extra parameter called "namespace" on the backend interface and
on FileSystemBlobsBackend. This parameter overrides default id
partitioning and uses a separate folder for a custom namespace.
-- Resolves: #8889
|
