| Age | Commit message (Collapse) | Author |
|---|
|
|
|
|
|
|
|
batch is slower than usual insert for a single doc, so, if a document
exceeds the buffer, commit the batch (if any) and put the huge load by
traditional insert.
refactor coming.
|
|
Batching is now decided by server, this commits enables it.
|
|
This allow different paths for raw data and metadata, avoiding
unnecessary json parsing.
|
|
There was an if without an else on error handler that avoided handling
errors that falled back current logic. Added a generic one to the tail
so we dont miss it.
|
|
We enabled chunking, which means that a use can upload his entire db on
a single request. This commit makes server enable this and throttle
download as Twisted cant control the payload producer code as its
synchronous and blocking code.
|
|
1) enable HTTP 1.1 chunked upload on server
2) make the client sync.py generate a list of function calls instead of
a list of full docs
3) disable encryption pool
4) make the doc encryption a list of function calls
5) create a twisted protocol for sending
6) make a producer that calls the doc generation as necessary
|
|
This commit finishes reversion into u1db original streaming protocol for
downloads.
|
|
It's not being used
|
|
Will put a file object on doc json string if read_content is False,
otherwise it will fetch and fill as usual. This is useful for improving
server througput on sync download stream by receiving a bulk-get without
attachments and consume the file-objects as they come.
|
|
If a doc doesnt have a content it means it was deleted. Sync stream was
unable to represent this state.
|
|
Check if the backend provides a commit method before calling or we will
break the tests with InMemoryDatabase
|
|
Make the client parse a 2-line doc on sync download stream.
|
|
couchdb lib returns a file object representing the attachment. This
commit dumps the read() call into the wsgi write() call. Doc
representation uses 2 lines also, separating metadata from content.
|
|
Temporary fix for server streaming
|
|
We were using 1 transaction per doc, which is bad.
Reference:
http://stackoverflow.com/questions/1711631/improve-insert-per-second-performance-of-sqlite
Code now uses 1 transaction for the whole sync.
|
|
Instead of getting the attachments as the generator runs, get_docs will
now get as needed. Also, deepcopy solves a memory issue where we were
feeding the couchdb lib view with blobs while modifying it
unintentionally.
|
|
Instead of concurrent download, we are going to download a stream. This
commit modifies server to support it.
|
|
Tag version 0.9.0
|
|
|
|
|
|
|
|
This should avoid tox virtualenv recreation.
|
|
When importing server, couch_state will load itself against couch_db url
configured on server. This fails when running on Docker as couchdb is in
another node.
|
|
This is necessary for keymanager and this image is shared, thus adding
here with a comment explaining why. Also explained why using
jessie-backports.
|
|
create_cmd lacked an explanation and check_schema_versions lacked
reasoning on why it defaults to False.
|
|
CouchServerState is spread across test codebase and this option is
intended to be used only on server startup. This commit makes it default
to False and explicitly set it to True on where it's necessary.
|
|
code-check is running with py3 randomly on CI, this commit should pin
it.
|
|
Instead of hardcoding a version. This should give us the flexibility of
changing images without changing code.
|
|
Current docker image is broken due missing libsqlcipher. This commit
adds it and jessie-backports due package needs.
Resolves: #8508
|
|
We discovered that class was registering a `finalClose` to be
executed on reactor shutdown.
On the multiuser scenario, a logout destroys Soledad and should
properly terminate everything related to it. That SQLCipherU1DBSync
instance was being held even after logout by the reactor so it
could call that `finalClose` on shutdown.
The `finalClose` only set running to False and set a `shutdownID` that
was not used anywhere else, so we removed it and moved setting
running to False to the `close` function method. That way we preserve
the functionality but let the instance be properly garbage collected
on logout.
|
|
|
|
|
|
Otherwise it will put the exception as an additional parameter.
|
|
|
|
Tests that were imported from u1db or created on top of that structure
were leaving temporary directories behind. This could cause problems in
test servers, either by filling the partition or by extrapolating the
maximum amount of files in a directory.
This commit replaces all usages of temporary directories in the old test
structure by pytest tmpdir fixture, which properly cares for removing
temporary directories.
|
|
|
|
|
|
In order to configure performance tests to run in a specific machine we
need to add a tagged job to .gitlab-ci.yml file. That job will only
execute the perf tests, and then we can have runners that will only run
those jobs.
|
|
|
|
|
|
add coverage reports too.
(hereby we swear not to write stupid tests just because it feels good to
have an increased coverage metric).
- Resolves: #8416
|
|
this is needed for some mail tests.
|
|
|
|
|
|
We were using 'x'*size as payload, but on real usage the payload will be
random. This commit randomizes the payload using a predefined seed, so
the random payload will be the same across benchmarks.
Using random payloads also improves accuracy of compression or encoding
impacts and we will be evaluating those changes for resouce usage
issues.
Also note that base64 is used on payload. That was needed for utf8
safety, but overhead was removed to leave payloads as defined by
benchmarks.
Base64 was chosen also due its popular usage on MIME encoding, which is
used on mail attachments (our current scenario).
|
|
TestSyncEncrypterPool.test_encrypt_doc_and_get_it_back was trying to do
an operation and asserting the number of attempts. This test is about
putting a doc on encrypter pool and getting it encrypted. If we dont
wait for the encryption operation to succeed, then complex
trial-and-error happens, but if we just ask twisted to wait for one
operation before going to the other, this is not needed.
-- Resolves: #8398
|
|
|
