| Age | Commit message (Collapse) | Author |
|---|
|
Sync very large files will now fail early because it was exploding
the memory, it needs some love, but for now I'll just make it fail
and open an issue to take a better look at it
|
|
Changes threading.lock to DeferredLock and checks syncing attribute by
looking into the lock state.
Also, applies more of startTwistedServer on tests that relies on
HTTP/1.1.
Fixes mock for events
|
|
Two spaces before class, not one
|
|
Some mixins were being called from u1db_tests but not used,
what was really needed was pulled from there and the remaining
dependencies were identified and used from the base
(u1db_tests.__init__.py)
|
|
Started looking at what we can do to remove the u1db test
dependencies directly, the parse was calling another class
setup that was in fact the u1db testcase, changed that to
testcase directly and it still works
|
|
U1DB test_http_app file was not being required anywhere and it's tests were
being skipped, removing it to clean up
|
|
Some tests still rely on python WSGI logic while others expect HTTP/1.1
from Twisted WSGI. While we can't use a single implementation for all,
both are needed.
startServer -> python default wsgi (http 1.0)
startTwistedServer -> Twisted WSGI (http 1.1 'production-like')
|
|
Soledad has a close method that wasn't calling http_target close. The
reference to sync exchange was being deleted without proper closing of
underlying resources.
|
|
Soledad was being served with python's default wsgi server, which uses
HTTP 1.0. Since Soledad server doesn't send length or chunked headers,
Twisted HTTP Client gets partial downloads when served with HTTP1.0.
Soledad is meant to be used with Twisted WSGI container on HTTP/1.1 and
the tests should reflect that.
|
|
|
|
- ability to disable sending (to get raw receive times)
- ability to specify size and number of payloads
- ability to point to a file to be used as raw source of payloads
|
|
|
|
|
|
Updating the profile-sync script to the latest deferred-based sync.
- Added a couple of options to have finer control about what output to
get.
- Add support for the plop profiler https://pypi.python.org/pypi/plop/
- To get meaningful plop profiles, make sure to disable the system stats
collection, like this::
./profile-sync.py --no-stats --plop -b /tmp/syncdata/ -p sikret user@provider
A good practice for this is having a pre-seeded soledad account
(currently you have to do that through the wizard, a cli will be very
nice to have in the coming future) with a known amount of data (for
instance, sending some mails with known payload weight).
It is VERY IMPORTANT that you *NEVER* process the data in this account
(ie, do not ever log in with a mail client, for instance, since that
will alter the original documents).
In order to have comparable results, you should always run this sync
script in similar conditions. Ideally, on a machine with runlevel 1.
Also, make sure of deleting the contents in the folder if you are
not using a temporal dir.
|
|
|
|
--use-leap-wheels sets --trusted-host (remove it when we have a proper
cert) and WHEELHOUSE to https://ftp.lizard.leap.se
Until we get ftp.lizard cname, use lizard as the wheels server.
- Related: #7339
|
|
All the response parse tests are passing now, response
with no entries was broken because it wasn't being treated
and the others were broken because of calls that no longer
existed
|
|
Created a setup for the http target tests
Fixed two tests relying on http target that were
outdated
Fixed a call for an exception that doesn't exist, it
won't break anymore if it gets to that exception
|
|
The old target is now http_target and the class name was changed from
SoledadSyncTarget to SoledadHTTPSyncTarget, fixed this on the tests
|
|
Line break before binary operator breaks PEP8, fixed
that in the client api.py
|
|
when copying a generic config file over in previous commit
|
|
before sqlcipher backend, or the attribute is not found.
this is a leftover of the recent refactor
|
|
|
|
|
|
|
|
|
|
|
|
SoledadCrypto had Soledad as parameter to be able to use
SoledadSecrets. SoledadSecrets had SoledadCrypto as parameter to use
*crypt_sym. This commit removes this circular dependency passing
directly the secret that SoledadCrypto cares about to the constructor
and removing the *crypt_sym methods from SoledadCrypto.
- Resolves: #7338
|
|
|
|
* change close method name to stop
* add start/stop methods to both enc/dec clases
* remove any delayed calls on pool shutdown
|
|
|
|
generate_wheels uses $WHEELHOUSE to generate and store the wheels for
requirements.pip and requirements-testing.pip (if it exists).
pip_install_requirements.sh installs requirements.pip from them if
possible (if not, then it fetches them from pypi) or, if passed the
--testing flag, it installs requirements-testing.pip.
Related: #7327
|
|
|
|
That is the right way TM to fix the events issue without explicitly
setting the flag, as kaliy suggested
|
|
|
|
|
|
|
|
The soledad tests were breaking after the change to zmq, the
event server was trying to create a zmq instance but there are
some missing files that prevented the tests from running
just fixed those and the tests run again
|
|
requirements-latest.pip will try to clone and install. Since it is meant
to be latest, I added a small change to specify the branch 'develop'.
|
|
The bolean operator must come before a line break, not after
according to pep8
|
|
With this, you can setup soledad for using locally
and running the tests with the latest head in a simpler
way
|
|
|
|
|
|
|
|
- update pip
- install base reqs, with insecure flags for dirspec and u1db
|
|
Because of how the incoming document queue is implemented, it could be the
case that a document was sent to async decryption queue more than once. This
commit creates a list of documents to be decrypted, so we avoid sending the
same document to the queue more than once.
|
|
|
|
The incoming documents events are meant to be used by a progress bar for
soledad sync, yet to be implemented. When deferred decryption was used, the
events were sent out of order, depending on the order of arrival of the
documents. This commit changes it so that the content of the emited events are
in order, so it is meaningful for the implementation of a progress bar.
Note that even after documents are received from the server, they will still
be decrypted asynchronously, so another signal could be implemented to signal
for the waiting of the decryption of incoming documents.
|
|
|
|
This is how a secret was stored in the secrets json file:
* each secret is symmetrically encrypted amd MACed with keys derived from
the user's passphrase.
* the encrypted secrets dictionary is then MACed with another key derived
* from the user's passphrase.
* each key is derived using scrypt and a unique random salt.
There are disadvantages to this approach:
* repeating scrypt many times is a waste of time.
* an attacker could crack whichever has weaker parameters, if they get out
of sync.
* if an attacker can modify the secret in a way it is good to decrypt the
database, then she can also modify the MAC.
The solution for this is:
* completelly eliminate the MAC from the storage secrets file.
* attempt to decrypt the database with whatever is got from the decryption
of the secret. If that is wrong, report an error.
Closes #6980.
|
