summaryrefslogtreecommitdiff
AgeCommit message (Collapse)Author
2015-08-12[tests] Sync very large files won't explode memoryBruno Wagner
Sync very large files will now fail early because it was exploding the memory, it needs some love, but for now I'll just make it fail and open an issue to take a better look at it
2015-08-12[bug] fixes concurrent sync and their testsVictor Shyba
Changes threading.lock to DeferredLock and checks syncing attribute by looking into the lock state. Also, applies more of startTwistedServer on tests that relies on HTTP/1.1. Fixes mock for events
2015-08-12[style] Fixed pep8 warningBruno Wagner
Two spaces before class, not one
2015-08-12[tests] Removed some of the dependencies on u1db_testsBruno Wagner
Some mixins were being called from u1db_tests but not used, what was really needed was pulled from there and the remaining dependencies were identified and used from the base (u1db_tests.__init__.py)
2015-08-12[tests] Removed dependency u1db test in parse testBruno Wagner
Started looking at what we can do to remove the u1db test dependencies directly, the parse was calling another class setup that was in fact the u1db testcase, changed that to testcase directly and it still works
2015-08-12[tests] U1db test file was not being used anywhereBruno Wagner
U1DB test_http_app file was not being required anywhere and it's tests were being skipped, removing it to clean up
2015-08-12[tests] Twisted WSGI is optionalVictor Shyba
Some tests still rely on python WSGI logic while others expect HTTP/1.1 from Twisted WSGI. While we can't use a single implementation for all, both are needed. startServer -> python default wsgi (http 1.0) startTwistedServer -> Twisted WSGI (http 1.1 'production-like')
2015-08-12[bug] close http_target when soledad closesVictor Shyba
Soledad has a close method that wasn't calling http_target close. The reference to sync exchange was being deleted without proper closing of underlying resources.
2015-08-12[test] Change test server to use TwistedVictor Shyba
Soledad was being served with python's default wsgi server, which uses HTTP 1.0. Since Soledad server doesn't send length or chunked headers, Twisted HTTP Client gets partial downloads when served with HTTP1.0. Soledad is meant to be used with Twisted WSGI container on HTTP/1.1 and the tests should reflect that.
2015-08-11[tests] add util function to instantiate dowser mem profilerKali Kaneko
2015-08-11[feat] control sending messages in scriptKali Kaneko
- ability to disable sending (to get raw receive times) - ability to specify size and number of payloads - ability to point to a file to be used as raw source of payloads
2015-08-11[feat] add support for twisted-aware theseus profilerKali Kaneko
2015-08-11[bug] disable events on profilingKali Kaneko
2015-08-11[feat] update profile-sync script, add plop profilingKali Kaneko
Updating the profile-sync script to the latest deferred-based sync. - Added a couple of options to have finer control about what output to get. - Add support for the plop profiler https://pypi.python.org/pypi/plop/ - To get meaningful plop profiles, make sure to disable the system stats collection, like this:: ./profile-sync.py --no-stats --plop -b /tmp/syncdata/ -p sikret user@provider A good practice for this is having a pre-seeded soledad account (currently you have to do that through the wizard, a cli will be very nice to have in the coming future) with a known amount of data (for instance, sending some mails with known payload weight). It is VERY IMPORTANT that you *NEVER* process the data in this account (ie, do not ever log in with a mail client, for instance, since that will alter the original documents). In order to have comparable results, you should always run this sync script in similar conditions. Ideally, on a machine with runlevel 1. Also, make sure of deleting the contents in the folder if you are not using a temporal dir.
2015-08-11[feat] emit a dict instead of a strIvan Alejandro
2015-08-06[feat] WHEELHOUSE can be a url + --use-leap-wheelsParménides GV
--use-leap-wheels sets --trusted-host (remove it when we have a proper cert) and WHEELHOUSE to https://ftp.lizard.leap.se Until we get ftp.lizard cname, use lizard as the wheels server. - Related: #7339
2015-08-05[tests] Fixed remaining parse testsBruno Wagner
All the response parse tests are passing now, response with no entries was broken because it wasn't being treated and the others were broken because of calls that no longer existed
2015-08-05[tests] Created a setup for the http target testsBruno Wagner
Created a setup for the http target tests Fixed two tests relying on http target that were outdated Fixed a call for an exception that doesn't exist, it won't break anymore if it gets to that exception
2015-08-05[tests] fixed the http target test importsBruno Wagner
The old target is now http_target and the class name was changed from SoledadSyncTarget to SoledadHTTPSyncTarget, fixed this on the tests
2015-08-05[style] Fixed PEP8 warningBruno Wagner
Line break before binary operator breaks PEP8, fixed that in the client api.py
2015-08-05[tests] re-add accidental removalKali Kaneko
when copying a generic config file over in previous commit
2015-08-04[bug] Soledad._crypto has to be initialized earlierKali Kaneko
before sqlcipher backend, or the attribute is not found. this is a leftover of the recent refactor
2015-08-04[style] pep8 cleanupKali Kaneko
2015-08-04[style] added pep8 excludes+ignoresKali Kaneko
2015-08-04[test] add test for many documents decryptiondrebs
2015-08-04[test] add test for receiving many docs on decpooldrebs
2015-08-04[feat] add running state method to enc/dec poolsdrebs
2015-08-04[refactor] remove circular dependency in SoledadCryptoRuben Pollan
SoledadCrypto had Soledad as parameter to be able to use SoledadSecrets. SoledadSecrets had SoledadCrypto as parameter to use *crypt_sym. This commit removes this circular dependency passing directly the secret that SoledadCrypto cares about to the constructor and removing the *crypt_sym methods from SoledadCrypto. - Resolves: #7338
2015-08-04[test] add initial enc/dec pool testsdrebs
2015-08-04[refactor] standardize start/stop of enc/dec poolsdrebs
* change close method name to stop * add start/stop methods to both enc/dec clases * remove any delayed calls on pool shutdown
2015-08-04[bug] add log msg to sent docs on syncdrebs
2015-08-03[feat] use wheels to install dependenciesParménides GV
generate_wheels uses $WHEELHOUSE to generate and store the wheels for requirements.pip and requirements-testing.pip (if it exists). pip_install_requirements.sh installs requirements.pip from them if possible (if not, then it fetches them from pypi) or, if passed the --testing flag, it installs requirements-testing.pip. Related: #7327
2015-08-03Merge remote-tracking branch 'leapcode/pr/237' into developKali Kaneko
2015-08-03[bug] BaseSoledadTest now inherit from BaseLeapTestBruno Wagner
That is the right way TM to fix the events issue without explicitly setting the flag, as kaliy suggested
2015-08-03[style] pep8Kali Kaneko
2015-08-03[feat] allow repeated sync in client db scriptdrebs
2015-08-03[feat] use dynamic mx server in spam scriptdrebs
2015-07-29[test] Added events enabled flag to False for the testsBruno Wagner
The soledad tests were breaking after the change to zmq, the event server was trying to create a zmq instance but there are some missing files that prevented the tests from running just fixed those and the tests run again
2015-07-29[bug] specify develop branch when using -eVictor Shyba
requirements-latest.pip will try to clone and install. Since it is meant to be latest, I added a small change to specify the branch 'develop'.
2015-07-29[style] Fixed pep8 warning on api.pyBruno Wagner
The bolean operator must come before a line break, not after according to pep8
2015-07-29[feat] Added requirements-latest pip fileBruno Wagner
With this, you can setup soledad for using locally and running the tests with the latest head in a simpler way
2015-07-29[bug] Added files created running tests to gitignoreBruno Wagner
2015-07-28[tests] add pep8 to requirements-testingKali Kaneko
2015-07-27[pkg] add AUTHORS file + one-liner to generate itKali Kaneko
2015-07-27[pkg] add script to install base requirementsKali Kaneko
- update pip - install base reqs, with insecure flags for dirspec and u1db
2015-07-27[bug] avoid double decryption of documentsdrebs
Because of how the incoming document queue is implemented, it could be the case that a document was sent to async decryption queue more than once. This commit creates a list of documents to be decrypted, so we avoid sending the same document to the queue more than once.
2015-07-27[feat] add log message for document encryptiondrebs
2015-07-27[bug] fix order of incoming document eventsdrebs
The incoming documents events are meant to be used by a progress bar for soledad sync, yet to be implemented. When deferred decryption was used, the events were sent out of order, depending on the order of arrival of the documents. This commit changes it so that the content of the emited events are in order, so it is meaningful for the implementation of a progress bar. Note that even after documents are received from the server, they will still be decrypted asynchronously, so another signal could be implemented to signal for the waiting of the decryption of incoming documents.
2015-07-27[feat] add smtp port and log level to spam scriptdrebs
2015-07-27[bug] remove mac from secrets filedrebs
This is how a secret was stored in the secrets json file: * each secret is symmetrically encrypted amd MACed with keys derived from the user's passphrase. * the encrypted secrets dictionary is then MACed with another key derived * from the user's passphrase. * each key is derived using scrypt and a unique random salt. There are disadvantages to this approach: * repeating scrypt many times is a waste of time. * an attacker could crack whichever has weaker parameters, if they get out of sync. * if an attacker can modify the secret in a way it is good to decrypt the database, then she can also modify the MAC. The solution for this is: * completelly eliminate the MAC from the storage secrets file. * attempt to decrypt the database with whatever is got from the decryption of the secret. If that is wrong, report an error. Closes #6980.