0.9.0 October 28 ++++++++++++++++ We are very pleased to announce Bitmask stable 0.9.0 :tada:. It's been 9 months since we released our latest stable version, we have been working a lot and trying out several release candidates in the way. Using the latest Bitmask you'll be able to use our encrypted email service, now in beta state! NOTE: beta means that we expect not to break but we don't promise you won't get any headache or lose some email, so please be careful. Currently we have a test provider for mail usage hosted on https://mail.bitmask.net this provider is already bundled with Bitmask for easy access on the wizard. ---- Some numbers on what we have been doing all this time: - we have closed **472** issues, - we have closed **379** pull requests, - adding up all the components changes we got **830** new commits ---- Here you have a list of the most notable changes since our latest stable release. Index of changes: * `Bitmask Client`_ (0.8.1 → 0.9.0) * `Soledad`_ (0.6.3 → 0.7.4) * `Keymanager`_ (0.3.8 → 0.4.3) * `Common`_ (0.3.10 → 0.4.4) * `Mail`_ (0.3.11 → 0.4.0) Bitmask Client ============== Features ~~~~~~~~ - `#4284 `_: Download specific smtp certificate from provider, instead of using the vpn one. - `#5526 `_: Make "check" button selected by default. - `#6359 `_: Adapt bitmask to the new events api on leap.common. - `#6360 `_: Use txzmq in backend. - `#6368 `_: Add support to the new async-api of keymanager. - `#6683 `_: Add ability to generate sumo tarball. - `#6713 `_: Add support for xfce-polkit agent. - `#6876 `_: Update api port for pinned riseup. - `#7139 `_: Use logbook zmq handler to centralize logging. - `#7140 `_: Implement a thread-safe zmq handler for logbook. - `#7141 `_: Add log handler to display colored logs on the terminal. - `#7142 `_: Add log handler to store logs on bitmask.log. - `#7143 `_: Adapt existing log filter/silencer to the new logbook handler. - `#7144 `_: Replace logging handler with logbook handler bitmask-wide. - `#7162 `_: Log LSB-release info if available. - `#7180 `_: Add log rotation for bitmask.log. - `#7184 `_: Forward twisted logs to logging and handle logging logs with logbook. - `#7250 `_: Enable ``--danger`` for stable versions. - `#7291 `_: Move the updater code from the launcher to the client. - `#7342 `_: Added ``apply_updates.py`` script for the pyinstaller bundle. - `#7353 `_: Add notifications of soledad sync progress to UI. - `#7356 `_: Allow to disable EIP component on build. - `#7414 `_: Remove taskthread dependency, replace with custom (and small) code. - `#7419 `_: Load credentials from environment variables and trigger login. - `#7471 `_: Disable email firewall if we are running inside a docker container. - Add support to the new async-api of soledad Bugfixes ~~~~~~~~ - `#6418 `_: Cannot change preseeded providers if checks for one fail. - `#6424 `_: Do not disable autostart if the quit is triggered by a system logout. - `#6536 `_, `#6568 `_, `#6691 `_: Refactor soledad sync to do it the twisted way. - `#6541 `_: Client must honor the ports specified in ``eip-service.json``. - `#6594 `_: Handle disabled registration on provider. - `#6654 `_: Regression fix, login attempt is made against previously selected provider. - `#6682 `_: Handle user cancel keyring open operation, this prevents a bitmask freeze. - `#6894 `_: Change ``ip`` command location to support Fedora/RHEL distros. - `#7093 `_: Fix controller attribute error. - `#7126 `_: Don't run the event server on the backend for the standalone bundle since the launcher takes care of that. - `#7149 `_: Start the events server when reactor is running. - `#7185 `_: Log contains exported PGP Private Key. - `#7222 `_: Run the zmq log subscriber in the background to avoid hitting the zmq's buffer limits. - `#7273 `_: Logbook subscriber stop fails if not started. - `#7273 `_: ZMQError: address already in use - logbook subscriber already started. - `#7281 `_: Support a provider not providing location for the eip gateways. - `#7319 `_: Raise the maxfiles limit in OSX - `#7343 `_: Clean up and fix the tests. - `#7415 `_: Fix wrong argument number on window raise event. - `#7448 `_: Fix hangs during logout. - `#7451 `_: Assign the timeout 'call later' before starting the sync to prevent race conditions. - `#7453 `_: After a complete sync show the user the amount of unread emails. - `#7470 `_: Fix bug with password change. - `#7474 `_: Track soledad ready state on a shared place for easy access. Enable password change window. - `#7503 `_: Handle soledad init fail after several retries. - `#7512 `_: Pass on standalone flag to common. - `#7512 `_: Store logs in the right place. - `#7512 `_: Store zmq certs in the right path. - Authenticate properly logout calls to API. - Fix soledad bootstrap sync retries. - Fix the bootstrap script for developers so it works on Fedora/RHEL systems where there is ``/usr/lib64`` for python libs. - Remove bubble argument from the logbook NullHandler ---- Soledad ======= soledad.client ~~~~~~~~~~~~~~ Features -------- - `#7353 `_: Improve how we send information on ``SOLEDAD_SYNC_SEND_STATUS`` and in ``SOLEDAD_SYNC_RECEIVE_STATUS``. - `#5895 `_: Store all incoming documents in the sync db. - `#6359 `_: Adapt soledad to the new events api on leap.common. - `#6400 `_: Include the IV in the encrypted document MAC. - `#6996 `_: Expose post-sync hooks via plugin system. - Add a pool of HTTP/HTTPS connections that is able to verify the server certificate against a given CA certificate. - Use twisted.enterprise.adbapi for access to the sync database. - Use twisted.web.client for client sync. Bugfixes -------- - `#5855 `_: Reset syncer connection when getting HTTP error during sync. - `#5975 `_: Wait for last post request to finish before starting a new one. - `#6437 `_: Use TLS v1 in soledad client. - `#6625 `_: Retry on sqlcipher thread timeouts. - `#6757 `_: Fix the order of insertion of documents when using workers for decrypting incoming documents during a sync. - `#6892 `_: Fix the log message when a local secret is not found so it's less confusing. - `#6980 `_: Remove MAC from secrets file. - `#7088 `_: Fix sync encrypter pool close queue error. - `#7302 `_: Increase http request timeout time to 90s. - `#7386 `_: Fix hanging sync by properly waiting db initialization on sync decrypter pool. - `#7503 `_: Do not signal sync completion if sync failed. - `#7503 `_: Handle soledad init fail after several retries. - Always initialize the sync db to allow for both asynchronous encryption and asynchronous decryption when syncing. - Avoid double decryption of documents. - Bugfix: move sync db and encpool creation to api. - Bugfix: refactor code loss. - Bugfix: set active secret before saving local file. - Bugfix: wrong sqlcipher passphrase now raises correctly. - Fallback to utf-8 if confidence on chardet guessing is too low. - Fix logging and graceful failing when exceptions are raised during sync. - Fix the order of the events emited for incoming documents. - Handle ``DatabaseDoesNotExist`` during sync. - Handle ``MissingDesignDocError`` after get_sync_info. - Handle missing design doc at GET (``get_sync_info``). Soledad server can handle this during sync. Misc (CI, tests, refactor, packaging) ------------------------------------- - `#2945 `_: Do not depend on pysqlite2. - `#6797 `_: Add dependency on Twisted. - `#7338 `_: refactor ``SoledadCrypto`` to remove circular dependency with ``SoledadSecrets``. - Add tests for enc/dec pool. - Improve helper scripts and dependencies listing. - Improve log messages when concurrently fetching documents from the server. - Lots of code restyling to pass CI tests. - Refactor asynchronous encryption/decryption code to its own file. - Refactor decription pool and http target to use a deferred instead of a waiting loop. - Refactor details of making an HTTP request body and headers out of the send/fetch logic. This also makes it easier to enable batching. - Refactor enc/dec pool to standardize start/stop of the pools. - Remove dependency on simplejson. - Split ``http_target`` into 4 modules, separating those responsibilities. soledad.server ~~~~~~~~~~~~~~ Features -------- - `#6785 `_: Use monthly token databases. - Lots of code restyling to pass CI tests. - Lots of work done to get tests passing. - Remove dependency on simplejson. Bugfixes -------- - `#6436 `_: Run daemon as user soledad. - `#6437 `_: Avoid use of SSLv3. - `#6557 `_: Fix server initscript location. - `#6797 `_: Add dependency on Twisted. - `#6833 `_: Remove unneeded parameters from ``CouchServerState`` initialization. - Fix a bug where `BadRequest` could be raised after everything was persisted. - Fix server daemon uid and gid by passing them to twistd on the initscript. soledad.common ~~~~~~~~~~~~~~ Features -------- - `#6359 `_: Adapt soledad to the new events api on leap.common. - Lots of code restyling to pass CI tests. - Lots of work done to get tests passing. - Refactor `couch.py` to separate persistence from logic while saving uploaded documents. Also simplify logic while checking for conflicts. - Remove dependency on simplejson. Bugfixes -------- - `#5896 `_: Include couch design docs source files in source distribution and only compile ``ddocs.py`` when building the package. - `#6671 `_: Bail out if ``cdocs/`` dir does not exist. - `#6833 `_: Remove unneeded parameters from ``CouchServerState`` initialization. ---- Keymanager ========== Features ~~~~~~~~ - `#5359 `_: Adapt to new events api on leap.common. - `#5932 `_: Add ``fetch_key`` method to fetch keys from a URI. - `#6211 `_: Upgrade keys if not successfully used and strict high validation level. - `#6212 `_: Multi uid support. - `#6240 `_: Upgrade key when signed by old key. - `#6262 `_: Keep old key after upgrade. - `#6299 `_: New soledad doc struct for encryption-keys. - `#6346 `_: Use addresses instead of keys for encrypt, decrypt, sign & verify. - `#6366 `_: Expose info about the signing key. - `#6368 `_: Port keymanager to the new soledad async API. - `#6815 `_: Fetched keys from other domain than its provider are set as 'Weak Chain' validation level. - `KeyManager.put_key` now accepts also ascii keys. Bugfixes ~~~~~~~~ - `#6022 `_: Fix call to python-gnupg's ``verify_file()`` method. - `#7188 `_: Remove the dependency on ``enum34``. - `#7274 `_: use async events api. - `#7410 `_: add logging to fetch_key. - `#7410 `_: catch request exceptions on key fetching. - `#7420 `_: don't repush a public key with different address. - `#7498 `_: self-repair the keyring if keys get duplicated. - Don't repush a public key with different addres - More verbosity in ``get_key`` wrong address log. - Return always ``KeyNotFound`` failure if fetch keys fails on an unknown error. - Use ``ca_bundle`` when fetching keys by url. Misc (CI, tests, refactor, packaging) ------------------------------------- - Cleanup API. - Packaging improvements. - Style changes. - Tests updates. ---- Common ====== Features ~~~~~~~~ - `#7188 `_: Modify ``leap.common.events`` to use ZMQ. Closes #6359. - Add a ``HTTPClient`` the twisted way. - Add close method for http agent. - Allow passing callback to HTTP client. - Bugfix: HTTP timeout was not being cleared on abort. - Bugfix: do not add a port string to non-tcp addresses. - Fix code style and tests. - Make https client use Twisted SSL validation and adds a reuse by default behavior on connection pool Bugfixes ~~~~~~~~ - `#6994 `_: Fix time comparison between local and UTC times that caused the VPN certificates not being correctly downloaded on time. - `#7089 `_: Fix regexp to allow ipc protocol in zmq sockets. - `#7130 `_: Remove extraneous data from events logs. - `#7234 `_: Add http request timeout. - `#7259 `_: Add a flag to disable events framework. - `#7274 `_: Expose async methods for events. - `#7512 `_: Consider standalone flag when saving events certificates. - Fix wrong ca_cert path inside bundle. - Workaround for deadlock problem in zmq auth. ---- Mail ==== Features ~~~~~~~~ - `#3879 `_: Parse OpenPGP header and import keys from it. - `#4692 `_: Don't add any footer to the emails. - `#5359 `_: Adapt to new events api on leap.common. - `#5937 `_: Discover public keys via attachment. - `#6357 `_: Create a ``OutgoingMail`` class that has the logic for encrypting, signing and sending messages. Factors that logic out of ``EncryptedMessage`` so it can be used by other clients. - `#6361 `_: Refactor email fetching outside IMAP to its own independient ``IncomingMail`` class. - `#6617 `_: Add public key as attachment. - `#6742 `_: Add listener for each email added to inbox in IncomingMail. - `#6996 `_: Ability to reindex local UIDs after a soledad sync. - Add very basic support for message sequence numbers. - Expose generic and protocol-agnostic public mail API. - Lots of style fixes and tests updates. - Make use of the twisted-based, async soledad API. - Send a BYE command to all open connections, so that the MUA is notified when the server is shutted down. Bugfixes ~~~~~~~~ - `#6601 `_: Port ``enum`` to ``enum34``. - `#7169 `_: Update SMTP gateway docs. - `#7244 `_: Fix nested multipart rendering. - `#7430 `_: If the auth token has expired signal the GUI to request her to log in again. - `#7471 `_: Disable local only tcp bind on docker containers to allow access to IMAP and SMTP. - `#7480 `_: Don't extract openpgp header if valid attached key. - Bugfix: Return the first cdoc if no body found - Bugfix: fix keyerror when inserting msg on ``pending_inserts`` dict. - Bugfix: fixed syntax error in ``models.py``.