[docs] revamp bitmask dev docs

26 files changed, 570 insertions, 259 deletions

diff --git a/docs/authors.rst b/docs/authors.rst
new file mode 100644
index 0000000..37f3122
--- /dev/null
+++ b/docs/authors.rst
@@ -0,0 +1,34 @@
+.. _authors:
+
+List of contributors
+================================================
+The following people have contributed in the history of Bitmask.
+
+All the Code is Copyright 2012-2016 LEAP Encryption Access Project 
+
+* Ruben Pollan meskio at sindominio dot net
+* drebs drebs at leap dot se
+* elijah elijah at riseup dot net
+* Tomás Touceda chiiph at leap dot se
+* Ivan Alejandro ivanalejandro0 at gmail dot com
+* Kali Kaneko kali at leap dot se
+* Micah Anderson micah at riseup dot net
+* kwadronaut kwadronaut at leap dot se
+* Duda Dornelles ddornell at thoughtworks dot com
+* Bruno Wagner Goncalves bwagner at thoughtworks.com
+* Parménides GV parmegv at sdf dot org
+* irregulator irregulator at riseup dot net
+* Paixu Aabuizia PaixuAabuizia at users dot noreply at github dot com
+* Neissi Lima neissi.lima at gmail dot com
+* k clair kclair at riseup dot net
+* antialias antialias at leap dot se
+* Jaromil jaromil at dyne dot org
+
+Updating the authors file
+-------------------------
+
+From the root of the ``bitmask-dev`` repo::
+
+  pkg/tools/get_authors.sh
+
+However, beware that some of the codebase from the legacy bitmask_client repo has not been preserved with the original authorship. Do not remove people here just because you don't see the commits they made :)
diff --git a/docs/bitmask-mail-index.rst b/docs/bitmask-mail-index.rst
deleted file mode 100644
index a2133f4..0000000
--- a/docs/bitmask-mail-index.rst
+++ /dev/null
@@ -1,96 +0,0 @@
-.. leap.mail documentation master file, created by
-   sphinx-quickstart on Mon Aug 25 19:19:48 2014.
-   You can adapt this file completely to your liking, but it should at least
-   contain the root `toctree` directive.
-
-leap.mail
-=========
-
-*decentralized and secure mail delivery and synchronization*
-
-This is the documentation for the ``leap.mail`` module. It is a `twisted`_
-package that allows to receive, process, send and access existing messages using
-the `LEAP`_ platform.
-
-One way to use this library is to let it launch two standard mail services,
-``smtp`` and ``imap``, that run as local proxies and interact with a remote
-``LEAP`` provider that offers *a soledad syncronization endpoint* and receives 
-the outgoing email. This is what `Bitmask`_ client does.
-
-From the release 0.4.0 on, it's also possible to use a protocol-agnostic email
-public API, so that third party mail clients can manipulate the data layer. This
-is what the awesome MUA in the `Pixelated`_ project is using.
-
-.. _`twisted`: https://twistedmatrix.com/trac/
-.. _`LEAP`: https://leap.se/en/docs
-.. _`Bitmask`: https://bitmask.net/en/features#email
-.. _`Pixelated`: https://pixelated-project.org/
-
-How does this all work?
------------------------
-
-All the underlying data storage and sync is handled by a library called
-`soledad`_, which handles encryption, storage and sync. Based on `u1db`_,
-documents are stored locally as local ``sqlcipher`` tables, and syncs against
-the soledad sync service in the provider.
-
-OpenPGP key generation and keyring management are handled by another leap
-python library: `keymanager`_.
-
-See :ref:`the life cycle of a leap email <mail_journey>` for an overview of the life cycle
-of an email through ``LEAP`` providers.
-
-.. _`Soledad`: https://leap.se/en/docs/design/soledad
-.. _`u1db`: https://en.wikipedia.org/wiki/U1DB
-.. _`keymanager`: https://github.com/leapcode/keymanager/
-
-
-Data model
-----------
-.. TODO clear document types documentation.
-
-The data model at the present moment consists of several *document types* that split email into
-different documents that are stored in ``Soledad``. The idea behind this is to
-keep clear the separation between *mutable* and *inmutable* parts, and still being able to
-reconstruct arbitrarily nested email structures easily.
-
-Documentation index
-===================
-
-..
-.. Contents:
-.. toctree::
-   :maxdepth: 2
-
-   hacking
-
-..   intro
-..   tutorial
-
-
-API documentation
------------------
-
-If you were looking for the documentation of the ``leap.mail`` module, you will
-find it here.
-
-Of special interest is the `public mail api`_, which should remain relatively
-stable across the next few releases.
-
-.. _`public mail api`: api/mail.html#module-mail
-
-
-.. toctree::
-   :maxdepth: 2
-
-   api/leap.mail
-
-
-
-Indices and tables
-==================
-
-* :ref:`genindex`
-* :ref:`modindex`
-* :ref:`search`
-
diff --git a/docs/bonafide/index.rst b/docs/bonafide/index.rst
new file mode 100644
index 0000000..1939a61
--- /dev/null
+++ b/docs/bonafide/index.rst
@@ -0,0 +1,14 @@
+:LastChangedDate: $LastChangedDate$
+:LastChangedRevision: $LastChangedRevision$
+:LastChangedBy: $LastChangedBy$
+
+.. _bonafide:
+
+
+Bonafide
+================================
+blah blah
+
+Using the library
+--------------------------------
+
diff --git a/docs/next-changelog.rst b/docs/changelog-next.rst
index f57e2ae..36b915b 100644
--- a/docs/next-changelog.rst
+++ b/docs/changelog-next.rst
@@ -26,8 +26,8 @@ Bugfixes
 - `#1235 <https://leap.se/code/issues/1235>`_: Description for the fixed stuff corresponding with issue #1235.
 - Bugfix without related issue number.
 
-Misc
-~~~~
+Other
+~~~~~
 - `#1236 <https://leap.se/code/issues/1236>`_: Description of the new feature corresponding with issue #1236.
 - Some change without issue number.
 
diff --git a/docs/changelog.rst b/docs/changelog.rst
new file mode 100644
index 0000000..34ee31f
--- /dev/null
+++ b/docs/changelog.rst
@@ -0,0 +1,3 @@
+Full Changelog
+--------------
+TODO: port here the changelog from the legacy repo.
diff --git a/docs/cli/index.rst b/docs/cli/index.rst
new file mode 100644
index 0000000..130a160
--- /dev/null
+++ b/docs/cli/index.rst
@@ -0,0 +1,13 @@
+:LastChangedDate: $LastChangedDate$
+:LastChangedRevision: $LastChangedRevision$
+:LastChangedBy: $LastChangedBy$
+
+.. _cli:
+
+
+Bitmask CLI
+================================
+The command line interface
+
+Using bitmask from the command line
+-----------------------------------
diff --git a/docs/conf.py b/docs/conf.py
index 52d2450..0339aca 100644
--- a/docs/conf.py
+++ b/docs/conf.py
@@ -51,7 +51,7 @@ master_doc = 'index'
 
 # General information about the project.
 project = u'Bitmask'
-copyright = u'2016, LEAP Encryption Access Project'
+copyright = u'2012-2016, LEAP Encryption Access Project'
 author = u'LEAP Encryption Access Project'
 
 # The version info for the project you're documenting, acts as replacement for
@@ -144,7 +144,7 @@ html_theme = 'alabaster'
 # The name of an image file (relative to this directory) to place at the top
 # of the sidebar.
 #
-# html_logo = None
+html_logo = '../pkg/branding/bitmask-sidebar.png'
 
 # The name of an image file (relative to this directory) to use as a favicon of
 # the docs.  This file should be a Windows icon file (.ico) being 16x16 or 32x32
diff --git a/docs/core_api_contract b/docs/core/core_api_contract
index b70fb8f..b70fb8f 100755
--- a/docs/core_api_contract
+++ b/docs/core/core_api_contract
diff --git a/docs/core/index.rst b/docs/core/index.rst
new file mode 100644
index 0000000..c52dcc1
--- /dev/null
+++ b/docs/core/index.rst
@@ -0,0 +1,13 @@
+:LastChangedDate: $LastChangedDate$
+:LastChangedRevision: $LastChangedRevision$
+:LastChangedBy: $LastChangedBy$
+
+.. _bitmask_core:
+
+Bitmask Core
+================================
+blah blah
+
+API documentation
+--------------------------------
+
diff --git a/docs/designdocs/index.rst b/docs/designdocs/index.rst
new file mode 100644
index 0000000..b6dd636
--- /dev/null
+++ b/docs/designdocs/index.rst
@@ -0,0 +1,13 @@
+:LastChangedDate: $LastChangedDate$
+:LastChangedRevision: $LastChangedRevision$
+:LastChangedBy: $LastChangedBy$
+
+.. _designdocs:
+
+List of design docs
+===================
+
+Here you can find a list of links to authoritative sources for the design documents
+
+* (lots of links to leap.se/docs)
+* Paper on the LEAP architecture https://satsymposium.org/papers/leap.pdf
diff --git a/docs/hacking/index.rst b/docs/hacking/index.rst
new file mode 100644
index 0000000..f1991af
--- /dev/null
+++ b/docs/hacking/index.rst
@@ -0,0 +1,77 @@
+:LastChangedDate: $LastChangedDate$
+:LastChangedRevision: $LastChangedRevision$
+:LastChangedBy: $LastChangedBy$
+
+Hacking
+=================================
+blah blah
+
+Running tests
+---------------------------------
+
+Tox is all you need::
+
+  tox
+
+Test when changes are made to common/soledad
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+If you modify soledad or leap.common packages::
+
+  tox -e py27-dev
+
+Setting up the development environment
+--------------------------------------
+
+Dependencies::
+
+  apt install ...
+
+There are different requirements files::
+
+  ...
+
+How to contribute
+---------------------------------
+
+Merge requests to https://0xacab/leap/bitmask-dev
+
+Coding conventions
+---------------------------------
+* pep8
+* pre-commit hook (more utils in docs/hooks folder)
+
+Pinning
+----------------------------------
+Only in the requirements files.
+
+Signing your commits
+---------------------------------
+* For contributors with commit access
+
+Developing on the gui
+---------------------------------
+blah blah. see some other README
+
+Developing on the Javascript UI
+---------------------------------
+blah blah. see the main README
+
+Developing on the Thunderbird Extension
+---------------------------------------
+blah blah
+
+Making a new release
+--------------------
+A checklist for the release process can be found :ref:`here <release>`
+
+Contribution ideas
+------------------
+Want to help?
+
+Some areas in which we always need contribution are:
+
+* Localization of the client (talk to elijah).
+* Multiplatform gitlab runners
+* Windows and OSX packaging
+* Windows Firewall integration for VPN
+* Migrating components to py3 (look for vshyba or kali).
diff --git a/docs/hacking/release.rst b/docs/hacking/release.rst
new file mode 100644
index 0000000..20f7c6e
--- /dev/null
+++ b/docs/hacking/release.rst
@@ -0,0 +1,102 @@
+.. _release:
+
+Bitmask Release Checklist
+=========================
+
+CI check
+--------
+* [ ] Check that all tests are passing!
+* [ ] Fix any broken tests.
+
+Version bumps and Tagging
+-------------------------
+* [ ] Update pkg/next-release
+* [ ] Update release-notes.rst in leap.bitmask if needed.
+* [ ] Update version in bitmask_client/pkg/linux/bitmask-root if needed.
+
+* [ ] Tag everything. Should be done for the following packages, in order:
+* [ ] 1. leap.common
+* [ ] 2. leap.keymanager
+* [ ] 3. leap.soledad
+* [ ] 4. leap.mail
+* [ ] 5. leap.bitmask
+* [ ] 6. leap.mx
+
+* NOTE: It's assumed that origin is the leap.se repo
+
+* [ ] git fetch origin
+* [ ] git tag -l, and see the latest tagged version (unless it's not a minor version bump, in which case, just bump to it)
+* [ ] export version: export RELEASE=0.9.0
+* [ ] git checkout `release/0.9.x`
+- NOTE: the release branch is created when the first release candidate
+  is tagged, after that the bugfixes and features that are meant to be
+  shipped with the specific version that we are targetting are merged in that branch
+* [ ] git checkout -b release/$RELEASE (this is a LOCAL branch, never published).
+* [ ] (maybe) cherry-pick specific commits
+* [ ] (maybe) add special fixes for this release
+
+* [ ] Review pkg/requirements.pip for everything, update if needed (that's why the order).
+  - See whatever has been introduced in changes/VERSION_COMPAT
+  - Reset changes/VERSION_COMPAT
+  - Bump all the leap-requirements altogether.
+* [ ] git commit -am "Update requirements file"
+
+* [ ] Merge changes/next-changelog.rst into the CHANGELOG
+  - NOTE: in leap.soledad, 3 sections (common, client, server).
+* [ ] reset changes/next-changelog.rst
+* [ ] git commit -S -m "[pkg] Update changelog"
+
+* [ ] git tag --sign $RELEASE -m "Tag version $RELEASE"
+
+* If everything went ok, push the changes, and merge back into master&develop:
+* [ ] git checkout release/0.9.x && git merge $RELEASE
+* [ ] git push origin release/0.9.x
+* [ ] git push origin $RELEASE
+* [ ] git checkout master && git pull origin master && git merge --no-edit $RELEASE
+* [ ] git checkout develop && git merge $RELEASE && git push origin develop
+
+Bundles
+-------
+* [ ] Build and upload bundles
+* [ ] Use 'make pyinst-linux' to build bundles.
+* [ ] Sign: make pyinst-sign
+* [ ] Upload bundle and signature to downloads.leap.se/client/<os>/Bitmask-<os>-<ver>.(tar.bz2,dmg,zip)
+* [ ] make pyinst-upload
+* [ ] Update symbolic link for latest upload and signature:
+* [ ] ~/public/client/Bitmask-<os>-latest
+* [ ] ~/public/client/Bitmask-<os>-latest.asc
+
+TUF: Relese candidate bundles: RC# (skipped for now)
+----------------------------------------------------
+
+* [ ] Upload the TUF unstable repo
+* [ ] Upload bundle to staging for release-candidate
+* [ ] Sign the bundles, move it to client downloads (micah)
+* [ ] Update symlinks for -latest
+* [ ] Fix all show stoppers
+
+TUF: Stable bundles (skipped for now)
+-------------------------------------
+* [ ] Upload the TUF Stable Repo to staging
+* [ ] Upload bundle to staging for stable
+* [ ] move and sign the TUF repo (kwadro)
+* [ ] Sign the bundles, move it to client downloads (micah)
+* [ ] Update symlinks for -latest
+  
+Debian packages
+---------------
+* TBD...
+
+Pypi upload
+---------------
+* [ ]  python setup.py sdist upload --sign -i kali@leap.se -r pypi
+
+Announcing
+---------------
+* [ ] Announce (use release-notes.rst)
+ * [ ] Mail leap@lists.riseup.net
+ * [ ] Twitter
+ * [ ] Gnusocial
+ * [ ] Post in leap.se
+ * [ ] reddit
+ * [ ] hackernews
diff --git a/docs/leap-autopep8.post-commit.hook b/docs/hooks/leap-autopep8.post-commit.hook
index cffb1d5..cffb1d5 100755
--- a/docs/leap-autopep8.post-commit.hook
+++ b/docs/hooks/leap-autopep8.post-commit.hook
diff --git a/docs/leap-autopep8.post-commit.hook.ADD b/docs/hooks/leap-autopep8.post-commit.hook.ADD
index b6e07ae..b6e07ae 100755
--- a/docs/leap-autopep8.post-commit.hook.ADD
+++ b/docs/hooks/leap-autopep8.post-commit.hook.ADD
diff --git a/docs/leap-commit-template b/docs/hooks/leap-commit-template
index 8a5c7cd..8a5c7cd 100644
--- a/docs/leap-commit-template
+++ b/docs/hooks/leap-commit-template
diff --git a/docs/leap-commit-template.README b/docs/hooks/leap-commit-template.README
index ce8809e..ce8809e 100644
--- a/docs/leap-commit-template.README
+++ b/docs/hooks/leap-commit-template.README
diff --git a/docs/leap-flake8.pre-commit.hook b/docs/hooks/leap-flake8.pre-commit.hook
index b00fd08..b00fd08 100755
--- a/docs/leap-flake8.pre-commit.hook
+++ b/docs/hooks/leap-flake8.pre-commit.hook
diff --git a/docs/index.rst b/docs/index.rst
index ad57479..ecd9d60 100644
--- a/docs/index.rst
+++ b/docs/index.rst
@@ -3,21 +3,76 @@
    You can adapt this file completely to your liking, but it should at least
    contain the root `toctree` directive.
 
-Bitmask
-===================================
-**your internet encryption toolkit**
+Bitmask: your internet encryption toolkit
+=========================================
 
-Contents:
+Release v\ |version|. (:ref:`Installation <install>` and :ref:`Known Issues
+<issues>`)
 
-.. toctree::
-   :maxdepth: 2
+What is Bitmask?
+-----------------------------------------
+**Bitmask** is the client for the services offered by `the LEAP Platform`_.  The
+services currently deployed are Encrypted Internet Proxy (VPN) and Encrypted Mail.
+
+Bitmask offers a **command-line interface** and a **multiplatform desktop
+client**. It can be also used as a **set of libraries** to communicate with the
+different services from third party applications.
+
+Bitmask is written in python using `Twisted`_  and licensed under the `GPL3`_. The
+Graphical User Interface is written in html+js and uses `PyQt5`_ for serving the
+application.
+
+.. _`the LEAP Platform`: https://github.com/leapcode/leap_platform
+.. _`Twisted`: https://twistedmatrix.com
+.. _`PyQt5`: https://pypi.python.org/pypi/PyQt5
+.. _`GPL3`: http://www.gnu.org/licenses/gpl.txt
+
+Understood! Show me the docs!
+-----------------------------------------
+
+These documents that you are reading are, mostly, a **guide for developers** that want to contribute to the development of Bitmask, and seek to understand better the code organization and the contribution process.
+
+The **authoritative users guide** lives at `bitmask.net`_.
+
+Other important documents about the LEAP Project can be found at the `Official LEAP documentation`_ site. If you ever need an offline copy, you can clone the `repo for the LEAP Docs site`_. That repo contains also the related LEAP Platform documentation and all the latest design documents. Enhancement contributions and new translations are always welcome! Just open a new merge request.
 
+On the contrary, this developers documentation you are reading right now is maintained in the `bitmask-dev`_ git repo itself, and `can also be checked online`_.
 
+Building the docs
+~~~~~~~~~~~~~~~~~
+
+if you want to build these docs locally, you can do::
+
+  make doc
+
+from the topmost folder in the `bitmask-dev`_ repo. Note that you need to have sphinx installed.
+
+.. _`bitmask.net`: https://bitmask.net/
+.. _`Official LEAP documentation`: https://leap.se/docs/
+.. _`repo for the LEAP Docs site`: https://0xacab.org/leap/leap_se
+.. _`bitmask-dev`: https://0xacab.org/leap/bitmask-dev
+.. _`can also be checked online`: https://bitmask.readthedocs.io
+
+
+Contents
+--------
+
+.. toctree::
+   :maxdepth: 2
 
-Indices and tables
-==================
+   installation/index
+   testing/index
+   knownissues
+   hacking/index
+   bundles/index
+   cli/index
+   core/index
+   bonafide/index
+   keymanager/index
+   mail/index
+   changelog
+   designdocs/index
+   authors
 
-* :ref:`genindex`
-* :ref:`modindex`
 * :ref:`search`
 
diff --git a/docs/installation/index.rst b/docs/installation/index.rst
new file mode 100644
index 0000000..ce119e6
--- /dev/null
+++ b/docs/installation/index.rst
@@ -0,0 +1,48 @@
+:LastChangedDate: $LastChangedDate$
+:LastChangedRevision: $LastChangedRevision$
+:LastChangedBy: $LastChangedBy$
+
+.. _install:
+
+
+Installation
+==================
+
+Users: go to the Bitmask Install guide https://bitmask.net/en/install
+
+Translators: improved/new translations welcome!
+
+
+.. toctree::
+   :hidden:
+
+Running latest code
+-------------------
+blah blah
+
+Building new bundles
+--------------------
+blah blah...
+nightly builds...
+building bleeding edge bundles...
+
+Debian packages
+---------------
+
+ubuntu:
+https://bitmask.net/en/install/linux#ubuntu-packages
+
+debian:
+https://bitmask.net/en/install/linux#debian-packages
+
+Experimental repo ...
+
+Building latest packages (TBD).
+
+
+Archlinux
+---------
+
+Not officially supported, but DoctorJellyFace maintains a PKGBUILD that can be found in the `AUR`_ repo.
+
+.. _`AUR`: https://aur.archlinux.org/packages/bitmask_client/
diff --git a/docs/keymanager-soledad-docs.rst b/docs/keymanager/index.rst
index 67055b2..fa3f0c8 100644
--- a/docs/keymanager-soledad-docs.rst
+++ b/docs/keymanager/index.rst
@@ -1,7 +1,12 @@
+.. _keymanager:
+
 =================
-Soledad Documents
+Keymanager
 =================
 
+Soledad documents
+-----------------
+
 KeyManager uses two types of documents for the keyring:
 
 * key document, that stores each gpg key.
diff --git a/docs/knownissues.rst b/docs/knownissues.rst
new file mode 100644
index 0000000..fed28bd
--- /dev/null
+++ b/docs/knownissues.rst
@@ -0,0 +1,29 @@
+.. _issues:
+
+===================
+Known Issues
+===================
+
+VPN
+-------------------
+
+* No VPN UI yet.
+* Only email is supported, but wizard allows you to login to providers that
+  only support VPN.
+
+Wizard
+-------------------
+
+* In the wizard log in / sign up page, the username field gets deselected.
+* The list of providers should have icons, be sortable, filterable.
+* The wizard should look more pretty.
+
+Main window
+-------------------
+
+* UI doesn't subscribe to events yet, won't get updated if user has logged out
+  via the command line interface.
+* Removing an account does not actually clean up all the files associated with
+  that account (need backend code).
+* Collapsing account list looks weird, and is state is not remembered (need
+  backend code).
diff --git a/docs/mail-hacking.rst b/docs/mail/hacking.rst
index 6c49c21..27087e5 100644
--- a/docs/mail-hacking.rst
+++ b/docs/mail/hacking.rst
@@ -1,14 +1,14 @@
 .. _hacking:
 
-========
-Hacking 
-========
+Hacking  on Bitmask Mail
+========================
 
 Some hints oriented to `leap.mail` hackers. These notes are mostly related to
 the imap server, although they probably will be useful for other pieces too.
 
 Don't panic! Just manhole into it
-=================================
+---------------------------------
+
 
 If you want to inspect the objects living in your application memory, in
 realtime, you can manhole into it.
@@ -29,7 +29,8 @@ Did I mention how *awesome* twisted is?? ``:)``
 
 
 Profiling
-=========
+----------
+
 If using ``twistd`` to launch the server, you can use twisted profiling
 capabities::
 
@@ -44,16 +45,9 @@ commands::
 
  LEAP_PROFILE_IMAPCMD=1 bitmask --debug
 
-Offline mode
-============
-
-The client has an ``--offline`` flag that will make the Mail services (imap,
-currently) not try to sync with remote replicas. Very useful during development,
-although you need to login with the remote server at least once before being
-able to use it.
 
 Mutt config
-===========
+------------
 
 You cannot live without mutt? You're lucky! Use the following minimal config
 with the imap service::
@@ -65,30 +59,10 @@ with the imap service::
  set imap_pass=MAHSIKRET
 
 
-Running the service with twistd
-===============================
-
-In order to run the mail service (currently, the imap server only), you will
-need a config with this info::
-
-  [leap_mail]
-  userid = "user@provider"
-  uuid = "deadbeefdeadabad"
-  passwd = "foobar" # Optional
-
-In the ``LEAP_MAIL_CONFIG`` enviroment variable. If you do not specify a password
-parameter, you'll be prompted for it.
 
-In order to get the user uid (uuid), look into the
-``~/.config/leap/leap-backend.conf`` file after you have logged in into your
-provider at least once.
-
-Run the twisted service::
-
-  LEAP_MAIL_CONFIG=~/.leapmailrc twistd -n -y imap-server.tac
-
-Now you can telnet into your local IMAP server and read your mail like a real
-programmer™::
+Debugging IMAP
+------------------------------
+After IMAP service is running, you can telnet into your local IMAP server and read your mail like a real programmer™::
 
   % telnet localhost 1984
   Trying 127.0.0.1...
@@ -112,6 +86,20 @@ Although you probably prefer to use ``offlineimap`` for tests::
   offlineimap -c LEAPofflineimapRC-tests
 
 
+Use ``ngrep`` to obtain live logs of the commands and responses::
+
+  sudo ngrep -d lo -W byline port 1984
+
+
+Thunderbird
+---------------------------
+
+To get verbose output from thunderbird/icedove, set the following environment
+variable::
+
+  NSPR_LOG_MODULES="imap:5" icedove
+
+
 Minimal offlineimap configuration
 ---------------------------------
 
@@ -182,14 +170,4 @@ running::
 You can find several message samples in the ``leap/mail/tests`` folder.
  
 
-Debugging IMAP commands
-=======================
 
-Use ``ngrep`` to obtain logs of the commands::
-
-  sudo ngrep -d lo -W byline port 1984
-
-To get verbose output from thunderbird/icedove, set the following environment
-variable::
-
-  NSPR_LOG_MODULES="imap:5" icedove
diff --git a/docs/mail/index.rst b/docs/mail/index.rst
new file mode 100644
index 0000000..22bdcf4
--- /dev/null
+++ b/docs/mail/index.rst
@@ -0,0 +1,88 @@
+:LastChangedDate: $LastChangedDate$
+:LastChangedRevision: $LastChangedRevision$
+:LastChangedBy: $LastChangedBy$
+
+.. _mail:
+
+Bitmask Mail
+================================
+*decentralized and secure mail delivery and synchronization*
+
+This is the documentation for the ``leap.mail`` module. It is a `twisted`_
+module, hanging from the ``leap.bitmask`` namespace, that allows to receive, process, send and access existing messages using the `LEAP`_ platform.
+
+One way to use this library is to let it launch two standard mail services,
+``smtp`` and ``imap``, that run as local proxies and interact with a remote
+``LEAP`` provider that offers *a soledad syncronization endpoint* and receives 
+the outgoing email. This is what `Bitmask`_ client does.
+
+From the mail release 0.4.0 on, it's also possible to use a protocol-agnostic email
+public API, so that third party mail clients can manipulate the data layer. This
+is what the awesome MUA in the `Pixelated`_ project is using.
+
+From release 0.10 on, the Bitmask Bundles will also ship a branded version of
+the Pixelated User Agent, that will be served locally. This will be one of the
+recommended ways of accessing the user emails. The other will be Thunderbird, by
+using the `Bitmask Thunderbird Extension`_.
+
+Note that this used to be a standalone python package, under the ``leap.mail``
+namespace. It was merged into bitmask repo, so it now lives in the
+``leap.bitmask.mail`` namespace. The `legacy repo`_ will no longer be updated.
+
+.. _`twisted`: https://twistedmatrix.com/trac/
+.. _`LEAP`: https://leap.se/en/docs
+.. _`Bitmask`: https://bitmask.net/en/features#email
+.. _`Pixelated`: https://pixelated-project.org/
+.. _`legacy repo`: https://github.com/leapcode/leap_mail/
+.. _`Bitmask Thunderbird Extension`: https://addons.mozilla.org/en-US/thunderbird/addon/bitmask/
+
+How does Bitmask Mail work?
+----------------------------
+
+All the underlying data storage and sync is handled by a library called
+`soledad`_, which handles encryption, storage and sync. Based on `u1db`_,
+documents are stored locally as local ``sqlcipher`` tables, and syncs against
+the soledad sync service in the provider.
+
+OpenPGP key generation and keyring management are handled by another leap
+python library: `keymanager`_.
+
+See :ref:`the life cycle of a leap email <mail_journey>` for an overview of the life cycle
+of an email through ``LEAP`` providers.
+
+.. _`Soledad`: https://leap.se/en/docs/design/soledad
+.. _`u1db`: https://en.wikipedia.org/wiki/U1DB
+.. _`keymanager`: https://github.com/leapcode/keymanager/
+
+The life cycle of a LEAP Email
+------------------------------
+
+For a better picture, you are invited to read about :ref:`the whole journey of a mail in the LEAP system <journey>`.
+
+
+Data model
+--------------------
+
+.. TODO clear document types documentation.
+
+The data model at the present moment consists of several *document types* that split email into different documents that are stored in ``Soledad``. The idea behind this is to
+keep clear the separation between *mutable* and *inmutable* parts, and still being able to
+reconstruct arbitrarily nested email structures easily.
+
+Authentication
+---------------------
+Currently, IMAP and SMTP are twisted services that are binded to ``localhost``. These services be initialized by the bitmask.core daemon, but they are not tied to any user session. When an use attempts to log in to those services, a ``twisted.cred`` pluggable authentication plugin will try to lookup a ``mail token`` that is stored inside the soledad encrypted storage.
+
+
+
+
+Mail development resources
+--------------------------
+
+Some old notes that might help you while developing or debugging bitmask mail
+issues.
+
+.. toctree::
+
+   hacking
+
diff --git a/docs/mail-journey.rst b/docs/mail/journey.rst
index 7e64f18..20fdf46 100644
--- a/docs/mail-journey.rst
+++ b/docs/mail/journey.rst
@@ -1,7 +1,13 @@
-.. _mail_journey:
+:LastChangedDate: $LastChangedDate$
+:LastChangedRevision: $LastChangedRevision$
+:LastChangedBy: $LastChangedBy$
+
+.. _journey:
+
 
 The life cycle of a LEAP Email
 ==============================
+
 The following are just some notes to facilitate the understanding of the
 leap.mail internals to developers and collaborators.
 
diff --git a/docs/release_checklist.wiki b/docs/release_checklist.wiki
deleted file mode 100644
index 95ff220..0000000
--- a/docs/release_checklist.wiki
+++ /dev/null
@@ -1,90 +0,0 @@
-= Bitmask Release Checklist =
-
-  == CI check ==
-  * [ ] Check that all tests are passing!
-  * [ ] Fix any broken tests.
-
-  == Version bumps and Tagging ==
-  * [ ] Update pkg/next-release
-  * [ ] Update release-notes.rst in leap.bitmask if needed.
-  * [ ] Update version in bitmask_client/pkg/linux/bitmask-root if needed.
-  
-  * [ ] Tag everything. Should be done for the following packages, in order:
-      * [ ] 1. leap.common
-      * [ ] 2. leap.keymanager
-      * [ ] 3. leap.soledad
-      * [ ] 4. leap.mail
-      * [ ] 5. leap.bitmask
-      * [ ] 6. leap.mx
-
-    * NOTE: It's assumed that origin is the leap.se repo
-     
-    * [ ] git fetch origin
-    * [ ] git tag -l, and see the latest tagged version (unless it's not a minor version bump, in which case, just bump to it)
-    * [ ] export version: export RELEASE=0.9.0
-    * [ ] git checkout `release/0.9.x`
-	- NOTE: the release branch is created when the first release candidate
-	  is tagged, after that the bugfixes and features that are meant to be
-	  shipped with the specific version that we are targetting are merged in that branch
-    * [ ] git checkout -b release/$RELEASE (this is a LOCAL branch, never published).
- 	* [ ] (maybe) cherry-pick specific commits
-	* [ ] (maybe) add special fixes for this release
-
-    * [ ] Review pkg/requirements.pip for everything, update if needed (that's why the order).
-          - See whatever has been introduced in changes/VERSION_COMPAT
-          - Reset changes/VERSION_COMPAT
-          - Bump all the leap-requirements altogether.
-    * [ ] git commit -am "Update requirements file"
-
-    * [ ] Merge changes/next-changelog.rst into the CHANGELOG
-          - NOTE: in leap.soledad, 3 sections (common, client, server).
-    * [ ] reset changes/next-changelog.rst
-    * [ ] git commit -S -m "[pkg] Update changelog"
-
-    * [ ] git tag --sign $RELEASE -m "Tag version $RELEASE"
-
-    * If everything went ok, push the changes, and merge back into master&develop:
-    	* [ ] git checkout release/0.9.x && git merge $RELEASE
-    	* [ ] git push origin release/0.9.x
-        * [ ] git push origin $RELEASE
-    	* [ ] git checkout master && git pull origin master && git merge --no-edit $RELEASE
-    	* [ ] git checkout develop && git merge $RELEASE && git push origin develop
-
-  == Bundles ==
-  * [ ] Build and upload bundles
-    * [ ] Use 'make pyinst-linux' to build bundles.
-    * [ ] Sign: make pyinst-sign
-    * [ ] Upload bundle and signature to downloads.leap.se/client/<os>/Bitmask-<os>-<ver>.(tar.bz2,dmg,zip)
-      * [ ] make pyinst-upload
-    * [ ] Update symbolic link for latest upload and signature:
-    	* [ ] ~/public/client/Bitmask-<os>-latest
-    	* [ ] ~/public/client/Bitmask-<os>-latest.asc
-
-    === TUF: Relese candidate bundles: RC# (skipped for now) ===
-	* [ ] Upload the TUF unstable repo
-	* [ ] Upload bundle to staging for release-candidate
-	* [ ] Sign the bundles, move it to client downloads (micah)
-	* [ ] Update symlinks for -latest
-	* [ ] Fix all show stoppers
-
-    === TUF: Stable bundles (skipped for now) ===
-	* [ ] Upload the TUF Stable Repo to staging
-	* [ ] Upload bundle to staging for stable
-	* [ ] move and sign the TUF repo (kwadro)
-	* [ ] Sign the bundles, move it to client downloads (micah)
-	* [ ] Update symlinks for -latest
-  
-  == Debian packages ==
-  * TBD...
-  
-  == Pypi upload ==
-  * [ ]  python setup.py sdist upload --sign -i kali@leap.se -r pypi
-
-  == Announcing ==
-  * [ ] Announce (use release-notes.rst)
-    * [ ] Mail leap@lists.riseup.net
-    * [ ] Twitter
-    * [ ] Gnusocial
-    * [ ] Post in leap.se
-    * [ ] reddit
-    * [ ] hackernews
diff --git a/docs/testing-rcs.README b/docs/testing/index.rst
index b0340f0..1adedba 100644
--- a/docs/testing-rcs.README
+++ b/docs/testing/index.rst
@@ -1,5 +1,25 @@
+:LastChangedDate: $LastChangedDate$
+:LastChangedRevision: $LastChangedRevision$
+:LastChangedBy: $LastChangedBy$
+
+.. _qa:
+
+Testing and Reporting
+================================
+
+Reporting bugs
+--------------------------------
+
+* Bug reports go into our `Issue Tracker`_. 
+* `Here`_ is some very good read about what constitutes a `good bug report`_.
+* Have also a look at the :ref:`Known Issues <issues>` page.
+
+.. _`Issue Tracker`: https://leap.se/code
+.. _`Here`: http://www.chiark.greenend.org.uk/~sgtatham/bugs.html
+.. _`good bug report`: http://www.chiark.greenend.org.uk/~sgtatham/bugs.html
+
 Tips for QA
-------------
+--------------------------------
 
 From time to time, we'll ask the community for help testing a new alpha release
 or a release candidate. Normally, we'll offer a link for the download of a
@@ -9,27 +29,26 @@ bugs.
 
 If you want to give a hand in this process, please follow the following tips:
 
-- Focus all your efforts, if possible, on whatever is *the* golden distro at
-  the time of the release.  This currently is: Ubuntu 14.04.x LTS, 64bits, with
-Unity as the default desktop environment.
+* Focus all your efforts, if possible, on whatever is *the* golden distro at
+  the time of the release.  This currently is: Ubuntu 16.04.x LTS, 64bits, with
+  Unity as the default desktop environment.
   It's very important to have a reference environment as bug-free as possible,
   before trying to solve issues that are present in other distributions or window
   managers.
-- Identify all issues that need help in the QA phase. You can do that going to
+* Identify all issues that need help in the QA phase. You can do that going to
   the bug tracker, and filtering all the issues for a given release that are in
   the QA state.
-- If the issue is solved in your tests for this alpha release, please add a
+* If the issue is solved in your tests for this alpha release, please add a
   comment to the issue stating the results of your tests, and the platform and
   desktop environment in which your tests took place.  But please do not change
   the QA status on the issue. We generally leave this role to the author of the
   original issue, or to the person playing the role of the release QA master.
-- Always test with a newly created account (specially relevant when testing
+* Always test with a newly created account (specially relevant when testing
   email candidates)
-- Always test with the reference Mail User Agent (currently, Thunderbird, in
+* Always test with the reference Mail User Agent (currently, Thunderbird, in
   whatever version is present in the reference distribution).
-- Remove also any thunderbird configuration, start a freshly configured account.
-- If you find a new bug, please make sure that it hasn't already been reported
+* Remove also any thunderbird configuration, start a freshly configured account.
+* If you find a new bug, please make sure that it hasn't already been reported
   in the issue tracker. If you are absolutely certain that you have found a new
   bug, please attach a log of a new bitmask session, which should contain
   *only* the behaviour needed to reproduce the bug you are reporting.
-