[docs] split development docs

the page was becoming too cluttered.

10 files changed, 266 insertions, 259 deletions

diff --git a/docs/designdocs/index.rst b/docs/designdocs/index.rst
index b6dd6362..cedf5a66 100644
--- a/docs/designdocs/index.rst
+++ b/docs/designdocs/index.rst
@@ -9,5 +9,5 @@ List of design docs
 
 Here you can find a list of links to authoritative sources for the design documents
 
-* (lots of links to leap.se/docs)
+* (all the links to leap.se/docs)
 * Paper on the LEAP architecture https://satsymposium.org/papers/leap.pdf
diff --git a/docs/hacking/architecture.rst b/docs/hacking/architecture.rst
new file mode 100644
index 00000000..c1e654ed
--- /dev/null
+++ b/docs/hacking/architecture.rst
@@ -0,0 +1,41 @@
+.. _architecture:
+
+The Bitmask Architecture
+========================
+
+The Core
+--------
+
+The main bitmask-dev repo orchestrates the launching if the bitmaskd daemon.
+This is a collection of services that launches the vpn and mail services.
+bitmask vpn, mail and keymanager are the main modules, and soledad is one of the
+main dependencies for the mail service.
+
+The Qt gui
+----------
+
+The Qt gui is a minimalistic wrapper that uses PyQt5 to launch the core and
+display a qt-webkit browser rendering the resources served by the core. Its main
+entrypoint is in ``gui/app.py``.
+
+The Javascript UI
+-----------------
+
+A modern javascript app is the main Bitmask Frontend. For instructions on how
+to develop with the js ui, refer to :ref:`how to develop for the UI<uidev>`.
+
+
+The Thunderbird Extension
+~~~~~~~~~~~~~~~~~~~~~~~~~
+
+The development for the Thunderbird Extension happens on `this repo`_.
+This extension gets published to the `mozilla addons page`_.
+
+.. _`this repo`: https://0xacab.org/leap/bitmask_thunderbird
+.. _`mozilla addons page`: https://addons.mozilla.org/en-US/thunderbird/addon/bitmask
+
+
+What Next?
+----------
+
+Have a look at the contribution guide.
diff --git a/docs/hacking/contributing.rst b/docs/hacking/contributing.rst
new file mode 100644
index 00000000..8d835932
--- /dev/null
+++ b/docs/hacking/contributing.rst
@@ -0,0 +1,60 @@
+:LastChangedDate: $LastChangedDate$
+:LastChangedRevision: $LastChangedRevision$
+:LastChangedBy: $LastChangedBy$
+
+.. _contributing:
+
+Contributing
+============
+
+* Send your merge requests to https://0xacab/leap/bitmask-dev, it will be
+  subject to code-review.
+* Please base your branch for master, and keep it rebased when you push.
+* After review, please squash your commits.
+ 
+
+Coding conventions
+---------------------------------
+
+* Follow pep8 for all the python code.
+* Git messages should be informative.
+* There is a pre-commit hook ready to be used in the ``docs/hooks`` folder,
+  alongside some other hooks to do autopep8 on each commit.
+
+.. include:: ../hooks/leap-commit-template.README
+   :literal:
+
+Dependencies
+----------------------------------
+
+We try hard not to introduce any new dependencies at this moment. If you really
+have to, the packages bitmask depends on have to be specified *both* in the
+setup.py and the pip requirements file.
+
+Don't introduce any pinning in the setup.py file, they should go in the
+requirements files (mainly ``pkg/requirements.pip``).
+
+
+Signing your commits
+---------------------------------
+
+For contributors with commit access, you **should** sign all your commits. If
+you are merging some code from external contributors, you should sign their
+commits.
+
+For handy alias for sign and signoff commits from external contributors add to
+your gitconfig::
+
+  [alias]
+  # Usage: git signoff-rebase [base-commit]
+  signoff-rebase = "!GIT_SEQUENCE_EDITOR='sed -i -re s/^pick/e/' sh -c 'git rebase -i $1 && while test -f .git/rebase-merge/interactive; do git commit --amend --signoff --no-edit && git rebase --continue; done' -"
+
+Merging code
+---------------------------------
+
+We avoid merge commits into master, they make a very messy history. Put this
+in your gitconfig to only allow the merges that can be resolved as a
+fast-forward::
+
+  [merge]
+  ff = only  
diff --git a/docs/hacking/devenv.rst b/docs/hacking/devenv.rst
new file mode 100644
index 00000000..d7ad4157
--- /dev/null
+++ b/docs/hacking/devenv.rst
@@ -0,0 +1,68 @@
+.. _devenv:
+
+Setting a Development Environment
+=================================
+
+
+Convencience script
+~~~~~~~~~~~~~~~~~~~
+
+There is an automated script that runs, sequentially, all the commands in the
+section below. In debian-based systems, you can get a fully working development
+environment with::
+
+  make dev-bootstrap
+
+To activate the freshly created virtualenv the next time, you must use `pew`_::
+
+  pew workon bitmask
+
+.. note:: the bootstrap script is, at the moment, quite opinionated. for
+          instance, it installs and depends on pew, it checks out the
+          bitmask-dev repo under ~/leap folder, and it assumes you are using
+          zsh. if you think it should allow more freedom of choices, feel free
+          to open a pull request.
+
+.. _`pew`: https://pypi.python.org/pypi/pew
+
+Manual instructions 
+-------------------
+
+Install the system-wide dependencies. For debian-based systems::
+
+  sudo apt install build-essential python-dev python-virtualenv \
+  libsqlcipher-dev libssl-dev libffi-dev \
+  python-pyqt5 python-pyqt5.qtwebkit
+
+If you are going to be running tests that involve creating a lot of OpenPGP
+keys, and specially in vms, the following is also recommended to speed up
+things::
+
+  sudo apt install haveged
+
+
+Clone the repo. The master branch has the latest code::
+
+  git clone https://0xacab.org/leap/bitmask-dev
+  cd bitmask-dev
+
+Create a virtualenv and activate it::
+
+  virtualenv venv
+  source venv/bin/activate
+
+By the way, if you plan to get into heavy development, you might want to
+consider using something like `pew`_, instead of the plain virtualenv.
+
+Now you should be able to install all the bitmask dependencies::
+
+  make dev-latest-all
+
+You can also install some dependencies that are going to be useful during
+development::
+
+  pip install -r pkg/requirements-dev.pip
+
+What next?
+-----------
+Check out the Bitmask Architecture.
diff --git a/docs/hacking/index.rst b/docs/hacking/index.rst
index f3bad11d..007e87f9 100644
--- a/docs/hacking/index.rst
+++ b/docs/hacking/index.rst
@@ -1,21 +1,20 @@
-:LastChangedDate: $LastChangedDate$
+:LastChangedDate: $LastChangedDate$ 
 :LastChangedRevision: $LastChangedRevision$
 :LastChangedBy: $LastChangedBy$
 
 Hacking
 ========================================
 
-So you want to hack on Bitmask?
-Thanks, and welcome!
+So you want to hack on Bitmask?  Thanks, and welcome!
 
 This document assumes a ``Linux`` environment.
 
-There are also ongoing documents with notes for setting up an :ref:`osx <osx-dev>` and a
-:ref:`windows <win-dev>` working environments too, contribution is very much
-welcome on those docs!
+There are also ongoing documents with notes for setting up an :ref:`OSX
+<osx-dev>` and a :ref:`Windows <win-dev>` working environments too,
+contribution is very much welcome on those docs!
 
 Running tests
----------------------------------
+-------------
 
 Tox is all you need::
 
@@ -33,214 +32,55 @@ If you are developing against a non-published branch of ``leap.common`` or
 This expects ``leap_common`` and ``soledad`` repos to be checked out in the
 parent folder.
 
-.. _devenv:
 
-Setting up the development environment
---------------------------------------
+Architecture
+------------ 
 
-Automated procedure
-~~~~~~~~~~~~~~~~~~~
+There is a small :ref:`explanation of the bitmask components <architecture>`
+that might be helpful to get you started
+with the different moving parts of the Bitmask codebase.
 
-There is an automated script that runs, sequentially, all the commands in the
-section below. In debian-based systems, you can get a fully working development
-environment with::
+User Interface
+--------------
 
-  make dev-bootstrap
+The :ref:`Javascript User Interface <uidev>` has its own guide for developers.
 
-To activate the freshly created virtualenv the next time, you must use `pew`_::
+Setup
+-----
 
-  pew workon bitmask
+Read how to :ref:`setup your python development environment <devenv>` to start coding in no time.
 
-.. note:: the bootstrap script is, at the moment, quite opinionated. for
-          instance, it installs and depends on pew, it checks out the
-          bitmask-dev repo under ~/leap folder, and it assumes you are using
-          zsh. if you think it should allow more freedom of choices, feel free
-          to open a pull request.
-
-Manual instructions 
-~~~~~~~~~~~~~~~~~~~
-
-Install the system-wide dependencies. For debian-based systems::
-
-  sudo apt install build-essential python-dev python-virtualenv \
-  libsqlcipher-dev libssl-dev libffi-dev \
-  python-pyqt5 python-pyqt5.qtwebkit
-
-If you are going to be running tests that involve creating a lot of OpenPGP
-keys, and specially in vms, the following is also recommended to speed up
-things::
-
-  sudo apt install haveged
-
-
-Clone the repo. The master branch has the latest code::
-
-  git clone https://0xacab.org/leap/bitmask-dev
-  cd bitmask-dev
-
-Create a virtualenv and activate it::
-
-  virtualenv venv
-  source venv/bin/activate
-
-By the way, if you plan to get into heavy development, you might want to
-consider using something like `pew`_, instead of the plain virtualenv.
-
-Now you should be able to install all the bitmask dependencies::
-
-  make dev-latest-all
-
-You can also install some dependencies that are going to be useful during
-development::
-
-  pip install -r pkg/requirements-dev.pip
-
-
-.. _`pew`: https://pypi.python.org/pypi/pew
-
-
-Main Bitmask Components
----------------------------------
-
-The Core
-~~~~~~~~
-
-The main bitmask-dev repo orchestrates the launching if the bitmaskd daemon.
-This is a collection of services that launches the vpn and mail services.
-bitmask vpn, mail and keymanager are the main modules, and soledad is one of the
-main dependencies for the mail service.
-
-The Qt gui
-~~~~~~~~~~
-
-The Qt gui is a minimalistic wrapper that uses PyQt5 to launch the core and
-display a qt-webkit browser rendering the resources served by the core. Its main
-entrypoint is in ``gui/app.py``.
-
-The Javascript UI
-~~~~~~~~~~~~~~~~~
-
-A modern javascript app is the main Bitmask Frontend. For instructions on how
-to develop with the js ui, refer to ``ui/README.md``.
-
-The Thunderbird Extension
-~~~~~~~~~~~~~~~~~~~~~~~~~
-
-The development for the Thunderbird Extension happens on `this repo`_.
-This extension gets published to the `mozilla addons page`_.
-
-.. _`this repo`: https://0xacab.org/leap/bitmask_thunderbird
-.. _`mozilla addons page`: https://addons.mozilla.org/en-US/thunderbird/addon/bitmask
-
-
-Debugging Bitmask
----------------------------------
+Debug
+-----
 
 A must-read for debugging the Bitmask Core daemon is the :ref:`manhole HowTo <manhole>`.
 
+Privileges
+----------
 
-Bitmask privileged runner
-----------------------------------
-
-For launching VPN and the firewall, Bitmask needs to run with administrative
-privileges.  In linux, ``bitmask_root`` is the component that runs with root
-privileges. We currently depend on ``pkexec`` and ``polkit`` to execute it as
-root. In order to do that, Bitmask needs to put some policykit helper files in a
-place that is root-writeable.
-
-If you have installed Bitmask from some distro package, these folders should be
-already in place. If you're running the Bitmask bundles, the first time you will
-be prompted to authenticate to allow these helpers to be copied over (or any
-time that these helpers change).
-
-However, if you're running bitmask in a headless environment, you will want to
-copy the helpers manually, without involving pkexec. To do that, use::
-
-  sudo `which bitmask_helpers` install 
-
-You can also uninstall them::
-
-  sudo `which bitmask_helpers` uninstall 
-
-
-.. note: split a Contributing page on its own, this is getting too long/messy
-
-How to contribute code
----------------------------------
-
-* Send your merge requests to https://0xacab/leap/bitmask-dev, it will be
-  subject to code-review.
-* Please base your branch for master, and keep it rebased when you push.
-* After review, please squash your commits.
- 
-
-Coding conventions
----------------------------------
-
-* Follow pep8 for all the python code.
-* Git messages should be informative.
-* There is a pre-commit hook ready to be used in the ``docs/hooks`` folder,
-  alongside some other hooks to do autopep8 on each commit.
-
-.. include:: ../hooks/leap-commit-template.README
-   :literal:
-
-Dependencies
-----------------------------------
-
-We try hard not to introduce any new dependencies at this moment. If you really
-have to, the packages bitmask depends on have to be specified *both* in the
-setup.py and the pip requirements file.
-
-Don't introduce any pinning in the setup.py file, they should go in the
-requirements files (mainly ``pkg/requirements.pip``).
-
-
-Signing your commits
----------------------------------
-
-For contributors with commit access, you **should** sign all your commits. If
-you are merging some code from external contributors, you should sign their
-commits.
-
-For handy alias for sign and signoff commits from external contributors add to
-your gitconfig::
-
-  [alias]
-  # Usage: git signoff-rebase [base-commit]
-  signoff-rebase = "!GIT_SEQUENCE_EDITOR='sed -i -re s/^pick/e/' sh -c 'git rebase -i $1 && while test -f .git/rebase-merge/interactive; do git commit --amend --signoff --no-edit && git rebase --continue; done' -"
-
-Merging code
----------------------------------
-
-We avoid merge commits into master, they make a very messy history. Put this
-in your gitconfig to only allow the merges that can be resolved as a
-fast-forward::
-
-  [merge]
-  ff = only  
+Bitmask VPN needs administrative privileges. For developing, you
+need to be sure that you have :ref:`installed the privileged helpers
+<privileges>` and that you keep them udpated.
 
 
-Making a new release
---------------------
+Contributing
+------------
 
-A checklist for the release process can be found :ref:`here <release>`
+There are some :ref:`guidelines for contributing code <contributing>` that you
+might want to check if you are insterested in developing with Bitmask.
 
-As part of the release we also tag upload snapshots of the ``leap.bitmask_js``
-package, in order to allow installation of the javascript application without
-needing to compile the javascript and html assets. This is done with::
 
-   cd ui
-   make dist-build
+Release
+-------
 
-and then you can upload it to pypi::
+We keep a :ref:`checklist <release-checklist>` for the release process.
 
-   make dist-upload
 
-Contribution ideas
-------------------
+Ideas
+-----
 
-Want to help? Come talk to us on irc or the mailing list!
+Want to help, but you don't know where to start? Come talk to us on irc or the
+mailing list!
 
 Some areas in which we always need contribution are:
 
diff --git a/docs/hacking/privileges.rst b/docs/hacking/privileges.rst
new file mode 100644
index 00000000..3b63d359
--- /dev/null
+++ b/docs/hacking/privileges.rst
@@ -0,0 +1,24 @@
+.. _privileges:
+
+Privilege handling
+==================
+
+For launching VPN and the firewall, Bitmask needs to run with administrative
+privileges.  In linux, ``bitmask_root`` is the component that runs with root
+privileges. We currently depend on ``pkexec`` and ``polkit`` to execute it as
+root. In order to do that, Bitmask needs to put some policykit helper files in a
+place that is root-writeable.
+
+If you have installed Bitmask from some distro package, these folders should be
+already in place. If you're running the Bitmask bundles, the first time you will
+be prompted to authenticate to allow these helpers to be copied over (or any
+time that these helpers change).
+
+However, if you're running bitmask in a headless environment, you will want to
+copy the helpers manually, without involving pkexec. To do that, use::
+
+  sudo `which bitmask_helpers` install 
+
+You can also uninstall them::
+
+  sudo `which bitmask_helpers` uninstall 
diff --git a/docs/hacking/release.rst b/docs/hacking/release.rst
index 20f7c6eb..aaf5b683 100644
--- a/docs/hacking/release.rst
+++ b/docs/hacking/release.rst
@@ -1,4 +1,4 @@
-.. _release:
+.. _release-checklist:
 
 Bitmask Release Checklist
 =========================
@@ -16,49 +16,29 @@ Version bumps and Tagging
 
 * [ ] Tag everything. Should be done for the following packages, in order:
 * [ ] 1. leap.common
-* [ ] 2. leap.keymanager
 * [ ] 3. leap.soledad
-* [ ] 4. leap.mail
 * [ ] 5. leap.bitmask
 * [ ] 6. leap.mx
 
-* NOTE: It's assumed that origin is the leap.se repo
-
 * [ ] git fetch origin
 * [ ] git tag -l, and see the latest tagged version (unless it's not a minor version bump, in which case, just bump to it)
 * [ ] export version: export RELEASE=0.9.0
-* [ ] git checkout `release/0.9.x`
-- NOTE: the release branch is created when the first release candidate
-  is tagged, after that the bugfixes and features that are meant to be
-  shipped with the specific version that we are targetting are merged in that branch
-* [ ] git checkout -b release/$RELEASE (this is a LOCAL branch, never published).
 * [ ] (maybe) cherry-pick specific commits
 * [ ] (maybe) add special fixes for this release
-
 * [ ] Review pkg/requirements.pip for everything, update if needed (that's why the order).
   - See whatever has been introduced in changes/VERSION_COMPAT
   - Reset changes/VERSION_COMPAT
   - Bump all the leap-requirements altogether.
 * [ ] git commit -am "Update requirements file"
-
-* [ ] Merge changes/next-changelog.rst into the CHANGELOG
-  - NOTE: in leap.soledad, 3 sections (common, client, server).
-* [ ] reset changes/next-changelog.rst
 * [ ] git commit -S -m "[pkg] Update changelog"
-
 * [ ] git tag --sign $RELEASE -m "Tag version $RELEASE"
-
-* If everything went ok, push the changes, and merge back into master&develop:
-* [ ] git checkout release/0.9.x && git merge $RELEASE
-* [ ] git push origin release/0.9.x
-* [ ] git push origin $RELEASE
-* [ ] git checkout master && git pull origin master && git merge --no-edit $RELEASE
-* [ ] git checkout develop && git merge $RELEASE && git push origin develop
+* If everything went ok, push the tag.
+* [ ] cd ui && make dist-build && make dist-upload
 
 Bundles
 -------
-* [ ] Build and upload bundles
-* [ ] Use 'make pyinst-linux' to build bundles.
+* [ ] Build and upload bundles:
+      [ ] make bundle_in_docker
 * [ ] Sign: make pyinst-sign
 * [ ] Upload bundle and signature to downloads.leap.se/client/<os>/Bitmask-<os>-<ver>.(tar.bz2,dmg,zip)
 * [ ] make pyinst-upload
@@ -66,26 +46,10 @@ Bundles
 * [ ] ~/public/client/Bitmask-<os>-latest
 * [ ] ~/public/client/Bitmask-<os>-latest.asc
 
-TUF: Relese candidate bundles: RC# (skipped for now)
-----------------------------------------------------
-
-* [ ] Upload the TUF unstable repo
-* [ ] Upload bundle to staging for release-candidate
-* [ ] Sign the bundles, move it to client downloads (micah)
-* [ ] Update symlinks for -latest
-* [ ] Fix all show stoppers
-
-TUF: Stable bundles (skipped for now)
--------------------------------------
-* [ ] Upload the TUF Stable Repo to staging
-* [ ] Upload bundle to staging for stable
-* [ ] move and sign the TUF repo (kwadro)
-* [ ] Sign the bundles, move it to client downloads (micah)
-* [ ] Update symlinks for -latest
-  
 Debian packages
 ---------------
-* TBD...
+* [ ] update changelog
+* [ ] upload staging packages to release component
 
 Pypi upload
 ---------------
@@ -98,5 +62,3 @@ Announcing
  * [ ] Twitter
  * [ ] Gnusocial
  * [ ] Post in leap.se
- * [ ] reddit
- * [ ] hackernews
diff --git a/docs/hacking/uidev.rst b/docs/hacking/uidev.rst
new file mode 100644
index 00000000..4ada825e
--- /dev/null
+++ b/docs/hacking/uidev.rst
@@ -0,0 +1,6 @@
+.. _uidev:
+
+Bitmask's Javascript User Interface
+====================================
+
+(stuff goes here).
diff --git a/docs/installation/index.rst b/docs/installation/index.rst
index 11561c30..39efb568 100644
--- a/docs/installation/index.rst
+++ b/docs/installation/index.rst
@@ -5,23 +5,28 @@
 .. _install:
 
 
-Installation and packaging 
-==========================
+Installation
+============
 
-Here you can find instructions for developers and advanced users. For **user instructions**, you should refer to the official `Bitmask Install Guide`_. You should only need to read the following sections if:
+Here you can find instructions for developers and advanced users. For **user
+instructions**, you should refer to the official `Bitmask Install Guide`_. You
+should only need to read the following sections if:
 
 * You plan to contribute code to bitmask core libraries.
 * You intend to develop the Bitmask JS User Interface.
 * You are a prospective maintainer for some platform yet unsupported.
-* Your platform is unsupported by the official packages, and you want to install the python packages in your system.
+* Your platform is unsupported by the official packages, and you want to
+  install the python packages in your system.
 
-If you want to contribute translations to some of these sections, please get in touch with us, it will be greatly appreciated to extend the community.
+If you want to contribute translations to some of these sections, please get in
+touch with us, it will be greatly appreciated to extend the community.
 
 .. _`Bitmask Install Guide`: https://bitmask.net/en/install
 
+.. _pip:
 
-Installing with pip
--------------------
+With Pip
+--------
 
 If we still do not provide packages for your platform (debian/ubuntu only at the moment), and for some reason you cannot run the bundles we offer for download, you still should be able to run bitmask downloading the packages from pypi. First you will need some dependencies in your system, that very probably will be provided by your package manager::
 
@@ -36,14 +41,15 @@ If you want also to use the pixelated MUA, you need to install an additional ext
   pip install leap.bitmask[pixelated]
 
 
-Running latest code
--------------------
+From git
+--------
 
-If you want to run latest code from git, you can refer to the :ref:`setting up the development environment <devenv>` section to
-learn how to run Bitmask from the latest code in the master branch.
+If you want to run latest code from git, you can refer to the :ref:`setting up
+the development environment <devenv>` section to learn how to run Bitmask from
+the latest code in the master branch.
 
-Building new bundles
---------------------
+Building bundles
+----------------
 
 The standalone bundles are built with PyInstaller.
 
@@ -76,7 +82,7 @@ A new bundle is created by the CI for every commit using this procedure
 involving docker, you can read more about the bundles in the :ref:`qa section
 <qa>`.
 
-Debian packages
+Debian & ubuntu 
 ---------------
 
 ubuntu:
diff --git a/docs/testing/index.rst b/docs/testing/index.rst
index 4e0febe9..8d72e9ac 100644
--- a/docs/testing/index.rst
+++ b/docs/testing/index.rst
@@ -4,8 +4,8 @@
 
 .. _qa:
 
-Testing and Reporting
-================================ 
+Testing & QA
+============ 
 
 `Latest bundles`_ for the next release cycle are automatically built by our
 Gitlab CI for every commit in master.
@@ -20,7 +20,7 @@ continuity of the accounts is made at this point):
 .. _`Latest bundles`: https://0xacab.org/leap/bitmask-dev/builds/artifacts/master/download?job=bitmask_latest_bundle
 
 Reporting bugs
---------------------------------
+--------------
 
 * Bug reports go into our `Issue Tracker`_. 
 * `Here`_ is some very good read about what constitutes a `good bug report`_.
@@ -37,7 +37,7 @@ If you want to give a hand testing the unreleased bundles, please follow the
 following tips:
 
 * Focus all your efforts, if possible, on whatever is *the* golden distro at
-  the time of the release.  This currently is: Ubuntu 16.04.x LTS, 64bits, with
+  the time of the release.  This currently is: Ubuntu 17.04.x (zesty), 64bits, with
   Unity as the default desktop environment.
   It's very important to have a reference environment as bug-free as possible,
   before trying to solve issues that are present in other distributions or window