diff options
Diffstat (limited to 'docs/hacking')
| -rw-r--r-- | docs/hacking/architecture.rst | 41 | ||||
| -rw-r--r-- | docs/hacking/contributing.rst | 60 | ||||
| -rw-r--r-- | docs/hacking/devenv.rst | 68 | ||||
| -rw-r--r-- | docs/hacking/index.rst | 230 | ||||
| -rw-r--r-- | docs/hacking/privileges.rst | 24 | ||||
| -rw-r--r-- | docs/hacking/release.rst | 52 | ||||
| -rw-r--r-- | docs/hacking/uidev.rst | 6 |
7 files changed, 241 insertions, 240 deletions
diff --git a/docs/hacking/architecture.rst b/docs/hacking/architecture.rst new file mode 100644 index 0000000..c1e654e --- /dev/null +++ b/docs/hacking/architecture.rst @@ -0,0 +1,41 @@ +.. _architecture: + +The Bitmask Architecture +======================== + +The Core +-------- + +The main bitmask-dev repo orchestrates the launching if the bitmaskd daemon. +This is a collection of services that launches the vpn and mail services. +bitmask vpn, mail and keymanager are the main modules, and soledad is one of the +main dependencies for the mail service. + +The Qt gui +---------- + +The Qt gui is a minimalistic wrapper that uses PyQt5 to launch the core and +display a qt-webkit browser rendering the resources served by the core. Its main +entrypoint is in ``gui/app.py``. + +The Javascript UI +----------------- + +A modern javascript app is the main Bitmask Frontend. For instructions on how +to develop with the js ui, refer to :ref:`how to develop for the UI<uidev>`. + + +The Thunderbird Extension +~~~~~~~~~~~~~~~~~~~~~~~~~ + +The development for the Thunderbird Extension happens on `this repo`_. +This extension gets published to the `mozilla addons page`_. + +.. _`this repo`: https://0xacab.org/leap/bitmask_thunderbird +.. _`mozilla addons page`: https://addons.mozilla.org/en-US/thunderbird/addon/bitmask + + +What Next? +---------- + +Have a look at the contribution guide. diff --git a/docs/hacking/contributing.rst b/docs/hacking/contributing.rst new file mode 100644 index 0000000..8d83593 --- /dev/null +++ b/docs/hacking/contributing.rst @@ -0,0 +1,60 @@ +:LastChangedDate: $LastChangedDate$ +:LastChangedRevision: $LastChangedRevision$ +:LastChangedBy: $LastChangedBy$ + +.. _contributing: + +Contributing +============ + +* Send your merge requests to https://0xacab/leap/bitmask-dev, it will be + subject to code-review. +* Please base your branch for master, and keep it rebased when you push. +* After review, please squash your commits. + + +Coding conventions +--------------------------------- + +* Follow pep8 for all the python code. +* Git messages should be informative. +* There is a pre-commit hook ready to be used in the ``docs/hooks`` folder, + alongside some other hooks to do autopep8 on each commit. + +.. include:: ../hooks/leap-commit-template.README + :literal: + +Dependencies +---------------------------------- + +We try hard not to introduce any new dependencies at this moment. If you really +have to, the packages bitmask depends on have to be specified *both* in the +setup.py and the pip requirements file. + +Don't introduce any pinning in the setup.py file, they should go in the +requirements files (mainly ``pkg/requirements.pip``). + + +Signing your commits +--------------------------------- + +For contributors with commit access, you **should** sign all your commits. If +you are merging some code from external contributors, you should sign their +commits. + +For handy alias for sign and signoff commits from external contributors add to +your gitconfig:: + + [alias] + # Usage: git signoff-rebase [base-commit] + signoff-rebase = "!GIT_SEQUENCE_EDITOR='sed -i -re s/^pick/e/' sh -c 'git rebase -i $1 && while test -f .git/rebase-merge/interactive; do git commit --amend --signoff --no-edit && git rebase --continue; done' -" + +Merging code +--------------------------------- + +We avoid merge commits into master, they make a very messy history. Put this +in your gitconfig to only allow the merges that can be resolved as a +fast-forward:: + + [merge] + ff = only diff --git a/docs/hacking/devenv.rst b/docs/hacking/devenv.rst new file mode 100644 index 0000000..d7ad415 --- /dev/null +++ b/docs/hacking/devenv.rst @@ -0,0 +1,68 @@ +.. _devenv: + +Setting a Development Environment +================================= + + +Convencience script +~~~~~~~~~~~~~~~~~~~ + +There is an automated script that runs, sequentially, all the commands in the +section below. In debian-based systems, you can get a fully working development +environment with:: + + make dev-bootstrap + +To activate the freshly created virtualenv the next time, you must use `pew`_:: + + pew workon bitmask + +.. note:: the bootstrap script is, at the moment, quite opinionated. for + instance, it installs and depends on pew, it checks out the + bitmask-dev repo under ~/leap folder, and it assumes you are using + zsh. if you think it should allow more freedom of choices, feel free + to open a pull request. + +.. _`pew`: https://pypi.python.org/pypi/pew + +Manual instructions +------------------- + +Install the system-wide dependencies. For debian-based systems:: + + sudo apt install build-essential python-dev python-virtualenv \ + libsqlcipher-dev libssl-dev libffi-dev \ + python-pyqt5 python-pyqt5.qtwebkit + +If you are going to be running tests that involve creating a lot of OpenPGP +keys, and specially in vms, the following is also recommended to speed up +things:: + + sudo apt install haveged + + +Clone the repo. The master branch has the latest code:: + + git clone https://0xacab.org/leap/bitmask-dev + cd bitmask-dev + +Create a virtualenv and activate it:: + + virtualenv venv + source venv/bin/activate + +By the way, if you plan to get into heavy development, you might want to +consider using something like `pew`_, instead of the plain virtualenv. + +Now you should be able to install all the bitmask dependencies:: + + make dev-latest-all + +You can also install some dependencies that are going to be useful during +development:: + + pip install -r pkg/requirements-dev.pip + +What next? +----------- +Check out the Bitmask Architecture. diff --git a/docs/hacking/index.rst b/docs/hacking/index.rst index f3bad11..007e87f 100644 --- a/docs/hacking/index.rst +++ b/docs/hacking/index.rst @@ -1,21 +1,20 @@ -:LastChangedDate: $LastChangedDate$ +:LastChangedDate: $LastChangedDate$ :LastChangedRevision: $LastChangedRevision$ :LastChangedBy: $LastChangedBy$ Hacking ======================================== -So you want to hack on Bitmask? -Thanks, and welcome! +So you want to hack on Bitmask? Thanks, and welcome! This document assumes a ``Linux`` environment. -There are also ongoing documents with notes for setting up an :ref:`osx <osx-dev>` and a -:ref:`windows <win-dev>` working environments too, contribution is very much -welcome on those docs! +There are also ongoing documents with notes for setting up an :ref:`OSX +<osx-dev>` and a :ref:`Windows <win-dev>` working environments too, +contribution is very much welcome on those docs! Running tests ---------------------------------- +------------- Tox is all you need:: @@ -33,214 +32,55 @@ If you are developing against a non-published branch of ``leap.common`` or This expects ``leap_common`` and ``soledad`` repos to be checked out in the parent folder. -.. _devenv: -Setting up the development environment --------------------------------------- +Architecture +------------ -Automated procedure -~~~~~~~~~~~~~~~~~~~ +There is a small :ref:`explanation of the bitmask components <architecture>` +that might be helpful to get you started +with the different moving parts of the Bitmask codebase. -There is an automated script that runs, sequentially, all the commands in the -section below. In debian-based systems, you can get a fully working development -environment with:: +User Interface +-------------- - make dev-bootstrap +The :ref:`Javascript User Interface <uidev>` has its own guide for developers. -To activate the freshly created virtualenv the next time, you must use `pew`_:: +Setup +----- - pew workon bitmask +Read how to :ref:`setup your python development environment <devenv>` to start coding in no time. -.. note:: the bootstrap script is, at the moment, quite opinionated. for - instance, it installs and depends on pew, it checks out the - bitmask-dev repo under ~/leap folder, and it assumes you are using - zsh. if you think it should allow more freedom of choices, feel free - to open a pull request. - -Manual instructions -~~~~~~~~~~~~~~~~~~~ - -Install the system-wide dependencies. For debian-based systems:: - - sudo apt install build-essential python-dev python-virtualenv \ - libsqlcipher-dev libssl-dev libffi-dev \ - python-pyqt5 python-pyqt5.qtwebkit - -If you are going to be running tests that involve creating a lot of OpenPGP -keys, and specially in vms, the following is also recommended to speed up -things:: - - sudo apt install haveged - - -Clone the repo. The master branch has the latest code:: - - git clone https://0xacab.org/leap/bitmask-dev - cd bitmask-dev - -Create a virtualenv and activate it:: - - virtualenv venv - source venv/bin/activate - -By the way, if you plan to get into heavy development, you might want to -consider using something like `pew`_, instead of the plain virtualenv. - -Now you should be able to install all the bitmask dependencies:: - - make dev-latest-all - -You can also install some dependencies that are going to be useful during -development:: - - pip install -r pkg/requirements-dev.pip - - -.. _`pew`: https://pypi.python.org/pypi/pew - - -Main Bitmask Components ---------------------------------- - -The Core -~~~~~~~~ - -The main bitmask-dev repo orchestrates the launching if the bitmaskd daemon. -This is a collection of services that launches the vpn and mail services. -bitmask vpn, mail and keymanager are the main modules, and soledad is one of the -main dependencies for the mail service. - -The Qt gui -~~~~~~~~~~ - -The Qt gui is a minimalistic wrapper that uses PyQt5 to launch the core and -display a qt-webkit browser rendering the resources served by the core. Its main -entrypoint is in ``gui/app.py``. - -The Javascript UI -~~~~~~~~~~~~~~~~~ - -A modern javascript app is the main Bitmask Frontend. For instructions on how -to develop with the js ui, refer to ``ui/README.md``. - -The Thunderbird Extension -~~~~~~~~~~~~~~~~~~~~~~~~~ - -The development for the Thunderbird Extension happens on `this repo`_. -This extension gets published to the `mozilla addons page`_. - -.. _`this repo`: https://0xacab.org/leap/bitmask_thunderbird -.. _`mozilla addons page`: https://addons.mozilla.org/en-US/thunderbird/addon/bitmask - - -Debugging Bitmask ---------------------------------- +Debug +----- A must-read for debugging the Bitmask Core daemon is the :ref:`manhole HowTo <manhole>`. +Privileges +---------- -Bitmask privileged runner ----------------------------------- - -For launching VPN and the firewall, Bitmask needs to run with administrative -privileges. In linux, ``bitmask_root`` is the component that runs with root -privileges. We currently depend on ``pkexec`` and ``polkit`` to execute it as -root. In order to do that, Bitmask needs to put some policykit helper files in a -place that is root-writeable. - -If you have installed Bitmask from some distro package, these folders should be -already in place. If you're running the Bitmask bundles, the first time you will -be prompted to authenticate to allow these helpers to be copied over (or any -time that these helpers change). - -However, if you're running bitmask in a headless environment, you will want to -copy the helpers manually, without involving pkexec. To do that, use:: - - sudo `which bitmask_helpers` install - -You can also uninstall them:: - - sudo `which bitmask_helpers` uninstall - - -.. note: split a Contributing page on its own, this is getting too long/messy - -How to contribute code ---------------------------------- - -* Send your merge requests to https://0xacab/leap/bitmask-dev, it will be - subject to code-review. -* Please base your branch for master, and keep it rebased when you push. -* After review, please squash your commits. - - -Coding conventions ---------------------------------- - -* Follow pep8 for all the python code. -* Git messages should be informative. -* There is a pre-commit hook ready to be used in the ``docs/hooks`` folder, - alongside some other hooks to do autopep8 on each commit. - -.. include:: ../hooks/leap-commit-template.README - :literal: - -Dependencies ----------------------------------- - -We try hard not to introduce any new dependencies at this moment. If you really -have to, the packages bitmask depends on have to be specified *both* in the -setup.py and the pip requirements file. - -Don't introduce any pinning in the setup.py file, they should go in the -requirements files (mainly ``pkg/requirements.pip``). - - -Signing your commits ---------------------------------- - -For contributors with commit access, you **should** sign all your commits. If -you are merging some code from external contributors, you should sign their -commits. - -For handy alias for sign and signoff commits from external contributors add to -your gitconfig:: - - [alias] - # Usage: git signoff-rebase [base-commit] - signoff-rebase = "!GIT_SEQUENCE_EDITOR='sed -i -re s/^pick/e/' sh -c 'git rebase -i $1 && while test -f .git/rebase-merge/interactive; do git commit --amend --signoff --no-edit && git rebase --continue; done' -" - -Merging code ---------------------------------- - -We avoid merge commits into master, they make a very messy history. Put this -in your gitconfig to only allow the merges that can be resolved as a -fast-forward:: - - [merge] - ff = only +Bitmask VPN needs administrative privileges. For developing, you +need to be sure that you have :ref:`installed the privileged helpers +<privileges>` and that you keep them udpated. -Making a new release --------------------- +Contributing +------------ -A checklist for the release process can be found :ref:`here <release>` +There are some :ref:`guidelines for contributing code <contributing>` that you +might want to check if you are insterested in developing with Bitmask. -As part of the release we also tag upload snapshots of the ``leap.bitmask_js`` -package, in order to allow installation of the javascript application without -needing to compile the javascript and html assets. This is done with:: - cd ui - make dist-build +Release +------- -and then you can upload it to pypi:: +We keep a :ref:`checklist <release-checklist>` for the release process. - make dist-upload -Contribution ideas ------------------- +Ideas +----- -Want to help? Come talk to us on irc or the mailing list! +Want to help, but you don't know where to start? Come talk to us on irc or the +mailing list! Some areas in which we always need contribution are: diff --git a/docs/hacking/privileges.rst b/docs/hacking/privileges.rst new file mode 100644 index 0000000..3b63d35 --- /dev/null +++ b/docs/hacking/privileges.rst @@ -0,0 +1,24 @@ +.. _privileges: + +Privilege handling +================== + +For launching VPN and the firewall, Bitmask needs to run with administrative +privileges. In linux, ``bitmask_root`` is the component that runs with root +privileges. We currently depend on ``pkexec`` and ``polkit`` to execute it as +root. In order to do that, Bitmask needs to put some policykit helper files in a +place that is root-writeable. + +If you have installed Bitmask from some distro package, these folders should be +already in place. If you're running the Bitmask bundles, the first time you will +be prompted to authenticate to allow these helpers to be copied over (or any +time that these helpers change). + +However, if you're running bitmask in a headless environment, you will want to +copy the helpers manually, without involving pkexec. To do that, use:: + + sudo `which bitmask_helpers` install + +You can also uninstall them:: + + sudo `which bitmask_helpers` uninstall diff --git a/docs/hacking/release.rst b/docs/hacking/release.rst index 20f7c6e..aaf5b68 100644 --- a/docs/hacking/release.rst +++ b/docs/hacking/release.rst @@ -1,4 +1,4 @@ -.. _release: +.. _release-checklist: Bitmask Release Checklist ========================= @@ -16,49 +16,29 @@ Version bumps and Tagging * [ ] Tag everything. Should be done for the following packages, in order: * [ ] 1. leap.common -* [ ] 2. leap.keymanager * [ ] 3. leap.soledad -* [ ] 4. leap.mail * [ ] 5. leap.bitmask * [ ] 6. leap.mx -* NOTE: It's assumed that origin is the leap.se repo - * [ ] git fetch origin * [ ] git tag -l, and see the latest tagged version (unless it's not a minor version bump, in which case, just bump to it) * [ ] export version: export RELEASE=0.9.0 -* [ ] git checkout `release/0.9.x` -- NOTE: the release branch is created when the first release candidate - is tagged, after that the bugfixes and features that are meant to be - shipped with the specific version that we are targetting are merged in that branch -* [ ] git checkout -b release/$RELEASE (this is a LOCAL branch, never published). * [ ] (maybe) cherry-pick specific commits * [ ] (maybe) add special fixes for this release - * [ ] Review pkg/requirements.pip for everything, update if needed (that's why the order). - See whatever has been introduced in changes/VERSION_COMPAT - Reset changes/VERSION_COMPAT - Bump all the leap-requirements altogether. * [ ] git commit -am "Update requirements file" - -* [ ] Merge changes/next-changelog.rst into the CHANGELOG - - NOTE: in leap.soledad, 3 sections (common, client, server). -* [ ] reset changes/next-changelog.rst * [ ] git commit -S -m "[pkg] Update changelog" - * [ ] git tag --sign $RELEASE -m "Tag version $RELEASE" - -* If everything went ok, push the changes, and merge back into master&develop: -* [ ] git checkout release/0.9.x && git merge $RELEASE -* [ ] git push origin release/0.9.x -* [ ] git push origin $RELEASE -* [ ] git checkout master && git pull origin master && git merge --no-edit $RELEASE -* [ ] git checkout develop && git merge $RELEASE && git push origin develop +* If everything went ok, push the tag. +* [ ] cd ui && make dist-build && make dist-upload Bundles ------- -* [ ] Build and upload bundles -* [ ] Use 'make pyinst-linux' to build bundles. +* [ ] Build and upload bundles: + [ ] make bundle_in_docker * [ ] Sign: make pyinst-sign * [ ] Upload bundle and signature to downloads.leap.se/client/<os>/Bitmask-<os>-<ver>.(tar.bz2,dmg,zip) * [ ] make pyinst-upload @@ -66,26 +46,10 @@ Bundles * [ ] ~/public/client/Bitmask-<os>-latest * [ ] ~/public/client/Bitmask-<os>-latest.asc -TUF: Relese candidate bundles: RC# (skipped for now) ----------------------------------------------------- - -* [ ] Upload the TUF unstable repo -* [ ] Upload bundle to staging for release-candidate -* [ ] Sign the bundles, move it to client downloads (micah) -* [ ] Update symlinks for -latest -* [ ] Fix all show stoppers - -TUF: Stable bundles (skipped for now) -------------------------------------- -* [ ] Upload the TUF Stable Repo to staging -* [ ] Upload bundle to staging for stable -* [ ] move and sign the TUF repo (kwadro) -* [ ] Sign the bundles, move it to client downloads (micah) -* [ ] Update symlinks for -latest - Debian packages --------------- -* TBD... +* [ ] update changelog +* [ ] upload staging packages to release component Pypi upload --------------- @@ -98,5 +62,3 @@ Announcing * [ ] Twitter * [ ] Gnusocial * [ ] Post in leap.se - * [ ] reddit - * [ ] hackernews diff --git a/docs/hacking/uidev.rst b/docs/hacking/uidev.rst new file mode 100644 index 0000000..4ada825 --- /dev/null +++ b/docs/hacking/uidev.rst @@ -0,0 +1,6 @@ +.. _uidev: + +Bitmask's Javascript User Interface +==================================== + +(stuff goes here). |