From 5519d02f609a87b0ca47a8e82c116811005b6277 Mon Sep 17 00:00:00 2001 From: "Kali Kaneko (leap communications)" Date: Mon, 3 Oct 2016 20:40:03 -0400 Subject: [docs] revamp bitmask dev docs --- docs/authors.rst | 34 +++++ docs/bitmask-mail-index.rst | 96 ------------- docs/bonafide/index.rst | 14 ++ docs/changelog-next.rst | 35 +++++ docs/changelog.rst | 3 + docs/cli/index.rst | 13 ++ docs/conf.py | 4 +- docs/core/core_api_contract | 41 ++++++ docs/core/index.rst | 13 ++ docs/core_api_contract | 41 ------ docs/designdocs/index.rst | 13 ++ docs/hacking/index.rst | 77 ++++++++++ docs/hacking/release.rst | 102 ++++++++++++++ docs/hooks/leap-autopep8.post-commit.hook | 15 ++ docs/hooks/leap-autopep8.post-commit.hook.ADD | 2 + docs/hooks/leap-commit-template | 7 + docs/hooks/leap-commit-template.README | 47 +++++++ docs/hooks/leap-flake8.pre-commit.hook | 7 + docs/index.rst | 75 ++++++++-- docs/installation/index.rst | 48 +++++++ docs/keymanager-soledad-docs.rst | 77 ---------- docs/keymanager/index.rst | 82 +++++++++++ docs/knownissues.rst | 29 ++++ docs/leap-autopep8.post-commit.hook | 15 -- docs/leap-autopep8.post-commit.hook.ADD | 2 - docs/leap-commit-template | 7 - docs/leap-commit-template.README | 47 ------- docs/leap-flake8.pre-commit.hook | 7 - docs/mail-hacking.rst | 195 -------------------------- docs/mail-journey.rst | 40 ------ docs/mail/hacking.rst | 173 +++++++++++++++++++++++ docs/mail/index.rst | 88 ++++++++++++ docs/mail/journey.rst | 46 ++++++ docs/next-changelog.rst | 35 ----- docs/release_checklist.wiki | 90 ------------ docs/testing-rcs.README | 35 ----- docs/testing/index.rst | 54 +++++++ 37 files changed, 1010 insertions(+), 699 deletions(-) create mode 100644 docs/authors.rst delete mode 100644 docs/bitmask-mail-index.rst create mode 100644 docs/bonafide/index.rst create mode 100644 docs/changelog-next.rst create mode 100644 docs/changelog.rst create mode 100644 docs/cli/index.rst create mode 100755 docs/core/core_api_contract create mode 100644 docs/core/index.rst delete mode 100755 docs/core_api_contract create mode 100644 docs/designdocs/index.rst create mode 100644 docs/hacking/index.rst create mode 100644 docs/hacking/release.rst create mode 100755 docs/hooks/leap-autopep8.post-commit.hook create mode 100755 docs/hooks/leap-autopep8.post-commit.hook.ADD create mode 100644 docs/hooks/leap-commit-template create mode 100644 docs/hooks/leap-commit-template.README create mode 100755 docs/hooks/leap-flake8.pre-commit.hook create mode 100644 docs/installation/index.rst delete mode 100644 docs/keymanager-soledad-docs.rst create mode 100644 docs/keymanager/index.rst create mode 100644 docs/knownissues.rst delete mode 100755 docs/leap-autopep8.post-commit.hook delete mode 100755 docs/leap-autopep8.post-commit.hook.ADD delete mode 100644 docs/leap-commit-template delete mode 100644 docs/leap-commit-template.README delete mode 100755 docs/leap-flake8.pre-commit.hook delete mode 100644 docs/mail-hacking.rst delete mode 100644 docs/mail-journey.rst create mode 100644 docs/mail/hacking.rst create mode 100644 docs/mail/index.rst create mode 100644 docs/mail/journey.rst delete mode 100644 docs/next-changelog.rst delete mode 100644 docs/release_checklist.wiki delete mode 100644 docs/testing-rcs.README create mode 100644 docs/testing/index.rst (limited to 'docs') diff --git a/docs/authors.rst b/docs/authors.rst new file mode 100644 index 0000000..37f3122 --- /dev/null +++ b/docs/authors.rst @@ -0,0 +1,34 @@ +.. _authors: + +List of contributors +================================================ +The following people have contributed in the history of Bitmask. + +All the Code is Copyright 2012-2016 LEAP Encryption Access Project + +* Ruben Pollan meskio at sindominio dot net +* drebs drebs at leap dot se +* elijah elijah at riseup dot net +* Tomás Touceda chiiph at leap dot se +* Ivan Alejandro ivanalejandro0 at gmail dot com +* Kali Kaneko kali at leap dot se +* Micah Anderson micah at riseup dot net +* kwadronaut kwadronaut at leap dot se +* Duda Dornelles ddornell at thoughtworks dot com +* Bruno Wagner Goncalves bwagner at thoughtworks.com +* Parménides GV parmegv at sdf dot org +* irregulator irregulator at riseup dot net +* Paixu Aabuizia PaixuAabuizia at users dot noreply at github dot com +* Neissi Lima neissi.lima at gmail dot com +* k clair kclair at riseup dot net +* antialias antialias at leap dot se +* Jaromil jaromil at dyne dot org + +Updating the authors file +------------------------- + +From the root of the ``bitmask-dev`` repo:: + + pkg/tools/get_authors.sh + +However, beware that some of the codebase from the legacy bitmask_client repo has not been preserved with the original authorship. Do not remove people here just because you don't see the commits they made :) diff --git a/docs/bitmask-mail-index.rst b/docs/bitmask-mail-index.rst deleted file mode 100644 index a2133f4..0000000 --- a/docs/bitmask-mail-index.rst +++ /dev/null @@ -1,96 +0,0 @@ -.. leap.mail documentation master file, created by - sphinx-quickstart on Mon Aug 25 19:19:48 2014. - You can adapt this file completely to your liking, but it should at least - contain the root `toctree` directive. - -leap.mail -========= - -*decentralized and secure mail delivery and synchronization* - -This is the documentation for the ``leap.mail`` module. It is a `twisted`_ -package that allows to receive, process, send and access existing messages using -the `LEAP`_ platform. - -One way to use this library is to let it launch two standard mail services, -``smtp`` and ``imap``, that run as local proxies and interact with a remote -``LEAP`` provider that offers *a soledad syncronization endpoint* and receives -the outgoing email. This is what `Bitmask`_ client does. - -From the release 0.4.0 on, it's also possible to use a protocol-agnostic email -public API, so that third party mail clients can manipulate the data layer. This -is what the awesome MUA in the `Pixelated`_ project is using. - -.. _`twisted`: https://twistedmatrix.com/trac/ -.. _`LEAP`: https://leap.se/en/docs -.. _`Bitmask`: https://bitmask.net/en/features#email -.. _`Pixelated`: https://pixelated-project.org/ - -How does this all work? ------------------------ - -All the underlying data storage and sync is handled by a library called -`soledad`_, which handles encryption, storage and sync. Based on `u1db`_, -documents are stored locally as local ``sqlcipher`` tables, and syncs against -the soledad sync service in the provider. - -OpenPGP key generation and keyring management are handled by another leap -python library: `keymanager`_. - -See :ref:`the life cycle of a leap email ` for an overview of the life cycle -of an email through ``LEAP`` providers. - -.. _`Soledad`: https://leap.se/en/docs/design/soledad -.. _`u1db`: https://en.wikipedia.org/wiki/U1DB -.. _`keymanager`: https://github.com/leapcode/keymanager/ - - -Data model ----------- -.. TODO clear document types documentation. - -The data model at the present moment consists of several *document types* that split email into -different documents that are stored in ``Soledad``. The idea behind this is to -keep clear the separation between *mutable* and *inmutable* parts, and still being able to -reconstruct arbitrarily nested email structures easily. - -Documentation index -=================== - -.. -.. Contents: -.. toctree:: - :maxdepth: 2 - - hacking - -.. intro -.. tutorial - - -API documentation ------------------ - -If you were looking for the documentation of the ``leap.mail`` module, you will -find it here. - -Of special interest is the `public mail api`_, which should remain relatively -stable across the next few releases. - -.. _`public mail api`: api/mail.html#module-mail - - -.. toctree:: - :maxdepth: 2 - - api/leap.mail - - - -Indices and tables -================== - -* :ref:`genindex` -* :ref:`modindex` -* :ref:`search` - diff --git a/docs/bonafide/index.rst b/docs/bonafide/index.rst new file mode 100644 index 0000000..1939a61 --- /dev/null +++ b/docs/bonafide/index.rst @@ -0,0 +1,14 @@ +:LastChangedDate: $LastChangedDate$ +:LastChangedRevision: $LastChangedRevision$ +:LastChangedBy: $LastChangedBy$ + +.. _bonafide: + + +Bonafide +================================ +blah blah + +Using the library +-------------------------------- + diff --git a/docs/changelog-next.rst b/docs/changelog-next.rst new file mode 100644 index 0000000..36b915b --- /dev/null +++ b/docs/changelog-next.rst @@ -0,0 +1,35 @@ +0.10.0 - xxx ++++++++++++++++++++++++++++++++ + +Please add lines to this file, they will be moved to the CHANGELOG.rst during +the next release. + +There are two template lines for each category, use them as reference. + +I've added a new category `Misc` so we can track doc/style/packaging stuff. + +Features +~~~~~~~~ +- `#7965 `_: Add basic keymanagement to the cli. +- `#8265 `_: Add a REST API and bitmask.js library for it. +- `#8400 `_: Add manual provider registration. +- `#8435 `_: Write service tokens to a file for email clients to read. +- `#8486 `_: Fetch smtp cert automatically if missing. +- `#8487 `_: Add change password command. +- Use mail_auth token in the core instead of imap/smtp tokens. + +- `#1234 `_: Description of the new feature corresponding with issue #1234. +- New feature without related issue number. + +Bugfixes +~~~~~~~~ +- `#1235 `_: Description for the fixed stuff corresponding with issue #1235. +- Bugfix without related issue number. + +Other +~~~~~ +- `#1236 `_: Description of the new feature corresponding with issue #1236. +- Some change without issue number. + +Known Issues +~~~~~~~~~~~~ diff --git a/docs/changelog.rst b/docs/changelog.rst new file mode 100644 index 0000000..34ee31f --- /dev/null +++ b/docs/changelog.rst @@ -0,0 +1,3 @@ +Full Changelog +-------------- +TODO: port here the changelog from the legacy repo. diff --git a/docs/cli/index.rst b/docs/cli/index.rst new file mode 100644 index 0000000..130a160 --- /dev/null +++ b/docs/cli/index.rst @@ -0,0 +1,13 @@ +:LastChangedDate: $LastChangedDate$ +:LastChangedRevision: $LastChangedRevision$ +:LastChangedBy: $LastChangedBy$ + +.. _cli: + + +Bitmask CLI +================================ +The command line interface + +Using bitmask from the command line +----------------------------------- diff --git a/docs/conf.py b/docs/conf.py index 52d2450..0339aca 100644 --- a/docs/conf.py +++ b/docs/conf.py @@ -51,7 +51,7 @@ master_doc = 'index' # General information about the project. project = u'Bitmask' -copyright = u'2016, LEAP Encryption Access Project' +copyright = u'2012-2016, LEAP Encryption Access Project' author = u'LEAP Encryption Access Project' # The version info for the project you're documenting, acts as replacement for @@ -144,7 +144,7 @@ html_theme = 'alabaster' # The name of an image file (relative to this directory) to place at the top # of the sidebar. # -# html_logo = None +html_logo = '../pkg/branding/bitmask-sidebar.png' # The name of an image file (relative to this directory) to use as a favicon of # the docs. This file should be a Windows icon file (.ico) being 16x16 or 32x32 diff --git a/docs/core/core_api_contract b/docs/core/core_api_contract new file mode 100755 index 0000000..b70fb8f --- /dev/null +++ b/docs/core/core_api_contract @@ -0,0 +1,41 @@ +#!/usr/bin/env python +# -*- coding: utf-8 -*- +# api_contract.py +# Copyright (C) 2016 LEAP Encryption Acess Project +# +# This program is free software: you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation, either version 3 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program. If not, see . +""" +Display a human-readable representation of the methods that compound the public +api for Bitmask Core. + +The values are meant to be type annotations. +""" + +if __name__ == "__main__": + from leap.bitmask.core.service import BitmaskBackend + from leap.bitmask.core import api + backend = BitmaskBackend() + + print '========= Bitmask Core API ==================' + print + + for key in api.registry: + human_key = key.replace('do_', '').lower() + value = api.registry[key] + + print("{}:\t\t{}".format( + human_key, + ' '.join([x for x in value]))) + print + print '=============================================' diff --git a/docs/core/index.rst b/docs/core/index.rst new file mode 100644 index 0000000..c52dcc1 --- /dev/null +++ b/docs/core/index.rst @@ -0,0 +1,13 @@ +:LastChangedDate: $LastChangedDate$ +:LastChangedRevision: $LastChangedRevision$ +:LastChangedBy: $LastChangedBy$ + +.. _bitmask_core: + +Bitmask Core +================================ +blah blah + +API documentation +-------------------------------- + diff --git a/docs/core_api_contract b/docs/core_api_contract deleted file mode 100755 index b70fb8f..0000000 --- a/docs/core_api_contract +++ /dev/null @@ -1,41 +0,0 @@ -#!/usr/bin/env python -# -*- coding: utf-8 -*- -# api_contract.py -# Copyright (C) 2016 LEAP Encryption Acess Project -# -# This program is free software: you can redistribute it and/or modify -# it under the terms of the GNU General Public License as published by -# the Free Software Foundation, either version 3 of the License, or -# (at your option) any later version. -# -# This program is distributed in the hope that it will be useful, -# but WITHOUT ANY WARRANTY; without even the implied warranty of -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -# GNU General Public License for more details. -# -# You should have received a copy of the GNU General Public License -# along with this program. If not, see . -""" -Display a human-readable representation of the methods that compound the public -api for Bitmask Core. - -The values are meant to be type annotations. -""" - -if __name__ == "__main__": - from leap.bitmask.core.service import BitmaskBackend - from leap.bitmask.core import api - backend = BitmaskBackend() - - print '========= Bitmask Core API ==================' - print - - for key in api.registry: - human_key = key.replace('do_', '').lower() - value = api.registry[key] - - print("{}:\t\t{}".format( - human_key, - ' '.join([x for x in value]))) - print - print '=============================================' diff --git a/docs/designdocs/index.rst b/docs/designdocs/index.rst new file mode 100644 index 0000000..b6dd636 --- /dev/null +++ b/docs/designdocs/index.rst @@ -0,0 +1,13 @@ +:LastChangedDate: $LastChangedDate$ +:LastChangedRevision: $LastChangedRevision$ +:LastChangedBy: $LastChangedBy$ + +.. _designdocs: + +List of design docs +=================== + +Here you can find a list of links to authoritative sources for the design documents + +* (lots of links to leap.se/docs) +* Paper on the LEAP architecture https://satsymposium.org/papers/leap.pdf diff --git a/docs/hacking/index.rst b/docs/hacking/index.rst new file mode 100644 index 0000000..f1991af --- /dev/null +++ b/docs/hacking/index.rst @@ -0,0 +1,77 @@ +:LastChangedDate: $LastChangedDate$ +:LastChangedRevision: $LastChangedRevision$ +:LastChangedBy: $LastChangedBy$ + +Hacking +================================= +blah blah + +Running tests +--------------------------------- + +Tox is all you need:: + + tox + +Test when changes are made to common/soledad +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +If you modify soledad or leap.common packages:: + + tox -e py27-dev + +Setting up the development environment +-------------------------------------- + +Dependencies:: + + apt install ... + +There are different requirements files:: + + ... + +How to contribute +--------------------------------- + +Merge requests to https://0xacab/leap/bitmask-dev + +Coding conventions +--------------------------------- +* pep8 +* pre-commit hook (more utils in docs/hooks folder) + +Pinning +---------------------------------- +Only in the requirements files. + +Signing your commits +--------------------------------- +* For contributors with commit access + +Developing on the gui +--------------------------------- +blah blah. see some other README + +Developing on the Javascript UI +--------------------------------- +blah blah. see the main README + +Developing on the Thunderbird Extension +--------------------------------------- +blah blah + +Making a new release +-------------------- +A checklist for the release process can be found :ref:`here ` + +Contribution ideas +------------------ +Want to help? + +Some areas in which we always need contribution are: + +* Localization of the client (talk to elijah). +* Multiplatform gitlab runners +* Windows and OSX packaging +* Windows Firewall integration for VPN +* Migrating components to py3 (look for vshyba or kali). diff --git a/docs/hacking/release.rst b/docs/hacking/release.rst new file mode 100644 index 0000000..20f7c6e --- /dev/null +++ b/docs/hacking/release.rst @@ -0,0 +1,102 @@ +.. _release: + +Bitmask Release Checklist +========================= + +CI check +-------- +* [ ] Check that all tests are passing! +* [ ] Fix any broken tests. + +Version bumps and Tagging +------------------------- +* [ ] Update pkg/next-release +* [ ] Update release-notes.rst in leap.bitmask if needed. +* [ ] Update version in bitmask_client/pkg/linux/bitmask-root if needed. + +* [ ] Tag everything. Should be done for the following packages, in order: +* [ ] 1. leap.common +* [ ] 2. leap.keymanager +* [ ] 3. leap.soledad +* [ ] 4. leap.mail +* [ ] 5. leap.bitmask +* [ ] 6. leap.mx + +* NOTE: It's assumed that origin is the leap.se repo + +* [ ] git fetch origin +* [ ] git tag -l, and see the latest tagged version (unless it's not a minor version bump, in which case, just bump to it) +* [ ] export version: export RELEASE=0.9.0 +* [ ] git checkout `release/0.9.x` +- NOTE: the release branch is created when the first release candidate + is tagged, after that the bugfixes and features that are meant to be + shipped with the specific version that we are targetting are merged in that branch +* [ ] git checkout -b release/$RELEASE (this is a LOCAL branch, never published). +* [ ] (maybe) cherry-pick specific commits +* [ ] (maybe) add special fixes for this release + +* [ ] Review pkg/requirements.pip for everything, update if needed (that's why the order). + - See whatever has been introduced in changes/VERSION_COMPAT + - Reset changes/VERSION_COMPAT + - Bump all the leap-requirements altogether. +* [ ] git commit -am "Update requirements file" + +* [ ] Merge changes/next-changelog.rst into the CHANGELOG + - NOTE: in leap.soledad, 3 sections (common, client, server). +* [ ] reset changes/next-changelog.rst +* [ ] git commit -S -m "[pkg] Update changelog" + +* [ ] git tag --sign $RELEASE -m "Tag version $RELEASE" + +* If everything went ok, push the changes, and merge back into master&develop: +* [ ] git checkout release/0.9.x && git merge $RELEASE +* [ ] git push origin release/0.9.x +* [ ] git push origin $RELEASE +* [ ] git checkout master && git pull origin master && git merge --no-edit $RELEASE +* [ ] git checkout develop && git merge $RELEASE && git push origin develop + +Bundles +------- +* [ ] Build and upload bundles +* [ ] Use 'make pyinst-linux' to build bundles. +* [ ] Sign: make pyinst-sign +* [ ] Upload bundle and signature to downloads.leap.se/client//Bitmask--.(tar.bz2,dmg,zip) +* [ ] make pyinst-upload +* [ ] Update symbolic link for latest upload and signature: +* [ ] ~/public/client/Bitmask--latest +* [ ] ~/public/client/Bitmask--latest.asc + +TUF: Relese candidate bundles: RC# (skipped for now) +---------------------------------------------------- + +* [ ] Upload the TUF unstable repo +* [ ] Upload bundle to staging for release-candidate +* [ ] Sign the bundles, move it to client downloads (micah) +* [ ] Update symlinks for -latest +* [ ] Fix all show stoppers + +TUF: Stable bundles (skipped for now) +------------------------------------- +* [ ] Upload the TUF Stable Repo to staging +* [ ] Upload bundle to staging for stable +* [ ] move and sign the TUF repo (kwadro) +* [ ] Sign the bundles, move it to client downloads (micah) +* [ ] Update symlinks for -latest + +Debian packages +--------------- +* TBD... + +Pypi upload +--------------- +* [ ] python setup.py sdist upload --sign -i kali@leap.se -r pypi + +Announcing +--------------- +* [ ] Announce (use release-notes.rst) + * [ ] Mail leap@lists.riseup.net + * [ ] Twitter + * [ ] Gnusocial + * [ ] Post in leap.se + * [ ] reddit + * [ ] hackernews diff --git a/docs/hooks/leap-autopep8.post-commit.hook b/docs/hooks/leap-autopep8.post-commit.hook new file mode 100755 index 0000000..cffb1d5 --- /dev/null +++ b/docs/hooks/leap-autopep8.post-commit.hook @@ -0,0 +1,15 @@ +#!/bin/sh + +# Auto pep8 correction as a post-commit hook. +# Thanks to http://victorlin.me/posts/2014/02/05/auto-post-commit-pep8-correction + +echo "[+] running autopep8..." +FILES=$(git diff HEAD^ HEAD --name-only --diff-filter=ACM | grep -e '\.py$') + +for f in $FILES +do + # auto pep8 correction + autopep8 --in-place $f +done + +git status diff --git a/docs/hooks/leap-autopep8.post-commit.hook.ADD b/docs/hooks/leap-autopep8.post-commit.hook.ADD new file mode 100755 index 0000000..b6e07ae --- /dev/null +++ b/docs/hooks/leap-autopep8.post-commit.hook.ADD @@ -0,0 +1,2 @@ + #!/bin/sh + cd .git/hooks && ln -s ../../docs/leap-autopep8.post-commit.hook post-commit diff --git a/docs/hooks/leap-commit-template b/docs/hooks/leap-commit-template new file mode 100644 index 0000000..8a5c7cd --- /dev/null +++ b/docs/hooks/leap-commit-template @@ -0,0 +1,7 @@ +[bug|feat|docs|style|refactor|test|pkg|i18n] ... +... + +- Resolves: #XYZ +- Related: #XYZ +- Documentation: #XYZ +- Releases: XYZ diff --git a/docs/hooks/leap-commit-template.README b/docs/hooks/leap-commit-template.README new file mode 100644 index 0000000..ce8809e --- /dev/null +++ b/docs/hooks/leap-commit-template.README @@ -0,0 +1,47 @@ +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +HOW TO USE THIS TEMPLATE: +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Run `git config commit.template docs/leap-commit-template` or +edit the .git/config for this project and add +`template = docs/leap-commit-template` +under the [commit] block + +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +COMMIT TEMPLATE FORMAT EXPLAINED +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +[type] + + +