Merge tag '0.8.0'

Release 0.8.0

8 files changed, 696 insertions, 0 deletions

diff --git a/doc/services/couchdb.md b/doc/services/couchdb.md
new file mode 100644
index 00000000..cc40dc32
--- /dev/null
+++ b/doc/services/couchdb.md
@@ -0,0 +1,159 @@
+@title = "couchdb"
+@summary = "Data storage for all user data."
+
+Topology
+------------------------
+
+Required:
+
+* Nodes with `couchdb` service must also have `soledad` service, if email is enabled.
+
+Suggested:
+
+* Nodes with `couchdb` service communicate heavily with `webapp` and `mx`.
+
+`couchdb` nodes do not need to be reachable from the public internet, although the `soledad` service does require this.
+
+Configuration
+----------------------------
+
+### Nighly dumps
+
+You can do a nightly couchdb data dump by adding this to your node config:
+
+    "couch": {
+      "backup": true
+    }
+
+Data will get dumped to `/var/backups/couchdb`.
+
+### Plain CouchDB
+
+BigCouch is not supported on Platform version 0.8 and higher: only plain CouchDB is possible. For earlier versions, you must do this in order to use plain CouchDB:
+
+    "couch": {
+      "master": true,
+      "pwhash_alg": "pbkdf2"
+    }
+
+Various Tasks
+-------------------------------------------------
+
+### Re-enabling blocked account
+
+When a user account gets destroyed from the webapp, there's still a leftover doc in the identities db so other people can't claim that account without an admin's intervention. You can remove this username reservation through the webapp.
+
+However, here is how you could do it manually, if you wanted to:
+
+grep the identities db for the email address:
+
+    curl -s --netrc-file /etc/couchdb/couchdb.netrc -X GET http://127.0.0.1:5984/identities/_all_docs?include_docs=true|grep test_127@bitmask.net
+
+lookup "id" and "rev" to delete the doc:
+
+    curl -s --netrc-file /etc/couchdb/couchdb.netrc -X DELETE 'http://127.0.0.1:5984/identities/b25cf10f935b58088f0d547fca823265?rev=2-715a9beba597a2ab01851676f12c3e4a'
+
+### How to find out which userstore belongs to which identity?
+
+    /usr/bin/curl -s --netrc-file /etc/couchdb/couchdb.netrc '127.0.0.1:5984/identities/_all_docs?include_docs=true' | grep testuser
+
+    {"id":"665e004870ee17aa4c94331ff3ecb173","key":"665e004870ee17aa4c94331ff3ecb173","value":{"rev":"2-2e335a75c4b79a5c2ef5c9950706fe1b"},"doc":{"_id":"665e004870ee17aa4c94331ff3ecb173","_rev":"2-2e335a75c4b79a5c2ef5c9950706fe1b","user_id":"665e004870ee17aa4c94331ff3cd59eb","address":"testuser@example.org","destination":"testuser@example.org","keys": ...
+
+* search for the "user_id" field
+* in this example testuser@example.org uses the database user-665e004870ee17aa4c94331ff3cd59eb
+
+
+### How much disk space is used by a userstore
+
+Beware that this returns the uncompacted disk size (see http://wiki.apache.org/couchdb/Compaction)
+
+    echo "`curl --netrc -s -X GET 'http://127.0.0.1:5984/user-dcd6492d74b90967b6b874100b7dbfcf'|json_pp|grep disk_size|cut -d: -f 2`/1024"|bc
+
+
+Deprecated BigCouch Tasks
+-----------------------------------------
+
+As of release 0.8, the LEAP platform no longer supports BigCouch. This information is kept here for historical reference.
+
+### Rebalance Cluster
+
+Bigcouch currently does not have automatic rebalancing.
+It will probably be added after merging into couchdb.
+If you add a node, or remove one node from the cluster,
+
+1. make sure you have a backup of all DBs !
+
+1. put the webapp into [[maintenance mode => services/webapp#maintenance-mode]]
+
+1. Stop all services that access the database:
+
+    ```
+    workstation$ leap ssh soledad-nodes
+    server# /etc/init.d/soledad-server stop
+
+    workstation$ leap ssh mx-node
+    server# /etc/init.d/postfix stop
+    server# /etc/init.d/leap-mx stop
+
+    workstation$ leap ssh webapp
+    server# /etc/init.d/nickserver stop
+    ```
+
+    Alternately, you can create a temporary firewall rule to block access (run on couchdb server):
+
+    ```
+    server# iptables -A INPUT -p tcp --dport 5984 --jump REJECT
+    ```
+
+1. dump the dbs:
+
+    ```
+    cd /srv/leap/couchdb/scripts
+    time ./couchdb_dumpall.sh
+    ```
+
+1. delete all dbs
+
+1. shut down old node
+
+1. check the couchdb members
+
+    ```
+    curl -s —netrc-file /etc/couchdb/couchdb.netrc -X GET http://127.0.0.1:5986/nodes/_all_docs
+    curl -s —netrc-file /etc/couchdb/couchdb.netrc http://127.0.0.1:5984/_membership
+    ```
+
+1. remove bigcouch from all nodes
+
+    ```
+    apt-get --purge remove bigcouch
+    ```
+
+1. deploy to all couch nodes
+
+    ```
+    leap deploy couchdb
+    ```
+
+1. most likely, deploy will fail because bigcouch will complain about not all nodes beeing connected. Let the deploy finish, restart the bigcouch service on all nodes and re-deploy:
+
+    ```
+    /etc/init.d/bigcouch restart
+    ```
+
+1. restore the backup
+
+    ```
+    cd /srv/leap/couchdb/scripts
+    time ./couchdb_restoreall.sh
+    ```
+
+### Migrating from BigCouch to plain CouchDB
+
+<%= render :partial => 'docs/platform/common/bigcouch_migration_begin.md' %>
+
+
+<%= render :partial => 'docs/platform/common/bigcouch_migration_end.md' %>
+
+
+<%= render :partial => 'docs/platform/common/bigcouch_migration_finish.md' %>
diff --git a/doc/services/en.md b/doc/services/en.md
new file mode 100644
index 00000000..5d0fec5f
--- /dev/null
+++ b/doc/services/en.md
@@ -0,0 +1,80 @@
+@nav_title = "Services"
+@title = "Guide to node services"
+@summary = ""
+@toc = true
+
+# Introduction
+
+Every node (server) must have one or more `services` defined that determines what role the node performs. For example:
+
+    workstation$ cat nodes/stallman.json
+    {
+      "ip_address": "199.99.99.1",
+      "services": ["webapp", "tor"]
+    }
+
+Here are common questions to ask when adding a new node to your provider:
+
+* **many or few?** Some services benefit from having many nodes, while some services are best run on only one or two nodes.
+* **required or optional?** Some services are required, while others can be left out.
+* **who does the node communicate with?** Some services communicate very heavily with other particular services. Nodes running these services should be close together.
+* **public or private network?** Some services communicate with the public internet, while others only need to communicate with other nodes in the infrastructure.
+
+# Available services
+
+<table class="table table-striped">
+<tr>
+  <th>Service</th>
+  <th>VPN</th>
+  <th>Email</th>
+  <th>Notes</th>
+</tr>
+<tr>
+  <td>webapp</td>
+  <td><i class="fa fa-circle"></i></td>
+  <td><i class="fa fa-circle"></i></td>
+  <td>User control panel, provider API, and support system.</td>
+</tr>
+<tr>
+  <td>couchdb</td>
+  <td><i class="fa fa-circle"></i></td>
+  <td><i class="fa fa-circle"></i></td>
+  <td>Data storage for everything. Private node.</td>
+<td></td>
+</tr>
+<tr>
+  <td>soledad</td>
+  <td><i class="fa fa-circle-o"></i></td>
+  <td><i class="fa fa-circle"></i></td>
+  <td>User data synchronization daemon. Usually paired with <code>couchdb</code> nodes.</td>
+<td></td>
+</tr>
+<tr>
+  <td>mx</td>
+  <td><i class="fa fa-circle-o"></i></td>
+  <td><i class="fa fa-circle"></i></td>
+  <td>Incoming and outgoing MX servers.</td>
+</tr>
+<tr>
+  <td>openvpn</td>
+  <td><i class="fa fa-circle"></i></td>
+  <td><i class="fa fa-circle-o"></i></td>
+  <td>OpenVPN gateways.</td>
+</tr>
+<tr>
+  <td>monitor</td>
+  <td><i class="fa fa-dot-circle-o"></i></td>
+  <td><i class="fa fa-dot-circle-o"></i></td>
+  <td>Nagios monitoring. This service must be on the webapp node.</td>
+</tr>
+<tr>
+  <td>tor</td>
+  <td><i class="fa fa-dot-circle-o"></i></td>
+  <td><i class="fa fa-dot-circle-o"></i></td>
+  <td>Tor exit node.</td>
+</tr>
+</table>
+
+Key: <i class="fa fa-circle"> Required</i>, <i class="fa fa-dot-circle-o"> Optional</i>, <i class="fa fa-circle-o"> Not Used</i>
+
+<%= child_summaries %>
\ No newline at end of file
diff --git a/doc/services/monitor.md b/doc/services/monitor.md
new file mode 100644
index 00000000..576b36a9
--- /dev/null
+++ b/doc/services/monitor.md
@@ -0,0 +1,36 @@
+@title = "monitor"
+@summary = "Nagios monitoring and continuous testing."
+
+The `monitor` node provides a nagios control panel that will give you a view into the health and status of all the servers and all the services. It will also spam you with alerts if something goes down.
+
+Topology
+--------------------------------------
+
+Currently, you can have zero or one `monitor` nodes defined. It is required that the monitor be on the webapp node. It was not designed to be run as a separate node service.
+
+Configuration
+-----------------------------------------------
+
+* `nagios.environments`: By default, the monitor node will monitor all servers in all environments. You can optionally restrict the environments to the ones you specify.
+
+For example:
+
+    {
+      "nagios": {
+        "environments": ["unstable", "production"]
+      }
+    }
+
+Access nagios web
+-----------------------------------------------
+
+*Determine the nagios URL*
+
+    $ leap ls --print domain.name,webapp.domain,ip_address monitor
+    > chameleon  chameleon.bitmask.net, demo.bitmask.net, 199.119.112.10
+
+In this case, you would open `https://demo.bitmask.net/cgi-bin/nagios3` in your browser (or alternately you could use 199.119.112.10 or chameleon.bitmask.net).
+
+*Determine the nagios password*
+
+The username for nagios is always `nagiosadmin`. The password is randomly generated and stored in `secrets.json` under the key `nagios_admin_password`. Note that the login is `nagiosadmin` without underscore, but the entry in secrets.json is with underscores.
diff --git a/doc/services/mx.md b/doc/services/mx.md
new file mode 100644
index 00000000..1a34b660
--- /dev/null
+++ b/doc/services/mx.md
@@ -0,0 +1,35 @@
+@title = "mx"
+@summary = "Incoming and outgoing MX servers."
+
+Topology
+-------------------
+
+`mx` nodes communicate with the public internet, clients, and `couchdb` nodes.
+
+Configuration
+--------------------
+
+### Aliases
+
+Using the `mx.aliases` property, you can specify your own hard-coded email aliases that precedence over the aliases in the user database. The `mx.aliases` property consists of a hash, where source address points to one or more destination addresses.
+
+For example:
+
+`services/mx.json`:
+
+    "mx": {
+      "aliases": {
+        "rook": "crow",
+        "robin": "robin@bird.org",
+        "flock": ["junco@bird.org", "robin", "crow"],
+        "chickadee@avian.org": "chickadee@bird.org",
+        "flicker": ["flicker@bird.org", "flicker@deliver.local"]
+      }
+    }
+
+This example demonstrates several of the features with `mx.aliases`:
+
+1. alias lists: by specifying an array of destination addresses, as in the case of "flock", the single email will get copied to each address.
+1. chained resolution: alias resolution will recursively continue until there are no more matching aliases. For example, "flock" is resolved to "robin", which then gets resolved to "robin@bird.org".
+1. virtual domains: by specifying the full domain, as in the case of "chickadee@avian.org", the alias will work for any domain you want. Of course, the MX record for that domain must point to appropriate MX servers, but otherwise you don't need to do any additional configuration.
+1. local delivery: for testing purposes, it is often useful to copy all incoming mail for a particular address and send those copies to another address. You can do this by adding "@deliver.local" as one of the destination addresses. When "@local.delivery" is found, alias resolution stops and the mail is delivered to that username.
diff --git a/doc/services/openvpn.md b/doc/services/openvpn.md
new file mode 100644
index 00000000..5f15ff07
--- /dev/null
+++ b/doc/services/openvpn.md
@@ -0,0 +1,49 @@
+@title = 'openvpn'
+@summary = "OpenVPN egress gateways"
+
+Topology
+------------------
+
+Currently, `openvpn` service should not be combined with other services on the same node.
+
+Unlike most of the other node types, the `openvpn` nodes do not need access to the database and does not ever communicate with any other nodes (except for the `monitor` node, if used). So, `openvpn` nodes can be placed anywhere without regard to the other nodes.
+
+Configuration
+---------------------
+
+*Essential configuration*
+
+* `openvpn.gateway_address`: The address that OpenVPN daemon is bound to and that VPN clients connect to.
+* `ip_address`: The main IP of the server, and the egress address for outgoing traffic.
+
+For example:
+
+    {
+      "ip_address": "1.1.1.1",
+      "openvpn": {
+        "gateway_address": "2.2.2.2"
+      }
+    }
+
+In this example, VPN clients will connect to 2.2.2.2, but their traffic will appear to come from 1.1.1.1.
+
+Why are two IP addresses needed? Without this, traffic between two VPN users on the same gateway will not get encrypted. This is because the VPN on every client must be configured to allow cleartext traffic for the IP address that is the VPN gateway.
+
+*Optional configuration*
+
+Here is the default configuration:
+
+    "openvpn": {
+      "configuration": {
+        "auth": "SHA1",
+        "cipher": "AES-128-CBC",
+        "fragment": 1400,
+        "keepalive": "10 30",
+        "tls-cipher": "DHE-RSA-AES128-SHA",
+        "tun-ipv6": true
+      },
+      "ports": ["80", "443", "53", "1194"],
+      "protocols": ["tcp", "udp"]
+    }
+
+You may want to change the ports so that only 443 or 80 are used. It is probably best to not modify the `openvpn.configuration` options for now.
\ No newline at end of file
diff --git a/doc/services/soledad.md b/doc/services/soledad.md
new file mode 100644
index 00000000..e2700d06
--- /dev/null
+++ b/doc/services/soledad.md
@@ -0,0 +1,12 @@
+@title = 'soledad'
+@summary = 'User data synchronization daemon'
+
+Topology
+--------------------
+
+Currently, the platform is designed for `soledad` and `couchdb` services to be combined (e.g. every `soledad` node should also be a `couchdb` node). `soledad` nodes might work in isolation, but this is not tested.
+
+Configuration
+------------------------
+
+There are no options to configure for `soledad` nodes.
diff --git a/doc/services/tor.md b/doc/services/tor.md
new file mode 100644
index 00000000..e64b0fe0
--- /dev/null
+++ b/doc/services/tor.md
@@ -0,0 +1,32 @@
+@title = 'tor'
+@summary = 'Tor exit node or hidden service'
+
+Topology
+------------------------
+
+Nodes with `tor` service will run a Tor exit or hidden service, depending on what other service it is paired with:
+
+* `tor` + `openvpn`: when combined with `openvpn` nodes, `tor` will create a Tor exit node to provide extra cover traffic for the VPN. This can be especially useful if there are VPN gateways without much traffic.
+* `tor` + `webapp`: when combined with a `webapp` node, the `tor` service will make the webapp and the API available via .onion hidden service.
+* `tor` stand alone: a regular Tor exit node.
+
+If activated, you can list the hidden service .onion addresses this way:
+
+   leap ls --print tor.hidden_service.address tor
+
+Then just add '.onion' to the end of the printed addresses.
+
+Configuration
+------------------------------
+
+* `tor.bandwidth_rate`: the max bandwidth allocated to Tor, in KB per second, when used as an exit node.
+
+For example:
+
+    {
+      "tor": {
+        "bandwidth_rate": 6550
+      }
+    }
+
+
diff --git a/doc/services/webapp.md b/doc/services/webapp.md
new file mode 100644
index 00000000..1c06d715
--- /dev/null
+++ b/doc/services/webapp.md
@@ -0,0 +1,293 @@
+@title = "webapp"
+@summary = "leap_web user management application and provider API."
+
+Introduction
+------------------------
+
+The service `webapp` will install the web application [[leap_web => https://leap.se/git/leap_web.git]]. It has performs the following functions:
+
+* REST API for user registration and authentication via the Bitmask client.
+* Admin interface to manage users.
+* Client certificate distribution and renewal.
+* User support help tickets.
+
+Coming soon:
+
+* Billing.
+* Customizable and localized user documentation.
+
+The leap_web application is written in Ruby on Rails 3, using CouchDB as the backend data store.
+
+Topology
+-------------------------
+
+Currently, the platform only supports a single `webapp` node, although we hope to change this in the future.
+
+* `webapp` nodes communicate heavily with `couchdb` nodes, but the two can be on separate servers.
+* The `monitor` service, if enabled, must be on the same node as `webapp`.
+
+Configuration
+--------------------------
+
+Essential options:
+
+* `webapp.admin`: An array of usernames that will be blessed with administrative permissions. These admins can delete users, answer help tickets, and so on. These usernames are for users that have registered through the webapp or through the Bitmask client application, NOT the sysadmin usernames lists in the provider directory `users`.
+
+Other options:
+
+* `webapp.engines`: A list of the engines you want enabled in leap_web. Currently, only "support" is available, and it is enabled by default.
+* `webapp.invite_required`: If true, registration requires an invite code. Default is `false`.
+
+For example, `services/webapp.json`:
+
+    {
+      "webapp": {
+        "admins": ["joehill", "ali", "mack_the_turtle"]
+      }
+    }
+
+By putting this in `services/webapp.json`, all the `webapp` nodes will inherit the same admin list.
+
+There are many options in `provider.json` that also control how the webapp behaves. See [[provider-configuration]] for details.
+
+Invite codes
+-------------------
+
+Enabling the invite code functionality will require new users to provide a valid invite code while signing up for a new account. This is turned off by default, allowing all new users to create an account.
+
+Set the `invite_code` option to `true` in `services/webapp.json`:
+
+    {
+      "webapp": {
+        "invite_required": true
+      }
+    }
+
+This only works with LEAP platform 0.8 or higher.
+
+Run `leap deploy` to enable the option.
+
+You can then generate invite codes by logging into the web application with an admin user.
+
+Alternately, you can also generate invite codes with the command line:
+
+    workstation$ leap ssh bumblebee
+    bumblebee# cd /srv/leap/webapp/
+    bumblebee# sudo -u leap-webapp RAILS_ENV=production bundle exec rake "generate_invites[NUM,USES]"
+
+Where `bumblebee` should be replaced with the name of your webapp node.
+
+The **NUM** specifies the amount of codes to generate. The **USES** parameter is optional: By default, all new invite codes can be used once and will then become invalid. If you provide another value for **USES**, you can set a different amount of maximum uses for the codes you generate.
+
+Customization
+---------------------------
+
+The provider directory `files/webapp` can be used to customize the appearance of the webapp. All the files in this directory will get sync'ed to the `/srv/leap/webapp/config/customization` directory of the deployed webapp node.
+
+Files in the `files/webapp` can override view files, locales, and stylesheets in the leap_web app:
+
+For example:
+
+    stylesheets/ -- override files in Rails.root/app/assets/stylesheets
+      tail.scss -- included before all others
+      head.scss -- included after all others
+
+    public/ -- overrides files in Rails.root/public
+      favicon.ico -- custom favicon
+      img/ -- customary directory to put images in
+
+    views/ -- overrides files Rails.root/app/views
+      home/
+        index.html.haml -- this file is what shows up on
+                           the home page
+      pages/
+        privacy-policy.en.md -- this file will override
+                                the default privacy policy
+        terms-of-service.en.md -- this file will override
+                                  the default TOS.
+
+    locales/ -- overrides files in Rails.root/config/locales
+      en.yml -- overrides for English
+      de.yml -- overrides for German
+      and so on...
+
+To interactively develop your customizations before you deploy them, you have two options:
+
+1. Edit a `webapp` node. This approach involves directly modifying the contents of the directory `/srv/leap/webapp/config/customization` on a deployed `webapp` node. This can, and probably should be, a "local" node. When doing this, you may need to restart leap_web in order for changes to take effect (`touch /srv/leap/webapp/tmp/restart.txt`).
+2. Alternately, you can install leap_web to run on your computer and edit files in `config/customization` locally. This approach does not require a provider or a `webapp` node. For more information, see the [leap_web README](https://github.com/leapcode/leap_web).
+
+NOTE: If you add a `tails.scss` or `head.scss` file, then you usually need to run `rake tmp:clear` and restart rails in order for the new stylesheet to get recognized. You should only need to do this once.
+
+Once you have what you want, then copy these files to the local provider directory `files/webapp` so that they will be installed each time you deploy.
+
+Customization tutorial
+----------------------------
+
+This mini-tutorial will walk you through creating a custom "branding" of the leap_web application. We will be creating a provider called "Prehistoric Computer."
+
+Here are the files we are going to create:
+
+    leap_web/config/customization
+    ├── locales
+    │   ├── en.yml
+    │   └── es.yml
+    ├── public
+    │   ├── favicon.ico
+    │   └── img
+    │       └── masthead.png
+    ├── stylesheets
+    │   └── tail.scss
+    └── views
+        └── pages
+            ├── privacy-policy.en.md
+            └── privacy-policy.es.md
+
+All these files are available in the source code in the [[customization.example => https://github.com/leapcode/leap_web/tree/develop/config/customization.example]] directory.
+
+Remember, these files may live different places:
+
+* `user@localmachine$ leap_web/config/customization`: This will be the path if you have checked out a local copy of leap_web.git and are running `rails server` locally in order to test your customizations.
+* `user@localmachine$ PROVIDER/files/webapp`: This is the local provider directory where the files should be put so that they get correctly deployed to webapp nodes.
+* `root@webappnode# /srv/leap/webapp/config/customization`: This is where the files in the local provider directory `PROVIDER/files/webapp` get copied to after a `leap deploy` to a live webapp nodes.
+
+### Override translations
+
+You can add additional locale files in order to change the text used in the existing application and to add translations for string that you added to the application.
+
+In this example, we will be altering the default text for the "login_info" string. In `config/locales/en/home.en.yml` there is this entry:
+
+    en:
+      login_info: "Log in to change your account settings, create support tickets, and manage payments."
+
+We are going to override this with some custom text in English and Spanish:
+
+`leap_web/config/customization/locale/en.yml`:
+
+    en:
+      login_info: Authenticate to change your "Prehistoric Computer" settings.
+
+`leap_web/config/customization/locale/es.yml`:
+
+    es:
+      login_info: Autenticar a cambiar la configuración de "Computer Prehistoria."
+
+Now, the home page of leap_web will use these new strings instead of the default. Remember that you must restart rails in order for new locale files to take effect.
+
+### Override static pages
+
+You can also override any of the static files included with leap_web, such as the privacy policy or terms of service.
+
+Here is how we would create a custom privacy policy in English and Spanish:
+
+`leap_web/config/customization/views/pages/privacy-policy.en.md`:
+
+    # Custom Privacy Policy
+    This is our privacy policy.
+
+`leap_web/config/customization/views/pages/privacy-policy.es.md`:
+
+    # Custom Política de Privacidad
+    Esta es nuestra política de privacidad.
+
+### Add a custom header
+
+Now we will add a custom header to every page. First, we add the images:
+
+    leap_web/config/customization
+        ├── public
+            ├── favicon.ico
+            └── img
+                └── masthead.png
+
+You can create your own, or use the example files in https://github.com/leapcode/leap_web/tree/develop/config/customization.example
+
+Now, we add some custom CSS so that we can style the masthead:
+
+`leap_web/config/customization/stylesheets/tail.scss`
+
+    $custom-color: #66bbaa;
+
+    a {
+      color: $custom-color;
+    }
+
+    //
+    // MASTHEAD
+    //
+
+    #masthead {
+      background-color: $custom-color;
+      border-bottom: none;
+
+      // make the masthead clickable by replacing the
+      // site name link with the masthead image:
+      .title {
+        padding: 0px;
+        .sitename a {
+          display: block;
+          background: url(/img/masthead.png) 0 0 no-repeat;
+          font-size: 0px;
+          height: 100px;
+          background-size: auto 100px;
+        }
+      }
+    }
+
+    // make the home page masthead slightly larger
+    body.home #masthead {
+      .sitename a {
+        height: 150px;
+        background-size: auto 150px;
+      }
+    }
+
+    //
+    // FOOTER
+    //
+
+    #footer .links {
+      background-color: $custom-color;
+    }
+
+NOTE: If you add a `tails.scss` or `head.scss` file, then you usually need to run `rake tmp:clear` and restart rails in order for the new stylesheet to get recognized. You should only need to do this once.
+
+
+Custom Fork
+----------------------------
+
+Sometimes it is easier to maintain your own fork of the leap_web app. You can keep your customizations in that fork instead of in the provider `files/webapp` directory. Or, perhaps you want to add an engine to the application that modifies the app's behavior.
+
+To deploy your own leap_web, modify the provider file `common.json`:
+
+    {
+      "sources": {
+        "webapp": {
+          "revision": "origin/develop",
+          "source": "https://github.com/leapcode/leap_web",
+          "type": "git"
+        }
+      }
+    }
+
+To target only particular environment, modify instead `common.ENV.json`, where ENV is the name of the environment.
+
+See https://github.com/leapcode/leap_web/blob/develop/doc/DEVELOP.md for notes on getting started hacking on leap_web.
+
+Maintenance mode
+------------------
+
+You can put the webapp into maintenance mode by simply dropping a html file to `/srv/leap/webapp/public/system/maintenance.html`. For example:
+
+    workstation$ leap ssh webappnode
+    server# echo "Temporarily down for maintenance. We will be back soon." > /srv/leap/webapp/public/system/maintenance.html
+
+Known problems
+---------------------------
+
+* Client certificates are generated without a CSR. The problem is that this makes the web
+  application extremely vulnerable to denial of service attacks. This was not an issue until we
+  started to allow the possibility of anonymously fetching a client certificate without
+  authenticating first.
+* By its very nature, the user database is vulnerable to enumeration attacks. These are
+  very hard to prevent, because our protocol is designed to allow query of a user database via
+  proxy in order to provide network perspective.