diff options
Diffstat (limited to 'pages/docs/platform/services')
| -rw-r--r-- | pages/docs/platform/services/couchdb.md | 88 | ||||
| -rw-r--r-- | pages/docs/platform/services/en.md | 83 | ||||
| -rw-r--r-- | pages/docs/platform/services/monitor.md | 36 | ||||
| -rw-r--r-- | pages/docs/platform/services/mx.md | 12 | ||||
| -rw-r--r-- | pages/docs/platform/services/openvpn.md | 49 | ||||
| -rw-r--r-- | pages/docs/platform/services/soledad.md | 12 | ||||
| -rw-r--r-- | pages/docs/platform/services/tor.md | 32 | ||||
| -rw-r--r-- | pages/docs/platform/services/webapp.md | 115 |
8 files changed, 427 insertions, 0 deletions
diff --git a/pages/docs/platform/services/couchdb.md b/pages/docs/platform/services/couchdb.md new file mode 100644 index 0000000..7bf4e7c --- /dev/null +++ b/pages/docs/platform/services/couchdb.md @@ -0,0 +1,88 @@ +@title = "couchdb" +@summary = "Data storage for all user data." + +Topology +------------------------ + +`couchdb` nodes communicate heavily with `webapp`, `mx`, and `soledad` nodes. Typically, `couchdb` nodes will also have the `soledad` service. + +`couchdb` nodes do not need to be reachable from the public internet, although the `soledad` service does require this. + +Configuration +---------------------------- + +There are no options that should be modified for `couchdb` nodes. + +NOTE: The LEAP platform is designed to support many database nodes. The goal is for you to be able to add nodes and remove nodes and everything should rebalance and work smoothly. Currently, however, we are using a broken CouchDB variant called BigCouch. Until we migrate off BigCouch, you should only have one `couchdb` node. More than one will work most of the time, but there are some bugs that can pop up and that are unfixed. + +Manual Tasks +--------------------- + +### Rebalance Cluster + +Bigcouch currently does not have automatic rebalancing. +It will probably be added after merging into couchdb. +If you add a node, or remove one node from the cluster, + +. make sure you have a backup of all DBs ! + + /srv/leap/couchdb/scripts/couchdb_dumpall.sh + + +. delete all dbs +. shut down old node +. check the couchdb members + + curl -s —netrc-file /etc/couchdb/couchdb.netrc -X GET http://127.0.0.1:5986/nodes/_all_docs + curl -s —netrc-file /etc/couchdb/couchdb.netrc http://127.0.0.1:5984/_membership + + +. remove bigcouch from all nodes + + apt-get --purge remove bigcouch + + +. deploy to all couch nodes + + leap deploy development +couchdb + +. most likely, deploy will fail because bigcouch will complain about not all nodes beeing connected. Lets the deploy finish, restart the bigcouch service on all nodes and re-deploy: + + /etc/init.d/bigcouch restart + + +. restore the backup + + /srv/leap/couchdb/scripts/couchdb_restoreall.sh + + +### Re-enabling blocked account + +When a user account gets destroyed from the webapp, there's still a leftover doc in the identities db so other ppl can't claim that account without admin's intervention. Here's how you delete that doc and therefore enable registration for that particular account again: + +. grep the identities db for the email address: + + curl -s --netrc-file /etc/couchdb/couchdb.netrc -X GET http://127.0.0.1:5984/identities/_all_docs?include_docs=true|grep test_127@bitmask.net + + +. lookup "id" and "rev" to delete the doc: + + curl -s --netrc-file /etc/couchdb/couchdb.netrc -X DELETE 'http://127.0.0.1:5984/identities/b25cf10f935b58088f0d547fca823265?rev=2-715a9beba597a2ab01851676f12c3e4a' + + +### How to find out which userstore belongs to which identity ? + + /usr/bin/curl -s --netrc-file /etc/couchdb/couchdb.netrc '127.0.0.1:5984/identities/_all_docs?include_docs=true' | grep testuser + + {"id":"665e004870ee17aa4c94331ff3ecb173","key":"665e004870ee17aa4c94331ff3ecb173","value":{"rev":"2-2e335a75c4b79a5c2ef5c9950706fe1b"},"doc":{"_id":"665e004870ee17aa4c94331ff3ecb173","_rev":"2-2e335a75c4b79a5c2ef5c9950706fe1b","user_id":"665e004870ee17aa4c94331ff3cd59eb","address":"testuser@example.org","destination":"testuser@example.org","keys": ... + +* search for the "user_id" field +* in this example testuser@example.org uses the database user-665e004870ee17aa4c94331ff3cd59eb + + +### How much disk space is used by a userstore + +Beware that this returns the uncompacted disk size (see http://wiki.apache.org/couchdb/Compaction) + + echo "`curl --netrc -s -X GET 'http://127.0.0.1:5984/user-dcd6492d74b90967b6b874100b7dbfcf'|json_pp|grep disk_size|cut -d: -f 2`/1024"|bc + diff --git a/pages/docs/platform/services/en.md b/pages/docs/platform/services/en.md new file mode 100644 index 0000000..aa4ffc7 --- /dev/null +++ b/pages/docs/platform/services/en.md @@ -0,0 +1,83 @@ +@nav_title = "Services" +@title = "Guide to node services" +@summary = "" +@toc = true + +# Introduction + +Every node (server) must have one or more `services` defined that determines what role the node performs. For example: + + +``` + cat nodes/stallman.json + { + "ip_address": "199.99.99.1", + "services": ["webapp", "tor"] + } +``` + +Here are common questions to ask when adding a new node to your provider: + +* **many or few?** Some services benefit from having many nodes, while some services are best run on only one or two nodes. +* **required or optional?** Some services are required, while others can be left out. +* **who does the node communicate with?** Some services communicate very heavily with other particular services. Nodes running these services should be close together. +* **public or private network?** Some services communicate with the public internet, while others only need to communicate with other nodes in the infrastructure. + +# Available services + +<table class="table table-striped"> +<tr> + <th>Service</th> + <th>VPN</th> + <th>Email</th> + <th>Notes</th> +</tr> +<tr> + <td>webapp</td> + <td><i class="fa fa-circle"></i></td> + <td><i class="fa fa-circle"></i></td> + <td>User control panel, provider API, and support system.</td> +</tr> +<tr> + <td>couchdb</td> + <td><i class="fa fa-circle"></i></td> + <td><i class="fa fa-circle"></i></td> + <td>Data storage for everything. Private node.</td> +<td></td> +</tr> +<tr> + <td>soledad</td> + <td><i class="fa fa-circle-o"></i></td> + <td><i class="fa fa-circle"></i></td> + <td>User data synchronization daemon. Usually paired with <code>couchdb</code> nodes.</td> +<td></td> +</tr> +<tr> + <td>mx</td> + <td><i class="fa fa-circle-o"></i></td> + <td><i class="fa fa-circle"></i></td> + <td>Incoming and outgoing MX servers.</td> +</tr> +<tr> + <td>openvpn</td> + <td><i class="fa fa-circle"></i></td> + <td><i class="fa fa-circle-o"></i></td> + <td>OpenVPN gateways.</td> +</tr> +<tr> + <td>monitor</td> + <td><i class="fa fa-dot-circle-o"></i></td> + <td><i class="fa fa-dot-circle-o"></i></td> + <td>Nagios monitoring. This service must be on the webapp node.</td> +</tr> +<tr> + <td>tor</td> + <td><i class="fa fa-dot-circle-o"></i></td> + <td><i class="fa fa-dot-circle-o"></i></td> + <td>Tor exit node.</td> +</tr> +</table> + +Key: <i class="fa fa-circle"> Required</i>, <i class="fa fa-dot-circle-o"> Optional</i>, <i class="fa fa-circle-o"> Not Used</i> + +<%= child_summaries %>
\ No newline at end of file diff --git a/pages/docs/platform/services/monitor.md b/pages/docs/platform/services/monitor.md new file mode 100644 index 0000000..576b36a --- /dev/null +++ b/pages/docs/platform/services/monitor.md @@ -0,0 +1,36 @@ +@title = "monitor" +@summary = "Nagios monitoring and continuous testing." + +The `monitor` node provides a nagios control panel that will give you a view into the health and status of all the servers and all the services. It will also spam you with alerts if something goes down. + +Topology +-------------------------------------- + +Currently, you can have zero or one `monitor` nodes defined. It is required that the monitor be on the webapp node. It was not designed to be run as a separate node service. + +Configuration +----------------------------------------------- + +* `nagios.environments`: By default, the monitor node will monitor all servers in all environments. You can optionally restrict the environments to the ones you specify. + +For example: + + { + "nagios": { + "environments": ["unstable", "production"] + } + } + +Access nagios web +----------------------------------------------- + +*Determine the nagios URL* + + $ leap ls --print domain.name,webapp.domain,ip_address monitor + > chameleon chameleon.bitmask.net, demo.bitmask.net, 199.119.112.10 + +In this case, you would open `https://demo.bitmask.net/cgi-bin/nagios3` in your browser (or alternately you could use 199.119.112.10 or chameleon.bitmask.net). + +*Determine the nagios password* + +The username for nagios is always `nagiosadmin`. The password is randomly generated and stored in `secrets.json` under the key `nagios_admin_password`. Note that the login is `nagiosadmin` without underscore, but the entry in secrets.json is with underscores. diff --git a/pages/docs/platform/services/mx.md b/pages/docs/platform/services/mx.md new file mode 100644 index 0000000..79f1c4e --- /dev/null +++ b/pages/docs/platform/services/mx.md @@ -0,0 +1,12 @@ +@title = "mx" +@summary = "Incoming and outgoing MX servers." + +Topology +------------------- + +`mx` nodes communicate with the public internet, clients, and `couchdb` nodes. + +Configuration +-------------------- + +There are not values that should be configured for `mx` nodes.
\ No newline at end of file diff --git a/pages/docs/platform/services/openvpn.md b/pages/docs/platform/services/openvpn.md new file mode 100644 index 0000000..5f15ff0 --- /dev/null +++ b/pages/docs/platform/services/openvpn.md @@ -0,0 +1,49 @@ +@title = 'openvpn' +@summary = "OpenVPN egress gateways" + +Topology +------------------ + +Currently, `openvpn` service should not be combined with other services on the same node. + +Unlike most of the other node types, the `openvpn` nodes do not need access to the database and does not ever communicate with any other nodes (except for the `monitor` node, if used). So, `openvpn` nodes can be placed anywhere without regard to the other nodes. + +Configuration +--------------------- + +*Essential configuration* + +* `openvpn.gateway_address`: The address that OpenVPN daemon is bound to and that VPN clients connect to. +* `ip_address`: The main IP of the server, and the egress address for outgoing traffic. + +For example: + + { + "ip_address": "1.1.1.1", + "openvpn": { + "gateway_address": "2.2.2.2" + } + } + +In this example, VPN clients will connect to 2.2.2.2, but their traffic will appear to come from 1.1.1.1. + +Why are two IP addresses needed? Without this, traffic between two VPN users on the same gateway will not get encrypted. This is because the VPN on every client must be configured to allow cleartext traffic for the IP address that is the VPN gateway. + +*Optional configuration* + +Here is the default configuration: + + "openvpn": { + "configuration": { + "auth": "SHA1", + "cipher": "AES-128-CBC", + "fragment": 1400, + "keepalive": "10 30", + "tls-cipher": "DHE-RSA-AES128-SHA", + "tun-ipv6": true + }, + "ports": ["80", "443", "53", "1194"], + "protocols": ["tcp", "udp"] + } + +You may want to change the ports so that only 443 or 80 are used. It is probably best to not modify the `openvpn.configuration` options for now.
\ No newline at end of file diff --git a/pages/docs/platform/services/soledad.md b/pages/docs/platform/services/soledad.md new file mode 100644 index 0000000..e2700d0 --- /dev/null +++ b/pages/docs/platform/services/soledad.md @@ -0,0 +1,12 @@ +@title = 'soledad' +@summary = 'User data synchronization daemon' + +Topology +-------------------- + +Currently, the platform is designed for `soledad` and `couchdb` services to be combined (e.g. every `soledad` node should also be a `couchdb` node). `soledad` nodes might work in isolation, but this is not tested. + +Configuration +------------------------ + +There are no options to configure for `soledad` nodes. diff --git a/pages/docs/platform/services/tor.md b/pages/docs/platform/services/tor.md new file mode 100644 index 0000000..e64b0fe --- /dev/null +++ b/pages/docs/platform/services/tor.md @@ -0,0 +1,32 @@ +@title = 'tor' +@summary = 'Tor exit node or hidden service' + +Topology +------------------------ + +Nodes with `tor` service will run a Tor exit or hidden service, depending on what other service it is paired with: + +* `tor` + `openvpn`: when combined with `openvpn` nodes, `tor` will create a Tor exit node to provide extra cover traffic for the VPN. This can be especially useful if there are VPN gateways without much traffic. +* `tor` + `webapp`: when combined with a `webapp` node, the `tor` service will make the webapp and the API available via .onion hidden service. +* `tor` stand alone: a regular Tor exit node. + +If activated, you can list the hidden service .onion addresses this way: + + leap ls --print tor.hidden_service.address tor + +Then just add '.onion' to the end of the printed addresses. + +Configuration +------------------------------ + +* `tor.bandwidth_rate`: the max bandwidth allocated to Tor, in KB per second, when used as an exit node. + +For example: + + { + "tor": { + "bandwidth_rate": 6550 + } + } + + diff --git a/pages/docs/platform/services/webapp.md b/pages/docs/platform/services/webapp.md new file mode 100644 index 0000000..4e0d171 --- /dev/null +++ b/pages/docs/platform/services/webapp.md @@ -0,0 +1,115 @@ +@title = "webapp" +@summary = "leap_web user management application and provider API." + +Introduction +------------------------ + +The service `webapp` will install the web application [[leap_web => https://leap.se/git/leap_web.git]]. It has performs the following functions: + +* REST API for user registration and authentication via the Bitmask client. +* Admin interface to manage users. +* Client certificate distribution and renewal. +* User support help tickets. + +Coming soon: + +* Billing. +* Customizable and localized user documentation. + +The leap_web application is written in Ruby on Rails 3, using CouchDB as the backend data store. + +Topology +------------------------- + +Currently, the platform only supports a single `webapp` node, although we hope to change this in the future. + +`webapp` nodes communicate heavily with `couchdb` nodes. + +Configuration +-------------------------- + +Essential options: + +* `webapp.admin`: An array of username that will be blessed with administrative permissions. These admins can delete users, answer help tickets, and so on. + +For example, `services/webapp.json`: + + { + "webapp": { + "admins": ["joehill", "ali", "mack_the_turtle"] + } + } + +By putting this in `services/webapp.json`, all the `webapp` nodes will inherit the same admin list. + +Customization +--------------------------- + +The provider directory `files/webapp` can be used to customize the appearance of the webapp. All the files in this directory will get sync'ed to the `/srv/leap/webapp/config/customization` directory of the deployed webapp node. + +Files in the `files/webapp` can override view files, locales, and stylesheets in the leap_web app: + +For example: + + stylesheets/ -- override files in Rails.root/app/assets/stylesheets + tail.scss -- included before all others + head.scss -- included after all others + + public/ -- overrides files in Rails.root/public + favicon.ico -- custom favicon + img/ -- customary directory to put images in + + views/ -- overrides files Rails.root/app/views + home/ + index.html.haml -- this file is what shows up on + the home page + pages/ + privacy-policy.en.md -- this file will override + the default privacy policy + terms-of-service.en.md -- this file will override + the default TOS. + + locales/ -- overrides files in Rails.root/config/locales + en.yml -- overrides for English + de.yml -- overrides for German + and so on... + +To play with these customizations, it is easiest to first modify the contents of the directory `/srv/leap/webapp/config/customization` on the `webapp` node. When doing this, you may need to restart leap_web in order for changes to take effect (`touch /srv/leap/webapp/tmp/restart.txt`). + +Sometimes a `rake tmp:clear` and a rails restart is required to pick up a new stylesheet. + +Once you have what you want, then copy these files to the local provider directory `files/webapp` so that they will be installed each time you deploy. + +Custom Fork +---------------------------- + +Sometimes it is easier to maintain your own fork of the leap_web app. You can keep your customizations in that fork instead of in the provider `files/webapp` directory. Or, perhaps you want to add an engine to the application that modifies the app's behavior. + +To deploy your own leap_web, modify the provider file `common.json`: + + { + "sources": { + "webapp": { + "revision": "origin/develop", + "source": "https://github.com/leapcode/leap_web", + "type": "git" + } + } + } + +To target only particular environment, modify instead `common.ENV.json`, where ENV is the name of the environment. + +See https://github.com/leapcode/leap_web/blob/develop/doc/DEVELOP.md for notes on getting started hacking on leap_web. + +Known problems +--------------------------- + +* Client certificates are generated without a CSR. The problem is that this makes the web + application extremely vulnerable to denial of service attacks. This was not an issue until we + started to allow the possibility of anonymously fetching a client certificate without + authenticating first. + +* By its very nature, the user database is vulnerable to enumeration attacks. These are + very hard to prevent, because our protocol is designed to allow query of a user database via + proxy in order to provide network perspective. + |