Merge branch 'develop'

Conflicts:
	platform.rb

Change-Id: Ic2e08e594d29a585691341c8667ac0b64933a505

1 files changed, 249 insertions, 0 deletions

diff --git a/doc/troubleshooting/where-to-look.md b/doc/troubleshooting/where-to-look.md
new file mode 100644
index 00000000..fbd95931
--- /dev/null
+++ b/doc/troubleshooting/where-to-look.md
@@ -0,0 +1,249 @@
+@title = 'Where to look for errors'
+@nav_title = 'Where to look'
+@toc = true
+
+
+General
+=======
+
+* Please increase verbosity when debugging / filing issues in our issue tracker. You can do this with adding i.e. `-v 5` after the `leap` cmd, i.e. `leap -v 2 deploy`.
+
+Webapp
+======
+
+Places to look for errors
+-------------------------
+
+* `/var/log/apache2/error.log`
+* `/srv/leap/webapp/log/production.log`
+* `/var/log/syslog` (watch out for stunnel issues)
+* `/var/log/leap/*`
+
+Is haproxy ok ?
+---------------
+
+
+    curl -s -X  GET "http://127.0.0.1:4096"
+
+Is couchdb accessible through stunnel ?
+---------------------------------------
+
+* Depending on how many couch nodes you have, increase the port for every test
+  (see /etc/haproxy/haproxy.cfg for the server/port mapping):
+
+
+    curl -s -X  GET "http://127.0.0.1:4000"
+    curl -s -X  GET "http://127.0.0.1:4001"
+    ...
+
+
+Check couchdb acl as admin
+--------------------------
+
+    mkdir /etc/couchdb
+    cat /srv/leap/webapp/config/couchdb.yml.admin  # see username and password
+    echo "machine 127.0.0.1 login admin password <PASSWORD>" > /etc/couchdb/couchdb-admin.netrc
+    chmod 600 /etc/couchdb/couchdb-admin.netrc
+
+    curl -s --netrc-file /etc/couchdb/couchdb-admin.netrc -X GET "http://127.0.0.1:4096"
+    curl -s --netrc-file /etc/couchdb/couchdb-admin.netrc -X GET "http://127.0.0.1:4096/_all_dbs"
+
+Check couchdb acl as unpriviledged user
+---------------------------------------
+
+    cat /srv/leap/webapp/config/couchdb.yml  # see username and password
+    echo "machine 127.0.0.1 login webapp password <PASSWORD>" > /etc/couchdb/couchdb-webapp.netrc
+    chmod 600 /etc/couchdb/couchdb-webapp.netrc
+
+    curl -s --netrc-file /etc/couchdb/couchdb-webapp.netrc -X GET "http://127.0.0.1:4096"
+    curl -s --netrc-file /etc/couchdb/couchdb-webapp.netrc -X GET "http://127.0.0.1:4096/_all_dbs"
+
+
+Check client config files
+-------------------------
+
+    https://example.net/provider.json
+    https://example.net/1/config/smtp-service.json
+    https://example.net/1/config/soledad-service.json
+    https://example.net/1/config/eip-service.json
+
+
+Soledad
+=======
+
+    /var/log/soledad.log
+
+
+Couchdb
+=======
+
+Places to look for errors
+-------------------------
+
+* `/opt/bigcouch/var/log/bigcouch.log`
+* `/var/log/syslog` (watch out for stunnel issues)
+
+
+
+Bigcouch membership
+-------------------
+
+* All nodes configured for the provider should appear here:
+
+<pre>
+    curl -s --netrc-file /etc/couchdb/couchdb.netrc -X GET 'http://127.0.0.1:5986/nodes/_all_docs'
+</pre>
+
+* All configured nodes should show up under "cluster_nodes", and the ones online and communicating with each other should appear under "all_nodes". This example output shows the configured cluster nodes `couch1.bitmask.net` and `couch2.bitmask.net`, but `couch2.bitmask.net` is currently not accessible from `couch1.bitmask.net`
+
+
+<pre>
+    curl -s --netrc-file /etc/couchdb/couchdb.netrc 'http://127.0.0.1:5984/_membership'
+    {"all_nodes":["bigcouch@couch1.bitmask.net"],"cluster_nodes":["bigcouch@couch1.bitmask.net","bigcouch@couch2.bitmask.net"]}
+</pre>
+
+* Sometimes a `/etc/init.d/bigcouch restart` on all nodes is needed, to register new nodes
+
+Databases
+---------
+
+* Following output shows all neccessary DBs that should be present. Note that the `user-0123456....` DBs are the data stores for a particular user.
+
+<pre>
+    curl -s --netrc-file /etc/couchdb/couchdb.netrc -X GET 'http://127.0.0.1:5984/_all_dbs'
+    ["customers","identities","sessions","shared","tickets","tokens","user-0","user-9d34680b01074c75c2ec58c7321f540c","user-9d34680b01074c75c2ec58c7325fb7ff","users"]
+</pre>
+
+
+
+
+Design Documents
+----------------
+
+* Is User `_design doc` available ?
+
+
+<pre>
+    curl -s --netrc-file /etc/couchdb/couchdb.netrc -X  GET "http://127.0.0.1:5984/users/_design/User"
+</pre>
+
+Is couchdb cluster backend accessible through stunnel ?
+-------------------------------------------------------
+
+* Find out how many connections are set up for the couchdb cluster backend:
+
+<pre>
+    grep "accept = 127.0.0.1" /etc/stunnel/*
+</pre>
+
+
+* Now connect to all of those local endpoints to see if they up. All these tests should return "localhost [127.0.0.1] 4000 (?) open"
+
+<pre>
+    nc -v 127.0.0.1 4000
+    nc -v 127.0.0.1 4001
+    ...
+</pre>
+
+
+MX
+==
+
+Places to look for errors
+-------------------------
+
+* `/var/log/mail.log`
+* `/var/log/leap_mx.log`
+* `/var/log/syslog` (watch out for stunnel issues)
+
+Is couchdb accessible through stunnel ?
+---------------------------------------
+
+* Depending on how many couch nodes you have, increase the port for every test
+  (see /etc/haproxy/haproxy.cfg for the server/port mapping):
+
+
+    curl -s -X  GET "http://127.0.0.1:4000"
+    curl -s -X  GET "http://127.0.0.1:4001"
+    ...
+
+Query leap-mx
+-------------
+
+* for useraccount
+
+
+<pre>
+    postmap -v -q  "joe@dev.bitmask.net" tcp:localhost:2244
+    ...
+    postmap: dict_tcp_lookup: send: get jow@dev.bitmask.net
+    postmap: dict_tcp_lookup: recv: 200
+    ...
+</pre>
+
+* for mailalias
+
+
+<pre>
+    postmap -v -q  "joe@dev.bitmask.net" tcp:localhost:4242
+    ...
+    postmap: dict_tcp_lookup: send: get joe@dev.bitmask.net
+    postmap: dict_tcp_lookup: recv: 200 f01bc1c70de7d7d80bc1ad77d987e73a
+    postmap: dict_tcp_lookup: found: f01bc1c70de7d7d80bc1ad77d987e73a
+    f01bc1c70de7d7d80bc1ad77d987e73a
+    ...
+</pre>
+
+
+Check couchdb acl as unpriviledged user
+---------------------------------------
+
+
+
+    cat /etc/leap/mx.conf  # see username and password
+    echo "machine 127.0.0.1 login leap_mx password <PASSWORD>" > /etc/couchdb/couchdb-leap_mx.netrc
+    chmod 600 /etc/couchdb/couchdb-leap_mx.netrc
+
+    curl -s --netrc-file /etc/couchdb/couchdb-leap_mx.netrc -X GET "http://127.0.0.1:4096/_all_dbs"   # pick one "user-<hash>" db
+    curl -s --netrc-file /etc/couchdb/couchdb-leap_mx.netrc -X GET "http://127.0.0.1:4096/user-de9c77a3d7efbc779c6c20da88e8fb9c"
+
+
+* you may check multiple times, cause 127.0.0.1:4096 is haproxy load-balancing the different couchdb nodes
+
+
+Mailspool
+---------
+
+* Any file in the leap_mx mailspool longer for a few seconds ?
+
+
+
+<pre>
+    ls -la /var/mail/vmail/Maildir/cur/
+</pre>
+
+* Any mails in postfix mailspool longer than a few seconds ?
+
+<pre>
+    mailq
+</pre>
+
+
+
+Testing mail delivery
+---------------------
+
+    swaks -f alice@example.org -t bob@example.net -s mx1.example.net --port 25
+    swaks -f varac@cdev.bitmask.net -t varac@cdev.bitmask.net -s chipmonk.cdev.bitmask.net --port 465 --tlsc
+    swaks -f alice@example.org -t bob@example.net -s mx1.example.net --port 587 --tls
+
+
+VPN
+===
+
+Places to look for errors
+-------------------------
+
+* `/var/log/syslog` (watch out for openvpn issues)
+
+