summaryrefslogtreecommitdiff
path: root/doc/troubleshooting.md
blob: bb2fc4b520f6ee64ac989790145c194773c0eb8c (plain)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
@title = 'Troubleshooting Guide'
@nav_title = 'Troubleshooting'
@toc = true


General
=======

* Please increase verbosity when debugging / filing issues in our issue tracker. You can do this with adding i.e. `-v 5` after the `leap` cmd, i.e. `leap -v 2 deploy`.

Webapp node
===========

Places to look for errors
-------------------------

* `/var/log/apache2/error.log`
* `/srv/leap/webapp/log/production.log`
* `/var/log/syslog` (watch out for stunnel issues)

Is haproxy ok ? 
---------------


    curl -s -X  GET "http://127.0.0.1:4096"

Is couchdb accessible through stunnel ?
---------------------------------------


    curl -s -X  GET "http://127.0.0.1:4000"


Check couchdb acl
-----------------


    mkdir /etc/couchdb
    cat /srv/leap/webapp/config/couchdb.yml.admin  # see username and password
    echo "machine 127.0.0.1 login admin password <PASSWORD>" > /etc/couchdb/couchdb-admin.netrc
    chmod 600 /etc/couchdb/couchdb-admin.netrc

    curl -s --netrc-file /etc/couchdb/couchdb-admin.netrc -X GET "http://127.0.0.1:4096"
    curl -s --netrc-file /etc/couchdb/couchdb-admin.netrc -X GET "http://127.0.0.1:4096/_all_dbs"
    

Couchdb node
============

Places to look for errors
-------------------------

* `/opt/bigcouch/var/log/bigcouch.log`
* `/var/log/syslog` (watch out for stunnel issues)


Bigcouch membership
-------------------

* All nodes configured for the provider should appear here:


    curl -s --netrc-file /etc/couchdb/couchdb.netrc -X GET 'http://127.0.0.1:5986/nodes/_all_docs'

* All configured nodes should show up under "cluster_nodes", and the ones online and communicating with each other should appear under "all_nodes". This example output shows the configured cluster nodes `couch1.bitmask.net` and `couch2.bitmask.net`, but `couch2.bitmask.net` is currently not accessible from `couch1.bitmask.net`


    curl -s --netrc-file /etc/couchdb/couchdb.netrc 'http://127.0.0.1:5984/_membership'
    {"all_nodes":["bigcouch@couch1.bitmask.net"],"cluster_nodes":["bigcouch@couch1.bitmask.net","bigcouch@couch2.bitmask.net"]}



Databases
---------

* Following output shows all neccessary DBs that should be present. Note that the `user-0123456....` DBs are the data stores for a particular user. 


    curl -s --netrc-file /etc/couchdb/couchdb.netrc -X GET 'http://127.0.0.1:5984/_all_dbs' 
    ["customers","identities","sessions","shared","tickets","tokens","user-0","user-9d34680b01074c75c2ec58c7321f540c","user-9d34680b01074c75c2ec58c7325fb7ff","users"]



Design Documents
----------------

* Is User `_design doc` available ?


    curl -s --netrc-file /etc/couchdb/couchdb.netrc -X  GET "http://127.0.0.1:5984/users/_design/User"



MX node
=======

Places to look for errors
-------------------------

* `/var/log/mail.log`
* `/var/log/leap_mx.log`
* `/var/log/syslog` (watch out for stunnel issues)


Query leap-mx
-------------

* for useraccount 


    postmap -v -q  "joe@dev.bitmask.net" tcp:localhost:2244
    ...
    postmap: dict_tcp_lookup: send: get jow@dev.bitmask.net
    postmap: dict_tcp_lookup: recv: 200 
    ...

* for mailalias


    postmap -v -q  "joe@dev.bitmask.net" tcp:localhost:4242
    ...
    postmap: dict_tcp_lookup: send: get joe@dev.bitmask.net
    postmap: dict_tcp_lookup: recv: 200 f01bc1c70de7d7d80bc1ad77d987e73a
    postmap: dict_tcp_lookup: found: f01bc1c70de7d7d80bc1ad77d987e73a
    f01bc1c70de7d7d80bc1ad77d987e73a
    ...



Mailspool
---------

* Any file in the mailspool longer for a few seconds ?


    ls -la /var/mail/vmail/Maildir/cur/


VPN node
========

Places to look for errors
-------------------------

* `/var/log/syslog` (watch out for openvpn issues)