1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
|
<!DOCTYPE html>
<html lang='en'>
<head>
<title>
couchdb - LEAP Platform Documentation
</title>
<meta content='width=device-width, initial-scale=1.0' name='viewport'>
<meta charset='UTF-8'>
<base href="" />
<style>
body {
background: #444;
display: flex;
flex-direction: row;
padding: 10px;
margin: 0px;
}
#sidebar {
flex: 0 0 250px;
background: white;
margin-right: 10px;
padding: 20px;
}
#sidebar ul {
list-style-type: none;
padding-left: 0px;
margin: 0;
}
#sidebar li { padding: 4px }
#sidebar li a { text-decoration: none }
#sidebar li.active { background: #444 }
#sidebar li.active a { color: white }
#sidebar li.level1 { padding-left: 20px }
#sidebar li.level2 { padding-left: 40px }
#main {
flex: 1 1 auto;
background: white;
padding: 20px;
}
#title-box {
padding-bottom: 20px;
border-bottom: 5px solid #eee;
}
#title-box h1 {
margin-top: 0px;
}
pre {
padding: 10px;
background: #eef;
}
code {
background: #eef;
}
table {border-collapse: collapse}
table td {
border: 1px solid #ccc;
padding: 4px;
vertical-align: top;
}
</style>
</head>
<body>
<div id='sidebar'>
<ul>
<li class=''>
<a href='../../../index.html'>Home</a>
</li>
<li class=' level0'>
<a class='' href='../../guide.html'>Guide</a>
</li>
<li class=' level0'>
<a class='' href='../../tutorials.html'>Tutorials</a>
</li>
<li class='semi-active level0'>
<a class='' href='../../services.html'>Services</a>
</li>
<li class='active level1'>
<a class='' href='../couchdb.html'>couchdb</a>
</li>
<li class=' level1'>
<a class='' href='../openvpn.html'>openvpn</a>
</li>
<li class=' level1'>
<a class='' href='../monitor.html'>monitor</a>
</li>
<li class=' level1'>
<a class='' href='../mx.html'>mx</a>
</li>
<li class=' level1'>
<a class='' href='../soledad.html'>soledad</a>
</li>
<li class=' level1'>
<a class='' href='../tor.html'>tor</a>
</li>
<li class=' level1'>
<a class='' href='../webapp.html'>webapp</a>
</li>
<li class=' level0'>
<a class='' href='../../upgrading.html'>Upgrading</a>
</li>
<li class=' level0'>
<a class='' href='../../troubleshooting.html'>Troubleshooting</a>
</li>
<li class=' level0'>
<a class='' href='../../details.html'>Details</a>
</li>
</ul>
</div>
<div id='main'>
<div id='title-box'>
<h1>couchdb</h1>
<div id='summary'>Data storage for all user data.</div>
</div>
<div id='content-box'>
<div id="TOC"><ol>
<li>
<a href="index.html#topology">Topology</a>
</li>
<li>
<a href="index.html#configuration">Configuration</a>
<ol>
<li>
<a href="index.html#nighly-dumps">Nighly dumps</a>
</li>
<li>
<a href="index.html#plain-couchdb">Plain CouchDB</a>
</li>
</ol>
</li>
<li>
<a href="index.html#various-tasks">Various Tasks</a>
<ol>
<li>
<a href="index.html#re-enabling-blocked-account">Re-enabling blocked account</a>
</li>
<li>
<a href="index.html#how-to-find-out-which-userstore-belongs-to-which-identity">How to find out which userstore belongs to which identity?</a>
</li>
<li>
<a href="index.html#how-much-disk-space-is-used-by-a-userstore">How much disk space is used by a userstore</a>
</li>
<li>
<a href="index.html#migrating-from-bigcouch-to-plain-couchdb">Migrating from BigCouch to plain CouchDB</a>
</li>
</ol>
</li>
</ol></div>
<h2><a name="topology"></a>Topology</h2>
<p>Required:</p>
<ul>
<li>Nodes with <code>couchdb</code> service must also have <code>soledad</code> service, if email is enabled.</li>
</ul>
<p>Suggested:</p>
<ul>
<li>Nodes with <code>couchdb</code> service communicate heavily with <code>webapp</code> and <code>mx</code>.</li>
</ul>
<p><code>couchdb</code> nodes do not need to be reachable from the public internet, although the <code>soledad</code> service does require this.</p>
<h2><a name="configuration"></a>Configuration</h2>
<h3><a name="nighly-dumps"></a>Nighly dumps</h3>
<p>You can do a nightly couchdb data dump by adding this to your node config:</p>
<pre><code>"couch": {
"backup": true
}
</code></pre>
<p>Data will get dumped to <code>/var/backups/couchdb</code>.</p>
<h3><a name="plain-couchdb"></a>Plain CouchDB</h3>
<p>BigCouch is not supported on Platform version 0.8 and higher: only plain CouchDB is possible. For earlier versions, you must do this in order to use plain CouchDB:</p>
<pre><code>"couch": {
"master": true,
"pwhash_alg": "pbkdf2"
}
</code></pre>
<h2><a name="various-tasks"></a>Various Tasks</h2>
<h3><a name="re-enabling-blocked-account"></a>Re-enabling blocked account</h3>
<p>When a user account gets destroyed from the webapp, there’s still a leftover doc in the identities db so other people can’t claim that account without an admin’s intervention. You can remove this username reservation through the webapp.</p>
<p>However, here is how you could do it manually, if you wanted to:</p>
<p>grep the identities db for the email address:</p>
<pre><code>curl -s --netrc-file /etc/couchdb/couchdb.netrc -X GET http://127.0.0.1:5984/identities/_all_docs?include_docs=true|grep test_127@bitmask.net
</code></pre>
<p>lookup “id” and “rev” to delete the doc:</p>
<pre><code>curl -s --netrc-file /etc/couchdb/couchdb.netrc -X DELETE 'http://127.0.0.1:5984/identities/b25cf10f935b58088f0d547fca823265?rev=2-715a9beba597a2ab01851676f12c3e4a'
</code></pre>
<h3><a name="how-to-find-out-which-userstore-belongs-to-which-identity"></a>How to find out which userstore belongs to which identity?</h3>
<pre><code>/usr/bin/curl -s --netrc-file /etc/couchdb/couchdb.netrc '127.0.0.1:5984/identities/_all_docs?include_docs=true' | grep testuser
{"id":"665e004870ee17aa4c94331ff3ecb173","key":"665e004870ee17aa4c94331ff3ecb173","value":{"rev":"2-2e335a75c4b79a5c2ef5c9950706fe1b"},"doc":{"_id":"665e004870ee17aa4c94331ff3ecb173","_rev":"2-2e335a75c4b79a5c2ef5c9950706fe1b","user_id":"665e004870ee17aa4c94331ff3cd59eb","address":"testuser@example.org","destination":"testuser@example.org","keys": ...
</code></pre>
<ul>
<li>search for the “user_id” field</li>
<li>in this example <a href="mailto:testuser@example.org">testuser@example.org</a> uses the database user-665e004870ee17aa4c94331ff3cd59eb</li>
</ul>
<h3><a name="how-much-disk-space-is-used-by-a-userstore"></a>How much disk space is used by a userstore</h3>
<p>Beware that this returns the uncompacted disk size (see <a href="http://wiki.apache.org/couchdb/Compaction">http://wiki.apache.org/couchdb/Compaction</a>)</p>
<pre><code>echo "`curl --netrc -s -X GET 'http://127.0.0.1:5984/user-dcd6492d74b90967b6b874100b7dbfcf'|json_pp|grep disk_size|cut -d: -f 2`/1024"|bc
</code></pre>
<h3><a name="migrating-from-bigcouch-to-plain-couchdb"></a>Migrating from BigCouch to plain CouchDB</h3>
<p><p>At the end of this process, you will have just <em>one</em> node with <code>services</code> property equal to <code>couchdb</code>. If you had a BigCouch cluster before, you will be removing all but one of those machines to consolidate them into one CouchDB machine.</p>
<ol>
<li><p>if you have multiple nodes with the <code>couchdb</code> service on them, pick one of them to be your CouchDB server, and remove the service from the others. If these machines were only doing BigCouch before, you can remove the nodes completely with <code>leap node rm <nodename></code> and then you can decommission the servers</p></li>
<li><p>put the webapp into <a href="../webapp.html#maintenance-mode">maintenance mode</a></p></li>
<li><p>turn off daemons that access the database. For example:</p>
<pre><code class="`"> workstation$ leap ssh <each soledad-node>
server# /etc/init.d/soledad-server stop
workstation$ leap ssh <mx-node>
server# /etc/init.d/postfix stop
server# /etc/init.d/leap-mx stop
workstation$ leap ssh <webapp-node>
server# /etc/init.d/nickserver stop
</code></pre>
<p> Alternately, you can create a temporary firewall rule to block access (run on couchdb server):</p>
<pre><code class="`"> server# iptables -A INPUT -p tcp --dport 5984 --jump REJECT
</code></pre></li>
<li><p>remove orphaned databases and do a backup of all remaining, active databases. This can take some time and will place several hundred megabytes of data into /var/backups/couchdb. The size and time depends on how many users there are on your system. For example, 15k users took approximately 25 minutes and 308M of space:</p>
<pre><code class="`"> workstation$ leap ssh <couchdb-node>
server# cd /srv/leap/couchdb/scripts
server# ./cleanup-user-dbs
server# time ./couchdb_dumpall.sh
</code></pre></li>
<li><p>stop bigcouch:</p>
<pre><code class="`"> server# /etc/init.d/bigcouch stop
server# pkill epmd
</code></pre></li>
<li><p>remove bigcouch:</p>
<pre><code class="`"> server# apt-get remove bigcouch
</code></pre></li>
<li><p>configure your couch node to use plain couchdb instead of bigcouch, you can do this by editing nodes/<couch-node>.json, look for this section:</p>
<pre><code class="`"> "couch": {
"mode": "plain"
}
</code></pre>
<p>change it, so it looks like this instead:</p>
<pre><code class="``"> "couch": {
"mode": "plain",
"pwhash_alg": "pbkdf2"
}
</code></pre></li>
</ol>
</p>
<p><ol>
<li><p>restore the backup, this will take approximately the same amount of time as the backup took above:</p>
<pre><code class="`"> server# cd /srv/leap/couchdb/scripts
server# time ./couchdb_restoreall.sh
</code></pre></li>
<li><p>start services again that were stopped in the beginning:</p>
<pre><code class="`"> workstation$ leap ssh soledad-nodes
server# /etc/init.d/soledad-server start
workstation$ leap ssh mx-node
server# /etc/init.d/postfix start
server# /etc/init.d/leap-mx start
workstation$ leap ssh webapp
server# /etc/init.d/nickserver start
</code></pre>
<p> Or, alternately, if you set up the firewall rule instead, now remove it:</p>
<pre><code class="`"> server# iptables -D INPUT -p tcp --dport 5984 --jump REJECT
</code></pre></li>
</ol>
</p>
<p><ol>
<li><p>check if everything is working, including running the test on your deployment machine:</p>
<pre><code class="`"> workstation$ leap test
</code></pre></li>
<li><p>Remove old bigcouch data dir <code>/opt</code> after you double checked everything is in place</p></li>
<li><p>Relax, enjoy a refreshing beverage.</p></li>
</ol>
</p>
</div>
</div>
</body>
</html>
|
