From 9e0d26d535e22c11eab758f52043d3e8e1549593 Mon Sep 17 00:00:00 2001 From: azul Date: Wed, 9 Apr 2014 13:00:03 +0200 Subject: Create tapicero.md Copy tapiceros readme into leap_doc. --- docs/design/tapicero.md | 76 +++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 76 insertions(+) create mode 100644 docs/design/tapicero.md diff --git a/docs/design/tapicero.md b/docs/design/tapicero.md new file mode 100644 index 0000000..359ee15 --- /dev/null +++ b/docs/design/tapicero.md @@ -0,0 +1,76 @@ +Tapicero - Creating per user databases on the couch for soledad +------------------------------------------------------------ + +``tapicero`` is a daemon that creates per user databases when users are added to the LEAP Platform. It watches the changes made to the users database and creates new databases accordingly. This way soledad does not need admin privileges. + +"Tapicero" is spanish for upholsterer - the person who creates your couch. + +This program is written in Ruby and is distributed under the following license: + +> GNU Affero General Public License +> Version 3.0 or higher +> http://www.gnu.org/licenses/agpl-3.0.html + +Installation +--------------------- + +Prerequisites: + + sudo apt-get install ruby ruby-dev couchdb + # for development, you will also need git, bundle, and rake. + +From source: + + git clone git://leap.se/tapicero + cd tapicero + bundle + rake build + sudo rake install + +From gem: + + sudo gem install tapicero + +Running +-------------------- + +Run in foreground to see if it works: + + tapicero run -- test/config/config.yaml + create a new record in the users database + observe /var/log/syslog or the logfile you specified + +Run as a deamon: + + tapicero start + tapicero stop + +Run once and then exit: + + tapicero --run-once + This will create per user databases for all users created since + the last run and then exit. + +Flags +--------------------- + +--run-once: + process the existing users and then exit + +--rerun: + also work on users that have been processed before + +--overwrite-security: + write the security settings even if the user database already has some + +Combining these flags you can migrate the security settings of all existing per user databases. + + +Configuration +--------------------- + +``tapicero`` reads the following configurations files, in this order: + +* ``$(tapicero_source)/config/default.yaml`` +* ``/etc/leap/tapicero.yaml`` +* Any file passed to ARGV like so ``tapicero start -- /etc/tapicero.yaml -- cgit v1.2.3 From e5064b8d005adb5602ec0c1d32b2ff86aff09011 Mon Sep 17 00:00:00 2001 From: azul Date: Thu, 10 Apr 2014 11:07:27 +0200 Subject: Add tapicero to menu --- menu.txt | 1 + 1 file changed, 1 insertion(+) diff --git a/menu.txt b/menu.txt index 52bff05..c1d8d3f 100644 --- a/menu.txt +++ b/menu.txt @@ -13,6 +13,7 @@ docs overview nicknym soledad + tapicero platform quick-start guide -- cgit v1.2.3 From 66ceedccb15ef4ef049b828e756c1df35d29de08 Mon Sep 17 00:00:00 2001 From: azul Date: Thu, 10 Apr 2014 11:25:06 +0200 Subject: Add database setup section to overview Explaining setup of 'normal' and per-user databases. link to tapicero, point out where the design docs come from and where they go. --- docs/design/overview.md | 12 +++++++++++- 1 file changed, 11 insertions(+), 1 deletion(-) diff --git a/docs/design/overview.md b/docs/design/overview.md index 9818aa1..27d8883 100644 --- a/docs/design/overview.md +++ b/docs/design/overview.md @@ -125,7 +125,17 @@ These are the databases we currently use: * tickets -- help tickets issued in the webapp * tokens -- created by the webapp on login, used by soledad to authenticate * users -- user records used by the webapp including the authentication data -* user-...id... -- client-encrypted data for the user with the given id accessed from the client via soledad +* user-...id... -- client-encrypted user data accessed from the client via soledad + +### Database Setup + +The main couch databases are initially created, seeded and updated when deploying the platform. + +The site_couchdb module contains the database description and security settings in `manifests/create_dbs.pp`. The design docs are seeded from the files in `files/designs/:db_name`. If these files change the next puppet deploy will update the databases accordingly. Both the webapp and soledad have scripts that will dump the required design docs so they can be included here. + +The per-user databases are created upon user registration by [Tapicero](https://leap.se/docs/design/tapicero). Tapicero also adds security and design documents. The design documents for per-user databases are stored in the [tapicero repository](https://github.com/leapcode/tapicero) in `designs`. Tapicero can be used to update existing user databases with new security settings and design documents. + +### BigCouch Like many NoSQL databases, BigCouch is inspired by [Amazon's Dynamo paper](http://www.allthingsdistributed.com/files/amazon-dynamo-sosp2007.pdf) and works by sharding each database among many servers using a circular ring hash. The number of shards might be greater than the number of servers, in which case each server would have multiple shards of the same database. Each server in the BigCouch cluster appears to contain the entire database, but actually it will just proxy the request to the actual database that has the content (if it does not have the document itself). -- cgit v1.2.3 From c2e5df6d37c3f44bf5cb16c44bee3d7da69a396e Mon Sep 17 00:00:00 2001 From: azul Date: Thu, 10 Apr 2014 12:17:06 +0200 Subject: Emphazise usage within leap platform for tapicero Sorting the sections so the usage scenarios for the platform come first. --- docs/design/tapicero.md | 98 ++++++++++++++++++++++++++++++------------------- 1 file changed, 60 insertions(+), 38 deletions(-) diff --git a/docs/design/tapicero.md b/docs/design/tapicero.md index 359ee15..cb7be7c 100644 --- a/docs/design/tapicero.md +++ b/docs/design/tapicero.md @@ -1,55 +1,56 @@ -Tapicero - Creating per user databases on the couch for soledad ------------------------------------------------------------- +@title = 'Tapicero' +@summary = 'Creating per-user databases on the couch for soledad.' +@toc = true -``tapicero`` is a daemon that creates per user databases when users are added to the LEAP Platform. It watches the changes made to the users database and creates new databases accordingly. This way soledad does not need admin privileges. +Tapicero +============== + +**Create databases for the leap platform users** + + +Tapicero is part of the leap platform. It's deployed to the couch nodes and watches the users database as a daemon. When a user is add it creates a new database for that user. It also removes these databases on user destruction. This way neither the webapp nor soledad need couch admin privileges. "Tapicero" is spanish for upholsterer - the person who creates your couch. -This program is written in Ruby and is distributed under the following license: +Running +-------------------- -> GNU Affero General Public License -> Version 3.0 or higher -> http://www.gnu.org/licenses/agpl-3.0.html +Tapicero is usually deployed with the leap platform and run as a daemon from an init script. It also serves as a tool to modify existing user databases. You can find it in `/srv/leap/tapicero` on the couch nodes or play with it on your own machine. -Installation ---------------------- +Run in foreground: -Prerequisites: + bundle exec /bin/tapicero run - sudo apt-get install ruby ruby-dev couchdb - # for development, you will also need git, bundle, and rake. +Run as a deamon: -From source: + bundle exec /bin/tapicero start + bundle exec /bin/tapicero stop - git clone git://leap.se/tapicero - cd tapicero - bundle - rake build - sudo rake install +Run once, process all changes so far and then exit: -From gem: + bundle exec tapicero --run-once - sudo gem install tapicero +Configuration +--------------------- -Running --------------------- +Tapicero reads the following configurations files, in this order: + +* ``$(tapicero_source)/config/default.yaml`` +* ``/etc/leap/tapicero.yaml`` +* Any file passed to ARGV like so ``tapicero start -- /etc/tapicero.yaml`` -Run in foreground to see if it works: +Files that come later will overwrite settings from the former. - tapicero run -- test/config/config.yaml - create a new record in the users database - observe /var/log/syslog or the logfile you specified +### Sequence File -Run as a deamon: +Tapicero keeps track of the last change processed in a sequence file. The location of the sequence file is configured as `seq_file` and defaults to `/var/log/leap/tapicero.seq` - tapicero start - tapicero stop +After restarting Tapicero it will only process changes that happened after the change with the sequence id given in the sequence file. This behaviour can be altered by using the --rerun flag or removing the sequence file. -Run once and then exit: +### Logging - tapicero --run-once - This will create per user databases for all users created since - the last run and then exit. +Tapicero logs it's activity to syslog in a production environment. Logging details can be configured via `log_level` +Configure `log_file` if you want to log to a file instead of syslog. Flags --------------------- @@ -66,11 +67,32 @@ Flags Combining these flags you can migrate the security settings of all existing per user databases. -Configuration +Installation --------------------- -``tapicero`` reads the following configurations files, in this order: +Tapicero is normally deployed as part of the leap platform. If you want to install it outside of this context these instructions are for you. -* ``$(tapicero_source)/config/default.yaml`` -* ``/etc/leap/tapicero.yaml`` -* Any file passed to ARGV like so ``tapicero start -- /etc/tapicero.yaml +Prerequisites: + + sudo apt-get install ruby ruby-dev couchdb + # for development, you will also need git, bundle, and rake. + +From source: + + git clone git://leap.se/tapicero + cd tapicero + bundle + bundle exec bin/tapicero {run, start, status, ...} + +From gem: + + sudo gem install tapicero + +License +-------- + +This program is written in Ruby and is distributed under the following license: + +> GNU Affero General Public License +> Version 3.0 or higher +> http://www.gnu.org/licenses/agpl-3.0.html -- cgit v1.2.3