17 files changed, 149 insertions, 29 deletions
diff --git a/CHANGES.md b/CHANGES.md
new file mode 100644
index 00000000..4382463b
--- /dev/null
+++ b/CHANGES.md
@@ -0,0 +1,68 @@
+Platform 0.7
+-------------------------------------
+
+CouchDB improvements: CouchDB is not designed to handle ephemeral data, like
+sessions, because documents are never really deleted (a tombstone document is
+always kept to record the deletion). To overcome this limitation, we now
+rotate the `sessions` and `tokens` databases monthly. The new database names
+are `tokens_XXX` and `sessions_XXX` where XXX is counter since the epoch that
+increments every month (not a calendar month, but a month's worth of seconds).
+
+Additionally, nagios monitor and `leap test run` now will create and destroy
+test users in the `tmp_users` database, which will get periodically deleted
+and recreated.
+
+Compatibility:
+
+* requires leap_cli version 1.7
+* requires bitmask client version >= 0.7
+
+Commits: https://leap.se/git/leap_platform.git/shortlog/refs/tags/0.7.0
+
+Upgrading:
+
+* `gem install leap_cli --version 1.7` or run leap_cli from current master branch.
+* `cd leap_platform; git pull; git checkout 0.7.0` or checkout current master branch.
+* `leap deploy`
+* `leap db destroy --db sessions,tokens` You can ignore message about needing
+  to redeploy (since, in this case, we just want to permanently delete those
+  databases).
+
+New features:
+
+* rotating couchdb databases
+* deployment logging: information on every deploy is logged to
+  `/var/log/leap`, including the user, leap_cli version, and platform version.
+* you must now run `leap deploy --downgrade` if you want to deploy an older
+  version over a newer platform version.
+* the install source each custom daemons (e.g. tapicero, etc) is now
+  configured on `common.json`.
+* you can configure apt sources in common.json
+* many bug fixes
+
+Platform 0.6
+-------------------------------------
+
+Compatibility:
+
+* requires leap_cli version 1.6
+* requires bitmask client version >= 0.5
+
+Commits: https://leap.se/git/leap_platform.git/shortlog/refs/tags/0.6.0
+
+New features:
+
+* single node deployment
+* include custom puppet modules and manifests
+* couch flexibility
+* stunnel rework
+* new debian repository structure
+* dependency pinning
+* leap_cli modularization
+* improved cert generation
+* monitoring improvements such as per-environment tooling and notifications
+* tor hidden service support
+* switch away from NIST curve and ensure TLSv1 is used
+* tests made significantly more robust
+* add support for webapp deployment to a subdomain
+* many, many bugfixes and stability improvements
diff --git a/README.md b/README.md
index a64962ca..9b33e9e8 100644
--- a/README.md
+++ b/README.md
@@ -42,7 +42,7 @@ You can't deploy new couchdb nodes after one or more have been deployed. Make *s
 User setup and ssh
 ------------------
 
-. if you aren't using a single ssh key, but have different ones, you will need to define the following at the top of your ~/.ssh/config: 
+. if you aren't using a single ssh key, but have different ones, you will need to define the following at the top of your ~/.ssh/config:
   HostName <ip address>
   IdentityFile <path to identity file>
 
@@ -52,7 +52,7 @@ User setup and ssh
 
 . At the moment, only ECDSA ssh host keys are supported. If you get the following error: `= FAILED ssh-keyscan: no hostkey alg (must be missing an ecdsa public host key)` then you should confirm that you have the following line defined in your server's **/etc/ssh/sshd_config**: `HostKey /etc/ssh/ssh_host_ecdsa_key`. If that file doesn't exist, run `ssh-keygen -t ecdsa -f /etc/ssh/ssh_host_ecdsa_key -N ""` in order to create it. If you made a change to your sshd_config, then you need to run `/etc/init.d/ssh restart` (see: https://leap.se/code/issues/2373)
 
-. To remove an admin's access to your servers, please remove the directory for that user under the `users/` subdirectory in your provider directory and then remove that user's ssh keys from files/ssh/authorized_keys. When finished you *must* run a `leap deploy` to update that information on the servers. 
+. To remove an admin's access to your servers, please remove the directory for that user under the `users/` subdirectory in your provider directory and then remove that user's ssh keys from files/ssh/authorized_keys. When finished you *must* run a `leap deploy` to update that information on the servers.
 
 . At the moment, it is only possible to add an admin who will have access to all LEAP servers (see: https://leap.se/code/issues/2280)
 
@@ -85,12 +85,10 @@ In order to validate the syntax and style guide compliance
 before you commit, see https://github.com/pixelated-project/puppet-git-hooks#installation
 
 
-Changelog
+Changes
 =========
 
-For a changelog of the current branch:
-
-    git log
+Read CHANGES.md or run `git log`.
 
 Authors and Credits
 ===================
diff --git a/bin/puppet_command b/bin/puppet_command
index bc4bfc5b..ce56fe51 100755
--- a/bin/puppet_command
+++ b/bin/puppet_command
@@ -132,7 +132,7 @@ def puppet_apply(options={}, &block)
   options = {:verbosity => @verbosity, :tags => @tags}.merge(options)
   manifest = options[:manifest] || SITE_MANIFEST
   modulepath = options[:module_path] || SITE_MODULES + CUSTOM_MODULES
-  fqdn = hiera_file['domain']['name']
+  fqdn = hiera_file['domain']['full']
   domain = hiera_file['domain']['full_suffix']
   Dir.chdir(PUPPET_DIRECTORY) do
     return run("FACTER_fqdn='#{fqdn}' FACTER_domain='#{domain}' #{PUPPET_BIN} apply #{custom_parameters(options)} --modulepath='#{modulepath}' #{PUPPET_PARAMETERS} #{manifest}", &block)
diff --git a/platform.rb b/platform.rb
index 1466b33c..2f7b36eb 100644
--- a/platform.rb
+++ b/platform.rb
@@ -4,7 +4,7 @@
 #
 
 Leap::Platform.define do
-  self.version = "0.6.1"
+  self.version = "0.7"
   self.compatible_cli = "1.6.3".."1.99"
 
   #
diff --git a/provider_base/common.json b/provider_base/common.json
index 6ad8a2e6..4a5406a0 100644
--- a/provider_base/common.json
+++ b/provider_base/common.json
@@ -75,12 +75,12 @@
     "tapicero": {
       "type": "git",
       "source": "https://leap.se/git/tapicero",
-      "revision": "origin/version/0.6.1"
+      "revision": "origin/version/0.7"
     },
     "webapp": {
       "type": "git",
       "source": "https://leap.se/git/leap_web",
-      "revision": "origin/version/0.6.1"
+      "revision": "origin/version/0.7"
     }
   }
 }
diff --git a/provider_base/services/openvpn.json b/provider_base/services/openvpn.json
index 127f5890..11cb0dc2 100644
--- a/provider_base/services/openvpn.json
+++ b/provider_base/services/openvpn.json
@@ -25,7 +25,7 @@
       "cipher": "AES-128-CBC",
       "keepalive": "10 30",
       "tun-ipv6": true,
-      "fragment": 1400
+      "fragment": 1500
     }
   },
   "obfsproxy": {
diff --git a/provider_base/services/webapp.json b/provider_base/services/webapp.json
index e80a758a..b0646579 100644
--- a/provider_base/services/webapp.json
+++ b/provider_base/services/webapp.json
@@ -4,11 +4,8 @@
     "forbidden_usernames": ["admin", "administrator", "arin-admin", "certmaster", "contact", "info", "maildrop", "postmaster", "ssladmin", "www-data"],
     "domain": "= domain.full_suffix",
     "modules": ["user", "billing", "help"],
-    "couchdb_webapp_user": {
-      "username": "= global.services[:couchdb].couch.users[:webapp].username",
-      "password": "= secret :couch_webapp_password",
-      "salt": "= hex_secret :couch_webapp_password_salt, 128"
-    },
+    "couchdb_webapp_user": "= global.services[:couchdb].couch.users[:webapp]",
+    "couchdb_admin_user": "= global.services[:couchdb].couch.users[:admin]",
     "customization_dir": "= file_path 'webapp'",
     "client_certificates": "= provider.ca.client_certificates",
     "allow_limited_certs": "= provider.service.allow_limited_bandwidth",
diff --git a/puppet/modules/site_apache/templates/vhosts.d/common.conf.erb b/puppet/modules/site_apache/templates/vhosts.d/common.conf.erb
index 87c40005..43bd1076 100644
--- a/puppet/modules/site_apache/templates/vhosts.d/common.conf.erb
+++ b/puppet/modules/site_apache/templates/vhosts.d/common.conf.erb
@@ -1,5 +1,7 @@
 <VirtualHost *:80>
-  ServerName <%= domain %>
+  ServerName <%= webapp_domain %>
+  ServerAlias <%= domain_name %>
+  ServerAlias <%= domain %>
   ServerAlias www.<%= domain %>
   RewriteEngine On
   RewriteRule ^.*$ https://<%= domain -%>%{REQUEST_URI} [R=permanent,L]
@@ -7,7 +9,6 @@
 </VirtualHost>
 
 <VirtualHost *:443>
-
   ServerName <%= webapp_domain %>
   ServerAlias <%= domain_name %>
   ServerAlias <%= domain %>
diff --git a/puppet/modules/site_check_mk/files/extra_service_conf.mk b/puppet/modules/site_check_mk/files/extra_service_conf.mk
index 61b0dd39..a4c6e769 100644
--- a/puppet/modules/site_check_mk/files/extra_service_conf.mk
+++ b/puppet/modules/site_check_mk/files/extra_service_conf.mk
@@ -5,14 +5,10 @@ extra_service_conf["max_check_attempts"] = [
 ]
 
 #
-# run check_mk_agent every 2 minutes if it terminates successfully.
+# run check_mk_agent every 10 minutes if it terminates successfully.
 # see https://leap.se/code/issues/6539 for the rationale
 #
-# update: temporarily set interval to 60 minutes until we solve the
-#         issue with the users db getting bloated with deleted
-#         test users.
-#
 extra_service_conf["normal_check_interval"] = [
-  ("60", ALL_HOSTS , "Check_MK" )
+  ("10", ALL_HOSTS , "Check_MK" )
 ]
 
diff --git a/puppet/modules/site_couchdb/files/designs/tmp_users/User.json b/puppet/modules/site_couchdb/files/designs/tmp_users/User.json
new file mode 120000
index 00000000..ed3d0af9
--- /dev/null
+++ b/puppet/modules/site_couchdb/files/designs/tmp_users/User.json
@@ -0,0 +1 @@
+../users/User.json
+\ No newline at end of file
diff --git a/puppet/modules/site_couchdb/manifests/create_dbs.pp b/puppet/modules/site_couchdb/manifests/create_dbs.pp
index 40148b8e..f9a08807 100644
--- a/puppet/modules/site_couchdb/manifests/create_dbs.pp
+++ b/puppet/modules/site_couchdb/manifests/create_dbs.pp
@@ -64,6 +64,13 @@ class site_couchdb::create_dbs {
     require => Couchdb::Query::Setup['localhost']
   }
 
+  ## tmp_users database
+  ## r/w: webapp
+  couchdb::create_db { 'tmp_users':
+    members => "{ \"names\": [], \"roles\": [\"replication\", \"users\"] }",
+    require => Couchdb::Query::Setup['localhost']
+  }
+
   ## messages db
   ## store messages to the clients such as payment reminders
   ## r/w: webapp
diff --git a/puppet/modules/site_static/manifests/domain.pp b/puppet/modules/site_static/manifests/domain.pp
index 6941b1a3..b9177f25 100644
--- a/puppet/modules/site_static/manifests/domain.pp
+++ b/puppet/modules/site_static/manifests/domain.pp
@@ -12,9 +12,18 @@ define site_static::domain (
 
   create_resources(site_static::location, $locations)
 
-  x509::cert { $domain: content => $cert }
-  x509::key  { $domain: content => $key }
-  x509::ca   { "${domain}_ca": content => $ca_cert }
+  x509::cert { $domain:
+    content => $cert,
+    notify => Service[apache]
+  }
+  x509::key { $domain:
+    content => $key,
+    notify => Service[apache]
+  }
+  x509::ca { "${domain}_ca":
+    content => $ca_cert,
+    notify => Service[apache]
+  }
 
   apache::vhost::file { $domain:
     content => template('site_static/apache.conf.erb')
diff --git a/puppet/modules/site_webapp/manifests/couchdb.pp b/puppet/modules/site_webapp/manifests/couchdb.pp
index 3ae4d266..1dbc745d 100644
--- a/puppet/modules/site_webapp/manifests/couchdb.pp
+++ b/puppet/modules/site_webapp/manifests/couchdb.pp
@@ -6,6 +6,8 @@ class site_webapp::couchdb {
   $couchdb_port            = '4096'
   $couchdb_webapp_user     = $webapp['couchdb_webapp_user']['username']
   $couchdb_webapp_password = $webapp['couchdb_webapp_user']['password']
+  $couchdb_admin_user      = $webapp['couchdb_admin_user']['username']
+  $couchdb_admin_password  = $webapp['couchdb_admin_user']['password']
 
   include x509::variables
 
@@ -17,6 +19,13 @@ class site_webapp::couchdb {
       mode    => '0600',
       require => Vcsrepo['/srv/leap/webapp'];
 
+    '/srv/leap/webapp/config/couchdb.admin.yml':
+      content => template('site_webapp/couchdb.admin.yml.erb'),
+      owner   => leap-webapp,
+      group   => leap-webapp,
+      mode    => '0600',
+      require => Vcsrepo['/srv/leap/webapp'];
+
     '/srv/leap/webapp/log':
       ensure  => directory,
       owner   => leap-webapp,
diff --git a/puppet/modules/site_webapp/manifests/cron.pp b/puppet/modules/site_webapp/manifests/cron.pp
index 811ad11d..bdf0fb74 100644
--- a/puppet/modules/site_webapp/manifests/cron.pp
+++ b/puppet/modules/site_webapp/manifests/cron.pp
@@ -2,6 +2,18 @@ class site_webapp::cron {
 
   # cron tasks that need to be performed to cleanup the database
   cron {
+    'rotate_databases':
+      command     => 'cd /srv/leap/webapp && bundle exec rake db:rotate',
+      environment => 'RAILS_ENV=production',
+      hour        => [0,6,12,18],
+      minute      => 0;
+
+    'delete_tmp_databases':
+      command     => 'cd /srv/leap/webapp && bundle exec rake db:deletetmp',
+      environment => 'RAILS_ENV=production',
+      hour        => 1,
+      minute      => 1;
+
     'remove_expired_sessions':
       command     => 'cd /srv/leap/webapp && bundle exec rake cleanup:sessions',
       environment => 'RAILS_ENV=production',
diff --git a/puppet/modules/site_webapp/manifests/init.pp b/puppet/modules/site_webapp/manifests/init.pp
index ea64048b..5071d9bc 100644
--- a/puppet/modules/site_webapp/manifests/init.pp
+++ b/puppet/modules/site_webapp/manifests/init.pp
@@ -50,7 +50,7 @@ class site_webapp {
     owner    => 'leap-webapp',
     group    => 'leap-webapp',
     require  => [ User['leap-webapp'], Group['leap-webapp'] ],
-    notify   => Exec['bundler_update']
+    notify   => [ Exec['bundler_update'], Exec['rotate_dbs'] ]
   }
 
   exec { 'bundler_update':
@@ -67,6 +67,19 @@ class site_webapp {
     notify  => Service['apache'];
   }
 
+  # this only needs to be called before the first time the web app is run.
+  # after that, the cron job will take care of running db:rotate regularly.
+  exec { 'rotate_dbs':
+    cwd     => '/srv/leap/webapp',
+    command => '/bin/bash -c "RAILS_ENV=production /usr/bin/bundle exec rake db:rotate"',
+    user    => 'leap-webapp',
+    timeout => 600,
+    refreshonly => true,
+    require => [
+      Vcsrepo['/srv/leap/webapp'],
+      Class['site_config::ruby::dev']];
+  }
+
   #
   # NOTE: in order to support a webapp that is running on a subpath and not the
   # root of the domain assets:precompile needs to be run with
diff --git a/puppet/modules/site_webapp/templates/couchdb.admin.yml.erb b/puppet/modules/site_webapp/templates/couchdb.admin.yml.erb
new file mode 100644
index 00000000..a0921add
--- /dev/null
+++ b/puppet/modules/site_webapp/templates/couchdb.admin.yml.erb
@@ -0,0 +1,9 @@
+production:
+  prefix: ""
+  protocol: 'http'
+  host: <%= @couchdb_host %>
+  port: <%= @couchdb_port %>
+  auto_update_design_doc: false
+  username: <%= @couchdb_admin_user %>
+  password: <%= @couchdb_admin_password %>
+
diff --git a/puppet/modules/tapicero/templates/tapicero.yaml.erb b/puppet/modules/tapicero/templates/tapicero.yaml.erb
index 40048a5e..fb3b93aa 100644
--- a/puppet/modules/tapicero/templates/tapicero.yaml.erb
+++ b/puppet/modules/tapicero/templates/tapicero.yaml.erb
@@ -17,7 +17,7 @@ connection:
 
 # file to store the last processed user record in so we can resume after
 # a restart:
-seq_file: "/var/lib/leap/tapicero/tapicero.seq"
+seq_dir: "/var/lib/leap/tapicero/"
 
 # Configure log_file like this if you want to log to a file instead of syslog:
 # log_file: "/var/leap/log/tapicero.log"