-- cgit v1.2.3 From 0a72bc6fd292bf9367b314fcb0347c4d35042f16 Mon Sep 17 00:00:00 2001 From: varac Date: Fri, 17 Jun 2016 09:44:10 +0200 Subject: New Build Badge from 0xacab.org for master branch --- README.md | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/README.md b/README.md index edc272d8..f2f10f1c 100644 --- a/README.md +++ b/README.md @@ -1,7 +1,9 @@ Leap Platform ============================= -[![Build Status](https://jenkins.leap.se/job/platform_develop/badge/icon)](https://jenkins.leap.se/job/platform_develop/) + +[![build status](https://0xacab.org/leap/platform/badges/master/build.svg)](https://0xacab.org/leap/platform/commits/master) + The LEAP Platform is set of complementary packages and server recipes to automate the maintenance of LEAP services in a hardened Debian environment. Its goal is to make it as painless as possible for sysadmins to deploy and maintain a service provider's infrastructure for secure communication. These recipes define an abstract service provider. It is a set of Puppet modules designed to work together to provide to sysadmins everything they need to manage a service provider infrastructure that provides secure communication services. -- cgit v1.2.3 From c8ea78a6ca823d94682c52627e90e7215ada2fff Mon Sep 17 00:00:00 2001 From: Micah Anderson Date: Thu, 3 Nov 2016 14:05:52 -0400 Subject: refresh docs using https://0xacab.org/leap/leap_se/blob/master/docs/README.md process --- docs/en/commands.html | 0 docs/en/guide/virtual-machines.html | 2 +- docs/en/guide/virtual-machines/index.html | 2 +- docs/en/services/couchdb.html | 2 +- docs/en/services/couchdb/index.html | 2 +- docs/en/services/mx.html | 4 ++-- docs/en/services/mx/index.html | 4 ++-- docs/index.html | 2 +- 8 files changed, 9 insertions(+), 9 deletions(-) delete mode 100644 docs/en/commands.html diff --git a/docs/en/commands.html b/docs/en/commands.html deleted file mode 100644 index e69de29b..00000000 diff --git a/docs/en/guide/virtual-machines.html b/docs/en/guide/virtual-machines.html index 5cee9a40..c522c181 100644 --- a/docs/en/guide/virtual-machines.html +++ b/docs/en/guide/virtual-machines.html @@ -142,7 +142,7 @@ Virtual Machines - LEAP Platform Documentation

Introduction

-

You can use the leap command line to easily remote virtual machines.

+

You can use the leap command line to easily manage remote virtual machines.

Note: there are two types of virtual machines that leap can handle:

diff --git a/docs/en/guide/virtual-machines/index.html b/docs/en/guide/virtual-machines/index.html index da0da107..4b2a2e0f 100644 --- a/docs/en/guide/virtual-machines/index.html +++ b/docs/en/guide/virtual-machines/index.html @@ -142,7 +142,7 @@ Virtual Machines - LEAP Platform Documentation

Introduction

-

You can use the leap command line to easily remote virtual machines.

+

You can use the leap command line to easily manage remote virtual machines.

Note: there are two types of virtual machines that leap can handle:

diff --git a/docs/en/services/couchdb.html b/docs/en/services/couchdb.html index 6de6455c..de50a692 100644 --- a/docs/en/services/couchdb.html +++ b/docs/en/services/couchdb.html @@ -215,7 +215,7 @@ couchdb - LEAP Platform Documentation diff --git a/docs/en/services/couchdb/index.html b/docs/en/services/couchdb/index.html index 10043db6..9eb7fcb8 100644 --- a/docs/en/services/couchdb/index.html +++ b/docs/en/services/couchdb/index.html @@ -215,7 +215,7 @@ couchdb - LEAP Platform Documentation diff --git a/docs/en/services/mx.html b/docs/en/services/mx.html index 8e08cfe0..8f5a36da 100644 --- a/docs/en/services/mx.html +++ b/docs/en/services/mx.html @@ -156,8 +156,8 @@ mx - LEAP Platform Documentation
  1. alias lists: by specifying an array of destination addresses, as in the case of “flock”, the single email will get copied to each address.
    2. -
  2. chained resolution: alias resolution will recursively continue until there are no more matching aliases. For example, “flock” is resolved to “robin”, which then gets resolved to “robin@bird.org”.
    3. -
  3. virtual domains: by specifying the full domain, as in the case of “chickadee@avian.org”, the alias will work for any domain you want. Of course, the MX record for that domain must point to appropriate MX servers, but otherwise you don’t need to do any additional configuration.
    4. +
  4. chained resolution: alias resolution will recursively continue until there are no more matching aliases. For example, “flock” is resolved to “robin”, which then gets resolved to “robin@bird.org”.
    5. +
  5. virtual domains: by specifying the full domain, as in the case of “chickadee@avian.org”, the alias will work for any domain you want. Of course, the MX record for that domain must point to appropriate MX servers, but otherwise you don’t need to do any additional configuration.
  6. local delivery: for testing purposes, it is often useful to copy all incoming mail for a particular address and send those copies to another address. You can do this by adding “@deliver.local” as one of the destination addresses. When “@local.delivery” is found, alias resolution stops and the mail is delivered to that username.
diff --git a/docs/en/services/mx/index.html b/docs/en/services/mx/index.html index 6899e0cc..e8e06e80 100644 --- a/docs/en/services/mx/index.html +++ b/docs/en/services/mx/index.html @@ -156,8 +156,8 @@ mx - LEAP Platform Documentation
  1. alias lists: by specifying an array of destination addresses, as in the case of “flock”, the single email will get copied to each address.
    2. -
  2. chained resolution: alias resolution will recursively continue until there are no more matching aliases. For example, “flock” is resolved to “robin”, which then gets resolved to “robin@bird.org”.
    3. -
  3. virtual domains: by specifying the full domain, as in the case of “chickadee@avian.org”, the alias will work for any domain you want. Of course, the MX record for that domain must point to appropriate MX servers, but otherwise you don’t need to do any additional configuration.
    4. +
  4. chained resolution: alias resolution will recursively continue until there are no more matching aliases. For example, “flock” is resolved to “robin”, which then gets resolved to “robin@bird.org”.
    5. +
  5. virtual domains: by specifying the full domain, as in the case of “chickadee@avian.org”, the alias will work for any domain you want. Of course, the MX record for that domain must point to appropriate MX servers, but otherwise you don’t need to do any additional configuration.
  6. local delivery: for testing purposes, it is often useful to copy all incoming mail for a particular address and send those copies to another address. You can do this by adding “@deliver.local” as one of the destination addresses. When “@local.delivery” is found, alias resolution stops and the mail is delivered to that username.
diff --git a/docs/index.html b/docs/index.html index 49465a9d..5ef4303c 100644 --- a/docs/index.html +++ b/docs/index.html @@ -152,7 +152,7 @@ Provider Platform - LEAP Platform Documentation

The leap command line tool

-

The leap command line tool is used by sysadmins to manage everything about a service provider’s infrastructure.

+

The leap command line tool is used by sysadmins to manage everything about a service provider’s infrastructure.

Keep these rules in mind:

-- cgit v1.2.3 From 971dae655910ccdb37e2612e38ee6fc58d5f6efa Mon Sep 17 00:00:00 2001 From: Micah Anderson Date: Fri, 4 Nov 2016 10:49:38 -0400 Subject: Additional entries/updates --- CHANGES.md | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-) diff --git a/CHANGES.md b/CHANGES.md index f447e9d5..3dc66746 100644 --- a/CHANGES.md +++ b/CHANGES.md @@ -9,17 +9,24 @@ New Features: * `leap vm` -- Support for managing remote virtual servers (AWS only, for now) * `leap cert renew` -- Integration with Let's Encrypt +* `leap open monitor` -- for handy access to nagios * improved documentation -- open docs/index.html to see Notable Changes: -* 58 bugs fixed +* 86 bugs fixed * Fixed security issues with VPN * More tests * Replaced git submodules with git subrepo * Nearly all the leap_cli code has been moved to leap_platform.git +* Command-line leap_cli cleanup to be more logically consistent * Better organization of the leap_platform.git directory structure * Removed ugly dependency on Capistrano +* Enabled DANE/TLSA validation +* Anti-spam improvements +* Performance improvements for couchdb +* Change from httpredir.debian.org to deb.debian.org +* Reduce duplicated logging Upgrading: -- cgit v1.2.3 From 4d8a1bc0ce304404e4d0cab0ce6a3c51ed035e71 Mon Sep 17 00:00:00 2001 From: varac Date: Tue, 8 Nov 2016 21:05:39 +0100 Subject: New nickserver is using fully qualified ruby path now --- tests/server-tests/white-box/webapp.rb | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tests/server-tests/white-box/webapp.rb b/tests/server-tests/white-box/webapp.rb index da1ec8c5..40c234d6 100644 --- a/tests/server-tests/white-box/webapp.rb +++ b/tests/server-tests/white-box/webapp.rb @@ -28,7 +28,7 @@ class Webapp < LeapTest def test_03_Are_daemons_running? assert_running '^/usr/sbin/apache2' - assert_running '^ruby /usr/bin/nickserver' + assert_running '^/usr/bin/ruby /usr/bin/nickserver' pass end -- cgit v1.2.3 From d8255eb07a8b96ad04fcade8486530db2e61dad4 Mon Sep 17 00:00:00 2001 From: varac Date: Wed, 9 Nov 2016 13:50:30 +0100 Subject: Don't do strict checking for nickserver Sometimes nickserver is listed with `ruby /usr/bin/nickserver start` in the process table, sometimes with `/usr/bin/ruby /usr/bin/nickserver start`. We should do proper checking with `systemctl status nickserver` to make sure the service is up though (https://leap.se/code/issues/8579). Meanwhile it's ok to not do strict checking. --- tests/server-tests/white-box/webapp.rb | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tests/server-tests/white-box/webapp.rb b/tests/server-tests/white-box/webapp.rb index 40c234d6..c46c9f96 100644 --- a/tests/server-tests/white-box/webapp.rb +++ b/tests/server-tests/white-box/webapp.rb @@ -28,7 +28,7 @@ class Webapp < LeapTest def test_03_Are_daemons_running? assert_running '^/usr/sbin/apache2' - assert_running '^/usr/bin/ruby /usr/bin/nickserver' + assert_running 'ruby /usr/bin/nickserver' pass end -- cgit v1.2.3 From ec4ec9d17c6fb08030a6178f7131f8a95cc9bdd5 Mon Sep 17 00:00:00 2001 From: varac Date: Thu, 10 Nov 2016 23:38:31 +0100 Subject: Use webapp 0.9 --- provider_base/common.json | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/provider_base/common.json b/provider_base/common.json index 893d5daf..c6ab18d5 100644 --- a/provider_base/common.json +++ b/provider_base/common.json @@ -86,7 +86,7 @@ "webapp": { "type": "git", "source": "https://leap.se/git/leap_web", - "revision": "origin/version/0.8" + "revision": "origin/version/0.9" } } } -- cgit v1.2.3 From c641fe08f26dce1c06ed61dc2d5a8b75486807fe Mon Sep 17 00:00:00 2001 From: Micah Anderson Date: Mon, 21 Nov 2016 21:58:09 -0500 Subject: Ignore non-existing locale.gen on init (#8649) --- bin/node_init | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/bin/node_init b/bin/node_init index b55cfed3..148ecc34 100755 --- a/bin/node_init +++ b/bin/node_init @@ -22,7 +22,7 @@ if ! egrep -q "$DEBIAN_VERSION" /etc/debian_version; then exit 1 fi mkdir -p $LEAP_DIR -if ! grep -q -e '^en_US.UTF-8' /etc/locale.gen; then +if ! grep -q -e '^en_US.UTF-8' /etc/locale.gen 2> /dev/null; then echo "en_US.UTF-8 UTF-8" >> /etc/locale.gen /usr/sbin/locale-gen fi -- cgit v1.2.3 From 678a211ca31a7801d8bef8a74ca30feaa16af508 Mon Sep 17 00:00:00 2001 From: drebs Date: Fri, 18 Nov 2016 18:03:26 -0200 Subject: add a timeout for the soledad sync test script (#8590) If for any reason a sync takes too long, the script will timeout and log an errro. --- tests/server-tests/helpers/soledad_sync.py | 15 ++++++++++++++- 1 file changed, 14 insertions(+), 1 deletion(-) diff --git a/tests/server-tests/helpers/soledad_sync.py b/tests/server-tests/helpers/soledad_sync.py index f4fc81ae..b674818d 100755 --- a/tests/server-tests/helpers/soledad_sync.py +++ b/tests/server-tests/helpers/soledad_sync.py @@ -35,6 +35,7 @@ flags.set_events_enabled(False) NUMDOCS = 1 USAGE = "Usage: %s uuid token server cert_file password" % sys.argv[0] +SYNC_TIMEOUT = 60 def bail(msg, exitcode): @@ -68,12 +69,23 @@ if __name__ == '__main__': s = get_soledad_instance( uuid, passphrase, tempdir, server, cert_file, token) + def syncWithTimeout(_): + d = s.sync() + reactor.callLater(SYNC_TIMEOUT, d.cancel) + return d + def onSyncDone(sync_result): print "SYNC_RESULT:", sync_result s.close() rm_tempdir() reactor.stop() + def trap_cancel(f): + f.trap(defer.CancelledError) + log.err("sync timed out after %s seconds" % SYNC_TIMEOUT) + rm_tempdir() + reactor.stop() + def log_and_exit(f): log.err(f) rm_tempdir() @@ -81,8 +93,9 @@ if __name__ == '__main__': def start_sync(): d = create_docs(s) - d.addCallback(lambda _: s.sync()) + d.addCallback(syncWithTimeout) d.addCallback(onSyncDone) + d.addErrback(trap_cancel) d.addErrback(log_and_exit) reactor.callWhenRunning(start_sync) -- cgit v1.2.3 From 389228df6ee52ce41cc83c2b91fe0b6572d4bc50 Mon Sep 17 00:00:00 2001 From: drebs Date: Fri, 18 Nov 2016 18:16:41 -0200 Subject: use lock to avoid running multiple soledad tests (#8590) If a soledad sync test script is already running, there's no need to run another one. This avoids having multiple test script hanging and eating up resources. We have seen this situation under development circumstances, when the soledad server has been modified in a way that the client hangs and never finishes. --- tests/server-tests/helpers/soledad_sync.py | 11 +++++++++++ 1 file changed, 11 insertions(+) diff --git a/tests/server-tests/helpers/soledad_sync.py b/tests/server-tests/helpers/soledad_sync.py index b674818d..a92ec68f 100755 --- a/tests/server-tests/helpers/soledad_sync.py +++ b/tests/server-tests/helpers/soledad_sync.py @@ -27,6 +27,7 @@ os.environ['SKIP_TWISTED_SSL_CHECK'] = '1' from twisted.internet import defer, reactor from twisted.python import log +from twisted.python.lockfile import FilesystemLock from client_side_db import get_soledad_instance from leap.common.events import flags @@ -43,6 +44,13 @@ def bail(msg, exitcode): sys.exit(exitcode) +def obtain_lock(): + scriptname = os.path.basename(__file__) + lockfile = os.path.join(tempfile.gettempdir(), scriptname + '.lock') + lock = FilesystemLock(lockfile) + return lock.lock() + + def create_docs(soledad): """ Populates the soledad database with dummy messages, so we can exercise @@ -65,6 +73,9 @@ if __name__ == '__main__': if len(sys.argv) < 6: bail(USAGE, 2) + if not obtain_lock(): + bail("another instance is already running", 1) + uuid, token, server, cert_file, passphrase = sys.argv[1:] s = get_soledad_instance( uuid, passphrase, tempdir, server, cert_file, token) -- cgit v1.2.3 From ffd23f263d15ce04d1610e87abbdca3554dadd75 Mon Sep 17 00:00:00 2001 From: varac Date: Thu, 8 Dec 2016 23:26:04 +0100 Subject: Lint site_config::files --- puppet/modules/site_config/manifests/files.pp | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/puppet/modules/site_config/manifests/files.pp b/puppet/modules/site_config/manifests/files.pp index d2ef8a98..e74ad567 100644 --- a/puppet/modules/site_config/manifests/files.pp +++ b/puppet/modules/site_config/manifests/files.pp @@ -3,10 +3,10 @@ class site_config::files { file { '/srv/leap': - ensure => directory, - owner => 'root', - group => 'root', - mode => '0711'; + ensure => directory, + owner => 'root', + group => 'root', + mode => '0711'; [ '/etc/leap', '/var/lib/leap']: ensure => directory, -- cgit v1.2.3 From 3c4b29162e17960108a92ecc71274ecc4c9c3f76 Mon Sep 17 00:00:00 2001 From: varac Date: Thu, 8 Dec 2016 09:52:46 +0100 Subject: Use webapp/nickserver:master on leap_platform:master (#8678) --- provider_base/common.json | 6 +++--- tests/platform-ci/ci-build.sh | 3 +-- tests/platform-ci/provider/common.json | 10 ---------- 3 files changed, 4 insertions(+), 15 deletions(-) diff --git a/provider_base/common.json b/provider_base/common.json index c6ab18d5..666fe923 100644 --- a/provider_base/common.json +++ b/provider_base/common.json @@ -71,11 +71,11 @@ "nickserver": { "type": "git", "source": "https://leap.se/git/nickserver", - "revision": "origin/version/0.9" + "revision": "origin/master" }, "platform": { "apt": { - "basic": "= 'http://deb.leap.se/' + Leap::Platform.major_version" + "basic": "http://deb.leap.se/experimental-platform" } }, "soledad": { @@ -86,7 +86,7 @@ "webapp": { "type": "git", "source": "https://leap.se/git/leap_web", - "revision": "origin/version/0.9" + "revision": "origin/master" } } } diff --git a/tests/platform-ci/ci-build.sh b/tests/platform-ci/ci-build.sh index 85557b3f..0dfbb5c3 100755 --- a/tests/platform-ci/ci-build.sh +++ b/tests/platform-ci/ci-build.sh @@ -43,8 +43,7 @@ NAME="citest${CI_BUILD_ID}" TAG='single' SERVICES='couchdb,soledad,mx,webapp,tor,monitor' -SEEDS='sources.platform.apt.basic:http://deb.leap.se/experimental-0.9 sources.webapp.revision:master sources.nickserver.revision:master' - +SEEDS='' # # Main diff --git a/tests/platform-ci/provider/common.json b/tests/platform-ci/provider/common.json index a13f8f75..2c63c085 100644 --- a/tests/platform-ci/provider/common.json +++ b/tests/platform-ci/provider/common.json @@ -1,12 +1,2 @@ { - "sources": { - "platform": { - "apt": { - "basic": "http://deb.leap.se/experimental-0.9" - } - }, - "nickserver": { - "revision": "develop" - } - } } -- cgit v1.2.3 From e767aa460fc64a317551012f1781c2105c572158 Mon Sep 17 00:00:00 2001 From: elijah Date: Mon, 19 Dec 2016 15:23:57 -0800 Subject: feature: add troubleshooting info to `leap user ls` It is hard to get ssh key setup right. This change makes it much easier to debug what the problem is. --- lib/leap_cli/commands/user.rb | 25 +++++++++++++++++++++++++ lib/leap_cli/ssh/key.rb | 11 ++++++----- 2 files changed, 31 insertions(+), 5 deletions(-) diff --git a/lib/leap_cli/commands/user.rb b/lib/leap_cli/commands/user.rb index 1ca92719..a10d5163 100644 --- a/lib/leap_cli/commands/user.rb +++ b/lib/leap_cli/commands/user.rb @@ -113,6 +113,20 @@ module LeapCli def do_list_users(global, options, args) require 'leap_cli/ssh' + ssh_keys = {} + Dir.glob("#{ENV['HOME']}/.ssh/*.pub").each do |keyfile| + key = SSH::Key.load(keyfile) + ssh_keys[key.fingerprint] = key if key + end + + ssh_agent_keys = {} + if !`which ssh-add`.empty? + `ssh-add -L`.split("\n").each do |keystring| + key = SSH::Key.load(keystring) + ssh_agent_keys[key.fingerprint] = key if key + end + end + Dir.glob(path([:user_ssh, '*'])).each do |keyfile| username = File.basename(File.dirname(keyfile)) log username, :color => :cyan do @@ -121,6 +135,14 @@ module LeapCli log 'SSH MD5 fingerprint: ' + key.fingerprint(:digest => :md5, :type => :ssh, :encoding => :hex) log 'SSH SHA256 fingerprint: ' + key.fingerprint(:digest => :sha256, :type => :ssh, :encoding => :base64) log 'DER MD5 fingerprint: ' + key.fingerprint(:digest => :md5, :type => :der, :encoding => :hex) + if ssh_keys[key.fingerprint] + log 'Matches local key: ' + ssh_keys[key.fingerprint].filename, color: :green + if ssh_agent_keys[key.fingerprint] + log 'Matches ssh-agent key: ' + ssh_agent_keys[key.fingerprint].summary(encoding: :base64), color: :green + else + log :error, 'No matching key in the ssh-agent' + end + end end end end @@ -154,6 +176,9 @@ module LeapCli end else key_index = 0 + log "Picking the only compatible ssh key: "+ ssh_keys[key_index].filename do + log ssh_keys[key_index].summary + end end return ssh_keys[key_index] diff --git a/lib/leap_cli/ssh/key.rb b/lib/leap_cli/ssh/key.rb index 76223b7e..108b6137 100644 --- a/lib/leap_cli/ssh/key.rb +++ b/lib/leap_cli/ssh/key.rb @@ -254,9 +254,9 @@ module LeapCli end if digest == "MD5" && encoding == :hex - return fp.scan(/../).join(':') + return fp.strip.scan(/../).join(':') else - return fp + return fp.strip end end @@ -267,11 +267,12 @@ module LeapCli Net::SSH::Buffer.from(:key, @key).to_s.split("\001\000").last.size * 8 end - def summary + def summary(type: :ssh, digest: :sha256, encoding: :hex) + fp = digest.to_s.upcase + ":" + self.fingerprint(type: type, digest: digest, encoding: encoding) if self.filename - "%s %s %s (%s)" % [self.type, self.bits, self.fingerprint, File.basename(self.filename)] + "%s %s %s (%s)" % [self.type, self.bits, fp, File.basename(self.filename)] else - "%s %s %s" % [self.type, self.bits, self.fingerprint] + "%s %s %s" % [self.type, self.bits, fp] end end -- cgit v1.2.3 From dc43b30079316ed41bf95eca902d5d65ba877888 Mon Sep 17 00:00:00 2001 From: elijah Date: Mon, 19 Dec 2016 15:28:11 -0800 Subject: bugfix: ensure let's encrypt errors make it to the user. --- lib/leap_cli/commands/ca.rb | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/lib/leap_cli/commands/ca.rb b/lib/leap_cli/commands/ca.rb index 3c5fc7d5..1c67ae67 100644 --- a/lib/leap_cli/commands/ca.rb +++ b/lib/leap_cli/commands/ca.rb @@ -281,11 +281,13 @@ module LeapCli; module Commands if status == 'valid' log 'authorized!', color: :green, style: :bold elsif status == 'error' - bail! :error, message + bail! :error, message.inspect elsif status == 'unauthorized' - bail!(:unauthorized, message, color: :yellow, style: :bold) do + bail!(:unauthorized, message.inspect, color: :yellow, style: :bold) do log 'You must first run `leap cert register` to register the account key with letsencrypt.org' end + else + bail!(:error, "unrecognized status: #{status.inspect}, #{message.inspect}") end log :fetching, "new certificate from letsencrypt.org" -- cgit v1.2.3 From 1afe3c5c107fc26d7ef0e8171c98b79463a15bcd Mon Sep 17 00:00:00 2001 From: elijah Date: Mon, 19 Dec 2016 15:29:38 -0800 Subject: bugfix: mx service does not require a commercial certificate --- provider_base/services/mx.json | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/provider_base/services/mx.json b/provider_base/services/mx.json index c7e99d85..2db773b5 100644 --- a/provider_base/services/mx.json +++ b/provider_base/services/mx.json @@ -37,7 +37,7 @@ }, "x509": { "use": true, - "use_commercial": true, + "use_commercial": false, "ca_cert": "= file :ca_cert, :missing => 'provider CA. Run `leap cert ca`'", "client_ca_cert": "= file :client_ca_cert, :missing => 'Certificate Authority. Run `leap cert ca`'", "client_ca_key": "= file :client_ca_key, :missing => 'Certificate Authority. Run `leap cert ca`'" -- cgit v1.2.3 From 866af737a3e641008c05a210d04a4dc5e5bcfbc4 Mon Sep 17 00:00:00 2001 From: elijah Date: Tue, 20 Dec 2016 10:12:39 -0800 Subject: bugfix: couchdb nodes should not require soledad. closes #8693 --- puppet/modules/site_couchdb/manifests/create_dbs.pp | 11 +++++++---- 1 file changed, 7 insertions(+), 4 deletions(-) diff --git a/puppet/modules/site_couchdb/manifests/create_dbs.pp b/puppet/modules/site_couchdb/manifests/create_dbs.pp index ddfb7d65..1c594bb6 100644 --- a/puppet/modules/site_couchdb/manifests/create_dbs.pp +++ b/puppet/modules/site_couchdb/manifests/create_dbs.pp @@ -1,5 +1,6 @@ # creates neccesary databases class site_couchdb::create_dbs { + $services = hiera('services', []) Class['site_couchdb::setup'] -> Class['site_couchdb::create_dbs'] @@ -42,10 +43,12 @@ class site_couchdb::create_dbs { ## shared database ## r/w: soledad - couchdb::create_db { 'shared': - members => "{ \"names\": [\"${site_couchdb::couchdb_soledad_user}\"], \"roles\": [\"replication\"] }", - require => Couchdb::Query::Setup['localhost'], - notify => Service['soledad-server']; + if member($services, 'soledad') { + couchdb::create_db { 'shared': + members => "{ \"names\": [\"${site_couchdb::couchdb_soledad_user}\"], \"roles\": [\"replication\"] }", + require => Couchdb::Query::Setup['localhost'], + notify => Service['soledad-server']; + } } ## tickets database -- cgit v1.2.3 From da70f97f9478281c296c3412dc1f25ada989eeb2 Mon Sep 17 00:00:00 2001 From: varac Date: Tue, 20 Dec 2016 22:34:16 +0100 Subject: [Vagrant] Remove versioning of provider config --- tests/example-provider/vagrant/configure-leap.sh | 7 ------- tests/example-provider/vagrant/install-platform.pp | 3 --- tests/example-provider/vagrant/vagrant.config | 2 -- 3 files changed, 12 deletions(-) diff --git a/tests/example-provider/vagrant/configure-leap.sh b/tests/example-provider/vagrant/configure-leap.sh index fd34d7ea..8bd591e0 100755 --- a/tests/example-provider/vagrant/configure-leap.sh +++ b/tests/example-provider/vagrant/configure-leap.sh @@ -35,10 +35,6 @@ echo '{ "webapp": { "admins": ["testadmin"] } }' > services/webapp.json $LEAP $OPTS compile -$GIT init -$GIT add . -$GIT commit -m'configured provider' - $LEAP $OPTS node init $NODE if [ $? -eq 1 ]; then echo 'node init failed' @@ -52,9 +48,6 @@ gem install rake $LEAP $OPTS -v 2 deploy -$GIT add . -$GIT commit -m'initialized and deployed provider' - # Vagrant: leap_mx fails to start on jessie # https://leap.se/code/issues/7755 # Workaround: we stop and start leap-mx after deploy and diff --git a/tests/example-provider/vagrant/install-platform.pp b/tests/example-provider/vagrant/install-platform.pp index 223853c1..9cefcf7c 100755 --- a/tests/example-provider/vagrant/install-platform.pp +++ b/tests/example-provider/vagrant/install-platform.pp @@ -10,6 +10,3 @@ class { '::leap::cli::install': file { [ '/srv/leap', '/srv/leap/configuration', '/var/log/leap' ]: ensure => directory } - -# install prerequisites for configuring the provider -include ::git diff --git a/tests/example-provider/vagrant/vagrant.config b/tests/example-provider/vagrant/vagrant.config index 60d2a52c..ff5dd38f 100644 --- a/tests/example-provider/vagrant/vagrant.config +++ b/tests/example-provider/vagrant/vagrant.config @@ -19,5 +19,3 @@ SUDO="sudo -u ${USER}" PROVIDERDIR="/home/${USER}/leap/configuration" PLATFORMDIR="/srv/leap_platform" LEAP="$SUDO /usr/local/bin/leap" -GIT="$SUDO git" - -- cgit v1.2.3 From 3d254dcd1899fa266d0170f18da7b556a8114302 Mon Sep 17 00:00:00 2001 From: varac Date: Tue, 20 Dec 2016 22:36:47 +0100 Subject: [Vagrant] Install leap_cli gem dependencies --- puppet/modules/leap/manifests/cli/install.pp | 16 ++++++++++++---- 1 file changed, 12 insertions(+), 4 deletions(-) diff --git a/puppet/modules/leap/manifests/cli/install.pp b/puppet/modules/leap/manifests/cli/install.pp index 25e87033..d009316b 100644 --- a/puppet/modules/leap/manifests/cli/install.pp +++ b/puppet/modules/leap/manifests/cli/install.pp @@ -1,13 +1,20 @@ # installs leap_cli on node class leap::cli::install ( $source = false ) { + + # nokogiri is a dependency gem of leap_cli and + # needs build tools in order to get compiled + ensure_packages (['gcc', 'make', 'zlib1g-dev']) + class { '::ruby': + install_dev => true, + require => [ Package['gcc'], Package['make'], Package['zlib1g-dev'] ] + } + + if $source { # needed for building leap_cli from source include ::git include ::rubygems - class { '::ruby': - install_dev => true - } class { 'bundler::install': install_method => 'package' } @@ -40,7 +47,8 @@ class leap::cli::install ( $source = false ) { else { package { 'leap_cli': ensure => installed, - provider => gem + provider => gem, + require => Class['ruby'] } } } -- cgit v1.2.3 From c0f489c4226c924fa1d96d12cba7eb5f63ccaf64 Mon Sep 17 00:00:00 2001 From: elijah Date: Wed, 21 Dec 2016 11:06:48 -0800 Subject: add command `leap node disable` and `leap node enable` --- lib/leap_cli/commands/node.rb | 27 +++++++++++++++++++++++++++ lib/leap_cli/config/environment.rb | 30 ++++++++++++++++++++++++++---- lib/leap_cli/config/node.rb | 4 ++-- 3 files changed, 55 insertions(+), 6 deletions(-) diff --git a/lib/leap_cli/commands/node.rb b/lib/leap_cli/commands/node.rb index 60540de9..9cde15bc 100644 --- a/lib/leap_cli/commands/node.rb +++ b/lib/leap_cli/commands/node.rb @@ -45,6 +45,23 @@ module LeapCli; module Commands do_node_rm(global_options, options, args) end end + + node.desc 'Mark a node as disabled.' + node.arg_name 'NAME' + node.command :disable do |cmd| + cmd.action do |global_options,options,args| + do_node_disable(global_options, options, args) + end + end + + node.desc 'Mark a node as enabled.' + node.arg_name 'NAME' + node.command :enable do |cmd| + cmd.action do |global_options,options,args| + do_node_enable(global_options, options, args) + end + end + end ## @@ -126,4 +143,14 @@ module LeapCli; module Commands remove_node_facts(node.name) end + def do_node_enable(global, options, args) + node = get_node_from_args(args, include_disabled: true) + node.update_json({}, remove: ["enabled"]) + end + + def do_node_disable(global, options, args) + node = get_node_from_args(args, include_disabled: true) + node.update_json("enabled" => false) + end + end; end diff --git a/lib/leap_cli/config/environment.rb b/lib/leap_cli/config/environment.rb index ce570839..0410ef5b 100644 --- a/lib/leap_cli/config/environment.rb +++ b/lib/leap_cli/config/environment.rb @@ -122,14 +122,25 @@ module LeapCli; module Config end # - # Alters the node's json config file. Unfortunately, doing this will - # strip out all the comments. + # Alters the node's json config file. As a side effect, all comments get + # moved to the top of the file. # - def update_node_json(node, new_values) + # NOTE: This does a shallow merge! In other words, a call like this... + # + # update_node_json(node, {"webapp" => {"domain" => "example.org"}) + # + # ...is probably not what you want, because it will entirely remove all + # existing entries under "webapp". + # + def update_node_json(node, new_values, options=nil) node_json_path = Path.named_path([:node_config, node.name]) + comments = read_comments(node_json_path) old_data = load_json(node_json_path, Config::Node) + options && options[:remove] && options[:remove].each do |key| + old_data.delete(key) + end new_data = old_data.merge(new_values) - new_contents = JSON.sorted_generate(new_data) + "\n" + new_contents = [comments, JSON.sorted_generate(new_data), "\n"].join Util::write_file! node_json_path, new_contents end @@ -152,6 +163,17 @@ module LeapCli; module Config results end + def read_comments(filename) + buffer = StringIO.new + File.open(filename, "rb", :encoding => 'UTF-8') do |f| + while (line = f.gets) + next unless line =~ /^\s*\/\// + buffer << line + end + end + return buffer.string.force_encoding('utf-8') + end + def load_json(filename, object_class, options={}) if !File.exist?(filename) return object_class.new(self) diff --git a/lib/leap_cli/config/node.rb b/lib/leap_cli/config/node.rb index 23abdee3..a7c5c1e4 100644 --- a/lib/leap_cli/config/node.rb +++ b/lib/leap_cli/config/node.rb @@ -169,8 +169,8 @@ module LeapCli; module Config # # modifies the config file nodes/NAME.json for this node. # - def update_json(new_values) - self.env.update_node_json(node, new_values) + def update_json(new_values, options=nil) + self.env.update_node_json(node, new_values, options) end # -- cgit v1.2.3 From 8ab553dfcaa1aff10123d15c908117b59d2d5b7d Mon Sep 17 00:00:00 2001 From: elijah Date: Wed, 21 Dec 2016 11:16:24 -0800 Subject: rename s/ca.rb/cert.rb/ --- lib/leap_cli/commands/ca.rb | 368 ------------------------------------------ lib/leap_cli/commands/cert.rb | 368 ++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 368 insertions(+), 368 deletions(-) delete mode 100644 lib/leap_cli/commands/ca.rb create mode 100644 lib/leap_cli/commands/cert.rb diff --git a/lib/leap_cli/commands/ca.rb b/lib/leap_cli/commands/ca.rb deleted file mode 100644 index 1c67ae67..00000000 --- a/lib/leap_cli/commands/ca.rb +++ /dev/null @@ -1,368 +0,0 @@ -module LeapCli; module Commands - - desc "Manage X.509 certificates" - command :cert do |cert| - - cert.desc 'Creates two Certificate Authorities (one for validating servers and one for validating clients).' - cert.long_desc 'See see what values are used in the generation of the certificates (like name and key size), run `leap inspect provider` and look for the "ca" property. To see the details of the created certs, run `leap inspect `.' - cert.command :ca do |ca| - ca.action do |global_options,options,args| - assert_config! 'provider.ca.name' - generate_new_certificate_authority(:ca_key, :ca_cert, provider.ca.name) - generate_new_certificate_authority(:client_ca_key, :client_ca_cert, provider.ca.name + ' (client certificates only!)') - end - end - - cert.desc 'Creates or renews a X.509 certificate/key pair for a single node or all nodes, but only if needed.' - cert.long_desc 'This command will a generate new certificate for a node if some value in the node has changed ' + - 'that is included in the certificate (like hostname or IP address), or if the old certificate will be expiring soon. ' + - 'Sometimes, you might want to force the generation of a new certificate, ' + - 'such as in the cases where you have changed a CA parameter for server certificates, like bit size or digest hash. ' + - 'In this case, use --force. If is empty, this command will apply to all nodes.' - cert.arg_name 'FILTER' - cert.command :update do |update| - update.switch 'force', :desc => 'Always generate new certificates', :negatable => false - update.action do |global_options,options,args| - update_certificates(manager.filter!(args), options) - end - end - - cert.desc 'Creates a Diffie-Hellman parameter file, needed for forward secret OpenVPN ciphers.' # (needed for server-side of some TLS connections) - cert.command :dh do |dh| - dh.action do |global_options,options,args| - generate_dh - end - end - - cert.desc "Creates a CSR for use in buying a commercial X.509 certificate." - cert.long_desc "Unless specified, the CSR is created for the provider's primary domain. "+ - "The properties used for this CSR come from `provider.ca.server_certificates`, "+ - "but may be overridden here." - cert.arg_name "DOMAIN" - cert.command :csr do |csr| - csr.flag 'domain', :arg_name => 'DOMAIN', :desc => 'Specify what domain to create the CSR for.' - csr.flag ['organization', 'O'], :arg_name => 'ORGANIZATION', :desc => "Override default O in distinguished name." - csr.flag ['unit', 'OU'], :arg_name => 'UNIT', :desc => "Set OU in distinguished name." - csr.flag 'email', :arg_name => 'EMAIL', :desc => "Set emailAddress in distinguished name." - csr.flag ['locality', 'L'], :arg_name => 'LOCALITY', :desc => "Set L in distinguished name." - csr.flag ['state', 'ST'], :arg_name => 'STATE', :desc => "Set ST in distinguished name." - csr.flag ['country', 'C'], :arg_name => 'COUNTRY', :desc => "Set C in distinguished name." - csr.flag :bits, :arg_name => 'BITS', :desc => "Override default certificate bit length" - csr.flag :digest, :arg_name => 'DIGEST', :desc => "Override default signature digest" - csr.action do |global_options,options,args| - generate_csr(global_options, options, args) - end - end - - cert.desc "Register an authorization key with the CA letsencrypt.org" - cert.long_desc "This only needs to be done once." - cert.command :register do |register| - register.action do |global, options, args| - do_register_key(global, options, args) - end - end - - cert.desc "Renews a certificate using the CA letsencrypt.org" - cert.arg_name "DOMAIN" - cert.command :renew do |renew| - renew.action do |global, options, args| - do_renew_cert(global, options, args) - end - end - - end - - protected - - # - # will generate new certificates for the specified nodes, if needed. - # - def update_certificates(nodes, options={}) - require 'leap_cli/x509' - assert_files_exist! :ca_cert, :ca_key, :msg => 'Run `leap cert ca` to create them' - assert_config! 'provider.ca.server_certificates.bit_size' - assert_config! 'provider.ca.server_certificates.digest' - assert_config! 'provider.ca.server_certificates.life_span' - assert_config! 'common.x509.use' - - nodes.each_node do |node| - node.warn_if_commercial_cert_will_soon_expire - if !node.x509.use - remove_file!([:node_x509_key, node.name]) - remove_file!([:node_x509_cert, node.name]) - elsif options[:force] || node.cert_needs_updating? - node.generate_cert - end - end - end - - # - # yields client key and cert suitable for testing - # - def generate_test_client_cert(prefix=nil) - require 'leap_cli/x509' - cert = CertificateAuthority::Certificate.new - cert.serial_number.number = cert_serial_number(provider.domain) - cert.subject.common_name = [prefix, random_common_name(provider.domain)].join - cert.not_before = X509.yesterday - cert.not_after = X509.yesterday.advance(:years => 1) - cert.key_material.generate_key(1024) # just for testing, remember! - cert.parent = client_ca_root - cert.sign! client_test_signing_profile - yield cert.key_material.private_key.to_pem, cert.to_pem - end - - private - - def generate_new_certificate_authority(key_file, cert_file, common_name) - require 'leap_cli/x509' - assert_files_missing! key_file, cert_file - assert_config! 'provider.ca.name' - assert_config! 'provider.ca.bit_size' - assert_config! 'provider.ca.life_span' - - root = X509.new_ca(provider.ca, common_name) - - write_file!(key_file, root.key_material.private_key.to_pem) - write_file!(cert_file, root.to_pem) - end - - def generate_dh - require 'leap_cli/x509' - long_running do - if cmd_exists?('certtool') - log 0, 'Generating DH parameters (takes a long time)...' - output = assert_run!('certtool --generate-dh-params --sec-param high') - output.sub!(/.*(-----BEGIN DH PARAMETERS-----.*-----END DH PARAMETERS-----).*/m, '\1') - output << "\n" - write_file!(:dh_params, output) - else - log 0, 'Generating DH parameters (takes a REALLY long time)...' - output = OpenSSL::PKey::DH.generate(3248).to_pem - write_file!(:dh_params, output) - end - end - end - - # - # hints: - # - # inspect CSR: - # openssl req -noout -text -in files/cert/x.csr - # - # generate CSR with openssl to see how it compares: - # openssl req -sha256 -nodes -newkey rsa:2048 -keyout example.key -out example.csr - # - # validate a CSR: - # http://certlogik.com/decoder/ - # - # nice details about CSRs: - # http://www.redkestrel.co.uk/Articles/CSR.html - # - def generate_csr(global_options, options, args) - require 'leap_cli/x509' - assert_config! 'provider.domain' - assert_config! 'provider.name' - assert_config! 'provider.default_language' - assert_config! 'provider.ca.server_certificates.bit_size' - assert_config! 'provider.ca.server_certificates.digest' - - server_certificates = provider.ca.server_certificates - options[:domain] ||= args.first || provider.domain - options[:organization] ||= provider.name[provider.default_language] - options[:country] ||= server_certificates['country'] - options[:state] ||= server_certificates['state'] - options[:locality] ||= server_certificates['locality'] - options[:bits] ||= server_certificates.bit_size - options[:digest] ||= server_certificates.digest - - unless global_options[:force] - assert_files_missing! [:commercial_key, options[:domain]], [:commercial_csr, options[:domain]], - :msg => 'If you really want to create a new key and CSR, remove these files first or run with --force.' - end - - X509.create_csr_and_cert(options) - end - - # - # letsencrypt.org - # - - def do_register_key(global, options, args) - require 'leap_cli/acme' - assert_config! 'provider.contacts.default' - contact = manager.provider.contacts.default.first - - if file_exists?(:acme_key) && !global[:force] - bail! do - log "the authorization key for letsencrypt.org already exists" - log "run with --force if you really want to register a new key." - end - else - private_key = Acme.new_private_key - registration = nil - - log(:registering, "letsencrypt.org authorization key using contact `%s`" % contact) do - acme = Acme.new(key: private_key) - registration = acme.register(contact) - if registration - log 'success!', :color => :green, :style => :bold - else - bail! "could not register authorization key." - end - end - - log :saving, "authorization key for letsencrypt.org" do - write_file!(:acme_key, private_key.to_pem) - write_file!(:acme_info, JSON.sorted_generate({ - id: registration.id, - contact: registration.contact, - key: registration.key, - uri: registration.uri - })) - log :warning, "keep key file private!" - end - end - end - - def assert_no_errors!(msg) - yield - rescue StandardError => exc - bail! :error, msg do - log exc.to_s - end - end - - def do_renew_cert(global, options, args) - require 'leap_cli/acme' - require 'leap_cli/ssh' - require 'socket' - require 'net/http' - - csr = nil - account_key = nil - cert = nil - acme = nil - - # - # sanity check the domain - # - domain = args.first - nodes = nodes_for_domain(domain) - domain_ready_for_acme!(domain) - - # - # load key material - # - assert_files_exist!([:commercial_key, domain], [:commercial_csr, domain], - :msg => 'Please create the CSR first with `leap cert csr %s`' % domain) - assert_no_errors!("Could not load #{path([:commercial_csr, domain])}") do - csr = Acme.load_csr(read_file!([:commercial_csr, domain])) - end - assert_files_exist!(:acme_key, - :msg => "Please run `leap cert register` first. This only needs to be done once.") - assert_no_errors!("Could not load #{path(:acme_key)}") do - account_key = Acme.load_private_key(read_file!(:acme_key)) - end - - # - # check authorization for this domain - # - log :checking, "authorization" - acme = Acme.new(domain: domain, key: account_key) - status, message = acme.authorize do |challenge| - log(:uploading, 'challenge to server %s' % domain) do - SSH.remote_command(nodes) do |ssh, host| - ssh.scripts.upload_acme_challenge(challenge.token, challenge.file_content) - end - end - log :waiting, "for letsencrypt.org to verify challenge" - end - if status == 'valid' - log 'authorized!', color: :green, style: :bold - elsif status == 'error' - bail! :error, message.inspect - elsif status == 'unauthorized' - bail!(:unauthorized, message.inspect, color: :yellow, style: :bold) do - log 'You must first run `leap cert register` to register the account key with letsencrypt.org' - end - else - bail!(:error, "unrecognized status: #{status.inspect}, #{message.inspect}") - end - - log :fetching, "new certificate from letsencrypt.org" - assert_no_errors!("could not renew certificate") do - cert = acme.get_certificate(csr) - end - log 'success', color: :green, style: :bold - write_file!([:commercial_cert, domain], cert.fullchain_to_pem) - log 'You should now run `leap deploy` to deploy the new certificate.' - end - - # - # Returns a hash of nodes that match this domain. It also checks: - # - # * a node configuration has this domain - # * the dns for the domain exists - # - # This method will bail if any checks fail. - # - def nodes_for_domain(domain) - bail! { log 'Argument DOMAIN is required' } if domain.nil? || domain.empty? - nodes = manager.nodes['dns.aliases' => domain] - if nodes.empty? - bail! :error, "There are no nodes configured for domain `%s`" % domain - end - begin - ips = Socket.getaddrinfo(domain, 'http').map {|record| record[2]}.uniq - nodes = nodes['ip_address' => ips] - if nodes.empty? - bail! do - log :error, "The domain `%s` resolves to [%s]" % [domain, ips.join(', ')] - log :error, "But there no nodes configured for this domain with these adddresses." - end - end - rescue SocketError - bail! :error, "Could not resolve the DNS for `#{domain}`. Without a DNS " + - "entry for this domain, authorization will not work." - end - return nodes - end - - # - # runs the following checks on the domain: - # - # * we are able to get /.well-known/acme-challenge/ok - # - # This method will bail if any checks fail. - # - def domain_ready_for_acme!(domain) - begin - uri = URI("https://#{domain}/.well-known/acme-challenge/ok") - options = { - use_ssl: true, - open_timeout: 5, - verify_mode: OpenSSL::SSL::VERIFY_NONE - } - Net::HTTP.start(uri.host, uri.port, options) do |http| - http.request(Net::HTTP::Get.new(uri)) do |response| - if !response.is_a?(Net::HTTPSuccess) - bail!(:error, "Could not GET %s" % uri) do - log "%s %s" % [response.code, response.message] - log "You may need to run `leap deploy`" - end - end - end - end - rescue Errno::ETIMEDOUT, Net::OpenTimeout - bail! :error, "Connection attempt timed out: %s" % uri - rescue Interrupt - bail! - rescue StandardError => exc - bail!(:error, "Could not GET %s" % uri) do - log exc.to_s - end - end - end - -end; end diff --git a/lib/leap_cli/commands/cert.rb b/lib/leap_cli/commands/cert.rb new file mode 100644 index 00000000..1c67ae67 --- /dev/null +++ b/lib/leap_cli/commands/cert.rb @@ -0,0 +1,368 @@ +module LeapCli; module Commands + + desc "Manage X.509 certificates" + command :cert do |cert| + + cert.desc 'Creates two Certificate Authorities (one for validating servers and one for validating clients).' + cert.long_desc 'See see what values are used in the generation of the certificates (like name and key size), run `leap inspect provider` and look for the "ca" property. To see the details of the created certs, run `leap inspect `.' + cert.command :ca do |ca| + ca.action do |global_options,options,args| + assert_config! 'provider.ca.name' + generate_new_certificate_authority(:ca_key, :ca_cert, provider.ca.name) + generate_new_certificate_authority(:client_ca_key, :client_ca_cert, provider.ca.name + ' (client certificates only!)') + end + end + + cert.desc 'Creates or renews a X.509 certificate/key pair for a single node or all nodes, but only if needed.' + cert.long_desc 'This command will a generate new certificate for a node if some value in the node has changed ' + + 'that is included in the certificate (like hostname or IP address), or if the old certificate will be expiring soon. ' + + 'Sometimes, you might want to force the generation of a new certificate, ' + + 'such as in the cases where you have changed a CA parameter for server certificates, like bit size or digest hash. ' + + 'In this case, use --force. If is empty, this command will apply to all nodes.' + cert.arg_name 'FILTER' + cert.command :update do |update| + update.switch 'force', :desc => 'Always generate new certificates', :negatable => false + update.action do |global_options,options,args| + update_certificates(manager.filter!(args), options) + end + end + + cert.desc 'Creates a Diffie-Hellman parameter file, needed for forward secret OpenVPN ciphers.' # (needed for server-side of some TLS connections) + cert.command :dh do |dh| + dh.action do |global_options,options,args| + generate_dh + end + end + + cert.desc "Creates a CSR for use in buying a commercial X.509 certificate." + cert.long_desc "Unless specified, the CSR is created for the provider's primary domain. "+ + "The properties used for this CSR come from `provider.ca.server_certificates`, "+ + "but may be overridden here." + cert.arg_name "DOMAIN" + cert.command :csr do |csr| + csr.flag 'domain', :arg_name => 'DOMAIN', :desc => 'Specify what domain to create the CSR for.' + csr.flag ['organization', 'O'], :arg_name => 'ORGANIZATION', :desc => "Override default O in distinguished name." + csr.flag ['unit', 'OU'], :arg_name => 'UNIT', :desc => "Set OU in distinguished name." + csr.flag 'email', :arg_name => 'EMAIL', :desc => "Set emailAddress in distinguished name." + csr.flag ['locality', 'L'], :arg_name => 'LOCALITY', :desc => "Set L in distinguished name." + csr.flag ['state', 'ST'], :arg_name => 'STATE', :desc => "Set ST in distinguished name." + csr.flag ['country', 'C'], :arg_name => 'COUNTRY', :desc => "Set C in distinguished name." + csr.flag :bits, :arg_name => 'BITS', :desc => "Override default certificate bit length" + csr.flag :digest, :arg_name => 'DIGEST', :desc => "Override default signature digest" + csr.action do |global_options,options,args| + generate_csr(global_options, options, args) + end + end + + cert.desc "Register an authorization key with the CA letsencrypt.org" + cert.long_desc "This only needs to be done once." + cert.command :register do |register| + register.action do |global, options, args| + do_register_key(global, options, args) + end + end + + cert.desc "Renews a certificate using the CA letsencrypt.org" + cert.arg_name "DOMAIN" + cert.command :renew do |renew| + renew.action do |global, options, args| + do_renew_cert(global, options, args) + end + end + + end + + protected + + # + # will generate new certificates for the specified nodes, if needed. + # + def update_certificates(nodes, options={}) + require 'leap_cli/x509' + assert_files_exist! :ca_cert, :ca_key, :msg => 'Run `leap cert ca` to create them' + assert_config! 'provider.ca.server_certificates.bit_size' + assert_config! 'provider.ca.server_certificates.digest' + assert_config! 'provider.ca.server_certificates.life_span' + assert_config! 'common.x509.use' + + nodes.each_node do |node| + node.warn_if_commercial_cert_will_soon_expire + if !node.x509.use + remove_file!([:node_x509_key, node.name]) + remove_file!([:node_x509_cert, node.name]) + elsif options[:force] || node.cert_needs_updating? + node.generate_cert + end + end + end + + # + # yields client key and cert suitable for testing + # + def generate_test_client_cert(prefix=nil) + require 'leap_cli/x509' + cert = CertificateAuthority::Certificate.new + cert.serial_number.number = cert_serial_number(provider.domain) + cert.subject.common_name = [prefix, random_common_name(provider.domain)].join + cert.not_before = X509.yesterday + cert.not_after = X509.yesterday.advance(:years => 1) + cert.key_material.generate_key(1024) # just for testing, remember! + cert.parent = client_ca_root + cert.sign! client_test_signing_profile + yield cert.key_material.private_key.to_pem, cert.to_pem + end + + private + + def generate_new_certificate_authority(key_file, cert_file, common_name) + require 'leap_cli/x509' + assert_files_missing! key_file, cert_file + assert_config! 'provider.ca.name' + assert_config! 'provider.ca.bit_size' + assert_config! 'provider.ca.life_span' + + root = X509.new_ca(provider.ca, common_name) + + write_file!(key_file, root.key_material.private_key.to_pem) + write_file!(cert_file, root.to_pem) + end + + def generate_dh + require 'leap_cli/x509' + long_running do + if cmd_exists?('certtool') + log 0, 'Generating DH parameters (takes a long time)...' + output = assert_run!('certtool --generate-dh-params --sec-param high') + output.sub!(/.*(-----BEGIN DH PARAMETERS-----.*-----END DH PARAMETERS-----).*/m, '\1') + output << "\n" + write_file!(:dh_params, output) + else + log 0, 'Generating DH parameters (takes a REALLY long time)...' + output = OpenSSL::PKey::DH.generate(3248).to_pem + write_file!(:dh_params, output) + end + end + end + + # + # hints: + # + # inspect CSR: + # openssl req -noout -text -in files/cert/x.csr + # + # generate CSR with openssl to see how it compares: + # openssl req -sha256 -nodes -newkey rsa:2048 -keyout example.key -out example.csr + # + # validate a CSR: + # http://certlogik.com/decoder/ + # + # nice details about CSRs: + # http://www.redkestrel.co.uk/Articles/CSR.html + # + def generate_csr(global_options, options, args) + require 'leap_cli/x509' + assert_config! 'provider.domain' + assert_config! 'provider.name' + assert_config! 'provider.default_language' + assert_config! 'provider.ca.server_certificates.bit_size' + assert_config! 'provider.ca.server_certificates.digest' + + server_certificates = provider.ca.server_certificates + options[:domain] ||= args.first || provider.domain + options[:organization] ||= provider.name[provider.default_language] + options[:country] ||= server_certificates['country'] + options[:state] ||= server_certificates['state'] + options[:locality] ||= server_certificates['locality'] + options[:bits] ||= server_certificates.bit_size + options[:digest] ||= server_certificates.digest + + unless global_options[:force] + assert_files_missing! [:commercial_key, options[:domain]], [:commercial_csr, options[:domain]], + :msg => 'If you really want to create a new key and CSR, remove these files first or run with --force.' + end + + X509.create_csr_and_cert(options) + end + + # + # letsencrypt.org + # + + def do_register_key(global, options, args) + require 'leap_cli/acme' + assert_config! 'provider.contacts.default' + contact = manager.provider.contacts.default.first + + if file_exists?(:acme_key) && !global[:force] + bail! do + log "the authorization key for letsencrypt.org already exists" + log "run with --force if you really want to register a new key." + end + else + private_key = Acme.new_private_key + registration = nil + + log(:registering, "letsencrypt.org authorization key using contact `%s`" % contact) do + acme = Acme.new(key: private_key) + registration = acme.register(contact) + if registration + log 'success!', :color => :green, :style => :bold + else + bail! "could not register authorization key." + end + end + + log :saving, "authorization key for letsencrypt.org" do + write_file!(:acme_key, private_key.to_pem) + write_file!(:acme_info, JSON.sorted_generate({ + id: registration.id, + contact: registration.contact, + key: registration.key, + uri: registration.uri + })) + log :warning, "keep key file private!" + end + end + end + + def assert_no_errors!(msg) + yield + rescue StandardError => exc + bail! :error, msg do + log exc.to_s + end + end + + def do_renew_cert(global, options, args) + require 'leap_cli/acme' + require 'leap_cli/ssh' + require 'socket' + require 'net/http' + + csr = nil + account_key = nil + cert = nil + acme = nil + + # + # sanity check the domain + # + domain = args.first + nodes = nodes_for_domain(domain) + domain_ready_for_acme!(domain) + + # + # load key material + # + assert_files_exist!([:commercial_key, domain], [:commercial_csr, domain], + :msg => 'Please create the CSR first with `leap cert csr %s`' % domain) + assert_no_errors!("Could not load #{path([:commercial_csr, domain])}") do + csr = Acme.load_csr(read_file!([:commercial_csr, domain])) + end + assert_files_exist!(:acme_key, + :msg => "Please run `leap cert register` first. This only needs to be done once.") + assert_no_errors!("Could not load #{path(:acme_key)}") do + account_key = Acme.load_private_key(read_file!(:acme_key)) + end + + # + # check authorization for this domain + # + log :checking, "authorization" + acme = Acme.new(domain: domain, key: account_key) + status, message = acme.authorize do |challenge| + log(:uploading, 'challenge to server %s' % domain) do + SSH.remote_command(nodes) do |ssh, host| + ssh.scripts.upload_acme_challenge(challenge.token, challenge.file_content) + end + end + log :waiting, "for letsencrypt.org to verify challenge" + end + if status == 'valid' + log 'authorized!', color: :green, style: :bold + elsif status == 'error' + bail! :error, message.inspect + elsif status == 'unauthorized' + bail!(:unauthorized, message.inspect, color: :yellow, style: :bold) do + log 'You must first run `leap cert register` to register the account key with letsencrypt.org' + end + else + bail!(:error, "unrecognized status: #{status.inspect}, #{message.inspect}") + end + + log :fetching, "new certificate from letsencrypt.org" + assert_no_errors!("could not renew certificate") do + cert = acme.get_certificate(csr) + end + log 'success', color: :green, style: :bold + write_file!([:commercial_cert, domain], cert.fullchain_to_pem) + log 'You should now run `leap deploy` to deploy the new certificate.' + end + + # + # Returns a hash of nodes that match this domain. It also checks: + # + # * a node configuration has this domain + # * the dns for the domain exists + # + # This method will bail if any checks fail. + # + def nodes_for_domain(domain) + bail! { log 'Argument DOMAIN is required' } if domain.nil? || domain.empty? + nodes = manager.nodes['dns.aliases' => domain] + if nodes.empty? + bail! :error, "There are no nodes configured for domain `%s`" % domain + end + begin + ips = Socket.getaddrinfo(domain, 'http').map {|record| record[2]}.uniq + nodes = nodes['ip_address' => ips] + if nodes.empty? + bail! do + log :error, "The domain `%s` resolves to [%s]" % [domain, ips.join(', ')] + log :error, "But there no nodes configured for this domain with these adddresses." + end + end + rescue SocketError + bail! :error, "Could not resolve the DNS for `#{domain}`. Without a DNS " + + "entry for this domain, authorization will not work." + end + return nodes + end + + # + # runs the following checks on the domain: + # + # * we are able to get /.well-known/acme-challenge/ok + # + # This method will bail if any checks fail. + # + def domain_ready_for_acme!(domain) + begin + uri = URI("https://#{domain}/.well-known/acme-challenge/ok") + options = { + use_ssl: true, + open_timeout: 5, + verify_mode: OpenSSL::SSL::VERIFY_NONE + } + Net::HTTP.start(uri.host, uri.port, options) do |http| + http.request(Net::HTTP::Get.new(uri)) do |response| + if !response.is_a?(Net::HTTPSuccess) + bail!(:error, "Could not GET %s" % uri) do + log "%s %s" % [response.code, response.message] + log "You may need to run `leap deploy`" + end + end + end + end + rescue Errno::ETIMEDOUT, Net::OpenTimeout + bail! :error, "Connection attempt timed out: %s" % uri + rescue Interrupt + bail! + rescue StandardError => exc + bail!(:error, "Could not GET %s" % uri) do + log exc.to_s + end + end + end + +end; end -- cgit v1.2.3 From 6da8d11ec2f000353e952ff95abe27dd8c8381c8 Mon Sep 17 00:00:00 2001 From: elijah Date: Wed, 21 Dec 2016 13:32:29 -0800 Subject: added command `leap ping` --- lib/leap_cli/commands/ping.rb | 58 +++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 58 insertions(+) create mode 100644 lib/leap_cli/commands/ping.rb diff --git a/lib/leap_cli/commands/ping.rb b/lib/leap_cli/commands/ping.rb new file mode 100644 index 00000000..4283d9b3 --- /dev/null +++ b/lib/leap_cli/commands/ping.rb @@ -0,0 +1,58 @@ +module LeapCli; module Commands + + desc "Ping nodes to see if they are alive." + long_desc "Attempts to ping each node in the FILTER set." + arg_name "FILTER" + command :ping do |c| + c.flag 'timeout', :arg_name => "TIMEOUT", + :default_value => 2, :desc => 'Wait at most TIMEOUT seconds.' + c.flag 'count', :arg_name => "COUNT", + :default_value => 2, :desc => 'Ping COUNT times.' + c.action do |global, options, args| + do_ping(global, options, args) + end + end + + private + + def do_ping(global, options, args) + assert_bin!('ping') + + timeout = [options[:timeout].to_i, 1].max + count = [options[:count].to_i, 1].max + nodes = nil + + if args && args.any? + node = manager.disabled_node(args.first) + if node + nodes = Config::ObjectList.new + nodes.add(node.name, node) + end + end + + nodes ||= manager.filter! args + + threads = [] + nodes.each_node do |node| + threads << Thread.new do + cmd = "ping -i 0.2 -n -q -W #{timeout} -c #{count} #{node.ip_address} 2>&1" + log(2, cmd) + output = `#{cmd}` + if $?.success? + last = output.split("\n").last + times = last.split('=').last.strip + min, avg, max, mdev = times.split('/') + log("ping #{min} ms", host: node.name, color: :green) + else + log(:failed, "to ping #{node.ip_address}", host: node.name) + end + end + end + threads.map(&:join) + + log("done") + end + +end; end + + -- cgit v1.2.3 From 627b8ef2e14dcbce2f057f163bee67cea8aa4443 Mon Sep 17 00:00:00 2001 From: elijah Date: Thu, 22 Dec 2016 09:46:33 -0800 Subject: COMPATIBILITY CHANGE: set platform version to 0.10 & require client 0.9.4 or later --- platform.rb | 2 +- provider_base/provider.json | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/platform.rb b/platform.rb index 2ff0a27f..935fa385 100644 --- a/platform.rb +++ b/platform.rb @@ -4,7 +4,7 @@ # Leap::Platform.define do - self.version = "0.9" + self.version = "0.10" self.compatible_cli = "1.9".."1.99" # diff --git a/provider_base/provider.json b/provider_base/provider.json index 81b2ea98..521c682f 100644 --- a/provider_base/provider.json +++ b/provider_base/provider.json @@ -58,7 +58,7 @@ } }, "client_version": { - "min": "0.7", + "min": "0.9.4", "max": null } } -- cgit v1.2.3 From 61c9d47ebcc7ee43bfcb1398d0166fb3544adc2b Mon Sep 17 00:00:00 2001 From: elijah Date: Thu, 22 Dec 2016 13:21:19 -0800 Subject: bugfix: don't block commercial certs for mx servers --- provider_base/services/mx.json | 1 - 1 file changed, 1 deletion(-) diff --git a/provider_base/services/mx.json b/provider_base/services/mx.json index 2db773b5..334e40de 100644 --- a/provider_base/services/mx.json +++ b/provider_base/services/mx.json @@ -37,7 +37,6 @@ }, "x509": { "use": true, - "use_commercial": false, "ca_cert": "= file :ca_cert, :missing => 'provider CA. Run `leap cert ca`'", "client_ca_cert": "= file :client_ca_cert, :missing => 'Certificate Authority. Run `leap cert ca`'", "client_ca_key": "= file :client_ca_key, :missing => 'Certificate Authority. Run `leap cert ca`'" -- cgit v1.2.3 From 6814e226d90944b4adde33ad4946ff7a400b413e Mon Sep 17 00:00:00 2001 From: varac Date: Fri, 23 Dec 2016 13:03:33 +0100 Subject: Change regex for soledad process check --- tests/server-tests/white-box/soledad.rb | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tests/server-tests/white-box/soledad.rb b/tests/server-tests/white-box/soledad.rb index d41bee58..b89145bc 100644 --- a/tests/server-tests/white-box/soledad.rb +++ b/tests/server-tests/white-box/soledad.rb @@ -10,7 +10,7 @@ class Soledad < LeapTest end def test_00_Is_Soledad_running? - assert_running '.*/usr/bin/twistd.*--wsgi=leap.soledad.server.application' + assert_running '/usr/bin/python /usr/bin/twistd --uid=soledad --gid=soledad --pidfile=/var/run/soledad.pid --syslog --prefix=soledad-server web --class=leap.soledad.server.resource.SoledadResource.*' pass end -- cgit v1.2.3 From 44cae3cf731d29fd1e882cf35526fb0e098914d2 Mon Sep 17 00:00:00 2001 From: varac Date: Fri, 23 Dec 2016 14:45:10 +0100 Subject: Use experimental-0.9 instead of experimental-platform experimental-platform is still WIP, see https://leap.se/code/issues/8437#note-8 for more details --- provider_base/common.json | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/provider_base/common.json b/provider_base/common.json index 666fe923..226ede19 100644 --- a/provider_base/common.json +++ b/provider_base/common.json @@ -75,7 +75,7 @@ }, "platform": { "apt": { - "basic": "http://deb.leap.se/experimental-platform" + "basic": "http://deb.leap.se/experimental-0.9" } }, "soledad": { -- cgit v1.2.3 From 889e206ac2dfab93adbc3ad70ab0ba4f883de2e2 Mon Sep 17 00:00:00 2001 From: drebs Date: Thu, 22 Dec 2016 20:23:21 -0200 Subject: bugfix: remove deprecated parameter from soledad test script --- tests/server-tests/helpers/client_side_db.py | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/tests/server-tests/helpers/client_side_db.py b/tests/server-tests/helpers/client_side_db.py index 2f8c220f..5842c007 100644 --- a/tests/server-tests/helpers/client_side_db.py +++ b/tests/server-tests/helpers/client_side_db.py @@ -55,8 +55,7 @@ def get_soledad_instance(uuid, passphrase, basedir, server_url, cert_file, local_db_path=local_db_path, server_url=server_url, cert_file=cert_file, - auth_token=token, - defer_encryption=True) + auth_token=token) def _get_api_info(provider): -- cgit v1.2.3 From d0fa7a7cc41c7111eab3cb2ebca2218f0b3a2812 Mon Sep 17 00:00:00 2001 From: Varac Date: Fri, 23 Dec 2016 14:22:25 +0000 Subject: Update README.md with master build badge --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index 4a045cdd..6e5cb68d 100644 --- a/README.md +++ b/README.md @@ -1,7 +1,7 @@ Leap Platform ============================= -[![Build Status](https://0xacab.org/leap/platform/badges/develop/build.svg)](https://0xacab.org/leap/platform/commits/develop) +[![Build Status](https://0xacab.org/leap/platform/badges/master/build.svg)](https://0xacab.org/leap/platform/commits/master) The LEAP Platform is set of complementary packages and server recipes to automate the maintenance of LEAP services in a hardened Debian environment. Its -- cgit v1.2.3 From c1f23241deb1f94eb35e6ef21e623d804fa2770e Mon Sep 17 00:00:00 2001 From: varac Date: Fri, 30 Dec 2016 18:44:27 +0100 Subject: Couchdb service should not require on soledad - Resolves: #8693 --- puppet/modules/site_couchdb/manifests/add_users.pp | 16 ++++++++++------ 1 file changed, 10 insertions(+), 6 deletions(-) diff --git a/puppet/modules/site_couchdb/manifests/add_users.pp b/puppet/modules/site_couchdb/manifests/add_users.pp index f12c5a5e..5c32c1e3 100644 --- a/puppet/modules/site_couchdb/manifests/add_users.pp +++ b/puppet/modules/site_couchdb/manifests/add_users.pp @@ -1,6 +1,8 @@ # add couchdb users for all services class site_couchdb::add_users { + $services = hiera('services', []) + Class['site_couchdb::create_dbs'] -> Class['site_couchdb::add_users'] @@ -29,12 +31,14 @@ class site_couchdb::add_users { ## soledad couchdb user ## r/w: user-, shared ## read: tokens - couchdb::add_user { $site_couchdb::couchdb_soledad_user: - roles => '["tokens"]', - pw => $site_couchdb::couchdb_soledad_pw, - salt => $site_couchdb::couchdb_soledad_salt, - require => Couchdb::Query::Setup['localhost'], - notify => Service['soledad-server']; + if member($services, 'soledad') { + couchdb::add_user { $site_couchdb::couchdb_soledad_user: + roles => '["tokens"]', + pw => $site_couchdb::couchdb_soledad_pw, + salt => $site_couchdb::couchdb_soledad_salt, + require => Couchdb::Query::Setup['localhost'], + notify => Service['soledad-server']; + } } ## webapp couchdb user -- cgit v1.2.3 From 46badc1744bd8ebbc2bd49f0dad0841d5961e5c6 Mon Sep 17 00:00:00 2001 From: varac Date: Tue, 3 Jan 2017 19:17:05 +0100 Subject: Revert "Use experimental-0.9 instead of experimental-platform" This reverts commit 44cae3cf731d29fd1e882cf35526fb0e098914d2. --- provider_base/common.json | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/provider_base/common.json b/provider_base/common.json index 226ede19..666fe923 100644 --- a/provider_base/common.json +++ b/provider_base/common.json @@ -75,7 +75,7 @@ }, "platform": { "apt": { - "basic": "http://deb.leap.se/experimental-0.9" + "basic": "http://deb.leap.se/experimental-platform" } }, "soledad": { -- cgit v1.2.3 From 9b8314f3d2707a80e6238bb173280de291ecd2f4 Mon Sep 17 00:00:00 2001 From: varac Date: Tue, 10 Jan 2017 19:43:30 +0100 Subject: New unbound runs with debug flag by default --- tests/server-tests/white-box/mx.rb | 2 +- tests/server-tests/white-box/openvpn.rb | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/tests/server-tests/white-box/mx.rb b/tests/server-tests/white-box/mx.rb index 0eeaacd0..ecc8686c 100644 --- a/tests/server-tests/white-box/mx.rb +++ b/tests/server-tests/white-box/mx.rb @@ -57,7 +57,7 @@ class Mx < LeapTest assert_running '^/usr/sbin/postfwd' assert_running 'postfwd2::cache$' assert_running 'postfwd2::policy$' - assert_running '^/usr/sbin/unbound$' + assert_running '^/usr/sbin/unbound' assert_running '^/usr/bin/freshclam' assert_running '^/usr/sbin/opendkim' if Dir.glob("/var/lib/clamav/main.{c[vl]d,inc}").size > 0 and Dir.glob("/var/lib/clamav/daily.{c[vl]d,inc}").size > 0 diff --git a/tests/server-tests/white-box/openvpn.rb b/tests/server-tests/white-box/openvpn.rb index 170d4503..d5cc2265 100644 --- a/tests/server-tests/white-box/openvpn.rb +++ b/tests/server-tests/white-box/openvpn.rb @@ -9,7 +9,7 @@ class OpenVPN < LeapTest def test_01_Are_daemons_running? assert_running '^/usr/sbin/openvpn .* /etc/openvpn/tcp_config.conf$' assert_running '^/usr/sbin/openvpn .* /etc/openvpn/udp_config.conf$' - assert_running '^/usr/sbin/unbound$' + assert_running '^/usr/sbin/unbound' pass end -- cgit v1.2.3 From dd189d2de941ec081261ced814a9c822e5ef02a1 Mon Sep 17 00:00:00 2001 From: elijah Date: Tue, 10 Jan 2017 10:45:36 -0800 Subject: bugfix: `leap user ls` now warns if the ssh keytype is unsupported --- lib/leap_cli/commands/user.rb | 24 +++++++++++++++--------- 1 file changed, 15 insertions(+), 9 deletions(-) diff --git a/lib/leap_cli/commands/user.rb b/lib/leap_cli/commands/user.rb index a10d5163..7fd5f52d 100644 --- a/lib/leap_cli/commands/user.rb +++ b/lib/leap_cli/commands/user.rb @@ -132,15 +132,21 @@ module LeapCli log username, :color => :cyan do log Path.relative_path(keyfile) key = SSH::Key.load(keyfile) - log 'SSH MD5 fingerprint: ' + key.fingerprint(:digest => :md5, :type => :ssh, :encoding => :hex) - log 'SSH SHA256 fingerprint: ' + key.fingerprint(:digest => :sha256, :type => :ssh, :encoding => :base64) - log 'DER MD5 fingerprint: ' + key.fingerprint(:digest => :md5, :type => :der, :encoding => :hex) - if ssh_keys[key.fingerprint] - log 'Matches local key: ' + ssh_keys[key.fingerprint].filename, color: :green - if ssh_agent_keys[key.fingerprint] - log 'Matches ssh-agent key: ' + ssh_agent_keys[key.fingerprint].summary(encoding: :base64), color: :green - else - log :error, 'No matching key in the ssh-agent' + if key.nil? + log :warning, "could not read ssh key #{keyfile}" do + log "currently, only these ssh key types are supported: " + SSH::Key::SUPPORTED_TYPES.join(", ") + end + else + log 'SSH MD5 fingerprint: ' + key.fingerprint(:digest => :md5, :type => :ssh, :encoding => :hex) + log 'SSH SHA256 fingerprint: ' + key.fingerprint(:digest => :sha256, :type => :ssh, :encoding => :base64) + log 'DER MD5 fingerprint: ' + key.fingerprint(:digest => :md5, :type => :der, :encoding => :hex) + if ssh_keys[key.fingerprint] + log 'Matches local key: ' + ssh_keys[key.fingerprint].filename, color: :green + if ssh_agent_keys[key.fingerprint] + log 'Matches ssh-agent key: ' + ssh_agent_keys[key.fingerprint].summary(encoding: :base64), color: :green + else + log :error, 'No matching key in the ssh-agent' + end end end end -- cgit v1.2.3 From 77d11c7ddeaeb123bf871bd2bfce0e5ace0c158e Mon Sep 17 00:00:00 2001 From: varac Date: Mon, 16 Jan 2017 11:48:53 +0100 Subject: Revert "Add systemd::enable define" This commit was moved to the systemd puppet repo. This reverts commit f5db49cf6b3ca0a5830b849c0aac074e371b95d9. --- puppet/modules/systemd/manifests/enable.pp | 8 -------- 1 file changed, 8 deletions(-) delete mode 100644 puppet/modules/systemd/manifests/enable.pp diff --git a/puppet/modules/systemd/manifests/enable.pp b/puppet/modules/systemd/manifests/enable.pp deleted file mode 100644 index e1bee18a..00000000 --- a/puppet/modules/systemd/manifests/enable.pp +++ /dev/null @@ -1,8 +0,0 @@ -# enables a systemd resource -define systemd::enable () { - - exec { "enable_systemd_${name}": - refreshonly => true, - command => "/bin/systemctl enable ${name}" - } -} -- cgit v1.2.3 From e9a3d71fd1c9fbdb1b468c633a6cbe310e3d6880 Mon Sep 17 00:00:00 2001 From: varac Date: Mon, 16 Jan 2017 14:39:14 +0100 Subject: git subrepo clone --force https://leap.se/git/puppet_systemd puppet/modules/systemd subrepo: subdir: "puppet/modules/systemd" merged: "f3c4059" upstream: origin: "https://leap.se/git/puppet_systemd" branch: "master" commit: "f3c4059" git-subrepo: version: "0.3.0" origin: "https://github.com/ingydotnet/git-subrepo.git" commit: "841aa43" --- puppet/modules/systemd/.fixtures.yml | 4 + puppet/modules/systemd/.gitrepo | 4 +- puppet/modules/systemd/.puppet-lint.rc | 2 +- puppet/modules/systemd/.travis.yml | 22 +-- puppet/modules/systemd/CHANGELOG.md | 20 +- puppet/modules/systemd/Gemfile | 18 +- puppet/modules/systemd/HISTORY.md | 15 +- puppet/modules/systemd/LICENSE | 201 +++++++++++++++++++++ puppet/modules/systemd/README.md | 51 +++++- puppet/modules/systemd/Rakefile | 10 +- puppet/modules/systemd/lib/facter/systemd.rb | 35 ++++ puppet/modules/systemd/manifests/enable.pp | 8 + puppet/modules/systemd/manifests/init.pp | 8 +- puppet/modules/systemd/manifests/service_limits.pp | 50 +++++ puppet/modules/systemd/manifests/tmpfile.pp | 20 ++ puppet/modules/systemd/manifests/unit_file.pp | 22 +++ puppet/modules/systemd/metadata.json | 2 +- .../systemd/spec/acceptance/nodesets/centos-5.yml | 16 ++ .../systemd/spec/acceptance/nodesets/centos-6.yml | 17 ++ .../systemd/spec/acceptance/nodesets/centos-7.yml | 15 ++ .../systemd/spec/acceptance/nodesets/debian-6.yml | 15 ++ .../systemd/spec/acceptance/nodesets/debian-7.yml | 15 ++ .../systemd/spec/acceptance/nodesets/debian-8.yml | 16 ++ .../spec/acceptance/nodesets/ubuntu-12.04.yml | 16 ++ .../spec/acceptance/nodesets/ubuntu-14.04.yml | 18 ++ .../spec/acceptance/nodesets/ubuntu-14.10.yml | 18 ++ .../spec/acceptance/nodesets/ubuntu-15.04.yml | 16 ++ .../spec/acceptance/nodesets/ubuntu-15.10.yml | 16 ++ .../spec/acceptance/nodesets/ubuntu-16.04.yml | 16 ++ .../modules/systemd/spec/defines/tmpfile_spec.rb | 48 +++++ .../modules/systemd/spec/defines/unit_file_spec.rb | 50 +++++ .../systemd/spec/unit/facter/systemd_spec.rb | 41 +++++ .../spec/unit/facter/systemd_version_spec.rb | 31 ++++ puppet/modules/systemd/templates/limits.erb | 26 +++ 34 files changed, 839 insertions(+), 43 deletions(-) create mode 100644 puppet/modules/systemd/.fixtures.yml create mode 100644 puppet/modules/systemd/LICENSE create mode 100644 puppet/modules/systemd/lib/facter/systemd.rb create mode 100644 puppet/modules/systemd/manifests/enable.pp create mode 100644 puppet/modules/systemd/manifests/service_limits.pp create mode 100644 puppet/modules/systemd/manifests/tmpfile.pp create mode 100644 puppet/modules/systemd/manifests/unit_file.pp create mode 100644 puppet/modules/systemd/spec/acceptance/nodesets/centos-5.yml create mode 100644 puppet/modules/systemd/spec/acceptance/nodesets/centos-6.yml create mode 100644 puppet/modules/systemd/spec/acceptance/nodesets/centos-7.yml create mode 100644 puppet/modules/systemd/spec/acceptance/nodesets/debian-6.yml create mode 100644 puppet/modules/systemd/spec/acceptance/nodesets/debian-7.yml create mode 100644 puppet/modules/systemd/spec/acceptance/nodesets/debian-8.yml create mode 100644 puppet/modules/systemd/spec/acceptance/nodesets/ubuntu-12.04.yml create mode 100644 puppet/modules/systemd/spec/acceptance/nodesets/ubuntu-14.04.yml create mode 100644 puppet/modules/systemd/spec/acceptance/nodesets/ubuntu-14.10.yml create mode 100644 puppet/modules/systemd/spec/acceptance/nodesets/ubuntu-15.04.yml create mode 100644 puppet/modules/systemd/spec/acceptance/nodesets/ubuntu-15.10.yml create mode 100644 puppet/modules/systemd/spec/acceptance/nodesets/ubuntu-16.04.yml create mode 100644 puppet/modules/systemd/spec/defines/tmpfile_spec.rb create mode 100644 puppet/modules/systemd/spec/defines/unit_file_spec.rb create mode 100644 puppet/modules/systemd/spec/unit/facter/systemd_spec.rb create mode 100644 puppet/modules/systemd/spec/unit/facter/systemd_version_spec.rb create mode 100644 puppet/modules/systemd/templates/limits.erb diff --git a/puppet/modules/systemd/.fixtures.yml b/puppet/modules/systemd/.fixtures.yml new file mode 100644 index 00000000..1d455a31 --- /dev/null +++ b/puppet/modules/systemd/.fixtures.yml @@ -0,0 +1,4 @@ +--- +fixtures: + symlinks: + systemd: "#{source_dir}" \ No newline at end of file diff --git a/puppet/modules/systemd/.gitrepo b/puppet/modules/systemd/.gitrepo index 1548a815..ea68e478 100644 --- a/puppet/modules/systemd/.gitrepo +++ b/puppet/modules/systemd/.gitrepo @@ -6,6 +6,6 @@ [subrepo] remote = https://leap.se/git/puppet_systemd branch = master - commit = 6d47fd4999fe03eba6fb11c4490dcbb90d937900 - parent = 56a771a3008d10720dd05fd815aeafbacdd1e08e + commit = f3c4059603a6ac19f132b0dc47b95e49d9ddc4ba + parent = 77d11c7ddeaeb123bf871bd2bfce0e5ace0c158e cmdver = 0.3.0 diff --git a/puppet/modules/systemd/.puppet-lint.rc b/puppet/modules/systemd/.puppet-lint.rc index d8f5c59e..e09d52f4 100644 --- a/puppet/modules/systemd/.puppet-lint.rc +++ b/puppet/modules/systemd/.puppet-lint.rc @@ -1,5 +1,5 @@ --fail-on-warnings --relative ---no-80chars +--no-140chars --no-documentation --no-class_inherits_from_params_class-check diff --git a/puppet/modules/systemd/.travis.yml b/puppet/modules/systemd/.travis.yml index 467045c5..1d1bedfc 100644 --- a/puppet/modules/systemd/.travis.yml +++ b/puppet/modules/systemd/.travis.yml @@ -1,22 +1,22 @@ --- language: ruby sudo: false +addons: + apt: + packages: + - libaugeas-dev + sources: + - augeas cache: bundler bundler_args: --without system_tests -script: ["bundle exec rake validate", "bundle exec rake lint", "bundle exec rake spec SPEC_OPTS='--format documentation'", "bundle exec rake metadata"] +script: ["bundle exec rake validate", "bundle exec rake lint", "bundle exec rake spec SPEC_OPTS='--format documentation'"] matrix: fast_finish: true include: - - rvm: 1.8.7 - env: PUPPET_GEM_VERSION="~> 3.0" FACTER_GEM_VERSION="~> 1.7.0" - - rvm: 1.9.3 - env: PUPPET_GEM_VERSION="~> 3.0" - - rvm: 2.0.0 - env: PUPPET_GEM_VERSION="~> 3.0" - - rvm: 2.0.0 - env: PUPPET_GEM_VERSION="~> 3.0" FUTURE_PARSER="yes" - - rvm: 2.1.6 + - rvm: 2.1.9 env: PUPPET_GEM_VERSION="~> 4.0" + - rvm: 2.3.1 + env: PUPPET_GEM_VERSION="~> 4" notifications: email: false deploy: @@ -29,4 +29,4 @@ deploy: # all_branches is required to use tags all_branches: true # Only publish if our main Ruby target builds - rvm: 1.9.3 + rvm: 2.1.9 diff --git a/puppet/modules/systemd/CHANGELOG.md b/puppet/modules/systemd/CHANGELOG.md index 11e84399..79b9e646 100644 --- a/puppet/modules/systemd/CHANGELOG.md +++ b/puppet/modules/systemd/CHANGELOG.md @@ -1,5 +1,22 @@ # Change Log +## [0.4.0](https://forge.puppetlabs.com/camptocamp/systemd/0.4.0) (2016-08-18) +[Full Changelog](https://github.com/camptocamp/puppet-systemd/compare/0.3.0...0.4.0) + +- Deprecate Ruby 1.8 tests +- Only use awk instead of grep and awk [\#9](https://github.com/camptocamp/puppet-systemd/pull/9) ([igalic](https://github.com/igalic)) +- Add LICENSE (fix #11) +- Add target param for the unit file [\#10](https://github.com/camptocamp/puppet-systemd/pull/10) ([tampakrap](https://github.com/tampakrap)) + +## [0.3.0](https://forge.puppetlabs.com/camptocamp/systemd/0.3.0) (2016-05-16) +[Full Changelog](https://github.com/camptocamp/puppet-systemd/compare/0.2.2...0.3.0) + +**Implemented enhancements:** + +- Shortcut for creating unit files / tmpfiles [\#4](https://github.com/camptocamp/puppet-systemd/pull/4) ([felixb](https://github.com/felixb)) +- Add systemd facts [\#6](https://github.com/camptocamp/puppet-systemd/pull/6) ([roidelapluie](https://github.com/roidelapluie)) + + ## [0.2.2](https://forge.puppetlabs.com/camptocamp/systemd/0.2.2) (2015-08-25) [Full Changelog](https://github.com/camptocamp/puppet-systemd/compare/0.2.1...0.2.2) @@ -60,6 +77,3 @@ \* *This Change Log was automatically generated by [github_changelog_generator](https://github.com/skywinder/Github-Changelog-Generator)* - - -\* *This Change Log was automatically generated by [github_changelog_generator](https://github.com/skywinder/Github-Changelog-Generator)* \ No newline at end of file diff --git a/puppet/modules/systemd/Gemfile b/puppet/modules/systemd/Gemfile index 0cb59337..377d0c16 100644 --- a/puppet/modules/systemd/Gemfile +++ b/puppet/modules/systemd/Gemfile @@ -2,7 +2,7 @@ source ENV['GEM_SOURCE'] || "https://rubygems.org" group :development, :unit_tests do gem 'rake', :require => false - gem 'rspec', '< 3.2', :require => false if RUBY_VERSION =~ /^1.8/ + gem 'rspec', :require => false gem 'rspec-puppet', :require => false gem 'puppetlabs_spec_helper', :require => false gem 'metadata-json-lint', :require => false @@ -10,26 +10,26 @@ group :development, :unit_tests do gem 'puppet-lint-unquoted_string-check', :require => false gem 'puppet-lint-empty_string-check', :require => false gem 'puppet-lint-spaceship_operator_without_tag-check', :require => false - gem 'puppet-lint-variable_contains_upcase', :require => false gem 'puppet-lint-absolute_classname-check', :require => false gem 'puppet-lint-undef_in_function-check', :require => false gem 'puppet-lint-leading_zero-check', :require => false gem 'puppet-lint-trailing_comma-check', :require => false gem 'puppet-lint-file_ensure-check', :require => false gem 'puppet-lint-version_comparison-check', :require => false - gem 'puppet-lint-fileserver-check', :require => false gem 'puppet-lint-file_source_rights-check', :require => false gem 'puppet-lint-alias-check', :require => false gem 'rspec-puppet-facts', :require => false - gem 'github_changelog_generator', :require => false, :git => 'https://github.com/raphink/github-changelog-generator.git', :branch => 'dev/all_patches' if RUBY_VERSION !~ /^1.8/ - gem 'puppet-blacksmith', :require => false if RUBY_VERSION !~ /^1.8/ + gem 'ruby-augeas', :require => false + gem 'puppet-blacksmith', :require => false if RUBY_VERSION !~ /^1\./ + gem 'json_pure', '< 2.0.2', :require => false end group :system_tests do - gem 'beaker', :require => false - gem 'beaker-rspec', :require => false - gem 'beaker_spec_helper', :require => false - gem 'serverspec', :require => false + gem 'beaker', :require => false + gem 'beaker-rspec', '> 5', :require => false + gem 'beaker_spec_helper', :require => false + gem 'serverspec', :require => false + gem 'specinfra', :require => false end if facterversion = ENV['FACTER_GEM_VERSION'] diff --git a/puppet/modules/systemd/HISTORY.md b/puppet/modules/systemd/HISTORY.md index c7bf2b4e..aee8ad5e 100644 --- a/puppet/modules/systemd/HISTORY.md +++ b/puppet/modules/systemd/HISTORY.md @@ -1,3 +1,14 @@ +# Change Log + +## [0.3.0](https://forge.puppetlabs.com/camptocamp/systemd/0.3.0) (2016-05-16) +[Full Changelog](https://github.com/camptocamp/puppet-systemd/compare/0.2.2...0.3.0) + +**Implemented enhancements:** + +- Shortcut for creating unit files / tmpfiles [\#4](https://github.com/camptocamp/puppet-systemd/pull/4) ([felixb](https://github.com/felixb)) +- Add systemd facts [\#6](https://github.com/camptocamp/puppet-systemd/pull/6) ([roidelapluie](https://github.com/roidelapluie)) + + ## [0.2.2](https://forge.puppetlabs.com/camptocamp/systemd/0.2.2) (2015-08-25) [Full Changelog](https://github.com/camptocamp/puppet-systemd/compare/0.2.1...0.2.2) @@ -5,6 +16,7 @@ - Add 'systemd-tmpfiles-create' [\#1](https://github.com/camptocamp/puppet-systemd/pull/1) ([roidelapluie](https://github.com/roidelapluie)) + ## [0.2.1](https://forge.puppetlabs.com/camptocamp/systemd/0.2.1) (2015-08-21) [Full Changelog](https://github.com/camptocamp/puppet-systemd/compare/0.2.0...0.2.1) @@ -56,7 +68,4 @@ - Confine rspec pinning to ruby 1.8 -\* *This Change Log was automatically generated by [github_changelog_generator](https://github.com/skywinder/Github-Changelog-Generator)* - - \* *This Change Log was automatically generated by [github_changelog_generator](https://github.com/skywinder/Github-Changelog-Generator)* diff --git a/puppet/modules/systemd/LICENSE b/puppet/modules/systemd/LICENSE new file mode 100644 index 00000000..8d968b6c --- /dev/null +++ b/puppet/modules/systemd/LICENSE @@ -0,0 +1,201 @@ + Apache License + Version 2.0, January 2004 + http://www.apache.org/licenses/ + + TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION + + 1. Definitions. + + "License" shall mean the terms and conditions for use, reproduction, + and distribution as defined by Sections 1 through 9 of this document. + + "Licensor" shall mean the copyright owner or entity authorized by + the copyright owner that is granting the License. + + "Legal Entity" shall mean the union of the acting entity and all + other entities that control, are controlled by, or are under common + control with that entity. For the purposes of this definition, + "control" means (i) the power, direct or indirect, to cause the + direction or management of such entity, whether by contract or + otherwise, or (ii) ownership of fifty percent (50%) or more of the + outstanding shares, or (iii) beneficial ownership of such entity. + + "You" (or "Your") shall mean an individual or Legal Entity + exercising permissions granted by this License. + + "Source" form shall mean the preferred form for making modifications, + including but not limited to software source code, documentation + source, and configuration files. + + "Object" form shall mean any form resulting from mechanical + transformation or translation of a Source form, including but + not limited to compiled object code, generated documentation, + and conversions to other media types. + + "Work" shall mean the work of authorship, whether in Source or + Object form, made available under the License, as indicated by a + copyright notice that is included in or attached to the work + (an example is provided in the Appendix below). + + "Derivative Works" shall mean any work, whether in Source or Object + form, that is based on (or derived from) the Work and for which the + editorial revisions, annotations, elaborations, or other modifications + represent, as a whole, an original work of authorship. For the purposes + of this License, Derivative Works shall not include works that remain + separable from, or merely link (or bind by name) to the interfaces of, + the Work and Derivative Works thereof. + + "Contribution" shall mean any work of authorship, including + the original version of the Work and any modifications or additions + to that Work or Derivative Works thereof, that is intentionally + submitted to Licensor for inclusion in the Work by the copyright owner + or by an individual or Legal Entity authorized to submit on behalf of + the copyright owner. For the purposes of this definition, "submitted" + means any form of electronic, verbal, or written communication sent + to the Licensor or its representatives, including but not limited to + communication on electronic mailing lists, source code control systems, + and issue tracking systems that are managed by, or on behalf of, the + Licensor for the purpose of discussing and improving the Work, but + excluding communication that is conspicuously marked or otherwise + designated in writing by the copyright owner as "Not a Contribution." + + "Contributor" shall mean Licensor and any individual or Legal Entity + on behalf of whom a Contribution has been received by Licensor and + subsequently incorporated within the Work. + + 2. Grant of Copyright License. Subject to the terms and conditions of + this License, each Contributor hereby grants to You a perpetual, + worldwide, non-exclusive, no-charge, royalty-free, irrevocable + copyright license to reproduce, prepare Derivative Works of, + publicly display, publicly perform, sublicense, and distribute the + Work and such Derivative Works in Source or Object form. + + 3. Grant of Patent License. Subject to the terms and conditions of + this License, each Contributor hereby grants to You a perpetual, + worldwide, non-exclusive, no-charge, royalty-free, irrevocable + (except as stated in this section) patent license to make, have made, + use, offer to sell, sell, import, and otherwise transfer the Work, + where such license applies only to those patent claims licensable + by such Contributor that are necessarily infringed by their + Contribution(s) alone or by combination of their Contribution(s) + with the Work to which such Contribution(s) was submitted. If You + institute patent litigation against any entity (including a + cross-claim or counterclaim in a lawsuit) alleging that the Work + or a Contribution incorporated within the Work constitutes direct + or contributory patent infringement, then any patent licenses + granted to You under this License for that Work shall terminate + as of the date such litigation is filed. + + 4. Redistribution. You may reproduce and distribute copies of the + Work or Derivative Works thereof in any medium, with or without + modifications, and in Source or Object form, provided that You + meet the following conditions: + + (a) You must give any other recipients of the Work or + Derivative Works a copy of this License; and + + (b) You must cause any modified files to carry prominent notices + stating that You changed the files; and + + (c) You must retain, in the Source form of any Derivative Works + that You distribute, all copyright, patent, trademark, and + attribution notices from the Source form of the Work, + excluding those notices that do not pertain to any part of + the Derivative Works; and + + (d) If the Work includes a "NOTICE" text file as part of its + distribution, then any Derivative Works that You distribute must + include a readable copy of the attribution notices contained + within such NOTICE file, excluding those notices that do not + pertain to any part of the Derivative Works, in at least one + of the following places: within a NOTICE text file distributed + as part of the Derivative Works; within the Source form or + documentation, if provided along with the Derivative Works; or, + within a display generated by the Derivative Works, if and + wherever such third-party notices normally appear. The contents + of the NOTICE file are for informational purposes only and + do not modify the License. You may add Your own attribution + notices within Derivative Works that You distribute, alongside + or as an addendum to the NOTICE text from the Work, provided + that such additional attribution notices cannot be construed + as modifying the License. + + You may add Your own copyright statement to Your modifications and + may provide additional or different license terms and conditions + for use, reproduction, or distribution of Your modifications, or + for any such Derivative Works as a whole, provided Your use, + reproduction, and distribution of the Work otherwise complies with + the conditions stated in this License. + + 5. Submission of Contributions. Unless You explicitly state otherwise, + any Contribution intentionally submitted for inclusion in the Work + by You to the Licensor shall be under the terms and conditions of + this License, without any additional terms or conditions. + Notwithstanding the above, nothing herein shall supersede or modify + the terms of any separate license agreement you may have executed + with Licensor regarding such Contributions. + + 6. Trademarks. This License does not grant permission to use the trade + names, trademarks, service marks, or product names of the Licensor, + except as required for reasonable and customary use in describing the + origin of the Work and reproducing the content of the NOTICE file. + + 7. Disclaimer of Warranty. Unless required by applicable law or + agreed to in writing, Licensor provides the Work (and each + Contributor provides its Contributions) on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or + implied, including, without limitation, any warranties or conditions + of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A + PARTICULAR PURPOSE. You are solely responsible for determining the + appropriateness of using or redistributing the Work and assume any + risks associated with Your exercise of permissions under this License. + + 8. Limitation of Liability. In no event and under no legal theory, + whether in tort (including negligence), contract, or otherwise, + unless required by applicable law (such as deliberate and grossly + negligent acts) or agreed to in writing, shall any Contributor be + liable to You for damages, including any direct, indirect, special, + incidental, or consequential damages of any character arising as a + result of this License or out of the use or inability to use the + Work (including but not limited to damages for loss of goodwill, + work stoppage, computer failure or malfunction, or any and all + other commercial damages or losses), even if such Contributor + has been advised of the possibility of such damages. + + 9. Accepting Warranty or Additional Liability. While redistributing + the Work or Derivative Works thereof, You may choose to offer, + and charge a fee for, acceptance of support, warranty, indemnity, + or other liability obligations and/or rights consistent with this + License. However, in accepting such obligations, You may act only + on Your own behalf and on Your sole responsibility, not on behalf + of any other Contributor, and only if You agree to indemnify, + defend, and hold each Contributor harmless for any liability + incurred by, or claims asserted against, such Contributor by reason + of your accepting any such warranty or additional liability. + + END OF TERMS AND CONDITIONS + + APPENDIX: How to apply the Apache License to your work. + + To apply the Apache License to your work, attach the following + boilerplate notice, with the fields enclosed by brackets "[]" + replaced with your own identifying information. (Don't include + the brackets!) The text should be enclosed in the appropriate + comment syntax for the file format. We also recommend that a + file or class name and description of purpose be included on the + same "printed page" as the copyright notice for easier + identification within third-party archives. + + Copyright [yyyy] [name of copyright owner] + + Licensed under the Apache License, Version 2.0 (the "License"); + you may not use this file except in compliance with the License. + You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. diff --git a/puppet/modules/systemd/README.md b/puppet/modules/systemd/README.md index f70bcb0c..51bf5cde 100644 --- a/puppet/modules/systemd/README.md +++ b/puppet/modules/systemd/README.md @@ -5,11 +5,23 @@ ## Overview -This module declares exec resources that you can use when you change systemd units or configuration files. +This module declares exec resources to create global sync points for reloading systemd. -## Examples +## Usage and examples -### systemctl --daemon-reload +There are two ways to use this module. + +### unit files + +Let this module handle file creation and systemd reloading. + +```puppet +::systemd::unit_file { 'foo.service': + source => "puppet:///modules/${module_name}/foo.service", +} +``` + +Or handle file creation yourself and trigger systemd. ```puppet include ::systemd @@ -23,7 +35,17 @@ file { '/usr/lib/systemd/system/foo.service': Exec['systemctl-daemon-reload'] ``` -### systemd-tmpfiles --create +### tmpfiles + +Let this module handle file creation and systemd reloading + +```puppet +::systemd::tmpfile { 'foo.conf': + source => "puppet:///modules/${module_name}/foo.conf", +} +``` + +Or handle file creation yourself and trigger systemd. ```puppet include ::systemd @@ -36,3 +58,24 @@ file { '/etc/tmpfiles.d/foo.conf': } ~> Exec['systemd-tmpfiles-create'] ``` + +### service limits + +Manage soft and hard limits on various resources for executed processes. + +```puppet +::systemd::service_limits { 'foo.service': + limits => { + LimitNOFILE => 8192, + LimitNPROC => 16384 + } +} +``` + +Or provide the configuration file yourself. Systemd reloading and restarting of the service are handled by the module. + +```puppet +::systemd::service_limits { 'foo.service': + source => "puppet:///modules/${module_name}/foo.conf", +} +``` diff --git a/puppet/modules/systemd/Rakefile b/puppet/modules/systemd/Rakefile index adcac180..aa7b8a15 100644 --- a/puppet/modules/systemd/Rakefile +++ b/puppet/modules/systemd/Rakefile @@ -4,20 +4,14 @@ require 'puppet-lint/tasks/puppet-lint' Rake::Task[:lint].clear PuppetLint::RakeTask.new :lint do |config| config.ignore_paths = ["spec/**/*.pp", "pkg/**/*.pp", "vendor/**/*.pp"] - config.disable_checks = ['80chars'] + config.disable_checks = ['140chars'] config.fail_on_warnings = true end PuppetSyntax.exclude_paths = ["spec/fixtures/**/*.pp", "vendor/**/*"] # Publishing tasks -unless RUBY_VERSION =~ /^1\.8/ +unless RUBY_VERSION =~ /^1\./ require 'puppet_blacksmith' require 'puppet_blacksmith/rake_tasks' - require 'github_changelog_generator/task' - GitHubChangelogGenerator::RakeTask.new :changelog do |config| - m = Blacksmith::Modulefile.new - config.future_release = m.version - config.release_url = "https://forge.puppetlabs.com/#{m.author}/#{m.name}/%s" - end end diff --git a/puppet/modules/systemd/lib/facter/systemd.rb b/puppet/modules/systemd/lib/facter/systemd.rb new file mode 100644 index 00000000..4361f775 --- /dev/null +++ b/puppet/modules/systemd/lib/facter/systemd.rb @@ -0,0 +1,35 @@ +# Fact: systemd +# +# Purpose: +# Determine whether SystemD is the init system on the node +# +# Resolution: +# Check the name of the process 1 (ps -p 1) +# +# Caveats: +# + +# Fact: systemd-version +# +# Purpose: +# Determine the version of systemd installed +# +# Resolution: +# Check the output of systemctl --version +# +# Caveats: +# + +Facter.add(:systemd) do + confine :kernel => :linux + setcode do + Facter::Util::Resolution.exec('ps -p 1 -o comm=') == 'systemd' + end +end + +Facter.add(:systemd_version) do + confine :systemd => true + setcode do + Facter::Util::Resolution.exec("systemctl --version | awk '/systemd/{ print $2 }'") + end +end diff --git a/puppet/modules/systemd/manifests/enable.pp b/puppet/modules/systemd/manifests/enable.pp new file mode 100644 index 00000000..e1bee18a --- /dev/null +++ b/puppet/modules/systemd/manifests/enable.pp @@ -0,0 +1,8 @@ +# enables a systemd resource +define systemd::enable () { + + exec { "enable_systemd_${name}": + refreshonly => true, + command => "/bin/systemctl enable ${name}" + } +} diff --git a/puppet/modules/systemd/manifests/init.pp b/puppet/modules/systemd/manifests/init.pp index 5e6ad792..e669f093 100644 --- a/puppet/modules/systemd/manifests/init.pp +++ b/puppet/modules/systemd/manifests/init.pp @@ -1,4 +1,8 @@ -class systemd { +# -- Class systemd +# This module allows triggering systemd commands once for all modules +class systemd ( + $service_limits = {} +){ Exec { refreshonly => true, @@ -15,4 +19,6 @@ class systemd { command => 'systemd-tmpfiles --create', } + create_resources('systemd::service_limits', $service_limits, {}) + } diff --git a/puppet/modules/systemd/manifests/service_limits.pp b/puppet/modules/systemd/manifests/service_limits.pp new file mode 100644 index 00000000..a9cdc25a --- /dev/null +++ b/puppet/modules/systemd/manifests/service_limits.pp @@ -0,0 +1,50 @@ +# -- Define: systemd::service_limits +# Creates a custom config file and reloads systemd +define systemd::service_limits( + $ensure = file, + $path = '/etc/systemd/system', + $limits = undef, + $source = undef, + $restart_service = true +) { + include ::systemd + + if $limits { + validate_hash($limits) + $content = template('systemd/limits.erb') + } + else { + $content = undef + } + + if $limits and $source { + fail('You may not supply both limits and source parameters to systemd::service_limits') + } elsif $limits == undef and $source == undef { + fail('You must supply either the limits or source parameter to systemd::service_limits') + } + + file { "${path}/${title}.d/": + ensure => 'directory', + owner => 'root', + group => 'root', + } + -> + file { "${path}/${title}.d/limits.conf": + ensure => $ensure, + content => $content, + source => $source, + owner => 'root', + group => 'root', + mode => '0444', + notify => Exec['systemctl-daemon-reload'], + } + + if $restart_service { + exec { "systemctl restart ${title}": + path => $::path, + refreshonly => true, + subscribe => File["${path}/${title}.d/limits.conf"], + require => Exec['systemctl-daemon-reload'], + } + } +} diff --git a/puppet/modules/systemd/manifests/tmpfile.pp b/puppet/modules/systemd/manifests/tmpfile.pp new file mode 100644 index 00000000..c4d1a05f --- /dev/null +++ b/puppet/modules/systemd/manifests/tmpfile.pp @@ -0,0 +1,20 @@ +# -- Define: systemd::tmpfile +# Creates a tmpfile and reloads systemd +define systemd::tmpfile( + $ensure = file, + $path = '/etc/tmpfiles.d', + $content = undef, + $source = undef, +) { + include ::systemd + + file { "${path}/${title}": + ensure => $ensure, + content => $content, + source => $source, + owner => 'root', + group => 'root', + mode => '0444', + notify => Exec['systemd-tmpfiles-create'], + } +} \ No newline at end of file diff --git a/puppet/modules/systemd/manifests/unit_file.pp b/puppet/modules/systemd/manifests/unit_file.pp new file mode 100644 index 00000000..94bc845b --- /dev/null +++ b/puppet/modules/systemd/manifests/unit_file.pp @@ -0,0 +1,22 @@ +# -- Define: systemd::unit_file +# Creates a unit file and reloads systemd +define systemd::unit_file( + $ensure = file, + $path = '/etc/systemd/system', + $content = undef, + $source = undef, + $target = undef, +) { + include ::systemd + + file { "${path}/${title}": + ensure => $ensure, + content => $content, + source => $source, + target => $target, + owner => 'root', + group => 'root', + mode => '0444', + notify => Exec['systemctl-daemon-reload'], + } +} diff --git a/puppet/modules/systemd/metadata.json b/puppet/modules/systemd/metadata.json index abdd481e..08951efb 100644 --- a/puppet/modules/systemd/metadata.json +++ b/puppet/modules/systemd/metadata.json @@ -1,6 +1,6 @@ { "name": "camptocamp-systemd", - "version": "0.2.2", + "version": "0.4.0", "author": "camptocamp", "summary": "Puppet Systemd module", "license": "Apache-2.0", diff --git a/puppet/modules/systemd/spec/acceptance/nodesets/centos-5.yml b/puppet/modules/systemd/spec/acceptance/nodesets/centos-5.yml new file mode 100644 index 00000000..a26f27fc --- /dev/null +++ b/puppet/modules/systemd/spec/acceptance/nodesets/centos-5.yml @@ -0,0 +1,16 @@ +HOSTS: + centos-5-x64: + default_apply_opts: + order: random + strict_variables: + platform: el-5-x86_64 + hypervisor : docker + image: tianon/centos:5.10 + docker_preserve_image: true + docker_cmd: '["/sbin/init"]' + docker_image_commands: + - 'yum install -y crontabs tar wget which' + - 'sed -i -e "/mingetty/d" /etc/inittab' +CONFIG: + type: aio + log_level: debug diff --git a/puppet/modules/systemd/spec/acceptance/nodesets/centos-6.yml b/puppet/modules/systemd/spec/acceptance/nodesets/centos-6.yml new file mode 100644 index 00000000..71e23cd8 --- /dev/null +++ b/puppet/modules/systemd/spec/acceptance/nodesets/centos-6.yml @@ -0,0 +1,17 @@ +HOSTS: + centos-6-x64: + default_apply_opts: + order: random + strict_variables: + platform: el-6-x86_64 + hypervisor : docker + image: centos:6 + docker_preserve_image: true + docker_cmd: '["/sbin/init"]' + docker_image_commands: + - 'rm -rf /var/run/network/*' + - 'yum install -y crontabs tar wget' + - 'rm /etc/init/tty.conf' +CONFIG: + type: aio + log_level: debug diff --git a/puppet/modules/systemd/spec/acceptance/nodesets/centos-7.yml b/puppet/modules/systemd/spec/acceptance/nodesets/centos-7.yml new file mode 100644 index 00000000..a8fa4686 --- /dev/null +++ b/puppet/modules/systemd/spec/acceptance/nodesets/centos-7.yml @@ -0,0 +1,15 @@ +HOSTS: + centos-7-x64: + default_apply_opts: + order: random + strict_variables: + platform: el-7-x86_64 + hypervisor : docker + image: centos:7 + docker_preserve_image: true + docker_cmd: '["/usr/sbin/init"]' + docker_image_commands: + - 'yum install -y crontabs tar wget iproute' +CONFIG: + type: aio + log_level: debug diff --git a/puppet/modules/systemd/spec/acceptance/nodesets/debian-6.yml b/puppet/modules/systemd/spec/acceptance/nodesets/debian-6.yml new file mode 100644 index 00000000..d7b02756 --- /dev/null +++ b/puppet/modules/systemd/spec/acceptance/nodesets/debian-6.yml @@ -0,0 +1,15 @@ +HOSTS: + debian-6-x64: + default_apply_opts: + order: random + strict_variables: + platform: debian-6-amd64 + hypervisor : docker + image: debian/eol:squeeze + docker_preserve_image: true + docker_cmd: '["/sbin/init"]' + docker_image_commands: + - 'apt-get install -y cron locales-all net-tools wget' +CONFIG: + type: aio + log_level: debug diff --git a/puppet/modules/systemd/spec/acceptance/nodesets/debian-7.yml b/puppet/modules/systemd/spec/acceptance/nodesets/debian-7.yml new file mode 100644 index 00000000..9591ea77 --- /dev/null +++ b/puppet/modules/systemd/spec/acceptance/nodesets/debian-7.yml @@ -0,0 +1,15 @@ +HOSTS: + debian-7-x64: + default_apply_opts: + order: random + strict_variables: + platform: debian-7-amd64 + hypervisor : docker + image: debian:7 + docker_preserve_image: true + docker_cmd: '["/sbin/init"]' + docker_image_commands: + - 'apt-get install -y cron locales-all net-tools wget' +CONFIG: + type: aio + log_level: debug diff --git a/puppet/modules/systemd/spec/acceptance/nodesets/debian-8.yml b/puppet/modules/systemd/spec/acceptance/nodesets/debian-8.yml new file mode 100644 index 00000000..5fb24c61 --- /dev/null +++ b/puppet/modules/systemd/spec/acceptance/nodesets/debian-8.yml @@ -0,0 +1,16 @@ +HOSTS: + debian-8-x64: + default_apply_opts: + order: random + strict_variables: + platform: debian-8-amd64 + hypervisor : docker + image: debian:8 + docker_preserve_image: true + docker_cmd: '["/sbin/init"]' + docker_image_commands: + - 'apt-get install -y cron locales-all net-tools wget' + - 'rm -f /usr/sbin/policy-rc.d' +CONFIG: + type: aio + log_level: debug diff --git a/puppet/modules/systemd/spec/acceptance/nodesets/ubuntu-12.04.yml b/puppet/modules/systemd/spec/acceptance/nodesets/ubuntu-12.04.yml new file mode 100644 index 00000000..594e1771 --- /dev/null +++ b/puppet/modules/systemd/spec/acceptance/nodesets/ubuntu-12.04.yml @@ -0,0 +1,16 @@ +HOSTS: + ubuntu-1204-x64: + default_apply_opts: + order: random + strict_variables: + platform: ubuntu-12.04-amd64 + hypervisor : docker + image: ubuntu:12.04 + docker_preserve_image: true + docker_cmd: '["/sbin/init"]' + docker_image_commands: + - 'apt-get install -y net-tools wget' + - 'locale-gen en_US.UTF-8' +CONFIG: + type: aio + log_level: debug diff --git a/puppet/modules/systemd/spec/acceptance/nodesets/ubuntu-14.04.yml b/puppet/modules/systemd/spec/acceptance/nodesets/ubuntu-14.04.yml new file mode 100644 index 00000000..2b293c99 --- /dev/null +++ b/puppet/modules/systemd/spec/acceptance/nodesets/ubuntu-14.04.yml @@ -0,0 +1,18 @@ +HOSTS: + ubuntu-1404-x64: + default_apply_opts: + order: random + strict_variables: + platform: ubuntu-14.04-amd64 + hypervisor : docker + image: ubuntu:14.04 + docker_preserve_image: true + docker_cmd: '["/sbin/init"]' + docker_image_commands: + - 'rm /usr/sbin/policy-rc.d' + - 'rm /sbin/initctl; dpkg-divert --rename --remove /sbin/initctl' + - 'apt-get install -y net-tools wget' + - 'locale-gen en_US.UTF-8' +CONFIG: + type: aio + log_level: debug diff --git a/puppet/modules/systemd/spec/acceptance/nodesets/ubuntu-14.10.yml b/puppet/modules/systemd/spec/acceptance/nodesets/ubuntu-14.10.yml new file mode 100644 index 00000000..7ce09b2a --- /dev/null +++ b/puppet/modules/systemd/spec/acceptance/nodesets/ubuntu-14.10.yml @@ -0,0 +1,18 @@ +HOSTS: + ubuntu-1410-x64: + default_apply_opts: + order: random + strict_variables: + platform: ubuntu-14.10-amd64 + hypervisor : docker + image: ubuntu:14.10 + docker_preserve_image: true + docker_cmd: '["/sbin/init"]' + docker_image_commands: + - 'rm /usr/sbin/policy-rc.d' + - 'rm /sbin/initctl; dpkg-divert --rename --remove /sbin/initctl' + - 'apt-get install -y net-tools wget' + - 'locale-gen en_US.UTF-8' +CONFIG: + type: aio + log_level: debug diff --git a/puppet/modules/systemd/spec/acceptance/nodesets/ubuntu-15.04.yml b/puppet/modules/systemd/spec/acceptance/nodesets/ubuntu-15.04.yml new file mode 100644 index 00000000..329f3319 --- /dev/null +++ b/puppet/modules/systemd/spec/acceptance/nodesets/ubuntu-15.04.yml @@ -0,0 +1,16 @@ +HOSTS: + ubuntu-1504-x64: + default_apply_opts: + order: random + strict_variables: + platform: ubuntu-15.04-amd64 + hypervisor : docker + image: ubuntu:15.04 + docker_preserve_image: true + docker_cmd: '["/sbin/init"]' + docker_image_commands: + - 'apt-get install -y net-tools wget' + - 'locale-gen en_US.UTF-8' +CONFIG: + type: aio + log_level: debug diff --git a/puppet/modules/systemd/spec/acceptance/nodesets/ubuntu-15.10.yml b/puppet/modules/systemd/spec/acceptance/nodesets/ubuntu-15.10.yml new file mode 100644 index 00000000..487795a3 --- /dev/null +++ b/puppet/modules/systemd/spec/acceptance/nodesets/ubuntu-15.10.yml @@ -0,0 +1,16 @@ +HOSTS: + ubuntu-1510-x64: + default_apply_opts: + order: random + strict_variables: + platform: ubuntu-15.10-amd64 + hypervisor : docker + image: ubuntu:15.10 + docker_preserve_image: true + docker_cmd: '["/sbin/init"]' + docker_image_commands: + - 'apt-get install -y net-tools wget' + - 'locale-gen en_US.UTF-8' +CONFIG: + type: aio + log_level: debug diff --git a/puppet/modules/systemd/spec/acceptance/nodesets/ubuntu-16.04.yml b/puppet/modules/systemd/spec/acceptance/nodesets/ubuntu-16.04.yml new file mode 100644 index 00000000..6c32b96d --- /dev/null +++ b/puppet/modules/systemd/spec/acceptance/nodesets/ubuntu-16.04.yml @@ -0,0 +1,16 @@ +HOSTS: + ubuntu-1604-x64: + default_apply_opts: + order: random + strict_variables: + platform: ubuntu-16.04-amd64 + hypervisor : docker + image: ubuntu:16.04 + docker_preserve_image: true + docker_cmd: '["/sbin/init"]' + docker_image_commands: + - 'apt-get install -y net-tools wget' + - 'locale-gen en_US.UTF-8' +CONFIG: + type: aio + log_level: debug diff --git a/puppet/modules/systemd/spec/defines/tmpfile_spec.rb b/puppet/modules/systemd/spec/defines/tmpfile_spec.rb new file mode 100644 index 00000000..4eb22acd --- /dev/null +++ b/puppet/modules/systemd/spec/defines/tmpfile_spec.rb @@ -0,0 +1,48 @@ +require 'spec_helper' + +describe 'systemd::tmpfile' do + + let(:facts) { { + :path => '/usr/bin', + } } + + context 'default params' do + + let(:title) { 'fancy.conf' } + + it 'creates the tmpfile' do + should contain_file('/etc/tmpfiles.d/fancy.conf').with({ + 'ensure' => 'file', + 'owner' => 'root', + 'group' => 'root', + 'mode' => '0444', + }) + end + + it 'triggers systemd daemon-reload' do + should contain_class('systemd') + should contain_file('/etc/tmpfiles.d/fancy.conf').with_notify("Exec[systemd-tmpfiles-create]") + end + end + + context 'with params' do + let(:title) { 'fancy.conf' } + + let(:params) { { + :ensure => 'absent', + :path => '/etc/tmpfiles.d/foo', + :content => 'some-content', + :source => 'some-source', + } } + + it 'creates the unit file' do + should contain_file('/etc/tmpfiles.d/foo/fancy.conf').with({ + 'ensure' => 'absent', + 'content' => 'some-content', + 'source' => 'some-source', + }) + end + + end + +end diff --git a/puppet/modules/systemd/spec/defines/unit_file_spec.rb b/puppet/modules/systemd/spec/defines/unit_file_spec.rb new file mode 100644 index 00000000..88a0122c --- /dev/null +++ b/puppet/modules/systemd/spec/defines/unit_file_spec.rb @@ -0,0 +1,50 @@ +require 'spec_helper' + +describe 'systemd::unit_file' do + + let(:facts) { { + :path => '/usr/bin', + } } + + context 'default params' do + + let(:title) { 'fancy.service' } + + it 'creates the unit file' do + should contain_file('/etc/systemd/system/fancy.service').with({ + 'ensure' => 'file', + 'owner' => 'root', + 'group' => 'root', + 'mode' => '0444', + }) + end + + it 'triggers systemd daemon-reload' do + should contain_class('systemd') + should contain_file('/etc/systemd/system/fancy.service').with_notify("Exec[systemctl-daemon-reload]") + end + end + + context 'with params' do + let(:title) { 'fancy.service' } + + let(:params) { { + :ensure => 'absent', + :path => '/usr/lib/systemd/system', + :content => 'some-content', + :source => 'some-source', + :target => 'some-target', + } } + + it 'creates the unit file' do + should contain_file('/usr/lib/systemd/system/fancy.service').with({ + 'ensure' => 'absent', + 'content' => 'some-content', + 'source' => 'some-source', + 'target' => 'some-target', + }) + end + + end + +end diff --git a/puppet/modules/systemd/spec/unit/facter/systemd_spec.rb b/puppet/modules/systemd/spec/unit/facter/systemd_spec.rb new file mode 100644 index 00000000..a7b62410 --- /dev/null +++ b/puppet/modules/systemd/spec/unit/facter/systemd_spec.rb @@ -0,0 +1,41 @@ +require "spec_helper" + +describe Facter::Util::Fact do + before { + Facter.clear + } + + describe "systemd" do + context 'returns true when systemd present' do + before do + Facter.fact(:kernel).stubs(:value).returns(:linux) + end + let(:facts) { {:kernel => :linux} } + it do + Facter::Util::Resolution.expects(:exec).with('ps -p 1 -o comm=').returns('systemd') + expect(Facter.value(:systemd)).to eq(true) + end + end + context 'returns false when systemd not present' do + before do + Facter.fact(:kernel).stubs(:value).returns(:linux) + end + let(:facts) { {:kernel => :linux} } + it do + Facter::Util::Resolution.expects(:exec).with('ps -p 1 -o comm=').returns('init') + expect(Facter.value(:systemd)).to eq(false) + end + end + + context 'returns nil when kernel is not linux' do + before do + Facter.fact(:kernel).stubs(:value).returns(:windows) + end + let(:facts) { {:kernel => :windows} } + it do + Facter::Util::Resolution.expects(:exec).with('ps -p 1 -o comm=').never + expect(Facter.value(:systemd)).to be_nil + end + end + end +end diff --git a/puppet/modules/systemd/spec/unit/facter/systemd_version_spec.rb b/puppet/modules/systemd/spec/unit/facter/systemd_version_spec.rb new file mode 100644 index 00000000..5007dc69 --- /dev/null +++ b/puppet/modules/systemd/spec/unit/facter/systemd_version_spec.rb @@ -0,0 +1,31 @@ +require "spec_helper" + +describe Facter::Util::Fact do + before { + Facter.clear + } + + describe "systemd_version" do + context 'returns version when systemd fact present' do + before do + Facter.fact(:systemd).stubs(:value).returns(true) + end + let(:facts) { {:systemd => true} } + it do + Facter::Util::Resolution.expects(:exec).with("systemctl --version | awk '/systemd/{ print $2 }'").returns('229') + expect(Facter.value(:systemd_version)).to eq('229') + end + end + context 'returns nil when systemd fact not present' do + before do + Facter.fact(:systemd).stubs(:value).returns(false) + end + let(:facts) { {:systemd => false } } + it do + Facter::Util::Resolution.stubs(:exec) + Facter::Util::Resolution.expects(:exec).with("systemctl --version | awk '/systemd/{ print $2 }'").never + expect(Facter.value(:systemd_version)).to eq(nil) + end + end + end +end diff --git a/puppet/modules/systemd/templates/limits.erb b/puppet/modules/systemd/templates/limits.erb new file mode 100644 index 00000000..3caf5867 --- /dev/null +++ b/puppet/modules/systemd/templates/limits.erb @@ -0,0 +1,26 @@ +# This file is created by Puppet +[Service] +<% +[ + 'LimitCPU', + 'LimitFSIZE', + 'LimitDATA', + 'LimitSTACK', + 'LimitCORE', + 'LimitRSS', + 'LimitNOFILE', + 'LimitAS', + 'LimitNPROC', + 'LimitMEMLOCK', + 'LimitLOCKS', + 'LimitSIGPENDING', + 'LimitMSGQUEUE', + 'LimitNICE', + 'LimitRTPRIO', + 'LimitRTTIME' +].each do |d| +if @limits[d] -%> +<%= d %>=<%= @limits[d] %> +<% +end +end %> -- cgit v1.2.3 From cfd1f9ee35f7b80b17c460b77709d3d5dc880638 Mon Sep 17 00:00:00 2001 From: Tulio Casagrande Date: Mon, 16 Jan 2017 15:15:41 -0200 Subject: Add apache auto-restart extension file --- puppet/modules/site_apache/files/auto_restart.conf | 2 ++ puppet/modules/site_apache/manifests/common.pp | 1 + puppet/modules/site_apache/manifests/common/extensions.pp | 14 ++++++++++++++ puppet/modules/site_apache/spec/classes/extensions.rb | 7 +++++++ puppet/modules/site_apache/spec/spec_helper.rb | 6 ++++++ 5 files changed, 30 insertions(+) create mode 100644 puppet/modules/site_apache/files/auto_restart.conf create mode 100644 puppet/modules/site_apache/manifests/common/extensions.pp create mode 100644 puppet/modules/site_apache/spec/classes/extensions.rb create mode 100644 puppet/modules/site_apache/spec/spec_helper.rb diff --git a/puppet/modules/site_apache/files/auto_restart.conf b/puppet/modules/site_apache/files/auto_restart.conf new file mode 100644 index 00000000..8a764e34 --- /dev/null +++ b/puppet/modules/site_apache/files/auto_restart.conf @@ -0,0 +1,2 @@ +[Service] +Restart=always diff --git a/puppet/modules/site_apache/manifests/common.pp b/puppet/modules/site_apache/manifests/common.pp index 208c15d5..c96932dd 100644 --- a/puppet/modules/site_apache/manifests/common.pp +++ b/puppet/modules/site_apache/manifests/common.pp @@ -28,5 +28,6 @@ class site_apache::common { include site_apache::common::tls include site_apache::common::acme + include site_apache::common::extensions } diff --git a/puppet/modules/site_apache/manifests/common/extensions.pp b/puppet/modules/site_apache/manifests/common/extensions.pp new file mode 100644 index 00000000..ddeafae0 --- /dev/null +++ b/puppet/modules/site_apache/manifests/common/extensions.pp @@ -0,0 +1,14 @@ +class site_apache::common::extensions { + + include ::systemd + file { '/etc/systemd/system/apache2.service.d/auto_restart.conf': + source => 'puppet:///modules/site_apache/auto_restart.conf', + owner => 'root', + group => 'root', + mode => '0644', + require => [ + Service['apache'] + ] + }~> + Exec['systemctl-daemon-reload'] +} diff --git a/puppet/modules/site_apache/spec/classes/extensions.rb b/puppet/modules/site_apache/spec/classes/extensions.rb new file mode 100644 index 00000000..164034c2 --- /dev/null +++ b/puppet/modules/site_apache/spec/classes/extensions.rb @@ -0,0 +1,7 @@ +require File.expand_path(File.join(File.dirname(__FILE__),'../spec_helper')) + +describe 'site_apache::common::extensions' do + it "should include apache autostart" do + should contain_file('/etc/systemd/system/apache2.service.d/auto_restart.conf').with_source('puppet:///modules/site_apache/apache_auto_restart.conf') + end +end diff --git a/puppet/modules/site_apache/spec/spec_helper.rb b/puppet/modules/site_apache/spec/spec_helper.rb new file mode 100644 index 00000000..dea9e892 --- /dev/null +++ b/puppet/modules/site_apache/spec/spec_helper.rb @@ -0,0 +1,6 @@ +require 'rspec-puppet' + +RSpec.configure do |c| + c.module_path = File.expand_path(File.join(File.dirname(__FILE__), '..', '..')) + c.color = true +end -- cgit v1.2.3 From 6748ba91d4d5fb77bb9034f8fc39e6735e2ef375 Mon Sep 17 00:00:00 2001 From: Tulio Casagrande Date: Mon, 16 Jan 2017 17:24:50 -0200 Subject: Update how exec is run --- puppet/modules/site_apache/manifests/common/extensions.pp | 8 +++----- 1 file changed, 3 insertions(+), 5 deletions(-) diff --git a/puppet/modules/site_apache/manifests/common/extensions.pp b/puppet/modules/site_apache/manifests/common/extensions.pp index ddeafae0..6e489ce8 100644 --- a/puppet/modules/site_apache/manifests/common/extensions.pp +++ b/puppet/modules/site_apache/manifests/common/extensions.pp @@ -6,9 +6,7 @@ class site_apache::common::extensions { owner => 'root', group => 'root', mode => '0644', - require => [ - Service['apache'] - ] - }~> - Exec['systemctl-daemon-reload'] + require => Service['apache'], + notify => Exec['systemctl-daemon-reload'] + } } -- cgit v1.2.3 From 65604b35e96c4ca6e83c2d90b1c512dfa7ff31e7 Mon Sep 17 00:00:00 2001 From: Tulio Casagrande Date: Mon, 16 Jan 2017 17:25:15 -0200 Subject: Remove spec_helper --- puppet/modules/site_apache/spec/classes/extensions.rb | 2 +- puppet/modules/site_apache/spec/spec_helper.rb | 6 ------ 2 files changed, 1 insertion(+), 7 deletions(-) delete mode 100644 puppet/modules/site_apache/spec/spec_helper.rb diff --git a/puppet/modules/site_apache/spec/classes/extensions.rb b/puppet/modules/site_apache/spec/classes/extensions.rb index 164034c2..632b2d72 100644 --- a/puppet/modules/site_apache/spec/classes/extensions.rb +++ b/puppet/modules/site_apache/spec/classes/extensions.rb @@ -1,4 +1,4 @@ -require File.expand_path(File.join(File.dirname(__FILE__),'../spec_helper')) +require 'spec_helper' describe 'site_apache::common::extensions' do it "should include apache autostart" do diff --git a/puppet/modules/site_apache/spec/spec_helper.rb b/puppet/modules/site_apache/spec/spec_helper.rb deleted file mode 100644 index dea9e892..00000000 --- a/puppet/modules/site_apache/spec/spec_helper.rb +++ /dev/null @@ -1,6 +0,0 @@ -require 'rspec-puppet' - -RSpec.configure do |c| - c.module_path = File.expand_path(File.join(File.dirname(__FILE__), '..', '..')) - c.color = true -end -- cgit v1.2.3 From 810df8983e3ada8ad4b0ebbc7e0e3cf01219e33c Mon Sep 17 00:00:00 2001 From: Tulio Casagrande Date: Tue, 17 Jan 2017 11:03:14 -0200 Subject: Rename extensions module to autorestart --- puppet/modules/site_apache/files/auto_restart.conf | 2 -- puppet/modules/site_apache/files/autorestart.conf | 2 ++ puppet/modules/site_apache/manifests/common.pp | 2 +- .../modules/site_apache/manifests/common/autorestart.pp | 15 +++++++++++++++ puppet/modules/site_apache/manifests/common/extensions.pp | 12 ------------ puppet/modules/site_apache/spec/classes/autorestart.rb | 7 +++++++ puppet/modules/site_apache/spec/classes/extensions.rb | 7 ------- 7 files changed, 25 insertions(+), 22 deletions(-) delete mode 100644 puppet/modules/site_apache/files/auto_restart.conf create mode 100644 puppet/modules/site_apache/files/autorestart.conf create mode 100644 puppet/modules/site_apache/manifests/common/autorestart.pp delete mode 100644 puppet/modules/site_apache/manifests/common/extensions.pp create mode 100644 puppet/modules/site_apache/spec/classes/autorestart.rb delete mode 100644 puppet/modules/site_apache/spec/classes/extensions.rb diff --git a/puppet/modules/site_apache/files/auto_restart.conf b/puppet/modules/site_apache/files/auto_restart.conf deleted file mode 100644 index 8a764e34..00000000 --- a/puppet/modules/site_apache/files/auto_restart.conf +++ /dev/null @@ -1,2 +0,0 @@ -[Service] -Restart=always diff --git a/puppet/modules/site_apache/files/autorestart.conf b/puppet/modules/site_apache/files/autorestart.conf new file mode 100644 index 00000000..8a764e34 --- /dev/null +++ b/puppet/modules/site_apache/files/autorestart.conf @@ -0,0 +1,2 @@ +[Service] +Restart=always diff --git a/puppet/modules/site_apache/manifests/common.pp b/puppet/modules/site_apache/manifests/common.pp index c96932dd..4847cbe3 100644 --- a/puppet/modules/site_apache/manifests/common.pp +++ b/puppet/modules/site_apache/manifests/common.pp @@ -28,6 +28,6 @@ class site_apache::common { include site_apache::common::tls include site_apache::common::acme - include site_apache::common::extensions + include site_apache::common::autorestart } diff --git a/puppet/modules/site_apache/manifests/common/autorestart.pp b/puppet/modules/site_apache/manifests/common/autorestart.pp new file mode 100644 index 00000000..aa13d4ef --- /dev/null +++ b/puppet/modules/site_apache/manifests/common/autorestart.pp @@ -0,0 +1,15 @@ +# +# Adds autorestart extension to apache on crash +# +class site_apache::common::autorestart { + + include ::systemd + file { '/etc/systemd/system/apache2.service.d/autorestart.conf': + source => 'puppet:///modules/site_apache/autorestart.conf', + owner => 'root', + group => 'root', + mode => '0644', + require => Service['apache'], + notify => Exec['systemctl-daemon-reload'] + } +} diff --git a/puppet/modules/site_apache/manifests/common/extensions.pp b/puppet/modules/site_apache/manifests/common/extensions.pp deleted file mode 100644 index 6e489ce8..00000000 --- a/puppet/modules/site_apache/manifests/common/extensions.pp +++ /dev/null @@ -1,12 +0,0 @@ -class site_apache::common::extensions { - - include ::systemd - file { '/etc/systemd/system/apache2.service.d/auto_restart.conf': - source => 'puppet:///modules/site_apache/auto_restart.conf', - owner => 'root', - group => 'root', - mode => '0644', - require => Service['apache'], - notify => Exec['systemctl-daemon-reload'] - } -} diff --git a/puppet/modules/site_apache/spec/classes/autorestart.rb b/puppet/modules/site_apache/spec/classes/autorestart.rb new file mode 100644 index 00000000..afa02ec9 --- /dev/null +++ b/puppet/modules/site_apache/spec/classes/autorestart.rb @@ -0,0 +1,7 @@ +require 'spec_helper' + +describe 'site_apache::common::autorestart' do + it "should include apache autorestart" do + should contain_file('/etc/systemd/system/apache2.service.d/autorestart.conf').with_source('puppet:///modules/site_apache/autorestart.conf') + end +end diff --git a/puppet/modules/site_apache/spec/classes/extensions.rb b/puppet/modules/site_apache/spec/classes/extensions.rb deleted file mode 100644 index 632b2d72..00000000 --- a/puppet/modules/site_apache/spec/classes/extensions.rb +++ /dev/null @@ -1,7 +0,0 @@ -require 'spec_helper' - -describe 'site_apache::common::extensions' do - it "should include apache autostart" do - should contain_file('/etc/systemd/system/apache2.service.d/auto_restart.conf').with_source('puppet:///modules/site_apache/apache_auto_restart.conf') - end -end -- cgit v1.2.3 From b6e19e290eb6395ce0a12bf2307282b00e7456ea Mon Sep 17 00:00:00 2001 From: Tulio Casagrande Date: Tue, 17 Jan 2017 14:26:27 -0200 Subject: Change autorestart to use systemd::unit_file --- puppet/modules/site_apache/manifests/common/autorestart.pp | 7 +------ 1 file changed, 1 insertion(+), 6 deletions(-) diff --git a/puppet/modules/site_apache/manifests/common/autorestart.pp b/puppet/modules/site_apache/manifests/common/autorestart.pp index aa13d4ef..f8213439 100644 --- a/puppet/modules/site_apache/manifests/common/autorestart.pp +++ b/puppet/modules/site_apache/manifests/common/autorestart.pp @@ -3,13 +3,8 @@ # class site_apache::common::autorestart { - include ::systemd - file { '/etc/systemd/system/apache2.service.d/autorestart.conf': + ::systemd::unit_file { '/etc/systemd/system/apache2.service.d/autorestart.conf': source => 'puppet:///modules/site_apache/autorestart.conf', - owner => 'root', - group => 'root', - mode => '0644', require => Service['apache'], - notify => Exec['systemctl-daemon-reload'] } } -- cgit v1.2.3 From 80e0ca380e92fe435622dbd35d1a5baedb6c3f92 Mon Sep 17 00:00:00 2001 From: Tulio Casagrande Date: Tue, 17 Jan 2017 15:18:30 -0200 Subject: Ensure the directory exists before creating the file with @aarni --- puppet/modules/site_apache/manifests/common/autorestart.pp | 12 ++++++++++-- puppet/modules/site_apache/spec/classes/autorestart.rb | 7 ------- puppet/modules/site_apache/spec/classes/autorestart_spec.rb | 7 +++++++ 3 files changed, 17 insertions(+), 9 deletions(-) delete mode 100644 puppet/modules/site_apache/spec/classes/autorestart.rb create mode 100644 puppet/modules/site_apache/spec/classes/autorestart_spec.rb diff --git a/puppet/modules/site_apache/manifests/common/autorestart.pp b/puppet/modules/site_apache/manifests/common/autorestart.pp index f8213439..0273f272 100644 --- a/puppet/modules/site_apache/manifests/common/autorestart.pp +++ b/puppet/modules/site_apache/manifests/common/autorestart.pp @@ -3,8 +3,16 @@ # class site_apache::common::autorestart { - ::systemd::unit_file { '/etc/systemd/system/apache2.service.d/autorestart.conf': + file { '/etc/systemd/system/apache2.service.d': + ensure => directory, + mode => '0755', + } + + ::systemd::unit_file { 'apache2.service.d/autorestart.conf': source => 'puppet:///modules/site_apache/autorestart.conf', - require => Service['apache'], + require => [ + File['/etc/systemd/system/apache2.service.d'], + Service['apache'], + ] } } diff --git a/puppet/modules/site_apache/spec/classes/autorestart.rb b/puppet/modules/site_apache/spec/classes/autorestart.rb deleted file mode 100644 index afa02ec9..00000000 --- a/puppet/modules/site_apache/spec/classes/autorestart.rb +++ /dev/null @@ -1,7 +0,0 @@ -require 'spec_helper' - -describe 'site_apache::common::autorestart' do - it "should include apache autorestart" do - should contain_file('/etc/systemd/system/apache2.service.d/autorestart.conf').with_source('puppet:///modules/site_apache/autorestart.conf') - end -end diff --git a/puppet/modules/site_apache/spec/classes/autorestart_spec.rb b/puppet/modules/site_apache/spec/classes/autorestart_spec.rb new file mode 100644 index 00000000..ad9c9f2e --- /dev/null +++ b/puppet/modules/site_apache/spec/classes/autorestart_spec.rb @@ -0,0 +1,7 @@ +require 'spec_helper' + +describe 'site_apache::common::autorestart' do + it "should include apache autorestart" do + should contain_file('apache2.service.d/autorestart.conf').with_source('puppet:///modules/site_apache/autorestart.conf') + end +end -- cgit v1.2.3 From 7e51872c854f48eaf372a6025f4550ca801cb447 Mon Sep 17 00:00:00 2001 From: varac Date: Mon, 16 Jan 2017 15:41:44 +0100 Subject: Use systemd unit file for nickserver [#8578] --- puppet/modules/site_nickserver/manifests/init.pp | 46 ++++++++---------------- 1 file changed, 15 insertions(+), 31 deletions(-) diff --git a/puppet/modules/site_nickserver/manifests/init.pp b/puppet/modules/site_nickserver/manifests/init.pp index ad97f829..12236b3b 100644 --- a/puppet/modules/site_nickserver/manifests/init.pp +++ b/puppet/modules/site_nickserver/manifests/init.pp @@ -101,42 +101,26 @@ class site_nickserver { # NICKSERVER DAEMON # - file { - '/usr/bin/nickserver': - ensure => link, - target => '/srv/leap/nickserver/bin/nickserver', - require => Vcsrepo['/srv/leap/nickserver']; - - '/etc/init.d/nickserver': - owner => root, - group => 0, - mode => '0755', - source => '/srv/leap/nickserver/dist/debian-init-script', - require => Vcsrepo['/srv/leap/nickserver']; + file { '/usr/bin/nickserver': + ensure => link, + target => '/srv/leap/nickserver/bin/nickserver', + require => Vcsrepo['/srv/leap/nickserver']; } - # register initscript at systemd on nodes newer than wheezy - # see https://leap.se/code/issues/7614 - case $::operatingsystemrelease { - /^7.*/: { } - default: { - exec { 'register_systemd_nickserver': - refreshonly => true, - command => '/bin/systemctl enable nickserver', - subscribe => File['/etc/init.d/nickserver'], - before => Service['nickserver']; - } - } + ::systemd::unit_file {'nickserver.service': + ensure => present, + source => '/srv/leap/nickserver/dist/nickserver.service', + subscribe => Vcsrepo['/srv/leap/nickserver'], + require => File['/usr/bin/nickserver']; } service { 'nickserver': - ensure => running, - enable => true, - hasrestart => true, - hasstatus => true, - require => [ - File['/etc/init.d/nickserver'], - File['/usr/bin/nickserver'], + ensure => running, + provider => 'systemd', + enable => true, + require => [ + Systemd::Unit_file['nickserver.service'], + Exec['systemctl-daemon-reload'], Class['Site_config::X509::Key'], Class['Site_config::X509::Cert'], Class['Site_config::X509::Ca'] ]; -- cgit v1.2.3 From b4df96ee1d654b4f56fe901e548e4a80207b60d9 Mon Sep 17 00:00:00 2001 From: varac Date: Mon, 16 Jan 2017 11:48:53 +0100 Subject: Revert "Add systemd::enable define" This commit was moved to the systemd puppet repo. This reverts commit f5db49cf6b3ca0a5830b849c0aac074e371b95d9. --- puppet/modules/systemd/manifests/enable.pp | 8 -------- 1 file changed, 8 deletions(-) delete mode 100644 puppet/modules/systemd/manifests/enable.pp diff --git a/puppet/modules/systemd/manifests/enable.pp b/puppet/modules/systemd/manifests/enable.pp deleted file mode 100644 index e1bee18a..00000000 --- a/puppet/modules/systemd/manifests/enable.pp +++ /dev/null @@ -1,8 +0,0 @@ -# enables a systemd resource -define systemd::enable () { - - exec { "enable_systemd_${name}": - refreshonly => true, - command => "/bin/systemctl enable ${name}" - } -} -- cgit v1.2.3 From 0117808a130c36f14f5b86879b52ed7ce2fa6d57 Mon Sep 17 00:00:00 2001 From: varac Date: Mon, 16 Jan 2017 14:39:14 +0100 Subject: git subrepo clone --force https://leap.se/git/puppet_systemd puppet/modules/systemd subrepo: subdir: "puppet/modules/systemd" merged: "f3c4059" upstream: origin: "https://leap.se/git/puppet_systemd" branch: "master" commit: "f3c4059" git-subrepo: version: "0.3.0" origin: "https://github.com/ingydotnet/git-subrepo.git" commit: "841aa43" --- puppet/modules/systemd/.fixtures.yml | 4 + puppet/modules/systemd/.gitrepo | 4 +- puppet/modules/systemd/.puppet-lint.rc | 2 +- puppet/modules/systemd/.travis.yml | 22 +-- puppet/modules/systemd/CHANGELOG.md | 20 +- puppet/modules/systemd/Gemfile | 18 +- puppet/modules/systemd/HISTORY.md | 15 +- puppet/modules/systemd/LICENSE | 201 +++++++++++++++++++++ puppet/modules/systemd/README.md | 51 +++++- puppet/modules/systemd/Rakefile | 10 +- puppet/modules/systemd/lib/facter/systemd.rb | 35 ++++ puppet/modules/systemd/manifests/enable.pp | 8 + puppet/modules/systemd/manifests/init.pp | 8 +- puppet/modules/systemd/manifests/service_limits.pp | 50 +++++ puppet/modules/systemd/manifests/tmpfile.pp | 20 ++ puppet/modules/systemd/manifests/unit_file.pp | 22 +++ puppet/modules/systemd/metadata.json | 2 +- .../systemd/spec/acceptance/nodesets/centos-5.yml | 16 ++ .../systemd/spec/acceptance/nodesets/centos-6.yml | 17 ++ .../systemd/spec/acceptance/nodesets/centos-7.yml | 15 ++ .../systemd/spec/acceptance/nodesets/debian-6.yml | 15 ++ .../systemd/spec/acceptance/nodesets/debian-7.yml | 15 ++ .../systemd/spec/acceptance/nodesets/debian-8.yml | 16 ++ .../spec/acceptance/nodesets/ubuntu-12.04.yml | 16 ++ .../spec/acceptance/nodesets/ubuntu-14.04.yml | 18 ++ .../spec/acceptance/nodesets/ubuntu-14.10.yml | 18 ++ .../spec/acceptance/nodesets/ubuntu-15.04.yml | 16 ++ .../spec/acceptance/nodesets/ubuntu-15.10.yml | 16 ++ .../spec/acceptance/nodesets/ubuntu-16.04.yml | 16 ++ .../modules/systemd/spec/defines/tmpfile_spec.rb | 48 +++++ .../modules/systemd/spec/defines/unit_file_spec.rb | 50 +++++ .../systemd/spec/unit/facter/systemd_spec.rb | 41 +++++ .../spec/unit/facter/systemd_version_spec.rb | 31 ++++ puppet/modules/systemd/templates/limits.erb | 26 +++ 34 files changed, 839 insertions(+), 43 deletions(-) create mode 100644 puppet/modules/systemd/.fixtures.yml create mode 100644 puppet/modules/systemd/LICENSE create mode 100644 puppet/modules/systemd/lib/facter/systemd.rb create mode 100644 puppet/modules/systemd/manifests/enable.pp create mode 100644 puppet/modules/systemd/manifests/service_limits.pp create mode 100644 puppet/modules/systemd/manifests/tmpfile.pp create mode 100644 puppet/modules/systemd/manifests/unit_file.pp create mode 100644 puppet/modules/systemd/spec/acceptance/nodesets/centos-5.yml create mode 100644 puppet/modules/systemd/spec/acceptance/nodesets/centos-6.yml create mode 100644 puppet/modules/systemd/spec/acceptance/nodesets/centos-7.yml create mode 100644 puppet/modules/systemd/spec/acceptance/nodesets/debian-6.yml create mode 100644 puppet/modules/systemd/spec/acceptance/nodesets/debian-7.yml create mode 100644 puppet/modules/systemd/spec/acceptance/nodesets/debian-8.yml create mode 100644 puppet/modules/systemd/spec/acceptance/nodesets/ubuntu-12.04.yml create mode 100644 puppet/modules/systemd/spec/acceptance/nodesets/ubuntu-14.04.yml create mode 100644 puppet/modules/systemd/spec/acceptance/nodesets/ubuntu-14.10.yml create mode 100644 puppet/modules/systemd/spec/acceptance/nodesets/ubuntu-15.04.yml create mode 100644 puppet/modules/systemd/spec/acceptance/nodesets/ubuntu-15.10.yml create mode 100644 puppet/modules/systemd/spec/acceptance/nodesets/ubuntu-16.04.yml create mode 100644 puppet/modules/systemd/spec/defines/tmpfile_spec.rb create mode 100644 puppet/modules/systemd/spec/defines/unit_file_spec.rb create mode 100644 puppet/modules/systemd/spec/unit/facter/systemd_spec.rb create mode 100644 puppet/modules/systemd/spec/unit/facter/systemd_version_spec.rb create mode 100644 puppet/modules/systemd/templates/limits.erb diff --git a/puppet/modules/systemd/.fixtures.yml b/puppet/modules/systemd/.fixtures.yml new file mode 100644 index 00000000..1d455a31 --- /dev/null +++ b/puppet/modules/systemd/.fixtures.yml @@ -0,0 +1,4 @@ +--- +fixtures: + symlinks: + systemd: "#{source_dir}" \ No newline at end of file diff --git a/puppet/modules/systemd/.gitrepo b/puppet/modules/systemd/.gitrepo index 1548a815..ea68e478 100644 --- a/puppet/modules/systemd/.gitrepo +++ b/puppet/modules/systemd/.gitrepo @@ -6,6 +6,6 @@ [subrepo] remote = https://leap.se/git/puppet_systemd branch = master - commit = 6d47fd4999fe03eba6fb11c4490dcbb90d937900 - parent = 56a771a3008d10720dd05fd815aeafbacdd1e08e + commit = f3c4059603a6ac19f132b0dc47b95e49d9ddc4ba + parent = 77d11c7ddeaeb123bf871bd2bfce0e5ace0c158e cmdver = 0.3.0 diff --git a/puppet/modules/systemd/.puppet-lint.rc b/puppet/modules/systemd/.puppet-lint.rc index d8f5c59e..e09d52f4 100644 --- a/puppet/modules/systemd/.puppet-lint.rc +++ b/puppet/modules/systemd/.puppet-lint.rc @@ -1,5 +1,5 @@ --fail-on-warnings --relative ---no-80chars +--no-140chars --no-documentation --no-class_inherits_from_params_class-check diff --git a/puppet/modules/systemd/.travis.yml b/puppet/modules/systemd/.travis.yml index 467045c5..1d1bedfc 100644 --- a/puppet/modules/systemd/.travis.yml +++ b/puppet/modules/systemd/.travis.yml @@ -1,22 +1,22 @@ --- language: ruby sudo: false +addons: + apt: + packages: + - libaugeas-dev + sources: + - augeas cache: bundler bundler_args: --without system_tests -script: ["bundle exec rake validate", "bundle exec rake lint", "bundle exec rake spec SPEC_OPTS='--format documentation'", "bundle exec rake metadata"] +script: ["bundle exec rake validate", "bundle exec rake lint", "bundle exec rake spec SPEC_OPTS='--format documentation'"] matrix: fast_finish: true include: - - rvm: 1.8.7 - env: PUPPET_GEM_VERSION="~> 3.0" FACTER_GEM_VERSION="~> 1.7.0" - - rvm: 1.9.3 - env: PUPPET_GEM_VERSION="~> 3.0" - - rvm: 2.0.0 - env: PUPPET_GEM_VERSION="~> 3.0" - - rvm: 2.0.0 - env: PUPPET_GEM_VERSION="~> 3.0" FUTURE_PARSER="yes" - - rvm: 2.1.6 + - rvm: 2.1.9 env: PUPPET_GEM_VERSION="~> 4.0" + - rvm: 2.3.1 + env: PUPPET_GEM_VERSION="~> 4" notifications: email: false deploy: @@ -29,4 +29,4 @@ deploy: # all_branches is required to use tags all_branches: true # Only publish if our main Ruby target builds - rvm: 1.9.3 + rvm: 2.1.9 diff --git a/puppet/modules/systemd/CHANGELOG.md b/puppet/modules/systemd/CHANGELOG.md index 11e84399..79b9e646 100644 --- a/puppet/modules/systemd/CHANGELOG.md +++ b/puppet/modules/systemd/CHANGELOG.md @@ -1,5 +1,22 @@ # Change Log +## [0.4.0](https://forge.puppetlabs.com/camptocamp/systemd/0.4.0) (2016-08-18) +[Full Changelog](https://github.com/camptocamp/puppet-systemd/compare/0.3.0...0.4.0) + +- Deprecate Ruby 1.8 tests +- Only use awk instead of grep and awk [\#9](https://github.com/camptocamp/puppet-systemd/pull/9) ([igalic](https://github.com/igalic)) +- Add LICENSE (fix #11) +- Add target param for the unit file [\#10](https://github.com/camptocamp/puppet-systemd/pull/10) ([tampakrap](https://github.com/tampakrap)) + +## [0.3.0](https://forge.puppetlabs.com/camptocamp/systemd/0.3.0) (2016-05-16) +[Full Changelog](https://github.com/camptocamp/puppet-systemd/compare/0.2.2...0.3.0) + +**Implemented enhancements:** + +- Shortcut for creating unit files / tmpfiles [\#4](https://github.com/camptocamp/puppet-systemd/pull/4) ([felixb](https://github.com/felixb)) +- Add systemd facts [\#6](https://github.com/camptocamp/puppet-systemd/pull/6) ([roidelapluie](https://github.com/roidelapluie)) + + ## [0.2.2](https://forge.puppetlabs.com/camptocamp/systemd/0.2.2) (2015-08-25) [Full Changelog](https://github.com/camptocamp/puppet-systemd/compare/0.2.1...0.2.2) @@ -60,6 +77,3 @@ \* *This Change Log was automatically generated by [github_changelog_generator](https://github.com/skywinder/Github-Changelog-Generator)* - - -\* *This Change Log was automatically generated by [github_changelog_generator](https://github.com/skywinder/Github-Changelog-Generator)* \ No newline at end of file diff --git a/puppet/modules/systemd/Gemfile b/puppet/modules/systemd/Gemfile index 0cb59337..377d0c16 100644 --- a/puppet/modules/systemd/Gemfile +++ b/puppet/modules/systemd/Gemfile @@ -2,7 +2,7 @@ source ENV['GEM_SOURCE'] || "https://rubygems.org" group :development, :unit_tests do gem 'rake', :require => false - gem 'rspec', '< 3.2', :require => false if RUBY_VERSION =~ /^1.8/ + gem 'rspec', :require => false gem 'rspec-puppet', :require => false gem 'puppetlabs_spec_helper', :require => false gem 'metadata-json-lint', :require => false @@ -10,26 +10,26 @@ group :development, :unit_tests do gem 'puppet-lint-unquoted_string-check', :require => false gem 'puppet-lint-empty_string-check', :require => false gem 'puppet-lint-spaceship_operator_without_tag-check', :require => false - gem 'puppet-lint-variable_contains_upcase', :require => false gem 'puppet-lint-absolute_classname-check', :require => false gem 'puppet-lint-undef_in_function-check', :require => false gem 'puppet-lint-leading_zero-check', :require => false gem 'puppet-lint-trailing_comma-check', :require => false gem 'puppet-lint-file_ensure-check', :require => false gem 'puppet-lint-version_comparison-check', :require => false - gem 'puppet-lint-fileserver-check', :require => false gem 'puppet-lint-file_source_rights-check', :require => false gem 'puppet-lint-alias-check', :require => false gem 'rspec-puppet-facts', :require => false - gem 'github_changelog_generator', :require => false, :git => 'https://github.com/raphink/github-changelog-generator.git', :branch => 'dev/all_patches' if RUBY_VERSION !~ /^1.8/ - gem 'puppet-blacksmith', :require => false if RUBY_VERSION !~ /^1.8/ + gem 'ruby-augeas', :require => false + gem 'puppet-blacksmith', :require => false if RUBY_VERSION !~ /^1\./ + gem 'json_pure', '< 2.0.2', :require => false end group :system_tests do - gem 'beaker', :require => false - gem 'beaker-rspec', :require => false - gem 'beaker_spec_helper', :require => false - gem 'serverspec', :require => false + gem 'beaker', :require => false + gem 'beaker-rspec', '> 5', :require => false + gem 'beaker_spec_helper', :require => false + gem 'serverspec', :require => false + gem 'specinfra', :require => false end if facterversion = ENV['FACTER_GEM_VERSION'] diff --git a/puppet/modules/systemd/HISTORY.md b/puppet/modules/systemd/HISTORY.md index c7bf2b4e..aee8ad5e 100644 --- a/puppet/modules/systemd/HISTORY.md +++ b/puppet/modules/systemd/HISTORY.md @@ -1,3 +1,14 @@ +# Change Log + +## [0.3.0](https://forge.puppetlabs.com/camptocamp/systemd/0.3.0) (2016-05-16) +[Full Changelog](https://github.com/camptocamp/puppet-systemd/compare/0.2.2...0.3.0) + +**Implemented enhancements:** + +- Shortcut for creating unit files / tmpfiles [\#4](https://github.com/camptocamp/puppet-systemd/pull/4) ([felixb](https://github.com/felixb)) +- Add systemd facts [\#6](https://github.com/camptocamp/puppet-systemd/pull/6) ([roidelapluie](https://github.com/roidelapluie)) + + ## [0.2.2](https://forge.puppetlabs.com/camptocamp/systemd/0.2.2) (2015-08-25) [Full Changelog](https://github.com/camptocamp/puppet-systemd/compare/0.2.1...0.2.2) @@ -5,6 +16,7 @@ - Add 'systemd-tmpfiles-create' [\#1](https://github.com/camptocamp/puppet-systemd/pull/1) ([roidelapluie](https://github.com/roidelapluie)) + ## [0.2.1](https://forge.puppetlabs.com/camptocamp/systemd/0.2.1) (2015-08-21) [Full Changelog](https://github.com/camptocamp/puppet-systemd/compare/0.2.0...0.2.1) @@ -56,7 +68,4 @@ - Confine rspec pinning to ruby 1.8 -\* *This Change Log was automatically generated by [github_changelog_generator](https://github.com/skywinder/Github-Changelog-Generator)* - - \* *This Change Log was automatically generated by [github_changelog_generator](https://github.com/skywinder/Github-Changelog-Generator)* diff --git a/puppet/modules/systemd/LICENSE b/puppet/modules/systemd/LICENSE new file mode 100644 index 00000000..8d968b6c --- /dev/null +++ b/puppet/modules/systemd/LICENSE @@ -0,0 +1,201 @@ + Apache License + Version 2.0, January 2004 + http://www.apache.org/licenses/ + + TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION + + 1. Definitions. + + "License" shall mean the terms and conditions for use, reproduction, + and distribution as defined by Sections 1 through 9 of this document. + + "Licensor" shall mean the copyright owner or entity authorized by + the copyright owner that is granting the License. + + "Legal Entity" shall mean the union of the acting entity and all + other entities that control, are controlled by, or are under common + control with that entity. For the purposes of this definition, + "control" means (i) the power, direct or indirect, to cause the + direction or management of such entity, whether by contract or + otherwise, or (ii) ownership of fifty percent (50%) or more of the + outstanding shares, or (iii) beneficial ownership of such entity. + + "You" (or "Your") shall mean an individual or Legal Entity + exercising permissions granted by this License. + + "Source" form shall mean the preferred form for making modifications, + including but not limited to software source code, documentation + source, and configuration files. + + "Object" form shall mean any form resulting from mechanical + transformation or translation of a Source form, including but + not limited to compiled object code, generated documentation, + and conversions to other media types. + + "Work" shall mean the work of authorship, whether in Source or + Object form, made available under the License, as indicated by a + copyright notice that is included in or attached to the work + (an example is provided in the Appendix below). + + "Derivative Works" shall mean any work, whether in Source or Object + form, that is based on (or derived from) the Work and for which the + editorial revisions, annotations, elaborations, or other modifications + represent, as a whole, an original work of authorship. For the purposes + of this License, Derivative Works shall not include works that remain + separable from, or merely link (or bind by name) to the interfaces of, + the Work and Derivative Works thereof. + + "Contribution" shall mean any work of authorship, including + the original version of the Work and any modifications or additions + to that Work or Derivative Works thereof, that is intentionally + submitted to Licensor for inclusion in the Work by the copyright owner + or by an individual or Legal Entity authorized to submit on behalf of + the copyright owner. For the purposes of this definition, "submitted" + means any form of electronic, verbal, or written communication sent + to the Licensor or its representatives, including but not limited to + communication on electronic mailing lists, source code control systems, + and issue tracking systems that are managed by, or on behalf of, the + Licensor for the purpose of discussing and improving the Work, but + excluding communication that is conspicuously marked or otherwise + designated in writing by the copyright owner as "Not a Contribution." + + "Contributor" shall mean Licensor and any individual or Legal Entity + on behalf of whom a Contribution has been received by Licensor and + subsequently incorporated within the Work. + + 2. Grant of Copyright License. Subject to the terms and conditions of + this License, each Contributor hereby grants to You a perpetual, + worldwide, non-exclusive, no-charge, royalty-free, irrevocable + copyright license to reproduce, prepare Derivative Works of, + publicly display, publicly perform, sublicense, and distribute the + Work and such Derivative Works in Source or Object form. + + 3. Grant of Patent License. Subject to the terms and conditions of + this License, each Contributor hereby grants to You a perpetual, + worldwide, non-exclusive, no-charge, royalty-free, irrevocable + (except as stated in this section) patent license to make, have made, + use, offer to sell, sell, import, and otherwise transfer the Work, + where such license applies only to those patent claims licensable + by such Contributor that are necessarily infringed by their + Contribution(s) alone or by combination of their Contribution(s) + with the Work to which such Contribution(s) was submitted. If You + institute patent litigation against any entity (including a + cross-claim or counterclaim in a lawsuit) alleging that the Work + or a Contribution incorporated within the Work constitutes direct + or contributory patent infringement, then any patent licenses + granted to You under this License for that Work shall terminate + as of the date such litigation is filed. + + 4. Redistribution. You may reproduce and distribute copies of the + Work or Derivative Works thereof in any medium, with or without + modifications, and in Source or Object form, provided that You + meet the following conditions: + + (a) You must give any other recipients of the Work or + Derivative Works a copy of this License; and + + (b) You must cause any modified files to carry prominent notices + stating that You changed the files; and + + (c) You must retain, in the Source form of any Derivative Works + that You distribute, all copyright, patent, trademark, and + attribution notices from the Source form of the Work, + excluding those notices that do not pertain to any part of + the Derivative Works; and + + (d) If the Work includes a "NOTICE" text file as part of its + distribution, then any Derivative Works that You distribute must + include a readable copy of the attribution notices contained + within such NOTICE file, excluding those notices that do not + pertain to any part of the Derivative Works, in at least one + of the following places: within a NOTICE text file distributed + as part of the Derivative Works; within the Source form or + documentation, if provided along with the Derivative Works; or, + within a display generated by the Derivative Works, if and + wherever such third-party notices normally appear. The contents + of the NOTICE file are for informational purposes only and + do not modify the License. You may add Your own attribution + notices within Derivative Works that You distribute, alongside + or as an addendum to the NOTICE text from the Work, provided + that such additional attribution notices cannot be construed + as modifying the License. + + You may add Your own copyright statement to Your modifications and + may provide additional or different license terms and conditions + for use, reproduction, or distribution of Your modifications, or + for any such Derivative Works as a whole, provided Your use, + reproduction, and distribution of the Work otherwise complies with + the conditions stated in this License. + + 5. Submission of Contributions. Unless You explicitly state otherwise, + any Contribution intentionally submitted for inclusion in the Work + by You to the Licensor shall be under the terms and conditions of + this License, without any additional terms or conditions. + Notwithstanding the above, nothing herein shall supersede or modify + the terms of any separate license agreement you may have executed + with Licensor regarding such Contributions. + + 6. Trademarks. This License does not grant permission to use the trade + names, trademarks, service marks, or product names of the Licensor, + except as required for reasonable and customary use in describing the + origin of the Work and reproducing the content of the NOTICE file. + + 7. Disclaimer of Warranty. Unless required by applicable law or + agreed to in writing, Licensor provides the Work (and each + Contributor provides its Contributions) on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or + implied, including, without limitation, any warranties or conditions + of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A + PARTICULAR PURPOSE. You are solely responsible for determining the + appropriateness of using or redistributing the Work and assume any + risks associated with Your exercise of permissions under this License. + + 8. Limitation of Liability. In no event and under no legal theory, + whether in tort (including negligence), contract, or otherwise, + unless required by applicable law (such as deliberate and grossly + negligent acts) or agreed to in writing, shall any Contributor be + liable to You for damages, including any direct, indirect, special, + incidental, or consequential damages of any character arising as a + result of this License or out of the use or inability to use the + Work (including but not limited to damages for loss of goodwill, + work stoppage, computer failure or malfunction, or any and all + other commercial damages or losses), even if such Contributor + has been advised of the possibility of such damages. + + 9. Accepting Warranty or Additional Liability. While redistributing + the Work or Derivative Works thereof, You may choose to offer, + and charge a fee for, acceptance of support, warranty, indemnity, + or other liability obligations and/or rights consistent with this + License. However, in accepting such obligations, You may act only + on Your own behalf and on Your sole responsibility, not on behalf + of any other Contributor, and only if You agree to indemnify, + defend, and hold each Contributor harmless for any liability + incurred by, or claims asserted against, such Contributor by reason + of your accepting any such warranty or additional liability. + + END OF TERMS AND CONDITIONS + + APPENDIX: How to apply the Apache License to your work. + + To apply the Apache License to your work, attach the following + boilerplate notice, with the fields enclosed by brackets "[]" + replaced with your own identifying information. (Don't include + the brackets!) The text should be enclosed in the appropriate + comment syntax for the file format. We also recommend that a + file or class name and description of purpose be included on the + same "printed page" as the copyright notice for easier + identification within third-party archives. + + Copyright [yyyy] [name of copyright owner] + + Licensed under the Apache License, Version 2.0 (the "License"); + you may not use this file except in compliance with the License. + You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. diff --git a/puppet/modules/systemd/README.md b/puppet/modules/systemd/README.md index f70bcb0c..51bf5cde 100644 --- a/puppet/modules/systemd/README.md +++ b/puppet/modules/systemd/README.md @@ -5,11 +5,23 @@ ## Overview -This module declares exec resources that you can use when you change systemd units or configuration files. +This module declares exec resources to create global sync points for reloading systemd. -## Examples +## Usage and examples -### systemctl --daemon-reload +There are two ways to use this module. + +### unit files + +Let this module handle file creation and systemd reloading. + +```puppet +::systemd::unit_file { 'foo.service': + source => "puppet:///modules/${module_name}/foo.service", +} +``` + +Or handle file creation yourself and trigger systemd. ```puppet include ::systemd @@ -23,7 +35,17 @@ file { '/usr/lib/systemd/system/foo.service': Exec['systemctl-daemon-reload'] ``` -### systemd-tmpfiles --create +### tmpfiles + +Let this module handle file creation and systemd reloading + +```puppet +::systemd::tmpfile { 'foo.conf': + source => "puppet:///modules/${module_name}/foo.conf", +} +``` + +Or handle file creation yourself and trigger systemd. ```puppet include ::systemd @@ -36,3 +58,24 @@ file { '/etc/tmpfiles.d/foo.conf': } ~> Exec['systemd-tmpfiles-create'] ``` + +### service limits + +Manage soft and hard limits on various resources for executed processes. + +```puppet +::systemd::service_limits { 'foo.service': + limits => { + LimitNOFILE => 8192, + LimitNPROC => 16384 + } +} +``` + +Or provide the configuration file yourself. Systemd reloading and restarting of the service are handled by the module. + +```puppet +::systemd::service_limits { 'foo.service': + source => "puppet:///modules/${module_name}/foo.conf", +} +``` diff --git a/puppet/modules/systemd/Rakefile b/puppet/modules/systemd/Rakefile index adcac180..aa7b8a15 100644 --- a/puppet/modules/systemd/Rakefile +++ b/puppet/modules/systemd/Rakefile @@ -4,20 +4,14 @@ require 'puppet-lint/tasks/puppet-lint' Rake::Task[:lint].clear PuppetLint::RakeTask.new :lint do |config| config.ignore_paths = ["spec/**/*.pp", "pkg/**/*.pp", "vendor/**/*.pp"] - config.disable_checks = ['80chars'] + config.disable_checks = ['140chars'] config.fail_on_warnings = true end PuppetSyntax.exclude_paths = ["spec/fixtures/**/*.pp", "vendor/**/*"] # Publishing tasks -unless RUBY_VERSION =~ /^1\.8/ +unless RUBY_VERSION =~ /^1\./ require 'puppet_blacksmith' require 'puppet_blacksmith/rake_tasks' - require 'github_changelog_generator/task' - GitHubChangelogGenerator::RakeTask.new :changelog do |config| - m = Blacksmith::Modulefile.new - config.future_release = m.version - config.release_url = "https://forge.puppetlabs.com/#{m.author}/#{m.name}/%s" - end end diff --git a/puppet/modules/systemd/lib/facter/systemd.rb b/puppet/modules/systemd/lib/facter/systemd.rb new file mode 100644 index 00000000..4361f775 --- /dev/null +++ b/puppet/modules/systemd/lib/facter/systemd.rb @@ -0,0 +1,35 @@ +# Fact: systemd +# +# Purpose: +# Determine whether SystemD is the init system on the node +# +# Resolution: +# Check the name of the process 1 (ps -p 1) +# +# Caveats: +# + +# Fact: systemd-version +# +# Purpose: +# Determine the version of systemd installed +# +# Resolution: +# Check the output of systemctl --version +# +# Caveats: +# + +Facter.add(:systemd) do + confine :kernel => :linux + setcode do + Facter::Util::Resolution.exec('ps -p 1 -o comm=') == 'systemd' + end +end + +Facter.add(:systemd_version) do + confine :systemd => true + setcode do + Facter::Util::Resolution.exec("systemctl --version | awk '/systemd/{ print $2 }'") + end +end diff --git a/puppet/modules/systemd/manifests/enable.pp b/puppet/modules/systemd/manifests/enable.pp new file mode 100644 index 00000000..e1bee18a --- /dev/null +++ b/puppet/modules/systemd/manifests/enable.pp @@ -0,0 +1,8 @@ +# enables a systemd resource +define systemd::enable () { + + exec { "enable_systemd_${name}": + refreshonly => true, + command => "/bin/systemctl enable ${name}" + } +} diff --git a/puppet/modules/systemd/manifests/init.pp b/puppet/modules/systemd/manifests/init.pp index 5e6ad792..e669f093 100644 --- a/puppet/modules/systemd/manifests/init.pp +++ b/puppet/modules/systemd/manifests/init.pp @@ -1,4 +1,8 @@ -class systemd { +# -- Class systemd +# This module allows triggering systemd commands once for all modules +class systemd ( + $service_limits = {} +){ Exec { refreshonly => true, @@ -15,4 +19,6 @@ class systemd { command => 'systemd-tmpfiles --create', } + create_resources('systemd::service_limits', $service_limits, {}) + } diff --git a/puppet/modules/systemd/manifests/service_limits.pp b/puppet/modules/systemd/manifests/service_limits.pp new file mode 100644 index 00000000..a9cdc25a --- /dev/null +++ b/puppet/modules/systemd/manifests/service_limits.pp @@ -0,0 +1,50 @@ +# -- Define: systemd::service_limits +# Creates a custom config file and reloads systemd +define systemd::service_limits( + $ensure = file, + $path = '/etc/systemd/system', + $limits = undef, + $source = undef, + $restart_service = true +) { + include ::systemd + + if $limits { + validate_hash($limits) + $content = template('systemd/limits.erb') + } + else { + $content = undef + } + + if $limits and $source { + fail('You may not supply both limits and source parameters to systemd::service_limits') + } elsif $limits == undef and $source == undef { + fail('You must supply either the limits or source parameter to systemd::service_limits') + } + + file { "${path}/${title}.d/": + ensure => 'directory', + owner => 'root', + group => 'root', + } + -> + file { "${path}/${title}.d/limits.conf": + ensure => $ensure, + content => $content, + source => $source, + owner => 'root', + group => 'root', + mode => '0444', + notify => Exec['systemctl-daemon-reload'], + } + + if $restart_service { + exec { "systemctl restart ${title}": + path => $::path, + refreshonly => true, + subscribe => File["${path}/${title}.d/limits.conf"], + require => Exec['systemctl-daemon-reload'], + } + } +} diff --git a/puppet/modules/systemd/manifests/tmpfile.pp b/puppet/modules/systemd/manifests/tmpfile.pp new file mode 100644 index 00000000..c4d1a05f --- /dev/null +++ b/puppet/modules/systemd/manifests/tmpfile.pp @@ -0,0 +1,20 @@ +# -- Define: systemd::tmpfile +# Creates a tmpfile and reloads systemd +define systemd::tmpfile( + $ensure = file, + $path = '/etc/tmpfiles.d', + $content = undef, + $source = undef, +) { + include ::systemd + + file { "${path}/${title}": + ensure => $ensure, + content => $content, + source => $source, + owner => 'root', + group => 'root', + mode => '0444', + notify => Exec['systemd-tmpfiles-create'], + } +} \ No newline at end of file diff --git a/puppet/modules/systemd/manifests/unit_file.pp b/puppet/modules/systemd/manifests/unit_file.pp new file mode 100644 index 00000000..94bc845b --- /dev/null +++ b/puppet/modules/systemd/manifests/unit_file.pp @@ -0,0 +1,22 @@ +# -- Define: systemd::unit_file +# Creates a unit file and reloads systemd +define systemd::unit_file( + $ensure = file, + $path = '/etc/systemd/system', + $content = undef, + $source = undef, + $target = undef, +) { + include ::systemd + + file { "${path}/${title}": + ensure => $ensure, + content => $content, + source => $source, + target => $target, + owner => 'root', + group => 'root', + mode => '0444', + notify => Exec['systemctl-daemon-reload'], + } +} diff --git a/puppet/modules/systemd/metadata.json b/puppet/modules/systemd/metadata.json index abdd481e..08951efb 100644 --- a/puppet/modules/systemd/metadata.json +++ b/puppet/modules/systemd/metadata.json @@ -1,6 +1,6 @@ { "name": "camptocamp-systemd", - "version": "0.2.2", + "version": "0.4.0", "author": "camptocamp", "summary": "Puppet Systemd module", "license": "Apache-2.0", diff --git a/puppet/modules/systemd/spec/acceptance/nodesets/centos-5.yml b/puppet/modules/systemd/spec/acceptance/nodesets/centos-5.yml new file mode 100644 index 00000000..a26f27fc --- /dev/null +++ b/puppet/modules/systemd/spec/acceptance/nodesets/centos-5.yml @@ -0,0 +1,16 @@ +HOSTS: + centos-5-x64: + default_apply_opts: + order: random + strict_variables: + platform: el-5-x86_64 + hypervisor : docker + image: tianon/centos:5.10 + docker_preserve_image: true + docker_cmd: '["/sbin/init"]' + docker_image_commands: + - 'yum install -y crontabs tar wget which' + - 'sed -i -e "/mingetty/d" /etc/inittab' +CONFIG: + type: aio + log_level: debug diff --git a/puppet/modules/systemd/spec/acceptance/nodesets/centos-6.yml b/puppet/modules/systemd/spec/acceptance/nodesets/centos-6.yml new file mode 100644 index 00000000..71e23cd8 --- /dev/null +++ b/puppet/modules/systemd/spec/acceptance/nodesets/centos-6.yml @@ -0,0 +1,17 @@ +HOSTS: + centos-6-x64: + default_apply_opts: + order: random + strict_variables: + platform: el-6-x86_64 + hypervisor : docker + image: centos:6 + docker_preserve_image: true + docker_cmd: '["/sbin/init"]' + docker_image_commands: + - 'rm -rf /var/run/network/*' + - 'yum install -y crontabs tar wget' + - 'rm /etc/init/tty.conf' +CONFIG: + type: aio + log_level: debug diff --git a/puppet/modules/systemd/spec/acceptance/nodesets/centos-7.yml b/puppet/modules/systemd/spec/acceptance/nodesets/centos-7.yml new file mode 100644 index 00000000..a8fa4686 --- /dev/null +++ b/puppet/modules/systemd/spec/acceptance/nodesets/centos-7.yml @@ -0,0 +1,15 @@ +HOSTS: + centos-7-x64: + default_apply_opts: + order: random + strict_variables: + platform: el-7-x86_64 + hypervisor : docker + image: centos:7 + docker_preserve_image: true + docker_cmd: '["/usr/sbin/init"]' + docker_image_commands: + - 'yum install -y crontabs tar wget iproute' +CONFIG: + type: aio + log_level: debug diff --git a/puppet/modules/systemd/spec/acceptance/nodesets/debian-6.yml b/puppet/modules/systemd/spec/acceptance/nodesets/debian-6.yml new file mode 100644 index 00000000..d7b02756 --- /dev/null +++ b/puppet/modules/systemd/spec/acceptance/nodesets/debian-6.yml @@ -0,0 +1,15 @@ +HOSTS: + debian-6-x64: + default_apply_opts: + order: random + strict_variables: + platform: debian-6-amd64 + hypervisor : docker + image: debian/eol:squeeze + docker_preserve_image: true + docker_cmd: '["/sbin/init"]' + docker_image_commands: + - 'apt-get install -y cron locales-all net-tools wget' +CONFIG: + type: aio + log_level: debug diff --git a/puppet/modules/systemd/spec/acceptance/nodesets/debian-7.yml b/puppet/modules/systemd/spec/acceptance/nodesets/debian-7.yml new file mode 100644 index 00000000..9591ea77 --- /dev/null +++ b/puppet/modules/systemd/spec/acceptance/nodesets/debian-7.yml @@ -0,0 +1,15 @@ +HOSTS: + debian-7-x64: + default_apply_opts: + order: random + strict_variables: + platform: debian-7-amd64 + hypervisor : docker + image: debian:7 + docker_preserve_image: true + docker_cmd: '["/sbin/init"]' + docker_image_commands: + - 'apt-get install -y cron locales-all net-tools wget' +CONFIG: + type: aio + log_level: debug diff --git a/puppet/modules/systemd/spec/acceptance/nodesets/debian-8.yml b/puppet/modules/systemd/spec/acceptance/nodesets/debian-8.yml new file mode 100644 index 00000000..5fb24c61 --- /dev/null +++ b/puppet/modules/systemd/spec/acceptance/nodesets/debian-8.yml @@ -0,0 +1,16 @@ +HOSTS: + debian-8-x64: + default_apply_opts: + order: random + strict_variables: + platform: debian-8-amd64 + hypervisor : docker + image: debian:8 + docker_preserve_image: true + docker_cmd: '["/sbin/init"]' + docker_image_commands: + - 'apt-get install -y cron locales-all net-tools wget' + - 'rm -f /usr/sbin/policy-rc.d' +CONFIG: + type: aio + log_level: debug diff --git a/puppet/modules/systemd/spec/acceptance/nodesets/ubuntu-12.04.yml b/puppet/modules/systemd/spec/acceptance/nodesets/ubuntu-12.04.yml new file mode 100644 index 00000000..594e1771 --- /dev/null +++ b/puppet/modules/systemd/spec/acceptance/nodesets/ubuntu-12.04.yml @@ -0,0 +1,16 @@ +HOSTS: + ubuntu-1204-x64: + default_apply_opts: + order: random + strict_variables: + platform: ubuntu-12.04-amd64 + hypervisor : docker + image: ubuntu:12.04 + docker_preserve_image: true + docker_cmd: '["/sbin/init"]' + docker_image_commands: + - 'apt-get install -y net-tools wget' + - 'locale-gen en_US.UTF-8' +CONFIG: + type: aio + log_level: debug diff --git a/puppet/modules/systemd/spec/acceptance/nodesets/ubuntu-14.04.yml b/puppet/modules/systemd/spec/acceptance/nodesets/ubuntu-14.04.yml new file mode 100644 index 00000000..2b293c99 --- /dev/null +++ b/puppet/modules/systemd/spec/acceptance/nodesets/ubuntu-14.04.yml @@ -0,0 +1,18 @@ +HOSTS: + ubuntu-1404-x64: + default_apply_opts: + order: random + strict_variables: + platform: ubuntu-14.04-amd64 + hypervisor : docker + image: ubuntu:14.04 + docker_preserve_image: true + docker_cmd: '["/sbin/init"]' + docker_image_commands: + - 'rm /usr/sbin/policy-rc.d' + - 'rm /sbin/initctl; dpkg-divert --rename --remove /sbin/initctl' + - 'apt-get install -y net-tools wget' + - 'locale-gen en_US.UTF-8' +CONFIG: + type: aio + log_level: debug diff --git a/puppet/modules/systemd/spec/acceptance/nodesets/ubuntu-14.10.yml b/puppet/modules/systemd/spec/acceptance/nodesets/ubuntu-14.10.yml new file mode 100644 index 00000000..7ce09b2a --- /dev/null +++ b/puppet/modules/systemd/spec/acceptance/nodesets/ubuntu-14.10.yml @@ -0,0 +1,18 @@ +HOSTS: + ubuntu-1410-x64: + default_apply_opts: + order: random + strict_variables: + platform: ubuntu-14.10-amd64 + hypervisor : docker + image: ubuntu:14.10 + docker_preserve_image: true + docker_cmd: '["/sbin/init"]' + docker_image_commands: + - 'rm /usr/sbin/policy-rc.d' + - 'rm /sbin/initctl; dpkg-divert --rename --remove /sbin/initctl' + - 'apt-get install -y net-tools wget' + - 'locale-gen en_US.UTF-8' +CONFIG: + type: aio + log_level: debug diff --git a/puppet/modules/systemd/spec/acceptance/nodesets/ubuntu-15.04.yml b/puppet/modules/systemd/spec/acceptance/nodesets/ubuntu-15.04.yml new file mode 100644 index 00000000..329f3319 --- /dev/null +++ b/puppet/modules/systemd/spec/acceptance/nodesets/ubuntu-15.04.yml @@ -0,0 +1,16 @@ +HOSTS: + ubuntu-1504-x64: + default_apply_opts: + order: random + strict_variables: + platform: ubuntu-15.04-amd64 + hypervisor : docker + image: ubuntu:15.04 + docker_preserve_image: true + docker_cmd: '["/sbin/init"]' + docker_image_commands: + - 'apt-get install -y net-tools wget' + - 'locale-gen en_US.UTF-8' +CONFIG: + type: aio + log_level: debug diff --git a/puppet/modules/systemd/spec/acceptance/nodesets/ubuntu-15.10.yml b/puppet/modules/systemd/spec/acceptance/nodesets/ubuntu-15.10.yml new file mode 100644 index 00000000..487795a3 --- /dev/null +++ b/puppet/modules/systemd/spec/acceptance/nodesets/ubuntu-15.10.yml @@ -0,0 +1,16 @@ +HOSTS: + ubuntu-1510-x64: + default_apply_opts: + order: random + strict_variables: + platform: ubuntu-15.10-amd64 + hypervisor : docker + image: ubuntu:15.10 + docker_preserve_image: true + docker_cmd: '["/sbin/init"]' + docker_image_commands: + - 'apt-get install -y net-tools wget' + - 'locale-gen en_US.UTF-8' +CONFIG: + type: aio + log_level: debug diff --git a/puppet/modules/systemd/spec/acceptance/nodesets/ubuntu-16.04.yml b/puppet/modules/systemd/spec/acceptance/nodesets/ubuntu-16.04.yml new file mode 100644 index 00000000..6c32b96d --- /dev/null +++ b/puppet/modules/systemd/spec/acceptance/nodesets/ubuntu-16.04.yml @@ -0,0 +1,16 @@ +HOSTS: + ubuntu-1604-x64: + default_apply_opts: + order: random + strict_variables: + platform: ubuntu-16.04-amd64 + hypervisor : docker + image: ubuntu:16.04 + docker_preserve_image: true + docker_cmd: '["/sbin/init"]' + docker_image_commands: + - 'apt-get install -y net-tools wget' + - 'locale-gen en_US.UTF-8' +CONFIG: + type: aio + log_level: debug diff --git a/puppet/modules/systemd/spec/defines/tmpfile_spec.rb b/puppet/modules/systemd/spec/defines/tmpfile_spec.rb new file mode 100644 index 00000000..4eb22acd --- /dev/null +++ b/puppet/modules/systemd/spec/defines/tmpfile_spec.rb @@ -0,0 +1,48 @@ +require 'spec_helper' + +describe 'systemd::tmpfile' do + + let(:facts) { { + :path => '/usr/bin', + } } + + context 'default params' do + + let(:title) { 'fancy.conf' } + + it 'creates the tmpfile' do + should contain_file('/etc/tmpfiles.d/fancy.conf').with({ + 'ensure' => 'file', + 'owner' => 'root', + 'group' => 'root', + 'mode' => '0444', + }) + end + + it 'triggers systemd daemon-reload' do + should contain_class('systemd') + should contain_file('/etc/tmpfiles.d/fancy.conf').with_notify("Exec[systemd-tmpfiles-create]") + end + end + + context 'with params' do + let(:title) { 'fancy.conf' } + + let(:params) { { + :ensure => 'absent', + :path => '/etc/tmpfiles.d/foo', + :content => 'some-content', + :source => 'some-source', + } } + + it 'creates the unit file' do + should contain_file('/etc/tmpfiles.d/foo/fancy.conf').with({ + 'ensure' => 'absent', + 'content' => 'some-content', + 'source' => 'some-source', + }) + end + + end + +end diff --git a/puppet/modules/systemd/spec/defines/unit_file_spec.rb b/puppet/modules/systemd/spec/defines/unit_file_spec.rb new file mode 100644 index 00000000..88a0122c --- /dev/null +++ b/puppet/modules/systemd/spec/defines/unit_file_spec.rb @@ -0,0 +1,50 @@ +require 'spec_helper' + +describe 'systemd::unit_file' do + + let(:facts) { { + :path => '/usr/bin', + } } + + context 'default params' do + + let(:title) { 'fancy.service' } + + it 'creates the unit file' do + should contain_file('/etc/systemd/system/fancy.service').with({ + 'ensure' => 'file', + 'owner' => 'root', + 'group' => 'root', + 'mode' => '0444', + }) + end + + it 'triggers systemd daemon-reload' do + should contain_class('systemd') + should contain_file('/etc/systemd/system/fancy.service').with_notify("Exec[systemctl-daemon-reload]") + end + end + + context 'with params' do + let(:title) { 'fancy.service' } + + let(:params) { { + :ensure => 'absent', + :path => '/usr/lib/systemd/system', + :content => 'some-content', + :source => 'some-source', + :target => 'some-target', + } } + + it 'creates the unit file' do + should contain_file('/usr/lib/systemd/system/fancy.service').with({ + 'ensure' => 'absent', + 'content' => 'some-content', + 'source' => 'some-source', + 'target' => 'some-target', + }) + end + + end + +end diff --git a/puppet/modules/systemd/spec/unit/facter/systemd_spec.rb b/puppet/modules/systemd/spec/unit/facter/systemd_spec.rb new file mode 100644 index 00000000..a7b62410 --- /dev/null +++ b/puppet/modules/systemd/spec/unit/facter/systemd_spec.rb @@ -0,0 +1,41 @@ +require "spec_helper" + +describe Facter::Util::Fact do + before { + Facter.clear + } + + describe "systemd" do + context 'returns true when systemd present' do + before do + Facter.fact(:kernel).stubs(:value).returns(:linux) + end + let(:facts) { {:kernel => :linux} } + it do + Facter::Util::Resolution.expects(:exec).with('ps -p 1 -o comm=').returns('systemd') + expect(Facter.value(:systemd)).to eq(true) + end + end + context 'returns false when systemd not present' do + before do + Facter.fact(:kernel).stubs(:value).returns(:linux) + end + let(:facts) { {:kernel => :linux} } + it do + Facter::Util::Resolution.expects(:exec).with('ps -p 1 -o comm=').returns('init') + expect(Facter.value(:systemd)).to eq(false) + end + end + + context 'returns nil when kernel is not linux' do + before do + Facter.fact(:kernel).stubs(:value).returns(:windows) + end + let(:facts) { {:kernel => :windows} } + it do + Facter::Util::Resolution.expects(:exec).with('ps -p 1 -o comm=').never + expect(Facter.value(:systemd)).to be_nil + end + end + end +end diff --git a/puppet/modules/systemd/spec/unit/facter/systemd_version_spec.rb b/puppet/modules/systemd/spec/unit/facter/systemd_version_spec.rb new file mode 100644 index 00000000..5007dc69 --- /dev/null +++ b/puppet/modules/systemd/spec/unit/facter/systemd_version_spec.rb @@ -0,0 +1,31 @@ +require "spec_helper" + +describe Facter::Util::Fact do + before { + Facter.clear + } + + describe "systemd_version" do + context 'returns version when systemd fact present' do + before do + Facter.fact(:systemd).stubs(:value).returns(true) + end + let(:facts) { {:systemd => true} } + it do + Facter::Util::Resolution.expects(:exec).with("systemctl --version | awk '/systemd/{ print $2 }'").returns('229') + expect(Facter.value(:systemd_version)).to eq('229') + end + end + context 'returns nil when systemd fact not present' do + before do + Facter.fact(:systemd).stubs(:value).returns(false) + end + let(:facts) { {:systemd => false } } + it do + Facter::Util::Resolution.stubs(:exec) + Facter::Util::Resolution.expects(:exec).with("systemctl --version | awk '/systemd/{ print $2 }'").never + expect(Facter.value(:systemd_version)).to eq(nil) + end + end + end +end diff --git a/puppet/modules/systemd/templates/limits.erb b/puppet/modules/systemd/templates/limits.erb new file mode 100644 index 00000000..3caf5867 --- /dev/null +++ b/puppet/modules/systemd/templates/limits.erb @@ -0,0 +1,26 @@ +# This file is created by Puppet +[Service] +<% +[ + 'LimitCPU', + 'LimitFSIZE', + 'LimitDATA', + 'LimitSTACK', + 'LimitCORE', + 'LimitRSS', + 'LimitNOFILE', + 'LimitAS', + 'LimitNPROC', + 'LimitMEMLOCK', + 'LimitLOCKS', + 'LimitSIGPENDING', + 'LimitMSGQUEUE', + 'LimitNICE', + 'LimitRTPRIO', + 'LimitRTTIME' +].each do |d| +if @limits[d] -%> +<%= d %>=<%= @limits[d] %> +<% +end +end %> -- cgit v1.2.3 From 5126584fe40f08b779a6f93e2d1b6b57bae29700 Mon Sep 17 00:00:00 2001 From: Tulio Casagrande Date: Mon, 16 Jan 2017 15:15:41 -0200 Subject: Add apache auto-restart extension file --- puppet/modules/site_apache/files/auto_restart.conf | 2 ++ puppet/modules/site_apache/manifests/common.pp | 1 + puppet/modules/site_apache/manifests/common/extensions.pp | 14 ++++++++++++++ puppet/modules/site_apache/spec/classes/extensions.rb | 7 +++++++ puppet/modules/site_apache/spec/spec_helper.rb | 6 ++++++ 5 files changed, 30 insertions(+) create mode 100644 puppet/modules/site_apache/files/auto_restart.conf create mode 100644 puppet/modules/site_apache/manifests/common/extensions.pp create mode 100644 puppet/modules/site_apache/spec/classes/extensions.rb create mode 100644 puppet/modules/site_apache/spec/spec_helper.rb diff --git a/puppet/modules/site_apache/files/auto_restart.conf b/puppet/modules/site_apache/files/auto_restart.conf new file mode 100644 index 00000000..8a764e34 --- /dev/null +++ b/puppet/modules/site_apache/files/auto_restart.conf @@ -0,0 +1,2 @@ +[Service] +Restart=always diff --git a/puppet/modules/site_apache/manifests/common.pp b/puppet/modules/site_apache/manifests/common.pp index 208c15d5..c96932dd 100644 --- a/puppet/modules/site_apache/manifests/common.pp +++ b/puppet/modules/site_apache/manifests/common.pp @@ -28,5 +28,6 @@ class site_apache::common { include site_apache::common::tls include site_apache::common::acme + include site_apache::common::extensions } diff --git a/puppet/modules/site_apache/manifests/common/extensions.pp b/puppet/modules/site_apache/manifests/common/extensions.pp new file mode 100644 index 00000000..ddeafae0 --- /dev/null +++ b/puppet/modules/site_apache/manifests/common/extensions.pp @@ -0,0 +1,14 @@ +class site_apache::common::extensions { + + include ::systemd + file { '/etc/systemd/system/apache2.service.d/auto_restart.conf': + source => 'puppet:///modules/site_apache/auto_restart.conf', + owner => 'root', + group => 'root', + mode => '0644', + require => [ + Service['apache'] + ] + }~> + Exec['systemctl-daemon-reload'] +} diff --git a/puppet/modules/site_apache/spec/classes/extensions.rb b/puppet/modules/site_apache/spec/classes/extensions.rb new file mode 100644 index 00000000..164034c2 --- /dev/null +++ b/puppet/modules/site_apache/spec/classes/extensions.rb @@ -0,0 +1,7 @@ +require File.expand_path(File.join(File.dirname(__FILE__),'../spec_helper')) + +describe 'site_apache::common::extensions' do + it "should include apache autostart" do + should contain_file('/etc/systemd/system/apache2.service.d/auto_restart.conf').with_source('puppet:///modules/site_apache/apache_auto_restart.conf') + end +end diff --git a/puppet/modules/site_apache/spec/spec_helper.rb b/puppet/modules/site_apache/spec/spec_helper.rb new file mode 100644 index 00000000..dea9e892 --- /dev/null +++ b/puppet/modules/site_apache/spec/spec_helper.rb @@ -0,0 +1,6 @@ +require 'rspec-puppet' + +RSpec.configure do |c| + c.module_path = File.expand_path(File.join(File.dirname(__FILE__), '..', '..')) + c.color = true +end -- cgit v1.2.3 From 85d2ae3fca3bc6fca00e9b5bff045e2130fd539c Mon Sep 17 00:00:00 2001 From: Tulio Casagrande Date: Mon, 16 Jan 2017 17:24:50 -0200 Subject: Update how exec is run --- puppet/modules/site_apache/manifests/common/extensions.pp | 8 +++----- 1 file changed, 3 insertions(+), 5 deletions(-) diff --git a/puppet/modules/site_apache/manifests/common/extensions.pp b/puppet/modules/site_apache/manifests/common/extensions.pp index ddeafae0..6e489ce8 100644 --- a/puppet/modules/site_apache/manifests/common/extensions.pp +++ b/puppet/modules/site_apache/manifests/common/extensions.pp @@ -6,9 +6,7 @@ class site_apache::common::extensions { owner => 'root', group => 'root', mode => '0644', - require => [ - Service['apache'] - ] - }~> - Exec['systemctl-daemon-reload'] + require => Service['apache'], + notify => Exec['systemctl-daemon-reload'] + } } -- cgit v1.2.3 From 4d35718a58a11c399cca7001ac3ae73cd292fabf Mon Sep 17 00:00:00 2001 From: Tulio Casagrande Date: Mon, 16 Jan 2017 17:25:15 -0200 Subject: Remove spec_helper --- puppet/modules/site_apache/spec/classes/extensions.rb | 2 +- puppet/modules/site_apache/spec/spec_helper.rb | 6 ------ 2 files changed, 1 insertion(+), 7 deletions(-) delete mode 100644 puppet/modules/site_apache/spec/spec_helper.rb diff --git a/puppet/modules/site_apache/spec/classes/extensions.rb b/puppet/modules/site_apache/spec/classes/extensions.rb index 164034c2..632b2d72 100644 --- a/puppet/modules/site_apache/spec/classes/extensions.rb +++ b/puppet/modules/site_apache/spec/classes/extensions.rb @@ -1,4 +1,4 @@ -require File.expand_path(File.join(File.dirname(__FILE__),'../spec_helper')) +require 'spec_helper' describe 'site_apache::common::extensions' do it "should include apache autostart" do diff --git a/puppet/modules/site_apache/spec/spec_helper.rb b/puppet/modules/site_apache/spec/spec_helper.rb deleted file mode 100644 index dea9e892..00000000 --- a/puppet/modules/site_apache/spec/spec_helper.rb +++ /dev/null @@ -1,6 +0,0 @@ -require 'rspec-puppet' - -RSpec.configure do |c| - c.module_path = File.expand_path(File.join(File.dirname(__FILE__), '..', '..')) - c.color = true -end -- cgit v1.2.3 From 740734a79b143b572da440e203dbadcb471f2e4e Mon Sep 17 00:00:00 2001 From: Tulio Casagrande Date: Tue, 17 Jan 2017 11:03:14 -0200 Subject: Rename extensions module to autorestart --- puppet/modules/site_apache/files/auto_restart.conf | 2 -- puppet/modules/site_apache/files/autorestart.conf | 2 ++ puppet/modules/site_apache/manifests/common.pp | 2 +- .../modules/site_apache/manifests/common/autorestart.pp | 15 +++++++++++++++ puppet/modules/site_apache/manifests/common/extensions.pp | 12 ------------ puppet/modules/site_apache/spec/classes/autorestart.rb | 7 +++++++ puppet/modules/site_apache/spec/classes/extensions.rb | 7 ------- 7 files changed, 25 insertions(+), 22 deletions(-) delete mode 100644 puppet/modules/site_apache/files/auto_restart.conf create mode 100644 puppet/modules/site_apache/files/autorestart.conf create mode 100644 puppet/modules/site_apache/manifests/common/autorestart.pp delete mode 100644 puppet/modules/site_apache/manifests/common/extensions.pp create mode 100644 puppet/modules/site_apache/spec/classes/autorestart.rb delete mode 100644 puppet/modules/site_apache/spec/classes/extensions.rb diff --git a/puppet/modules/site_apache/files/auto_restart.conf b/puppet/modules/site_apache/files/auto_restart.conf deleted file mode 100644 index 8a764e34..00000000 --- a/puppet/modules/site_apache/files/auto_restart.conf +++ /dev/null @@ -1,2 +0,0 @@ -[Service] -Restart=always diff --git a/puppet/modules/site_apache/files/autorestart.conf b/puppet/modules/site_apache/files/autorestart.conf new file mode 100644 index 00000000..8a764e34 --- /dev/null +++ b/puppet/modules/site_apache/files/autorestart.conf @@ -0,0 +1,2 @@ +[Service] +Restart=always diff --git a/puppet/modules/site_apache/manifests/common.pp b/puppet/modules/site_apache/manifests/common.pp index c96932dd..4847cbe3 100644 --- a/puppet/modules/site_apache/manifests/common.pp +++ b/puppet/modules/site_apache/manifests/common.pp @@ -28,6 +28,6 @@ class site_apache::common { include site_apache::common::tls include site_apache::common::acme - include site_apache::common::extensions + include site_apache::common::autorestart } diff --git a/puppet/modules/site_apache/manifests/common/autorestart.pp b/puppet/modules/site_apache/manifests/common/autorestart.pp new file mode 100644 index 00000000..aa13d4ef --- /dev/null +++ b/puppet/modules/site_apache/manifests/common/autorestart.pp @@ -0,0 +1,15 @@ +# +# Adds autorestart extension to apache on crash +# +class site_apache::common::autorestart { + + include ::systemd + file { '/etc/systemd/system/apache2.service.d/autorestart.conf': + source => 'puppet:///modules/site_apache/autorestart.conf', + owner => 'root', + group => 'root', + mode => '0644', + require => Service['apache'], + notify => Exec['systemctl-daemon-reload'] + } +} diff --git a/puppet/modules/site_apache/manifests/common/extensions.pp b/puppet/modules/site_apache/manifests/common/extensions.pp deleted file mode 100644 index 6e489ce8..00000000 --- a/puppet/modules/site_apache/manifests/common/extensions.pp +++ /dev/null @@ -1,12 +0,0 @@ -class site_apache::common::extensions { - - include ::systemd - file { '/etc/systemd/system/apache2.service.d/auto_restart.conf': - source => 'puppet:///modules/site_apache/auto_restart.conf', - owner => 'root', - group => 'root', - mode => '0644', - require => Service['apache'], - notify => Exec['systemctl-daemon-reload'] - } -} diff --git a/puppet/modules/site_apache/spec/classes/autorestart.rb b/puppet/modules/site_apache/spec/classes/autorestart.rb new file mode 100644 index 00000000..afa02ec9 --- /dev/null +++ b/puppet/modules/site_apache/spec/classes/autorestart.rb @@ -0,0 +1,7 @@ +require 'spec_helper' + +describe 'site_apache::common::autorestart' do + it "should include apache autorestart" do + should contain_file('/etc/systemd/system/apache2.service.d/autorestart.conf').with_source('puppet:///modules/site_apache/autorestart.conf') + end +end diff --git a/puppet/modules/site_apache/spec/classes/extensions.rb b/puppet/modules/site_apache/spec/classes/extensions.rb deleted file mode 100644 index 632b2d72..00000000 --- a/puppet/modules/site_apache/spec/classes/extensions.rb +++ /dev/null @@ -1,7 +0,0 @@ -require 'spec_helper' - -describe 'site_apache::common::extensions' do - it "should include apache autostart" do - should contain_file('/etc/systemd/system/apache2.service.d/auto_restart.conf').with_source('puppet:///modules/site_apache/apache_auto_restart.conf') - end -end -- cgit v1.2.3 From e1767f48c04a469f00a414c2bd85ec2ed0eceb7f Mon Sep 17 00:00:00 2001 From: Tulio Casagrande Date: Tue, 17 Jan 2017 14:26:27 -0200 Subject: Change autorestart to use systemd::unit_file --- puppet/modules/site_apache/manifests/common/autorestart.pp | 7 +------ 1 file changed, 1 insertion(+), 6 deletions(-) diff --git a/puppet/modules/site_apache/manifests/common/autorestart.pp b/puppet/modules/site_apache/manifests/common/autorestart.pp index aa13d4ef..f8213439 100644 --- a/puppet/modules/site_apache/manifests/common/autorestart.pp +++ b/puppet/modules/site_apache/manifests/common/autorestart.pp @@ -3,13 +3,8 @@ # class site_apache::common::autorestart { - include ::systemd - file { '/etc/systemd/system/apache2.service.d/autorestart.conf': + ::systemd::unit_file { '/etc/systemd/system/apache2.service.d/autorestart.conf': source => 'puppet:///modules/site_apache/autorestart.conf', - owner => 'root', - group => 'root', - mode => '0644', require => Service['apache'], - notify => Exec['systemctl-daemon-reload'] } } -- cgit v1.2.3 From c74a1f3c1eea0bb359bfa3bd7d6b37e720f9a991 Mon Sep 17 00:00:00 2001 From: Tulio Casagrande Date: Tue, 17 Jan 2017 15:18:30 -0200 Subject: Ensure the directory exists before creating the file with @aarni --- puppet/modules/site_apache/manifests/common/autorestart.pp | 12 ++++++++++-- puppet/modules/site_apache/spec/classes/autorestart.rb | 7 ------- puppet/modules/site_apache/spec/classes/autorestart_spec.rb | 7 +++++++ 3 files changed, 17 insertions(+), 9 deletions(-) delete mode 100644 puppet/modules/site_apache/spec/classes/autorestart.rb create mode 100644 puppet/modules/site_apache/spec/classes/autorestart_spec.rb diff --git a/puppet/modules/site_apache/manifests/common/autorestart.pp b/puppet/modules/site_apache/manifests/common/autorestart.pp index f8213439..0273f272 100644 --- a/puppet/modules/site_apache/manifests/common/autorestart.pp +++ b/puppet/modules/site_apache/manifests/common/autorestart.pp @@ -3,8 +3,16 @@ # class site_apache::common::autorestart { - ::systemd::unit_file { '/etc/systemd/system/apache2.service.d/autorestart.conf': + file { '/etc/systemd/system/apache2.service.d': + ensure => directory, + mode => '0755', + } + + ::systemd::unit_file { 'apache2.service.d/autorestart.conf': source => 'puppet:///modules/site_apache/autorestart.conf', - require => Service['apache'], + require => [ + File['/etc/systemd/system/apache2.service.d'], + Service['apache'], + ] } } diff --git a/puppet/modules/site_apache/spec/classes/autorestart.rb b/puppet/modules/site_apache/spec/classes/autorestart.rb deleted file mode 100644 index afa02ec9..00000000 --- a/puppet/modules/site_apache/spec/classes/autorestart.rb +++ /dev/null @@ -1,7 +0,0 @@ -require 'spec_helper' - -describe 'site_apache::common::autorestart' do - it "should include apache autorestart" do - should contain_file('/etc/systemd/system/apache2.service.d/autorestart.conf').with_source('puppet:///modules/site_apache/autorestart.conf') - end -end diff --git a/puppet/modules/site_apache/spec/classes/autorestart_spec.rb b/puppet/modules/site_apache/spec/classes/autorestart_spec.rb new file mode 100644 index 00000000..ad9c9f2e --- /dev/null +++ b/puppet/modules/site_apache/spec/classes/autorestart_spec.rb @@ -0,0 +1,7 @@ +require 'spec_helper' + +describe 'site_apache::common::autorestart' do + it "should include apache autorestart" do + should contain_file('apache2.service.d/autorestart.conf').with_source('puppet:///modules/site_apache/autorestart.conf') + end +end -- cgit v1.2.3 From 98a19ce148800d0945fbddf59f5bafbb09748fd5 Mon Sep 17 00:00:00 2001 From: varac Date: Tue, 31 Jan 2017 15:53:18 +0100 Subject: Platform CI: Dont run bundle install in parallel Closes: #8684 --- .gitlab-ci.yml | 18 +++++++++++++++--- tests/platform-ci/setup.sh | 2 +- 2 files changed, 16 insertions(+), 4 deletions(-) diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml index c6cbb666..ab2d5aa5 100644 --- a/.gitlab-ci.yml +++ b/.gitlab-ci.yml @@ -1,13 +1,25 @@ +image: leapcode/ruby + +# This is for caching the gems not only between the stages, but also persistent +# on the gitlab-runner so we don't need to install from scratch on every pipeline +cache: + key: "$CI_BUILD_REF_NAME" + untracked: true + paths: + - tests/platform-ci/vendor/ + before_script: - - echo 'Running global before_script' - cd tests/platform-ci - - ./setup.sh stages: + - setup - syntax - build -image: leapcode/ruby +setup: + stage: setup + script: + - ./setup.sh lint: stage: syntax diff --git a/tests/platform-ci/setup.sh b/tests/platform-ci/setup.sh index 39ef3130..99f735b7 100755 --- a/tests/platform-ci/setup.sh +++ b/tests/platform-ci/setup.sh @@ -1,4 +1,4 @@ #!/bin/sh which bundle || /usr/bin/apt install bundle -/usr/local/bin/bundle install --binstubs --path=/var/cache/gitlab-runner/ --with=test --jobs "$(nproc)" +/usr/local/bin/bundle install --binstubs --path=vendor --with=test --jobs "$(nproc)" -- cgit v1.2.3 From 9e68982b4ef8af087e8792e502d37632d1a9a0e8 Mon Sep 17 00:00:00 2001 From: elijah Date: Thu, 9 Feb 2017 16:34:56 -0800 Subject: tests: check process by either process scan or service name. closes #8753 --- tests/server-tests/helpers/os_helper.rb | 28 +++++++++++++++++++++++----- tests/server-tests/white-box/couchdb.rb | 4 ++-- tests/server-tests/white-box/mx.rb | 20 ++++++++++---------- tests/server-tests/white-box/openvpn.rb | 6 +++--- tests/server-tests/white-box/soledad.rb | 2 +- tests/server-tests/white-box/webapp.rb | 4 ++-- 6 files changed, 41 insertions(+), 23 deletions(-) diff --git a/tests/server-tests/helpers/os_helper.rb b/tests/server-tests/helpers/os_helper.rb index 9923d5b1..6a71388c 100644 --- a/tests/server-tests/helpers/os_helper.rb +++ b/tests/server-tests/helpers/os_helper.rb @@ -20,11 +20,29 @@ class LeapTest }.compact end - def assert_running(process, options={}) - processes = pgrep(process) - assert processes.any?, "No running process for #{process}" - if options[:single] - assert processes.length == 1, "More than one process for #{process}" + # + # passes if the specified process is runnin. + # + # arguments: + # + # match => VALUE -- scan process table for VALUE + # service => VALUE -- call systemctl is-active VALUE + # + # single => true|false -- if true, there must be one result + # + def assert_running(match:nil, service:nil, single:false) + if match + processes = pgrep(match) + assert processes.any?, "No running process for #{match}" + if single + assert processes.length == 1, "More than one process for #{match}" + end + elsif service + `systemctl is-active #{service} 2>&1` + if $?.exitstatus != 0 + output = `systemctl status #{service} 2>&1` + fail "Service '#{service}' is not running:\n#{output}" + end end end diff --git a/tests/server-tests/white-box/couchdb.rb b/tests/server-tests/white-box/couchdb.rb index 44a2769b..dcf71bc7 100644 --- a/tests/server-tests/white-box/couchdb.rb +++ b/tests/server-tests/white-box/couchdb.rb @@ -9,9 +9,9 @@ class CouchDB < LeapTest end def test_00_Are_daemons_running? - assert_running 'bin/beam' + assert_running match: 'bin/beam' if multimaster? - assert_running 'bin/epmd' + assert_running match: 'bin/epmd' end pass end diff --git a/tests/server-tests/white-box/mx.rb b/tests/server-tests/white-box/mx.rb index ecc8686c..432f4e54 100644 --- a/tests/server-tests/white-box/mx.rb +++ b/tests/server-tests/white-box/mx.rb @@ -52,17 +52,17 @@ class Mx < LeapTest end def test_04_Are_MX_daemons_running? - assert_running '.*/usr/bin/twistd.*mx.tac' - assert_running '^/usr/lib/postfix/master$' - assert_running '^/usr/sbin/postfwd' - assert_running 'postfwd2::cache$' - assert_running 'postfwd2::policy$' - assert_running '^/usr/sbin/unbound' - assert_running '^/usr/bin/freshclam' - assert_running '^/usr/sbin/opendkim' + assert_running match: '.*/usr/bin/twistd.*mx.tac' + assert_running match: '^/usr/lib/postfix/master$' + assert_running match: '^/usr/sbin/postfwd' + assert_running match: 'postfwd2::cache$' + assert_running match: 'postfwd2::policy$' + assert_running match: '^/usr/sbin/unbound' + assert_running match: '^/usr/bin/freshclam' + assert_running match: '^/usr/sbin/opendkim' if Dir.glob("/var/lib/clamav/main.{c[vl]d,inc}").size > 0 and Dir.glob("/var/lib/clamav/daily.{c[vl]d,inc}").size > 0 - assert_running '^/usr/sbin/clamd' - assert_running '^/usr/sbin/clamav-milter' + assert_running match: '^/usr/sbin/clamd' + assert_running match: '^/usr/sbin/clamav-milter' pass else skip "Downloading the clamav signature files (/var/lib/clamav/{daily,main}.{c[vl]d,inc}) is still in progress, so clamd is not running." diff --git a/tests/server-tests/white-box/openvpn.rb b/tests/server-tests/white-box/openvpn.rb index d5cc2265..4eed7eb9 100644 --- a/tests/server-tests/white-box/openvpn.rb +++ b/tests/server-tests/white-box/openvpn.rb @@ -7,9 +7,9 @@ class OpenVPN < LeapTest end def test_01_Are_daemons_running? - assert_running '^/usr/sbin/openvpn .* /etc/openvpn/tcp_config.conf$' - assert_running '^/usr/sbin/openvpn .* /etc/openvpn/udp_config.conf$' - assert_running '^/usr/sbin/unbound' + assert_running match: '^/usr/sbin/openvpn .* /etc/openvpn/tcp_config.conf$' + assert_running match: '^/usr/sbin/openvpn .* /etc/openvpn/udp_config.conf$' + assert_running match: '^/usr/sbin/unbound' pass end diff --git a/tests/server-tests/white-box/soledad.rb b/tests/server-tests/white-box/soledad.rb index b89145bc..112d6b9b 100644 --- a/tests/server-tests/white-box/soledad.rb +++ b/tests/server-tests/white-box/soledad.rb @@ -10,7 +10,7 @@ class Soledad < LeapTest end def test_00_Is_Soledad_running? - assert_running '/usr/bin/python /usr/bin/twistd --uid=soledad --gid=soledad --pidfile=/var/run/soledad.pid --syslog --prefix=soledad-server web --class=leap.soledad.server.resource.SoledadResource.*' + assert_running service: 'soledad-server' pass end diff --git a/tests/server-tests/white-box/webapp.rb b/tests/server-tests/white-box/webapp.rb index c46c9f96..e48df524 100644 --- a/tests/server-tests/white-box/webapp.rb +++ b/tests/server-tests/white-box/webapp.rb @@ -27,8 +27,8 @@ class Webapp < LeapTest end def test_03_Are_daemons_running? - assert_running '^/usr/sbin/apache2' - assert_running 'ruby /usr/bin/nickserver' + assert_running match: '^/usr/sbin/apache2' + assert_running match: 'ruby /usr/bin/nickserver' pass end -- cgit v1.2.3 From 83b5b3f78953560e53547a8fa21181e09b124744 Mon Sep 17 00:00:00 2001 From: varac Date: Wed, 15 Feb 2017 11:17:44 +0100 Subject: Remove old leap-keyring package --- puppet/modules/site_config/manifests/remove.pp | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/puppet/modules/site_config/manifests/remove.pp b/puppet/modules/site_config/manifests/remove.pp index 443df9c2..be6cdfd8 100644 --- a/puppet/modules/site_config/manifests/remove.pp +++ b/puppet/modules/site_config/manifests/remove.pp @@ -2,6 +2,11 @@ class site_config::remove { include site_config::remove::files + package { 'leap-keyring': + ensure => purged, + } + + case $::operatingsystemrelease { /^8.*/: { include site_config::remove::jessie -- cgit v1.2.3 From 0255d8a42fc2c37cfaa660a43936ae546b6178ef Mon Sep 17 00:00:00 2001 From: varac Date: Mon, 18 Apr 2016 15:04:05 +0200 Subject: [feat] dont use backports for passenger anymore --- puppet/modules/site_apt/manifests/preferences/passenger.pp | 14 -------------- puppet/modules/site_config/manifests/remove/jessie.pp | 2 +- 2 files changed, 1 insertion(+), 15 deletions(-) delete mode 100644 puppet/modules/site_apt/manifests/preferences/passenger.pp diff --git a/puppet/modules/site_apt/manifests/preferences/passenger.pp b/puppet/modules/site_apt/manifests/preferences/passenger.pp deleted file mode 100644 index 8cd41f91..00000000 --- a/puppet/modules/site_apt/manifests/preferences/passenger.pp +++ /dev/null @@ -1,14 +0,0 @@ -# -# currently, this is only used by static_site to get passenger v4. -# -# UPGRADE: this is not needed for jessie. -# -class site_apt::preferences::passenger { - - apt::preferences_snippet { 'passenger': - package => 'libapache2-mod-passenger', - release => "${::lsbdistcodename}-backports", - priority => 999; - } - -} diff --git a/puppet/modules/site_config/manifests/remove/jessie.pp b/puppet/modules/site_config/manifests/remove/jessie.pp index e9497baf..a3ac19b7 100644 --- a/puppet/modules/site_config/manifests/remove/jessie.pp +++ b/puppet/modules/site_config/manifests/remove/jessie.pp @@ -7,7 +7,7 @@ class site_config::remove::jessie { } apt::preferences_snippet { - [ 'facter', 'obfsproxy', 'python-twisted', 'unbound' ]: + [ 'facter', 'obfsproxy', 'python-twisted', 'unbound', 'passenger' ]: ensure => absent; } -- cgit v1.2.3 From 8b8ddde128d949f041f62dcf26ac65bfcf4b0875 Mon Sep 17 00:00:00 2001 From: varac Date: Mon, 18 Apr 2016 15:10:06 +0200 Subject: [feat] dont use backports for rsyslog anymore --- .../site_apt/manifests/preferences/rsyslog.pp | 13 ---------- .../modules/site_config/manifests/remove/jessie.pp | 5 ++-- puppet/modules/site_config/manifests/syslog.pp | 30 ++++++++-------------- 3 files changed, 14 insertions(+), 34 deletions(-) delete mode 100644 puppet/modules/site_apt/manifests/preferences/rsyslog.pp diff --git a/puppet/modules/site_apt/manifests/preferences/rsyslog.pp b/puppet/modules/site_apt/manifests/preferences/rsyslog.pp deleted file mode 100644 index bfeaa7da..00000000 --- a/puppet/modules/site_apt/manifests/preferences/rsyslog.pp +++ /dev/null @@ -1,13 +0,0 @@ -class site_apt::preferences::rsyslog { - - apt::preferences_snippet { - 'rsyslog_anon_depends': - package => 'libestr0 librelp0 rsyslog*', - priority => '999', - pin => 'release a=wheezy-backports', - before => Class['rsyslog::install']; - - 'fixed_rsyslog_anon_package': - ensure => absent; - } -} diff --git a/puppet/modules/site_config/manifests/remove/jessie.pp b/puppet/modules/site_config/manifests/remove/jessie.pp index a3ac19b7..2fdc4794 100644 --- a/puppet/modules/site_config/manifests/remove/jessie.pp +++ b/puppet/modules/site_config/manifests/remove/jessie.pp @@ -7,8 +7,9 @@ class site_config::remove::jessie { } apt::preferences_snippet { - [ 'facter', 'obfsproxy', 'python-twisted', 'unbound', 'passenger' ]: - ensure => absent; + [ 'facter', 'obfsproxy', 'python-twisted', 'unbound', 'passenger', + 'rsyslog_anon_depends' ]: + ensure => absent; } } diff --git a/puppet/modules/site_config/manifests/syslog.pp b/puppet/modules/site_config/manifests/syslog.pp index 591e0601..096d5d77 100644 --- a/puppet/modules/site_config/manifests/syslog.pp +++ b/puppet/modules/site_config/manifests/syslog.pp @@ -1,25 +1,17 @@ # configure rsyslog on all nodes class site_config::syslog { - # only pin rsyslog packages to backports on wheezy - case $::operatingsystemrelease { - /^7.*/: { - include ::site_apt::preferences::rsyslog - } - # on jessie+ systems, systemd and journald are enabled, - # and journald logs IP addresses, so we need to disable - # it until a solution is found, (#7863): - # https://github.com/systemd/systemd/issues/2447 - default: { - include ::journald - augeas { - 'disable_journald': - incl => '/etc/systemd/journald.conf', - lens => 'Puppet.lns', - changes => 'set /files/etc/systemd/journald.conf/Journal/Storage \'none\'', - notify => Service['systemd-journald']; - } - } + # on jessie+ systems, systemd and journald are enabled, + # and journald logs IP addresses, so we need to disable + # it until a solution is found, (#7863): + # https://github.com/systemd/systemd/issues/2447 + include ::journald + augeas { + 'disable_journald': + incl => '/etc/systemd/journald.conf', + lens => 'Puppet.lns', + changes => 'set /files/etc/systemd/journald.conf/Journal/Storage \'none\'', + notify => Service['systemd-journald']; } class { '::rsyslog::client': -- cgit v1.2.3 From 61b0127cb0357e187a930cba1cc0e1ace149ed55 Mon Sep 17 00:00:00 2001 From: varac Date: Mon, 18 Apr 2016 15:14:04 +0200 Subject: [feat] only care for apache >= 2.4 --- puppet/modules/site_apache/manifests/common.pp | 16 ++++------------ puppet/modules/site_nagios/manifests/server/apache.pp | 5 +---- 2 files changed, 5 insertions(+), 16 deletions(-) diff --git a/puppet/modules/site_apache/manifests/common.pp b/puppet/modules/site_apache/manifests/common.pp index 4847cbe3..74116575 100644 --- a/puppet/modules/site_apache/manifests/common.pp +++ b/puppet/modules/site_apache/manifests/common.pp @@ -13,18 +13,10 @@ class site_apache::common { # needed for the mod_ssl config include apache::module::mime - # load mods depending on apache version - if ( $::lsbdistcodename == 'jessie' ) { - # apache >= 2.4, debian jessie - # needed for mod_ssl config - include apache::module::socache_shmcb - # generally needed - include apache::module::mpm_prefork - } else { - # apache < 2.4, debian wheezy - # for "Order" directive, i.e. main apache2.conf - include apache::module::authz_host - } + # needed for mod_ssl config + include apache::module::socache_shmcb + # generally needed + include apache::module::mpm_prefork include site_apache::common::tls include site_apache::common::acme diff --git a/puppet/modules/site_nagios/manifests/server/apache.pp b/puppet/modules/site_nagios/manifests/server/apache.pp index 82962e89..98d38122 100644 --- a/puppet/modules/site_nagios/manifests/server/apache.pp +++ b/puppet/modules/site_nagios/manifests/server/apache.pp @@ -17,9 +17,6 @@ class site_nagios::server::apache { include apache::module::php5 include apache::module::cgi - # apache >= 2.4, debian jessie - if ( $::lsbdistcodename == 'jessie' ) { - include apache::module::authn_core - } + include apache::module::authn_core } -- cgit v1.2.3 From 43124b2481a8c37221a25b9cbc633433bb30b0b0 Mon Sep 17 00:00:00 2001 From: varac Date: Mon, 18 Apr 2016 15:18:13 +0200 Subject: assume systemd is always present now --- .../site_openvpn/manifests/server_config.pp | 23 +++++++++------------- 1 file changed, 9 insertions(+), 14 deletions(-) diff --git a/puppet/modules/site_openvpn/manifests/server_config.pp b/puppet/modules/site_openvpn/manifests/server_config.pp index 15e6fb38..f33ab17c 100644 --- a/puppet/modules/site_openvpn/manifests/server_config.pp +++ b/puppet/modules/site_openvpn/manifests/server_config.pp @@ -209,20 +209,15 @@ define site_openvpn::server_config( server => $openvpn_configname; } - # register openvpn services at systemd on nodes newer than wheezy + # register openvpn services at systemd # see https://leap.se/code/issues/7798 - case $::operatingsystemrelease { - /^7.*/: { } - default: { - exec { "enable_systemd_${openvpn_configname}": - refreshonly => true, - command => "/bin/systemctl enable openvpn@${openvpn_configname}", - subscribe => File["/etc/openvpn/${openvpn_configname}.conf"], - notify => Service["openvpn@${openvpn_configname}"]; - } - service { "openvpn@${openvpn_configname}": - ensure => running - } - } + exec { "enable_systemd_${openvpn_configname}": + refreshonly => true, + command => "/bin/systemctl enable openvpn@${openvpn_configname}", + subscribe => File["/etc/openvpn/${openvpn_configname}.conf"], + notify => Service["openvpn@${openvpn_configname}"]; + } + service { "openvpn@${openvpn_configname}": + ensure => running } } -- cgit v1.2.3 From e982bbf4d35529e59fbff386284d5478c7f5eb66 Mon Sep 17 00:00:00 2001 From: varac Date: Mon, 18 Apr 2016 15:18:52 +0200 Subject: no build_essential packages for wheeyz anymore --- puppet/modules/site_config/manifests/packages/build_essential.pp | 6 ------ 1 file changed, 6 deletions(-) diff --git a/puppet/modules/site_config/manifests/packages/build_essential.pp b/puppet/modules/site_config/manifests/packages/build_essential.pp index 2b3e13b9..5b9a2602 100644 --- a/puppet/modules/site_config/manifests/packages/build_essential.pp +++ b/puppet/modules/site_config/manifests/packages/build_essential.pp @@ -16,12 +16,6 @@ class site_config::packages::build_essential inherits ::site_config::packages { } } - /^7.*/: { - Package[ 'gcc-4.7','g++-4.7', 'cpp-4.7' ] { - ensure => present - } - } - default: { } } -- cgit v1.2.3 From 03235f93f32edf225128d24325b486531138e486 Mon Sep 17 00:00:00 2001 From: varac Date: Mon, 18 Apr 2016 15:23:28 +0200 Subject: [feat] always set smtpd_relay_restrictions now that we deprecate wheezy, we can always set smtpd_relay_restrictions --- puppet/modules/site_postfix/manifests/mx.pp | 12 ++---------- 1 file changed, 2 insertions(+), 10 deletions(-) diff --git a/puppet/modules/site_postfix/manifests/mx.pp b/puppet/modules/site_postfix/manifests/mx.pp index 2dac85f5..e94320c9 100644 --- a/puppet/modules/site_postfix/manifests/mx.pp +++ b/puppet/modules/site_postfix/manifests/mx.pp @@ -140,21 +140,13 @@ class site_postfix::mx { # greater verbosity for debugging, take out for production #include site_postfix::debug - case $::operatingsystemrelease { - /^7.*/: { - $smtpd_relay_restrictions='' - } - default: { - $smtpd_relay_restrictions=" -o smtpd_relay_restrictions=\$smtps_relay_restrictions\n" - } - } - $mastercf_tail = " smtps inet n - - - - smtpd -o smtpd_tls_wrappermode=yes -o smtpd_tls_security_level=encrypt -o tls_preempt_cipherlist=yes -${smtpd_relay_restrictions} -o smtpd_recipient_restrictions=\$smtps_recipient_restrictions + -o smtpd_relay_restrictions=\$smtps_relay_restrictions + -o smtpd_recipient_restrictions=\$smtps_recipient_restrictions -o smtpd_helo_restrictions=\$smtps_helo_restrictions -o smtpd_client_restrictions= -o cleanup_service_name=clean_smtps -- cgit v1.2.3 From ab5f9ccfe525e39dc1d6c77f12ad68878038f2c0 Mon Sep 17 00:00:00 2001 From: varac Date: Mon, 18 Apr 2016 15:27:10 +0200 Subject: Dont apply specific ssh parameters for wheezy --- puppet/modules/site_sshd/manifests/init.pp | 7 +------ 1 file changed, 1 insertion(+), 6 deletions(-) diff --git a/puppet/modules/site_sshd/manifests/init.pp b/puppet/modules/site_sshd/manifests/init.pp index a9202da4..7d5c728a 100644 --- a/puppet/modules/site_sshd/manifests/init.pp +++ b/puppet/modules/site_sshd/manifests/init.pp @@ -57,13 +57,8 @@ class site_sshd { # therefore we don't use it here, but include all other options # that would be applied by the 'hardened' parameter # not all options are available on wheezy - if ( $::lsbdistcodename == 'wheezy' ) { - $tail_additional_options = 'Ciphers aes256-ctr + $tail_additional_options = 'Ciphers chacha20-poly1305@openssh.com,aes256-gcm@openssh.com,aes256-ctr MACs hmac-sha2-512,hmac-sha2-256,hmac-ripemd160' - } else { - $tail_additional_options = 'Ciphers chacha20-poly1305@openssh.com,aes256-gcm@openssh.com,aes256-ctr -MACs hmac-sha2-512,hmac-sha2-256,hmac-ripemd160' - } ## ## SSHD SERVER CONFIGURATION -- cgit v1.2.3 From 15d2a330d3382306b80e3bd773604f0a6f0c79d5 Mon Sep 17 00:00:00 2001 From: varac Date: Tue, 21 Feb 2017 21:17:46 +0100 Subject: Cleanup modified Gemfile.lock before pulling nickserver vcsrepo Resolves: #8492 --- puppet/modules/site_nickserver/manifests/init.pp | 17 +++++++++++++---- 1 file changed, 13 insertions(+), 4 deletions(-) diff --git a/puppet/modules/site_nickserver/manifests/init.pp b/puppet/modules/site_nickserver/manifests/init.pp index 12236b3b..8ef47b07 100644 --- a/puppet/modules/site_nickserver/manifests/init.pp +++ b/puppet/modules/site_nickserver/manifests/init.pp @@ -61,21 +61,30 @@ class site_nickserver { require => Group['nickserver']; } + # Eariler we used bundle install without --deployment + exec { 'clean_git_repo': + cwd => '/srv/leap/nickserver', + user => 'nickserver', + command => '/usr/bin/git checkout Gemfile.lock', + onlyif => '/usr/bin/git status | /bin/grep -q "modified: *Gemfile.lock"', + require => Package['git'] + } + vcsrepo { '/srv/leap/nickserver': - ensure => present, + ensure => latest, revision => $sources['nickserver']['revision'], provider => $sources['nickserver']['type'], source => $sources['nickserver']['source'], owner => 'nickserver', group => 'nickserver', - require => [ User['nickserver'], Group['nickserver'] ], + require => [ User['nickserver'], Group['nickserver'], Exec['clean_git_repo'] ], notify => Exec['nickserver_bundler_update']; } exec { 'nickserver_bundler_update': cwd => '/srv/leap/nickserver', - command => '/bin/bash -c "/usr/bin/bundle check || /usr/bin/bundle install --path vendor/bundle"', - unless => '/usr/bin/bundle check', + command => '/usr/bin/bundle install --deployment', + unless => '/bin/bash -c "/usr/bin/bundle config --local frozen 1; /usr/bin/bundle check"', user => 'nickserver', timeout => 600, require => [ -- cgit v1.2.3 From dea0638d978b4fab1f6ebe9f20ba51a3cb0effdd Mon Sep 17 00:00:00 2001 From: varac Date: Mon, 27 Feb 2017 10:09:09 +0100 Subject: Install stunnel4 from jessie-backports The jessie version randonly closes the connection prematurely see https://0xacab.org/leap/platform/issues/8746 - Resolves: #8746 --- puppet/modules/site_stunnel/manifests/init.pp | 10 +++++++++- 1 file changed, 9 insertions(+), 1 deletion(-) diff --git a/puppet/modules/site_stunnel/manifests/init.pp b/puppet/modules/site_stunnel/manifests/init.pp index a874721f..5f53d576 100644 --- a/puppet/modules/site_stunnel/manifests/init.pp +++ b/puppet/modules/site_stunnel/manifests/init.pp @@ -5,6 +5,15 @@ class site_stunnel { + # Install stunnel4 from jessie-backports because the + # jessie version randonly closes the connection prematurely + # see https://0xacab.org/leap/platform/issues/8746 + apt::preferences_snippet { 'stunnel4': + package => 'stunnel4', + release => "${::lsbdistcodename}-backports", + priority => 999; + } + # include the generic stunnel module # increase the number of open files to allow for 800 connections class { 'stunnel': default_extra => 'ulimit -n 4096' } @@ -45,4 +54,3 @@ class site_stunnel { include site_stunnel::override_service } - -- cgit v1.2.3 From bb4dd153bbf1174a95017d0046ea9e1320fd81a9 Mon Sep 17 00:00:00 2001 From: varac Date: Thu, 23 Feb 2017 11:05:46 +0100 Subject: Linted couchdb.pp --- puppet/modules/site_webapp/manifests/couchdb.pp | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/puppet/modules/site_webapp/manifests/couchdb.pp b/puppet/modules/site_webapp/manifests/couchdb.pp index 71450370..175255af 100644 --- a/puppet/modules/site_webapp/manifests/couchdb.pp +++ b/puppet/modules/site_webapp/manifests/couchdb.pp @@ -1,3 +1,4 @@ +# Configures webapp couchdb config class site_webapp::couchdb { $webapp = hiera('webapp') @@ -22,8 +23,8 @@ class site_webapp::couchdb { # couchdb.admin.yml is a symlink to prevent the vcsrepo resource # from changing its user permissions every time. '/srv/leap/webapp/config/couchdb.admin.yml': - ensure => 'link', - target => '/etc/leap/couchdb.admin.yml', + ensure => 'link', + target => '/etc/leap/couchdb.admin.yml', require => Vcsrepo['/srv/leap/webapp']; '/etc/leap/couchdb.admin.yml': -- cgit v1.2.3 From cce9af1fce42c29bf062cccfc46ef356d83a6328 Mon Sep 17 00:00:00 2001 From: varac Date: Thu, 23 Feb 2017 11:42:47 +0100 Subject: [8144] Remove Haproxy We used haproxy because we had multiple bigcouch nodes but now with a single couchdb node this is not needed anymore. - Resolves: #8144 --- bin/debug.sh | 5 +- lib/leap_cli/macros/haproxy.rb | 73 ---------- provider_base/services/mx.json | 6 - provider_base/services/webapp.json | 6 - puppet/modules/haproxy/.fixtures.yml | 5 - puppet/modules/haproxy/.gemfile | 5 - puppet/modules/haproxy/.gitrepo | 11 -- puppet/modules/haproxy/.travis.yml | 23 ---- puppet/modules/haproxy/CHANGELOG | 5 - puppet/modules/haproxy/Modulefile | 12 -- puppet/modules/haproxy/README.md | 87 ------------ puppet/modules/haproxy/Rakefile | 1 - puppet/modules/haproxy/manifests/balancermember.pp | 95 ------------- puppet/modules/haproxy/manifests/init.pp | 149 --------------------- puppet/modules/haproxy/manifests/listen.pp | 95 ------------- puppet/modules/haproxy/manifests/params.pp | 65 --------- .../modules/haproxy/spec/classes/haproxy_spec.rb | 138 ------------------- .../haproxy/spec/defines/balancermember_spec.rb | 82 ------------ puppet/modules/haproxy/spec/defines/listen_spec.rb | 53 -------- puppet/modules/haproxy/spec/spec.opts | 6 - puppet/modules/haproxy/spec/spec_helper.rb | 1 - .../modules/haproxy/templates/haproxy-base.cfg.erb | 21 --- .../haproxy/templates/haproxy_balancermember.erb | 3 - .../haproxy/templates/haproxy_listen_block.erb | 10 -- puppet/modules/haproxy/tests/init.pp | 69 ---------- .../site_check_mk/manifests/agent/haproxy.pp | 15 --- .../modules/site_config/manifests/remove/webapp.pp | 12 ++ .../modules/site_haproxy/files/haproxy-stats.cfg | 6 - puppet/modules/site_haproxy/manifests/init.pp | 41 ------ puppet/modules/site_haproxy/templates/couch.erb | 32 ----- .../modules/site_haproxy/templates/haproxy.cfg.erb | 11 -- puppet/modules/site_mx/manifests/init.pp | 1 - puppet/modules/site_nickserver/manifests/init.pp | 10 +- puppet/modules/site_webapp/manifests/couchdb.pp | 4 +- puppet/modules/site_webapp/manifests/init.pp | 3 +- 35 files changed, 21 insertions(+), 1140 deletions(-) delete mode 100644 lib/leap_cli/macros/haproxy.rb delete mode 100644 puppet/modules/haproxy/.fixtures.yml delete mode 100644 puppet/modules/haproxy/.gemfile delete mode 100644 puppet/modules/haproxy/.gitrepo delete mode 100644 puppet/modules/haproxy/.travis.yml delete mode 100644 puppet/modules/haproxy/CHANGELOG delete mode 100644 puppet/modules/haproxy/Modulefile delete mode 100644 puppet/modules/haproxy/README.md delete mode 100644 puppet/modules/haproxy/Rakefile delete mode 100644 puppet/modules/haproxy/manifests/balancermember.pp delete mode 100644 puppet/modules/haproxy/manifests/init.pp delete mode 100644 puppet/modules/haproxy/manifests/listen.pp delete mode 100644 puppet/modules/haproxy/manifests/params.pp delete mode 100644 puppet/modules/haproxy/spec/classes/haproxy_spec.rb delete mode 100644 puppet/modules/haproxy/spec/defines/balancermember_spec.rb delete mode 100644 puppet/modules/haproxy/spec/defines/listen_spec.rb delete mode 100644 puppet/modules/haproxy/spec/spec.opts delete mode 100644 puppet/modules/haproxy/spec/spec_helper.rb delete mode 100644 puppet/modules/haproxy/templates/haproxy-base.cfg.erb delete mode 100644 puppet/modules/haproxy/templates/haproxy_balancermember.erb delete mode 100644 puppet/modules/haproxy/templates/haproxy_listen_block.erb delete mode 100644 puppet/modules/haproxy/tests/init.pp delete mode 100644 puppet/modules/site_check_mk/manifests/agent/haproxy.pp delete mode 100644 puppet/modules/site_haproxy/files/haproxy-stats.cfg delete mode 100644 puppet/modules/site_haproxy/manifests/init.pp delete mode 100644 puppet/modules/site_haproxy/templates/couch.erb delete mode 100644 puppet/modules/site_haproxy/templates/haproxy.cfg.erb diff --git a/bin/debug.sh b/bin/debug.sh index d6f37542..35bcfa3e 100755 --- a/bin/debug.sh +++ b/bin/debug.sh @@ -2,7 +2,7 @@ # debug script to be run on remote servers # called from leap_cli with the 'leap debug' cmd -apps='(leap|pixelated|stunnel|couch|soledad|haproxy)' +apps='(leap|pixelated|stunnel|couch|soledad)' facts='(apt_running |^architecture |^augeasversion |^couchdb_.* |^debian_.* |^dhcp_enabled |^domain |^facterversion |^filesystems |^fqdn |^hardwaremodel |^hostname |^interface.* |^ipaddress.* |^is_pe |^is_virtual |^kernel.* |^lib |^lsb.* |^memory.* |^mtu_.* |^netmask.* |^network_.* |^operatingsystem |^os.* |^path |^physicalprocessorcount |^processor.* |^ps |^puppetversion |^root_home |^rsyslog_version |^rubysitedir |^rubyversion |^selinux |^ssh_version |^swapfree.* |^swapsize.* |^type |^virtual)' @@ -24,6 +24,3 @@ ps aux|egrep "$apps" echo -e '\n\n' echo -e "Last deploy:\n" tail -2 /var/log/leap/deploy-summary.log - - - diff --git a/lib/leap_cli/macros/haproxy.rb b/lib/leap_cli/macros/haproxy.rb deleted file mode 100644 index 3fef24c4..00000000 --- a/lib/leap_cli/macros/haproxy.rb +++ /dev/null @@ -1,73 +0,0 @@ -# encoding: utf-8 - -## -## HAPROXY -## - -module LeapCli - module Macro - - # - # creates a hash suitable for configuring haproxy. the key is the node name of the server we are proxying to. - # - # * node_list - a hash of nodes for the haproxy servers - # * stunnel_client - contains the mappings to local ports for each server node. - # * non_stunnel_port - in case self is included in node_list, the port to connect to. - # - # 1000 weight is used for nodes in the same location. - # 100 otherwise. - # - def haproxy_servers(node_list, stunnel_clients, non_stunnel_port=nil) - default_weight = 10 - local_weight = 100 - - # record the hosts_file - hostnames(node_list) - - # create a simple map for node name -> local stunnel accept port - accept_ports = stunnel_clients.inject({}) do |hsh, stunnel_entry| - name = stunnel_entry.first.sub(/_[0-9]+$/, '') - hsh[name] = stunnel_entry.last['accept_port'] - hsh - end - - # if one the nodes in the node list is ourself, then there will not be a stunnel to it, - # but we need to include it anyway in the haproxy config. - if node_list[self.name] && non_stunnel_port - accept_ports[self.name] = non_stunnel_port - end - - # create the first pass of the servers hash - servers = node_list.values.inject(Config::ObjectList.new) do |hsh, node| - # make sure we have a port to talk to - unless accept_ports[node.name] - error "haproxy needs a local port to talk to when connecting to #{node.name}" - end - weight = default_weight - try { - weight = local_weight if self.location.name == node.location.name - } - hsh[node.name] = Config::Object[ - 'backup', false, - 'host', 'localhost', - 'port', accept_ports[node.name], - 'weight', weight - ] - if node.services.include?('couchdb') - hsh[node.name]['writable'] = node.couch.mode != 'mirror' - end - hsh - end - - # if there are some local servers, make the others backup - if servers.detect{|k,v| v.weight == local_weight} - servers.each do |k,server| - server['backup'] = server['weight'] == default_weight - end - end - - return servers - end - - end -end diff --git a/provider_base/services/mx.json b/provider_base/services/mx.json index 334e40de..17861d18 100644 --- a/provider_base/services/mx.json +++ b/provider_base/services/mx.json @@ -19,12 +19,6 @@ "couch_client": "= stunnel_client(nodes_like_me[:services => :couchdb], global.services[:couchdb].couch.port)" } }, - "haproxy": { - "couch": { - "listen_port": 4096, - "servers": "= haproxy_servers(nodes_like_me[:services => :couchdb], stunnel.clients.couch_client, global.services[:couchdb].couch.port)" - } - }, "couchdb_leap_mx_user": { "username": "= global.services[:couchdb].couch.users[:leap_mx].username", "password": "= secret :couch_leap_mx_password", diff --git a/provider_base/services/webapp.json b/provider_base/services/webapp.json index feca9524..0fd62795 100644 --- a/provider_base/services/webapp.json +++ b/provider_base/services/webapp.json @@ -45,12 +45,6 @@ "couch_client": "= stunnel_client(nodes_like_me[:services => :couchdb], global.services[:couchdb].couch.port)" } }, - "haproxy": { - "couch": { - "listen_port": 4096, - "servers": "= haproxy_servers(nodes_like_me[:services => :couchdb], stunnel.clients.couch_client, global.services[:couchdb].couch.port)" - } - }, "definition_files": { "provider": "= file :provider_json_template", "eip_service": "= file [:eip_service_json_template, 'v'+webapp.api_version.to_s]", diff --git a/puppet/modules/haproxy/.fixtures.yml b/puppet/modules/haproxy/.fixtures.yml deleted file mode 100644 index 8d6f22d6..00000000 --- a/puppet/modules/haproxy/.fixtures.yml +++ /dev/null @@ -1,5 +0,0 @@ -fixtures: - repositories: - concat: "git://github.com/ripienaar/puppet-concat.git" - symlinks: - haproxy: "#{source_dir}" diff --git a/puppet/modules/haproxy/.gemfile b/puppet/modules/haproxy/.gemfile deleted file mode 100644 index 9aad840c..00000000 --- a/puppet/modules/haproxy/.gemfile +++ /dev/null @@ -1,5 +0,0 @@ -source :rubygems - -puppetversion = ENV.key?('PUPPET_VERSION') ? "= #{ENV['PUPPET_VERSION']}" : ['>= 2.7'] -gem 'puppet', puppetversion -gem 'puppetlabs_spec_helper', '>= 0.1.0' diff --git a/puppet/modules/haproxy/.gitrepo b/puppet/modules/haproxy/.gitrepo deleted file mode 100644 index ed92831a..00000000 --- a/puppet/modules/haproxy/.gitrepo +++ /dev/null @@ -1,11 +0,0 @@ -; DO NOT EDIT (unless you know what you are doing) -; -; This subdirectory is a git "subrepo", and this file is maintained by the -; git-subrepo command. See https://github.com/git-commands/git-subrepo#readme -; -[subrepo] - remote = https://leap.se/git/puppet_haproxy - branch = master - commit = af322a73c013f80a958ab7d5d31d0c75cf6d0523 - parent = 04279dd8d1390d61d696d2c14817199304ccd4d8 - cmdver = 0.3.0 diff --git a/puppet/modules/haproxy/.travis.yml b/puppet/modules/haproxy/.travis.yml deleted file mode 100644 index fdbc95dc..00000000 --- a/puppet/modules/haproxy/.travis.yml +++ /dev/null @@ -1,23 +0,0 @@ -language: ruby -rvm: - - 1.8.7 - - 1.9.3 -script: "rake spec" -branches: - only: - - master -env: - - PUPPET_VERSION=2.6.17 - - PUPPET_VERSION=2.7.19 - #- PUPPET_VERSION=3.0.1 # Breaks due to rodjek/rspec-puppet#58 -notifications: - email: false -gemfile: .gemfile -matrix: - exclude: - - rvm: 1.9.3 - gemfile: .gemfile - env: PUPPET_VERSION=2.6.17 - - rvm: 1.8.7 - gemfile: .gemfile - env: PUPPET_VERSION=3.0.1 diff --git a/puppet/modules/haproxy/CHANGELOG b/puppet/modules/haproxy/CHANGELOG deleted file mode 100644 index 0b6d670f..00000000 --- a/puppet/modules/haproxy/CHANGELOG +++ /dev/null @@ -1,5 +0,0 @@ -2012-10-12 - Version 0.2.0 -- Initial public release -- Backwards incompatible changes all around -- No longer needs ordering passed for more than one listener -- Accepts multiple listen ips/ports/server_names diff --git a/puppet/modules/haproxy/Modulefile b/puppet/modules/haproxy/Modulefile deleted file mode 100644 index e729739b..00000000 --- a/puppet/modules/haproxy/Modulefile +++ /dev/null @@ -1,12 +0,0 @@ -name 'puppetlabs-haproxy' -version '0.2.0' -source 'git://github.com/puppetlabs/puppetlabs-haproxy' -author 'Puppet Labs' -license 'Apache License, Version 2.0' -summary 'Haproxy Module' -description 'An Haproxy module for Redhat family OSes using Storeconfigs' -project_page 'http://github.com/puppetlabs/puppetlabs-haproxy' - -## Add dependencies, if any: -# dependency 'username/name', '>= 1.2.0' -dependency 'ripienaar/concat', '>= 0.1.0' diff --git a/puppet/modules/haproxy/README.md b/puppet/modules/haproxy/README.md deleted file mode 100644 index d209e9ab..00000000 --- a/puppet/modules/haproxy/README.md +++ /dev/null @@ -1,87 +0,0 @@ -PuppetLabs Module for haproxy -============================= - -HAProxy is an HA proxying daemon for load-balancing to clustered services. It -can proxy TCP directly, or other kinds of traffic such as HTTP. - -Dependencies ------------- - -Tested and built on Debian, Ubuntu and CentOS - -Currently requires the ripienaar/concat module on the Puppet Forge and uses storeconfigs on the Puppet Master to export/collect resources -from all balancer members. - -Basic Usage ------------ - -This haproxy uses storeconfigs to collect and realize balancer member servers -on a load balancer server. - -*To install and configure HAProxy server listening on port 8140* - -```puppet -node 'haproxy-server' { - class { 'haproxy': } - haproxy::listen { 'puppet00': - ipaddress => $::ipaddress, - ports => '8140', - } -} -``` - -*To add backend loadbalance members* - -```puppet -node 'webserver01' { - @@haproxy::balancermember { $fqdn: - listening_service => 'puppet00', - server_names => $::hostname, - ipaddresses => $::ipaddress, - ports => '8140', - options => 'check' - } -} -``` - -Configuring haproxy options ---------------------------- - -The base `haproxy` class can accept two parameters which will configure basic -behaviour of the haproxy server daemon: - -- `global_options` to configure the `global` section in `haproxy.cfg` -- `defaults_options` to configure the `defaults` section in `haproxy.cfg` - -Configuring haproxy daemon listener ------------------------------------ - -One `haproxy::listen` defined resource should be defined for each HAProxy loadbalanced set of backend servers. The title of the `haproxy::listen` resource is the key to which balancer members will be proxied to. The `ipaddress` field should be the public ip address which the loadbalancer will be contacted on. The `ports` attribute can accept an array or comma-separated list of ports which should be proxied to the `haproxy::balancermemeber` nodes. - -Configuring haproxy loadbalanced member nodes ---------------------------------------------- - -The `haproxy::balacemember` defined resource should be exported from each node -which is serving loadbalanced traffic. the `listening_service` attribute will -associate it with `haproxy::listen` directives on the haproxy node. -`ipaddresses` and `ports` will be assigned to the member to be contacted on. If an array of `ipaddresses` and `server_names` are provided then they will be added to the config in lock-step. - - -Copyright and License ---------------------- - -Copyright (C) 2012 [Puppet Labs](https://www.puppetlabs.com/) Inc - -Puppet Labs can be contacted at: info@puppetlabs.com - -Licensed under the Apache License, Version 2.0 (the "License"); -you may not use this file except in compliance with the License. -You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - -Unless required by applicable law or agreed to in writing, software -distributed under the License is distributed on an "AS IS" BASIS, -WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -See the License for the specific language governing permissions and -limitations under the License. diff --git a/puppet/modules/haproxy/Rakefile b/puppet/modules/haproxy/Rakefile deleted file mode 100644 index cd3d3799..00000000 --- a/puppet/modules/haproxy/Rakefile +++ /dev/null @@ -1 +0,0 @@ -require 'puppetlabs_spec_helper/rake_tasks' diff --git a/puppet/modules/haproxy/manifests/balancermember.pp b/puppet/modules/haproxy/manifests/balancermember.pp deleted file mode 100644 index a0e27539..00000000 --- a/puppet/modules/haproxy/manifests/balancermember.pp +++ /dev/null @@ -1,95 +0,0 @@ -# == Define Resource Type: haproxy::balancermember -# -# This type will setup a balancer member inside a listening service -# configuration block in /etc/haproxy/haproxy.cfg on the load balancer. -# currently it only has the ability to specify the instance name, -# ip address, port, and whether or not it is a backup. More features -# can be added as needed. The best way to implement this is to export -# this resource for all haproxy balancer member servers, and then collect -# them on the main haproxy load balancer. -# -# === Requirement/Dependencies: -# -# Currently requires the ripienaar/concat module on the Puppet Forge and -# uses storeconfigs on the Puppet Master to export/collect resources -# from all balancer members. -# -# === Parameters -# -# [*name*] -# The title of the resource is arbitrary and only utilized in the concat -# fragment name. -# -# [*listening_service*] -# The haproxy service's instance name (or, the title of the -# haproxy::listen resource). This must match up with a declared -# haproxy::listen resource. -# -# [*ports*] -# An array or commas-separated list of ports for which the balancer member -# will accept connections from the load balancer. Note that cookie values -# aren't yet supported, but shouldn't be difficult to add to the -# configuration. If you use an array in server_names and ipaddresses, the -# same port is used for all balancermembers. -# -# [*server_names*] -# The name of the balancer member server as known to haproxy in the -# listening service's configuration block. This defaults to the -# hostname. Can be an array of the same length as ipaddresses, -# in which case a balancermember is created for each pair of -# server_names and ipaddresses (in lockstep). -# -# [*ipaddresses*] -# The ip address used to contact the balancer member server. -# Can be an array, see documentation to server_names. -# -# [*options*] -# An array of options to be specified after the server declaration -# in the listening service's configuration block. -# -# -# === Examples -# -# Exporting the resource for a balancer member: -# -# @@haproxy::balancermember { 'haproxy': -# listening_service => 'puppet00', -# ports => '8140', -# server_names => $::hostname, -# ipaddresses => $::ipaddress, -# options => 'check', -# } -# -# -# Collecting the resource on a load balancer -# -# Haproxy::Balancermember <<| listening_service == 'puppet00' |>> -# -# Creating the resource for multiple balancer members at once -# (for single-pass installation of haproxy without requiring a first -# pass to export the resources if you know the members in advance): -# -# haproxy::balancermember { 'haproxy': -# listening_service => 'puppet00', -# ports => '8140', -# server_names => ['server01', 'server02'], -# ipaddresses => ['192.168.56.200', '192.168.56.201'], -# options => 'check', -# } -# -# (this resource can be declared anywhere) -# -define haproxy::balancermember ( - $listening_service, - $ports, - $server_names = $::hostname, - $ipaddresses = $::ipaddress, - $options = '' -) { - # Template uses $ipaddresses, $server_name, $ports, $option - concat::fragment { "${listening_service}_balancermember_${name}": - order => "20-${listening_service}-${name}", - target => '/etc/haproxy/haproxy.cfg', - content => template('haproxy/haproxy_balancermember.erb'), - } -} diff --git a/puppet/modules/haproxy/manifests/init.pp b/puppet/modules/haproxy/manifests/init.pp deleted file mode 100644 index b91591a3..00000000 --- a/puppet/modules/haproxy/manifests/init.pp +++ /dev/null @@ -1,149 +0,0 @@ -# == Class: haproxy -# -# A Puppet module, using storeconfigs, to model an haproxy configuration. -# Currently VERY limited - Pull requests accepted! -# -# === Requirement/Dependencies: -# -# Currently requires the ripienaar/concat module on the Puppet Forge and -# uses storeconfigs on the Puppet Master to export/collect resources -# from all balancer members. -# -# === Parameters -# -# [*enable*] -# Chooses whether haproxy should be installed or ensured absent. -# Currently ONLY accepts valid boolean true/false values. -# -# [*version*] -# Allows you to specify what version of the package to install. -# Default is simply 'present' -# -# [*global_options*] -# A hash of all the haproxy global options. If you want to specify more -# than one option (i.e. multiple timeout or stats options), pass those -# options as an array and you will get a line for each of them in the -# resultant haproxy.cfg file. -# -# [*defaults_options*] -# A hash of all the haproxy defaults options. If you want to specify more -# than one option (i.e. multiple timeout or stats options), pass those -# options as an array and you will get a line for each of them in the -# resultant haproxy.cfg file. -# -# -# === Examples -# -# class { 'haproxy': -# enable => true, -# global_options => { -# 'log' => "${::ipaddress} local0", -# 'chroot' => '/var/lib/haproxy', -# 'pidfile' => '/var/run/haproxy.pid', -# 'maxconn' => '4000', -# 'user' => 'haproxy', -# 'group' => 'haproxy', -# 'daemon' => '', -# 'stats' => 'socket /var/lib/haproxy/stats' -# }, -# defaults_options => { -# 'log' => 'global', -# 'stats' => 'enable', -# 'option' => 'redispatch', -# 'retries' => '3', -# 'timeout' => [ -# 'http-request 10s', -# 'queue 1m', -# 'connect 10s', -# 'client 1m', -# 'server 1m', -# 'check 10s' -# ], -# 'maxconn' => '8000' -# }, -# } -# -class haproxy ( - $manage_service = true, - $enable = true, - $version = 'present', - $global_options = $haproxy::params::global_options, - $defaults_options = $haproxy::params::defaults_options -) inherits haproxy::params { - include concat::setup - - package { 'haproxy': - ensure => $enable ? { - true => $version, - false => absent, - }, - name => 'haproxy', - } - - if $enable { - concat { '/etc/haproxy/haproxy.cfg': - owner => '0', - group => '0', - mode => '0644', - require => Package['haproxy'], - notify => $manage_service ? { - true => Service['haproxy'], - false => undef, - }, - } - - # Simple Header - concat::fragment { '00-header': - target => '/etc/haproxy/haproxy.cfg', - order => '01', - content => "# This file managed by Puppet\n", - } - - # Template uses $global_options, $defaults_options - concat::fragment { 'haproxy-base': - target => '/etc/haproxy/haproxy.cfg', - order => '10', - content => template('haproxy/haproxy-base.cfg.erb'), - } - - if ($::osfamily == 'Debian') { - file { '/etc/default/haproxy': - content => 'ENABLED=1', - require => Package['haproxy'], - before => $manage_service ? { - true => Service['haproxy'], - false => undef, - }, - } - } - - file { $global_options['chroot']: - ensure => directory, - owner => $global_options['user'], - group => $global_options['group'], - mode => '0550', - require => Package['haproxy'] - } - - } - - if $manage_service { - service { 'haproxy': - ensure => $enable ? { - true => running, - false => stopped, - }, - enable => $enable ? { - true => true, - false => false, - }, - name => 'haproxy', - hasrestart => true, - hasstatus => true, - require => [ - Concat['/etc/haproxy/haproxy.cfg'], - File[$global_options['chroot']], - ], - } - } -} diff --git a/puppet/modules/haproxy/manifests/listen.pp b/puppet/modules/haproxy/manifests/listen.pp deleted file mode 100644 index 00636e3d..00000000 --- a/puppet/modules/haproxy/manifests/listen.pp +++ /dev/null @@ -1,95 +0,0 @@ -# == Define Resource Type: haproxy::listen -# -# This type will setup a listening service configuration block inside -# the haproxy.cfg file on an haproxy load balancer. Each listening service -# configuration needs one or more load balancer member server (that can be -# declared with the haproxy::balancermember defined resource type). Using -# storeconfigs, you can export the haproxy::balancermember resources on all -# load balancer member servers, and then collect them on a single haproxy -# load balancer server. -# -# === Requirement/Dependencies: -# -# Currently requires the ripienaar/concat module on the Puppet Forge and -# uses storeconfigs on the Puppet Master to export/collect resources -# from all balancer members. -# -# === Parameters -# -# [*name*] -# The namevar of the defined resource type is the listening service's name. -# This name goes right after the 'listen' statement in haproxy.cfg -# -# [*ports*] -# Ports on which the proxy will listen for connections on the ip address -# specified in the virtual_ip parameter. Accepts either a single -# comma-separated string or an array of strings which may be ports or -# hyphenated port ranges. -# -# [*ipaddress*] -# The ip address the proxy binds to. Empty addresses, '*', and '0.0.0.0' -# mean that the proxy listens to all valid addresses on the system. -# -# [*mode*] -# The mode of operation for the listening service. Valid values are 'tcp', -# HTTP', and 'health'. -# -# [*options*] -# A hash of options that are inserted into the listening service -# configuration block. -# -# [*collect_exported*] -# Boolean, default 'true'. True means 'collect exported @@balancermember resources' -# (for the case when every balancermember node exports itself), false means -# 'rely on the existing declared balancermember resources' (for the case when you -# know the full set of balancermembers in advance and use haproxy::balancermember -# with array arguments, which allows you to deploy everything in 1 run) -# -# -# === Examples -# -# Exporting the resource for a balancer member: -# -# haproxy::listen { 'puppet00': -# ipaddress => $::ipaddress, -# ports => '18140', -# mode => 'tcp', -# options => { -# 'option' => [ -# 'tcplog', -# 'ssl-hello-chk' -# ], -# 'balance' => 'roundrobin' -# }, -# } -# -# === Authors -# -# Gary Larizza -# -define haproxy::listen ( - $ports, - $ipaddress = [$::ipaddress], - $mode = 'tcp', - $collect_exported = true, - $options = { - 'option' => [ - 'tcplog', - 'ssl-hello-chk' - ], - 'balance' => 'roundrobin' - } -) { - # Template uses: $name, $ipaddress, $ports, $options - concat::fragment { "${name}_listen_block": - order => "20-${name}-00", - target => '/etc/haproxy/haproxy.cfg', - content => template('haproxy/haproxy_listen_block.erb'), - } - - if $collect_exported { - Haproxy::Balancermember <<| listening_service == $name |>> - } - # else: the resources have been created and they introduced their - # concat fragments. We don't have to do anything about them. -} diff --git a/puppet/modules/haproxy/manifests/params.pp b/puppet/modules/haproxy/manifests/params.pp deleted file mode 100644 index 53442ddc..00000000 --- a/puppet/modules/haproxy/manifests/params.pp +++ /dev/null @@ -1,65 +0,0 @@ -# == Class: haproxy::params -# -# This is a container class holding default parameters for for haproxy class. -# currently, only the Redhat family is supported, but this can be easily -# extended by changing package names and configuration file paths. -# -class haproxy::params { - case $osfamily { - Redhat: { - $global_options = { - 'log' => "${::ipaddress} local0", - 'chroot' => '/var/lib/haproxy', - 'pidfile' => '/var/run/haproxy.pid', - 'maxconn' => '4000', - 'user' => 'haproxy', - 'group' => 'haproxy', - 'daemon' => '', - 'stats' => 'socket /var/lib/haproxy/stats' - } - $defaults_options = { - 'log' => 'global', - 'stats' => 'enable', - 'option' => 'redispatch', - 'retries' => '3', - 'timeout' => [ - 'http-request 10s', - 'queue 1m', - 'connect 10s', - 'client 1m', - 'server 1m', - 'check 10s', - ], - 'maxconn' => '8000' - } - } - Debian: { - $global_options = { - 'log' => "${::ipaddress} local0", - 'chroot' => '/var/lib/haproxy', - 'pidfile' => '/var/run/haproxy.pid', - 'maxconn' => '4000', - 'user' => 'haproxy', - 'group' => 'haproxy', - 'daemon' => '', - 'stats' => 'socket /var/lib/haproxy/stats' - } - $defaults_options = { - 'log' => 'global', - 'stats' => 'enable', - 'option' => 'redispatch', - 'retries' => '3', - 'timeout' => [ - 'http-request 10s', - 'queue 1m', - 'connect 10s', - 'client 1m', - 'server 1m', - 'check 10s', - ], - 'maxconn' => '8000' - } - } - default: { fail("The $::osfamily operating system is not supported with the haproxy module") } - } -} diff --git a/puppet/modules/haproxy/spec/classes/haproxy_spec.rb b/puppet/modules/haproxy/spec/classes/haproxy_spec.rb deleted file mode 100644 index 4b5902ce..00000000 --- a/puppet/modules/haproxy/spec/classes/haproxy_spec.rb +++ /dev/null @@ -1,138 +0,0 @@ -require 'spec_helper' - -describe 'haproxy', :type => :class do - let(:default_facts) do - { - :concat_basedir => '/dne', - :ipaddress => '10.10.10.10' - } - end - context 'on supported platforms' do - describe 'for OS-agnostic configuration' do - ['Debian', 'RedHat'].each do |osfamily| - context "on #{osfamily} family operatingsystems" do - let(:facts) do - { :osfamily => osfamily }.merge default_facts - end - let(:params) do - {'enable' => true} - end - it { should include_class('concat::setup') } - it 'should install the haproxy package' do - subject.should contain_package('haproxy').with( - 'ensure' => 'present' - ) - end - it 'should install the haproxy service' do - subject.should contain_service('haproxy').with( - 'ensure' => 'running', - 'enable' => 'true', - 'hasrestart' => 'true', - 'hasstatus' => 'true', - 'require' => [ - 'Concat[/etc/haproxy/haproxy.cfg]', - 'File[/var/lib/haproxy]' - ] - ) - end - it 'should set up /etc/haproxy/haproxy.cfg as a concat resource' do - subject.should contain_concat('/etc/haproxy/haproxy.cfg').with( - 'owner' => '0', - 'group' => '0', - 'mode' => '0644' - ) - end - it 'should manage the chroot directory' do - subject.should contain_file('/var/lib/haproxy').with( - 'ensure' => 'directory' - ) - end - it 'should contain a header concat fragment' do - subject.should contain_concat__fragment('00-header').with( - 'target' => '/etc/haproxy/haproxy.cfg', - 'order' => '01', - 'content' => "# This file managed by Puppet\n" - ) - end - it 'should contain a haproxy-base concat fragment' do - subject.should contain_concat__fragment('haproxy-base').with( - 'target' => '/etc/haproxy/haproxy.cfg', - 'order' => '10' - ) - end - describe 'Base concat fragment contents' do - let(:contents) { param_value(subject, 'concat::fragment', 'haproxy-base', 'content').split("\n") } - it 'should contain global and defaults sections' do - contents.should include('global') - contents.should include('defaults') - end - it 'should log to an ip address for local0' do - contents.should be_any { |match| match =~ / log \d+(\.\d+){3} local0/ } - end - it 'should specify the default chroot' do - contents.should include(' chroot /var/lib/haproxy') - end - it 'should specify the correct user' do - contents.should include(' user haproxy') - end - it 'should specify the correct group' do - contents.should include(' group haproxy') - end - it 'should specify the correct pidfile' do - contents.should include(' pidfile /var/run/haproxy.pid') - end - end - end - context "on #{osfamily} family operatingsystems without managing the service" do - let(:facts) do - { :osfamily => osfamily }.merge default_facts - end - let(:params) do - { - 'enable' => true, - 'manage_service' => false, - } - end - it { should include_class('concat::setup') } - it 'should install the haproxy package' do - subject.should contain_package('haproxy').with( - 'ensure' => 'present' - ) - end - it 'should install the haproxy service' do - subject.should_not contain_service('haproxy') - end - end - end - end - describe 'for OS-specific configuration' do - context 'only on Debian family operatingsystems' do - let(:facts) do - { :osfamily => 'Debian' }.merge default_facts - end - it 'should manage haproxy service defaults' do - subject.should contain_file('/etc/default/haproxy').with( - 'before' => 'Service[haproxy]', - 'require' => 'Package[haproxy]' - ) - verify_contents(subject, '/etc/default/haproxy', ['ENABLED=1']) - end - end - context 'only on RedHat family operatingsystems' do - let(:facts) do - { :osfamily => 'RedHat' }.merge default_facts - end - end - end - end - context 'on unsupported operatingsystems' do - let(:facts) do - { :osfamily => 'RainbowUnicorn' }.merge default_facts - end - it do - expect { - should contain_service('haproxy') - }.to raise_error(Puppet::Error, /operating system is not supported with the haproxy module/) - end - end -end diff --git a/puppet/modules/haproxy/spec/defines/balancermember_spec.rb b/puppet/modules/haproxy/spec/defines/balancermember_spec.rb deleted file mode 100644 index 74bc7a8b..00000000 --- a/puppet/modules/haproxy/spec/defines/balancermember_spec.rb +++ /dev/null @@ -1,82 +0,0 @@ -require 'spec_helper' - -describe 'haproxy::balancermember' do - let(:title) { 'tyler' } - let(:facts) do - { - :ipaddress => '1.1.1.1', - :hostname => 'dero' - } - end - - context 'with a single balancermember option' do - let(:params) do - { - :name => 'tyler', - :listening_service => 'croy', - :ports => '18140', - :options => 'check' - } - end - - it { should contain_concat__fragment('croy_balancermember_tyler').with( - 'order' => '20-croy-tyler', - 'target' => '/etc/haproxy/haproxy.cfg', - 'content' => " server dero 1.1.1.1:18140 check\n\n" - ) } - end - - context 'with multiple balancermember options' do - let(:params) do - { - :name => 'tyler', - :listening_service => 'croy', - :ports => '18140', - :options => ['check', 'close'] - } - end - - it { should contain_concat__fragment('croy_balancermember_tyler').with( - 'order' => '20-croy-tyler', - 'target' => '/etc/haproxy/haproxy.cfg', - 'content' => " server dero 1.1.1.1:18140 check close\n\n" - ) } - end - - context 'with multiple servers' do - let(:params) do - { - :name => 'tyler', - :listening_service => 'croy', - :ports => '18140', - :server_names => ['server01', 'server02'], - :ipaddresses => ['192.168.56.200', '192.168.56.201'], - :options => ['check'] - } - end - - it { should contain_concat__fragment('croy_balancermember_tyler').with( - 'order' => '20-croy-tyler', - 'target' => '/etc/haproxy/haproxy.cfg', - 'content' => " server server01 192.168.56.200:18140 check\n server server02 192.168.56.201:18140 check\n\n" - ) } - end - context 'with multiple servers and multiple ports' do - let(:params) do - { - :name => 'tyler', - :listening_service => 'croy', - :ports => ['18140','18150'], - :server_names => ['server01', 'server02'], - :ipaddresses => ['192.168.56.200', '192.168.56.201'], - :options => ['check'] - } - end - - it { should contain_concat__fragment('croy_balancermember_tyler').with( - 'order' => '20-croy-tyler', - 'target' => '/etc/haproxy/haproxy.cfg', - 'content' => " server server01 192.168.56.200:18140,192.168.56.200:18150 check\n server server02 192.168.56.201:18140,192.168.56.201:18150 check\n\n" - ) } - end -end diff --git a/puppet/modules/haproxy/spec/defines/listen_spec.rb b/puppet/modules/haproxy/spec/defines/listen_spec.rb deleted file mode 100644 index 31dd4c85..00000000 --- a/puppet/modules/haproxy/spec/defines/listen_spec.rb +++ /dev/null @@ -1,53 +0,0 @@ -require 'spec_helper' - -describe 'haproxy::listen' do - let(:title) { 'tyler' } - let(:facts) {{ :ipaddress => '1.1.1.1' }} - context "when only one port is provided" do - let(:params) do - { - :name => 'croy', - :ports => '18140' - } - end - - it { should contain_concat__fragment('croy_listen_block').with( - 'order' => '20-croy-00', - 'target' => '/etc/haproxy/haproxy.cfg', - 'content' => "listen croy\n\n bind 1.1.1.1:18140\n\n balance roundrobin\n option tcplog\n option ssl-hello-chk\n" - ) } - end - context "when an array of ports is provided" do - let(:params) do - { - :name => 'apache', - :ipaddress => '23.23.23.23', - :ports => [ - '80', - '443', - ] - } - end - - it { should contain_concat__fragment('apache_listen_block').with( - 'order' => '20-apache-00', - 'target' => '/etc/haproxy/haproxy.cfg', - 'content' => "listen apache\n\n bind 23.23.23.23:80\n\n bind 23.23.23.23:443\n\n balance roundrobin\n option tcplog\n option ssl-hello-chk\n" - ) } - end - context "when a comma-separated list of ports is provided" do - let(:params) do - { - :name => 'apache', - :ipaddress => '23.23.23.23', - :ports => '80,443' - } - end - - it { should contain_concat__fragment('apache_listen_block').with( - 'order' => '20-apache-00', - 'target' => '/etc/haproxy/haproxy.cfg', - 'content' => "listen apache\n\n bind 23.23.23.23:80\n\n bind 23.23.23.23:443\n\n balance roundrobin\n option tcplog\n option ssl-hello-chk\n" - ) } - end -end diff --git a/puppet/modules/haproxy/spec/spec.opts b/puppet/modules/haproxy/spec/spec.opts deleted file mode 100644 index 91cd6427..00000000 --- a/puppet/modules/haproxy/spec/spec.opts +++ /dev/null @@ -1,6 +0,0 @@ ---format -s ---colour ---loadby -mtime ---backtrace diff --git a/puppet/modules/haproxy/spec/spec_helper.rb b/puppet/modules/haproxy/spec/spec_helper.rb deleted file mode 100644 index 2c6f5664..00000000 --- a/puppet/modules/haproxy/spec/spec_helper.rb +++ /dev/null @@ -1 +0,0 @@ -require 'puppetlabs_spec_helper/module_spec_helper' diff --git a/puppet/modules/haproxy/templates/haproxy-base.cfg.erb b/puppet/modules/haproxy/templates/haproxy-base.cfg.erb deleted file mode 100644 index f25d5c34..00000000 --- a/puppet/modules/haproxy/templates/haproxy-base.cfg.erb +++ /dev/null @@ -1,21 +0,0 @@ -global -<% @global_options.sort.each do |key,val| -%> -<% if val.is_a?(Array) -%> -<% val.each do |item| -%> - <%= key %> <%= item %> -<% end -%> -<% else -%> - <%= key %> <%= val %> -<% end -%> -<% end -%> - -defaults -<% @defaults_options.sort.each do |key,val| -%> -<% if val.is_a?(Array) -%> -<% val.each do |item| -%> - <%= key %> <%= item %> -<% end -%> -<% else -%> - <%= key %> <%= val %> -<% end -%> -<% end -%> diff --git a/puppet/modules/haproxy/templates/haproxy_balancermember.erb b/puppet/modules/haproxy/templates/haproxy_balancermember.erb deleted file mode 100644 index 1d03f565..00000000 --- a/puppet/modules/haproxy/templates/haproxy_balancermember.erb +++ /dev/null @@ -1,3 +0,0 @@ -<% Array(ipaddresses).zip(Array(server_names)).each do |ipaddress,host| -%> - server <%= host %> <%= ipaddress %>:<%= Array(ports).collect {|x|x.split(',')}.flatten.join(",#{ipaddress}:") %> <%= Array(options).join(" ") %> -<% end %> diff --git a/puppet/modules/haproxy/templates/haproxy_listen_block.erb b/puppet/modules/haproxy/templates/haproxy_listen_block.erb deleted file mode 100644 index 129313f1..00000000 --- a/puppet/modules/haproxy/templates/haproxy_listen_block.erb +++ /dev/null @@ -1,10 +0,0 @@ -listen <%= name %> - mode <%= mode %> -<% Array(ipaddress).uniq.each do |virtual_ip| (ports.is_a?(Array) ? ports : Array(ports.split(","))).each do |port| %> - bind <%= virtual_ip %>:<%= port %> -<% end end %> -<% options.sort.each do |key, val| -%> -<% Array(val).each do |item| -%> - <%= key %> <%= item %> -<% end -%> -<% end -%> diff --git a/puppet/modules/haproxy/tests/init.pp b/puppet/modules/haproxy/tests/init.pp deleted file mode 100644 index 77590ac8..00000000 --- a/puppet/modules/haproxy/tests/init.pp +++ /dev/null @@ -1,69 +0,0 @@ -# Declare haproxy base class with configuration options -class { 'haproxy': - enable => true, - global_options => { - 'log' => "${::ipaddress} local0", - 'chroot' => '/var/lib/haproxy', - 'pidfile' => '/var/run/haproxy.pid', - 'maxconn' => '4000', - 'user' => 'haproxy', - 'group' => 'haproxy', - 'daemon' => '', - 'stats' => 'socket /var/lib/haproxy/stats', - }, - defaults_options => { - 'log' => 'global', - 'stats' => 'enable', - 'option' => 'redispatch', - 'retries' => '3', - 'timeout' => [ - 'http-request 10s', - 'queue 1m', - 'connect 10s', - 'client 1m', - 'server 1m', - 'check 10s', - ], - 'maxconn' => '8000', - }, -} - -# Export a balancermember server, note that the listening_service parameter -# will/must correlate with an haproxy::listen defined resource type. -@@haproxy::balancermember { $fqdn: - order => '21', - listening_service => 'puppet00', - server_name => $::hostname, - balancer_ip => $::ipaddress, - balancer_port => '8140', - balancermember_options => 'check' -} - -# Declare a couple of Listening Services for haproxy.cfg -# Note that the balancermember server resources are being collected in -# the haproxy::config defined resource type with the following line: -# Haproxy::Balancermember <<| listening_service == $name |>> -haproxy::listen { 'puppet00': - order => '20', - ipaddress => $::ipaddress, - ports => '18140', - options => { - 'option' => [ - 'tcplog', - 'ssl-hello-chk', - ], - 'balance' => 'roundrobin', - }, -} -haproxy::listen { 'stats': - order => '30', - ipaddress => '', - ports => '9090', - options => { - 'mode' => 'http', - 'stats' => [ - 'uri /', - 'auth puppet:puppet' - ], - }, -} diff --git a/puppet/modules/site_check_mk/manifests/agent/haproxy.pp b/puppet/modules/site_check_mk/manifests/agent/haproxy.pp deleted file mode 100644 index 6d52efba..00000000 --- a/puppet/modules/site_check_mk/manifests/agent/haproxy.pp +++ /dev/null @@ -1,15 +0,0 @@ -class site_check_mk::agent::haproxy { - - include site_check_mk::agent::package::nagios_plugins_contrib - - # local nagios plugin checks via mrpe - augeas { 'haproxy': - incl => '/etc/check_mk/mrpe.cfg', - lens => 'Spacevars.lns', - changes => [ - 'rm /files/etc/check_mk/mrpe.cfg/Haproxy', - 'set Haproxy \'/usr/lib/nagios/plugins/check_haproxy -u "http://localhost:8000/haproxy;csv"\'' ], - require => File['/etc/check_mk/mrpe.cfg']; - } - -} diff --git a/puppet/modules/site_config/manifests/remove/webapp.pp b/puppet/modules/site_config/manifests/remove/webapp.pp index 58f59815..963eb705 100644 --- a/puppet/modules/site_config/manifests/remove/webapp.pp +++ b/puppet/modules/site_config/manifests/remove/webapp.pp @@ -4,4 +4,16 @@ class site_config::remove::webapp { '/etc/apache/sites-enabled/leap_webapp.conf': notify => Service['apache']; } + + # Ensure haproxy is removed + package { 'haproxy': + ensure => purged, + } + augeas { 'haproxy': + incl => '/etc/check_mk/mrpe.cfg', + lens => 'Spacevars.lns', + changes => [ 'rm /files/etc/check_mk/mrpe.cfg/Haproxy' ], + require => File['/etc/check_mk/mrpe.cfg']; + } + } diff --git a/puppet/modules/site_haproxy/files/haproxy-stats.cfg b/puppet/modules/site_haproxy/files/haproxy-stats.cfg deleted file mode 100644 index e6335ba2..00000000 --- a/puppet/modules/site_haproxy/files/haproxy-stats.cfg +++ /dev/null @@ -1,6 +0,0 @@ -# provide access to stats for the nagios plugin -listen stats 127.0.0.1:8000 - mode http - stats enable - stats uri /haproxy - diff --git a/puppet/modules/site_haproxy/manifests/init.pp b/puppet/modules/site_haproxy/manifests/init.pp deleted file mode 100644 index b28ce80e..00000000 --- a/puppet/modules/site_haproxy/manifests/init.pp +++ /dev/null @@ -1,41 +0,0 @@ -class site_haproxy { - $haproxy = hiera('haproxy') - - class { 'haproxy': - enable => true, - manage_service => true, - global_options => { - 'log' => '127.0.0.1 local0', - 'maxconn' => '4096', - 'stats' => 'socket /var/run/haproxy.sock user haproxy group haproxy', - 'chroot' => '/usr/share/haproxy', - 'user' => 'haproxy', - 'group' => 'haproxy', - 'daemon' => '' - }, - defaults_options => { - 'log' => 'global', - 'retries' => '3', - 'option' => 'redispatch', - 'timeout connect' => '4000', - 'timeout client' => '20000', - 'timeout server' => '20000' - } - } - - # monitor haproxy - concat::fragment { 'stats': - target => '/etc/haproxy/haproxy.cfg', - order => '90', - source => 'puppet:///modules/site_haproxy/haproxy-stats.cfg'; - } - - # Template uses $haproxy - concat::fragment { 'leap_haproxy_webapp_couchdb': - target => '/etc/haproxy/haproxy.cfg', - order => '20', - content => template('site_haproxy/haproxy.cfg.erb'), - } - - include site_check_mk::agent::haproxy -} diff --git a/puppet/modules/site_haproxy/templates/couch.erb b/puppet/modules/site_haproxy/templates/couch.erb deleted file mode 100644 index f42e8368..00000000 --- a/puppet/modules/site_haproxy/templates/couch.erb +++ /dev/null @@ -1,32 +0,0 @@ -frontend couch - bind localhost:<%= @listen_port %> - mode http - option httplog - option dontlognull - option http-server-close # use client keep-alive, but close server connection. - use_backend couch_read if METH_GET - default_backend couch_write - -backend couch_write - mode http - balance roundrobin - option httpchk GET / # health check using simple get to root - option allbackups # balance among all backups, not just one. - default-server inter 3000 fastinter 1000 downinter 1000 rise 2 fall 1 -<%- @servers.sort.each do |name,server| -%> -<%- next unless server['writable'] -%> - # <%=name%> - server couchdb_<%=server['port']%> <%=server['host']%>:<%=server['port']%> <%='backup' if server['backup']%> weight <%=server['weight']%> check -<%- end -%> - -backend couch_read - mode http - balance roundrobin - option httpchk GET / # health check using simple get to root - option allbackups # balance among all backups, not just one. - default-server inter 3000 fastinter 1000 downinter 1000 rise 2 fall 1 -<%- @servers.sort.each do |name,server| -%> - # <%=name%> - server couchdb_<%=server['port']%> <%=server['host']%>:<%=server['port']%> <%='backup' if server['backup']%> weight <%=server['weight']%> check -<%- end -%> - diff --git a/puppet/modules/site_haproxy/templates/haproxy.cfg.erb b/puppet/modules/site_haproxy/templates/haproxy.cfg.erb deleted file mode 100644 index 8311b1a5..00000000 --- a/puppet/modules/site_haproxy/templates/haproxy.cfg.erb +++ /dev/null @@ -1,11 +0,0 @@ -<%- @haproxy.each do |frontend, options| -%> -<%- if options['servers'] -%> - -## -## <%= frontend %> -## - -<%= scope.function_templatewlv(["site_haproxy/#{frontend}.erb", options]) %> -<%- end -%> -<%- end -%> - diff --git a/puppet/modules/site_mx/manifests/init.pp b/puppet/modules/site_mx/manifests/init.pp index c910a45a..5876e555 100644 --- a/puppet/modules/site_mx/manifests/init.pp +++ b/puppet/modules/site_mx/manifests/init.pp @@ -13,7 +13,6 @@ class site_mx { include ::site_stunnel include ::site_postfix::mx - include ::site_haproxy include ::site_shorewall::mx include ::site_shorewall::service::smtp include ::leap_mx diff --git a/puppet/modules/site_nickserver/manifests/init.pp b/puppet/modules/site_nickserver/manifests/init.pp index 8ef47b07..cab13522 100644 --- a/puppet/modules/site_nickserver/manifests/init.pp +++ b/puppet/modules/site_nickserver/manifests/init.pp @@ -1,9 +1,8 @@ # -# TODO: currently, this is dependent on some things that are set up in +# TODO: currently, this is dependent on one thing that is set up in # site_webapp # -# (1) HAProxy -> couchdb -# (2) Apache +# (1) Apache # # It would be good in the future to make nickserver installable independently of # site_webapp. @@ -29,10 +28,9 @@ class site_nickserver { # the port that nickserver is actually running on $nickserver_local_port = '64250' - # couchdb is available on localhost via haproxy, which is bound to 4096. + # couchdb is available on localhost via stunnel, which is bound to 4000. $couchdb_host = 'localhost' - # See site_webapp/templates/haproxy_couchdb.cfg.erg - $couchdb_port = '4096' + $couchdb_port = '4000' $sources = hiera('sources') diff --git a/puppet/modules/site_webapp/manifests/couchdb.pp b/puppet/modules/site_webapp/manifests/couchdb.pp index 175255af..ffe364c6 100644 --- a/puppet/modules/site_webapp/manifests/couchdb.pp +++ b/puppet/modules/site_webapp/manifests/couchdb.pp @@ -2,9 +2,9 @@ class site_webapp::couchdb { $webapp = hiera('webapp') - # haproxy listener on port localhost:4096, see site_webapp::haproxy + # stunnel endpoint on port localhost:4000 $couchdb_host = 'localhost' - $couchdb_port = '4096' + $couchdb_port = '4000' $couchdb_webapp_user = $webapp['couchdb_webapp_user']['username'] $couchdb_webapp_password = $webapp['couchdb_webapp_user']['password'] $couchdb_admin_user = $webapp['couchdb_admin_user']['username'] diff --git a/puppet/modules/site_webapp/manifests/init.pp b/puppet/modules/site_webapp/manifests/init.pp index 83cf99a9..1ae80012 100644 --- a/puppet/modules/site_webapp/manifests/init.pp +++ b/puppet/modules/site_webapp/manifests/init.pp @@ -19,7 +19,6 @@ class site_webapp { include ::site_config::ruby::dev include ::site_webapp::apache include ::site_webapp::couchdb - include ::site_haproxy include ::site_webapp::cron include ::site_config::default include ::site_config::x509::cert @@ -106,7 +105,9 @@ class site_webapp { '/srv/leap/webapp/public/ca.crt': ensure => link, require => Vcsrepo['/srv/leap/webapp'], + # lint:ignore:variable_is_lowercase target => "${x509::variables::local_CAs}/${site_config::params::ca_name}.crt"; + # lint:endignore "/srv/leap/webapp/public/${api_version}": ensure => directory, -- cgit v1.2.3 From 5035b80537d4f6d4f4d57a3a429d12fc4ca04d54 Mon Sep 17 00:00:00 2001 From: varac Date: Thu, 23 Feb 2017 11:52:10 +0100 Subject: [8144] Remove Haproxy tests --- tests/server-tests/helpers/couchdb_helper.rb | 31 +--------------------------- tests/server-tests/helpers/http_helper.rb | 4 ++-- tests/server-tests/white-box/mx.rb | 10 --------- tests/server-tests/white-box/webapp.rb | 10 --------- 4 files changed, 3 insertions(+), 52 deletions(-) diff --git a/tests/server-tests/helpers/couchdb_helper.rb b/tests/server-tests/helpers/couchdb_helper.rb index efb2c2bf..0b6671ee 100644 --- a/tests/server-tests/helpers/couchdb_helper.rb +++ b/tests/server-tests/helpers/couchdb_helper.rb @@ -30,35 +30,6 @@ class LeapTest end end - # - # generates a couchdb url for accessing couchdb via haproxy - # - # example properties: - # - # haproxy: - # couch: - # listen_port: 4096 - # servers: - # panda: - # backup: false - # host: localhost - # port: 4000 - # weight: 100 - # writable: true - # - def couchdb_url_via_haproxy(path="", options=nil) - path = path.gsub('"', '%22') - if options && options[:username] && options[:password] - userpart = "%{username}:%{password}@" % options - else - userpart = "" - end - port = assert_property('haproxy.couch.listen_port') - return URLString.new("http://#{userpart}localhost:#{port}#{path}").tap { |url| - url.memo = '(via haproxy)' - } - end - # # generates a couchdb url for when couchdb is running locally. # @@ -140,4 +111,4 @@ class LeapTest end end -end \ No newline at end of file +end diff --git a/tests/server-tests/helpers/http_helper.rb b/tests/server-tests/helpers/http_helper.rb index 0d0bb7d5..3a1df9e7 100644 --- a/tests/server-tests/helpers/http_helper.rb +++ b/tests/server-tests/helpers/http_helper.rb @@ -5,7 +5,7 @@ class LeapTest # # In order to easily provide detailed error messages, it is useful # to append a memo to a url string that details what this url is for - # (e.g. stunnel, haproxy, etc). + # (e.g. stunnel, etc). # # So, the url happens to be a UrlString, the memo field is used # if there is an error in assert_get. @@ -154,4 +154,4 @@ class LeapTest request end -end \ No newline at end of file +end diff --git a/tests/server-tests/white-box/mx.rb b/tests/server-tests/white-box/mx.rb index 432f4e54..dfad0eed 100644 --- a/tests/server-tests/white-box/mx.rb +++ b/tests/server-tests/white-box/mx.rb @@ -24,16 +24,6 @@ class Mx < LeapTest pass end - def test_02_Can_contact_couchdb_via_haproxy? - if property('haproxy.couch') - url = couchdb_url_via_haproxy("", couch_url_options) - assert_get(url) do |body| - assert_match /"couchdb":"Welcome"/, body, "Request to #{url} should return couchdb welcome message." - end - pass - end - end - # # this test picks a random identity document, then queries # using the by_address view for that same document again. diff --git a/tests/server-tests/white-box/webapp.rb b/tests/server-tests/white-box/webapp.rb index e48df524..b1ceddb1 100644 --- a/tests/server-tests/white-box/webapp.rb +++ b/tests/server-tests/white-box/webapp.rb @@ -16,16 +16,6 @@ class Webapp < LeapTest pass end - def test_02_Can_contact_couchdb_via_haproxy? - if property('haproxy.couch') - url = couchdb_url_via_haproxy("", url_options) - assert_get(url) do |body| - assert_match /"couchdb":"Welcome"/, body, "Request to #{url} should return couchdb welcome message." - end - pass - end - end - def test_03_Are_daemons_running? assert_running match: '^/usr/sbin/apache2' assert_running match: 'ruby /usr/bin/nickserver' -- cgit v1.2.3 From 8c1c4c102936dd779c74d615763e7adef7033ec1 Mon Sep 17 00:00:00 2001 From: varac Date: Wed, 15 Mar 2017 00:56:47 +0100 Subject: Direct connection when couch runs locally --- lib/leap_cli/macros/stunnel.rb | 14 +++++++++++++- provider_base/services/mx.json | 1 + provider_base/services/webapp.json | 1 + puppet/modules/site_webapp/manifests/couchdb.pp | 2 +- 4 files changed, 16 insertions(+), 2 deletions(-) diff --git a/lib/leap_cli/macros/stunnel.rb b/lib/leap_cli/macros/stunnel.rb index 821bda38..59a38fad 100644 --- a/lib/leap_cli/macros/stunnel.rb +++ b/lib/leap_cli/macros/stunnel.rb @@ -87,6 +87,18 @@ module LeapCli } end + # + # what it the port of the couchdb we should connect to. + # host will always be localhost. + # + def couchdb_port + if services.include?('couchdb') + couch.port + else + stunnel.clients.couch_client.values.first.accept_port + end + end + private # @@ -103,4 +115,4 @@ module LeapCli end end -end \ No newline at end of file +end diff --git a/provider_base/services/mx.json b/provider_base/services/mx.json index 17861d18..480d7c6e 100644 --- a/provider_base/services/mx.json +++ b/provider_base/services/mx.json @@ -24,6 +24,7 @@ "password": "= secret :couch_leap_mx_password", "salt": "= hex_secret :couch_leap_mx_password_salt, 128" }, + "couchdb_port": "= couchdb_port", "mynetworks": "= host_ips(nodes)", "rbls": ["zen.spamhaus.org"], "clamav": { diff --git a/provider_base/services/webapp.json b/provider_base/services/webapp.json index 0fd62795..064d5b1a 100644 --- a/provider_base/services/webapp.json +++ b/provider_base/services/webapp.json @@ -11,6 +11,7 @@ "tickets", "vmail", "www-data"], "domain": "= provider.domain", "modules": ["user", "billing", "help"], + "couchdb_port": "= couchdb_port", "couchdb_webapp_user": "= global.services[:couchdb].couch.users[:webapp]", "couchdb_admin_user": "= global.services[:couchdb].couch.users[:admin]", "customization_dir": "= file_path 'webapp'", diff --git a/puppet/modules/site_webapp/manifests/couchdb.pp b/puppet/modules/site_webapp/manifests/couchdb.pp index ffe364c6..e1947048 100644 --- a/puppet/modules/site_webapp/manifests/couchdb.pp +++ b/puppet/modules/site_webapp/manifests/couchdb.pp @@ -4,7 +4,7 @@ class site_webapp::couchdb { $webapp = hiera('webapp') # stunnel endpoint on port localhost:4000 $couchdb_host = 'localhost' - $couchdb_port = '4000' + $couchdb_port = $webapp['couchdb_port'] $couchdb_webapp_user = $webapp['couchdb_webapp_user']['username'] $couchdb_webapp_password = $webapp['couchdb_webapp_user']['password'] $couchdb_admin_user = $webapp['couchdb_admin_user']['username'] -- cgit v1.2.3 From 2568ae95455b1800b784971440c609d3680f8dfd Mon Sep 17 00:00:00 2001 From: varac Date: Thu, 16 Mar 2017 00:44:44 +0100 Subject: Direct couch connection if running on same host --- puppet/modules/leap_mx/manifests/init.pp | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/puppet/modules/leap_mx/manifests/init.pp b/puppet/modules/leap_mx/manifests/init.pp index d758e3ab..558b5404 100644 --- a/puppet/modules/leap_mx/manifests/init.pp +++ b/puppet/modules/leap_mx/manifests/init.pp @@ -6,7 +6,7 @@ class leap_mx { $couchdb_password = $leap_mx['password'] $couchdb_host = 'localhost' - $couchdb_port = '4096' + $couchdb_port = hiera('couchdb_port') $sources = hiera('sources') -- cgit v1.2.3 From c331da4033e574a88afef175c1ef0a6a28558ea8 Mon Sep 17 00:00:00 2001 From: varac Date: Sun, 12 Mar 2017 19:38:16 +0100 Subject: Try new packages from exerimental-gitbuildpackage --- provider_base/common.json | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/provider_base/common.json b/provider_base/common.json index 666fe923..dfdc8ff4 100644 --- a/provider_base/common.json +++ b/provider_base/common.json @@ -75,7 +75,7 @@ }, "platform": { "apt": { - "basic": "http://deb.leap.se/experimental-platform" + "basic": "http://deb.leap.se/experimental-gitbuildpackage" } }, "soledad": { -- cgit v1.2.3 From 168013abf257df1576bc69f907729db60c1fb04a Mon Sep 17 00:00:00 2001 From: varac Date: Tue, 14 Mar 2017 00:04:15 +0100 Subject: Make platform apt dist/component configurable --- provider_base/common.json | 3 ++- puppet/modules/site_apt/manifests/init.pp | 7 +++++++ puppet/modules/site_apt/manifests/leap_repo.pp | 2 +- 3 files changed, 10 insertions(+), 2 deletions(-) diff --git a/provider_base/common.json b/provider_base/common.json index dfdc8ff4..2cf9cf72 100644 --- a/provider_base/common.json +++ b/provider_base/common.json @@ -75,7 +75,8 @@ }, "platform": { "apt": { - "basic": "http://deb.leap.se/experimental-gitbuildpackage" + "basic": "http://deb.leap.se/0.9", + "component": "main" } }, "soledad": { diff --git a/puppet/modules/site_apt/manifests/init.pp b/puppet/modules/site_apt/manifests/init.pp index 26bd2c6a..798d0b84 100644 --- a/puppet/modules/site_apt/manifests/init.pp +++ b/puppet/modules/site_apt/manifests/init.pp @@ -12,6 +12,13 @@ class site_apt { # leap repo url $platform_sources = $sources['platform'] $apt_url_platform_basic = $platform_sources['apt']['basic'] + $apt_platform_component = $platform_sources['apt']['component'] + + if ( $platform_sources['apt']['codename'] == '') { + $apt_platform_codename = $::lsbdistcodename + } else { + $apt_platform_codename = $platform_sources['apt']['codename'] + } # needed on jessie hosts for getting pnp4nagios from testing if ( $::operatingsystemmajrelease == '8' ) { diff --git a/puppet/modules/site_apt/manifests/leap_repo.pp b/puppet/modules/site_apt/manifests/leap_repo.pp index 5eedce45..3d95d8b6 100644 --- a/puppet/modules/site_apt/manifests/leap_repo.pp +++ b/puppet/modules/site_apt/manifests/leap_repo.pp @@ -5,7 +5,7 @@ class site_apt::leap_repo { $major_version = $platform['major_version'] apt::sources_list { 'leap.list': - content => "deb ${::site_apt::apt_url_platform_basic} ${::lsbdistcodename} main\n", + content => "deb ${::site_apt::apt_url_platform_basic} ${::site_apt::apt_platform_codename} ${::site_apt::apt_platform_component}\n", before => Exec[refresh_apt] } -- cgit v1.2.3 From 4dbb2c726b7594685ed7857a3f2f89d9a08f36ff Mon Sep 17 00:00:00 2001 From: varac Date: Tue, 14 Mar 2017 00:04:47 +0100 Subject: Use http://deb.leap.se/platform jessie snapshots for platform CI --- provider_base/common.json | 4 ++-- tests/platform-ci/provider/common.json | 8 ++++++++ 2 files changed, 10 insertions(+), 2 deletions(-) diff --git a/provider_base/common.json b/provider_base/common.json index 2cf9cf72..41e1daa3 100644 --- a/provider_base/common.json +++ b/provider_base/common.json @@ -75,8 +75,8 @@ }, "platform": { "apt": { - "basic": "http://deb.leap.se/0.9", - "component": "main" + "basic": "http://deb.leap.se/platform", + "component": "snapshots" } }, "soledad": { diff --git a/tests/platform-ci/provider/common.json b/tests/platform-ci/provider/common.json index 2c63c085..e5096c47 100644 --- a/tests/platform-ci/provider/common.json +++ b/tests/platform-ci/provider/common.json @@ -1,2 +1,10 @@ { + "sources": { + "platform": { + "apt": { + "basic": "http://deb.leap.se/platform", + "component": "snapshots" + } + } + } } -- cgit v1.2.3 From 44f20f7c3907d500adde0edc87c90b2cd339acea Mon Sep 17 00:00:00 2001 From: Azul Date: Wed, 22 Mar 2017 10:10:16 +0100 Subject: webapp: add secret_key_base to config This replaces the secret_token from rails 4.1 on. Both are used for securing cookies in the browser. The secret_key_base will also encrypt the cookies while the token will only sign them. Keeping the token in there for now allows us to migrate existing sessions / cookies to the new secrets. We can remove it in the next version once all providers have run with secret_key_base for a while. --- provider_base/services/webapp.json | 1 + puppet/modules/site_webapp/manifests/init.pp | 1 + puppet/modules/site_webapp/templates/config.yml.erb | 1 + 3 files changed, 3 insertions(+) diff --git a/provider_base/services/webapp.json b/provider_base/services/webapp.json index 064d5b1a..ede3bf66 100644 --- a/provider_base/services/webapp.json +++ b/provider_base/services/webapp.json @@ -23,6 +23,7 @@ "invite_required": "= provider.enrollment_policy == 'invite'", "default_service_level": "= provider.service.default_service_level", "service_levels": "= service_levels()", + "secret_key_base": "= secret :webapp_secret_key_base", "secret_token": "= secret :webapp_secret_token", "api_version": 1, "secure": false, diff --git a/puppet/modules/site_webapp/manifests/init.pp b/puppet/modules/site_webapp/manifests/init.pp index 1ae80012..deb8e8c8 100644 --- a/puppet/modules/site_webapp/manifests/init.pp +++ b/puppet/modules/site_webapp/manifests/init.pp @@ -10,6 +10,7 @@ class site_webapp { $provider_domain = $node_domain['full_suffix'] $webapp = hiera('webapp') $api_version = $webapp['api_version'] + $secret_key_base = $webapp['secret_key_base'] $secret_token = $webapp['secret_token'] $tor = hiera('tor', false) $sources = hiera('sources') diff --git a/puppet/modules/site_webapp/templates/config.yml.erb b/puppet/modules/site_webapp/templates/config.yml.erb index dd55d3e9..1a802f4c 100644 --- a/puppet/modules/site_webapp/templates/config.yml.erb +++ b/puppet/modules/site_webapp/templates/config.yml.erb @@ -8,6 +8,7 @@ production = { "force_ssl" => @webapp['secure'], "client_ca_key" => "%s/%s.key" % [scope.lookupvar('x509::variables::keys'), scope.lookupvar('site_config::params::client_ca_name')], "client_ca_cert" => "%s/%s.crt" % [scope.lookupvar('x509::variables::local_CAs'), scope.lookupvar('site_config::params::client_ca_name')], + "secret_key_base" => @secret_key_base, "secret_token" => @secret_token, "client_cert_lifespan" => cert_options['life_span'], "client_cert_bit_size" => cert_options['bit_size'].to_i, -- cgit v1.2.3 From 747d3e9b55c8b7b7d98a63474b6de82d7114c389 Mon Sep 17 00:00:00 2001 From: varac Date: Wed, 29 Mar 2017 12:46:39 +0200 Subject: Run leap info after deploy --- tests/platform-ci/ci-build.sh | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tests/platform-ci/ci-build.sh b/tests/platform-ci/ci-build.sh index 0dfbb5c3..af1dba0f 100755 --- a/tests/platform-ci/ci-build.sh +++ b/tests/platform-ci/ci-build.sh @@ -78,10 +78,10 @@ $LEAP_CMD compile "$TAG" $LEAP_CMD vm status "$TAG" $LEAP_CMD node init "$TAG" -$LEAP_CMD info "${TAG}" # Deploy and test $LEAP_CMD deploy "$TAG" +$LEAP_CMD info "${TAG}" $LEAP_CMD test "$TAG" # if everything succeeds, destroy the vm -- cgit v1.2.3 From b6d23b4051587cd4dd69259ef7ead680fc66ce95 Mon Sep 17 00:00:00 2001 From: Micah Anderson Date: Thu, 20 Apr 2017 14:32:39 -0400 Subject: Ensure leap command is setup properly for CI Add a `leap help` command at the end of the CI setup.sh to ensure that the command is setup properly before continuing --- tests/platform-ci/setup.sh | 1 + 1 file changed, 1 insertion(+) diff --git a/tests/platform-ci/setup.sh b/tests/platform-ci/setup.sh index 99f735b7..e92dddc7 100755 --- a/tests/platform-ci/setup.sh +++ b/tests/platform-ci/setup.sh @@ -2,3 +2,4 @@ which bundle || /usr/bin/apt install bundle /usr/local/bin/bundle install --binstubs --path=vendor --with=test --jobs "$(nproc)" +/usr/local/bin/bundle exec leap -v2 --yes help -- cgit v1.2.3 From 92f069fc456260c0cc394ab280e61a560ccb3345 Mon Sep 17 00:00:00 2001 From: Micah Anderson Date: Thu, 20 Apr 2017 14:35:03 -0400 Subject: Enhance ci-build.sh for latest CI builds. . Reorganize script to allow for multiple builds . Add latest build, pulling from the ibex provider . Run the build as the cirunner unprivileged user . Set pipefail because job is run within a pipe . Change name of 'build' stage to 'deploy' . Setup an environment for the latest CI deployment --- .gitlab-ci.yml | 32 ++++++++---- tests/platform-ci/ci-build.sh | 118 ++++++++++++++++++++++++++---------------- 2 files changed, 97 insertions(+), 53 deletions(-) diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml index ab2d5aa5..8d3afaa5 100644 --- a/.gitlab-ci.yml +++ b/.gitlab-ci.yml @@ -1,4 +1,4 @@ -image: leapcode/ruby +image: 0xacab.org:4567/leap/gitlab-buildpackage:ruby # This is for caching the gems not only between the stages, but also persistent # on the gitlab-runner so we don't need to install from scratch on every pipeline @@ -14,7 +14,7 @@ before_script: stages: - setup - syntax - - build + - deploy setup: stage: setup @@ -24,27 +24,27 @@ setup: lint: stage: syntax script: - - /usr/local/bin/bundle exec rake lint + - su -c '/usr/local/bin/bundle exec rake lint' cirunner syntax: stage: syntax script: - - /usr/local/bin/bundle exec rake syntax + - su -c '/usr/local/bin/bundle exec rake syntax' cirunner validate: stage: syntax script: - - /usr/local/bin/bundle exec rake validate + - su -c '/usr/local/bin/bundle exec rake validate' cirunner templates: stage: syntax script: - - /usr/local/bin/bundle exec rake templates + - su -c '/usr/local/bin/bundle exec rake templates' cirunner catalog: stage: syntax script: - - /usr/local/bin/bundle exec rake catalog + - su -c '/usr/local/bin/bundle exec rake catalog' cirunner #rspec: # stage: rspec @@ -52,6 +52,20 @@ catalog: # - /usr/local/bin/bundle exec rake spec build: - stage: build + stage: deploy script: - - /usr/bin/unbuffer ./ci-build.sh | /usr/bin/ts -s + - su -c '/usr/bin/unbuffer ./ci-build.sh | /usr/bin/ts -s' cirunner + +# Latest job will only run on the master branch, which means all merge requests +# that are created from branches don't get to deploy to the latest-ci server. +# When a merge request is merged, then the latest job will deploy the code to +# the latest provider, and the deployment will be recorded in an environment +# named 'latest' +latest: + stage: deploy + environment: + name: latest + only: + - master + script: + - su -c '/usr/bin/unbuffer ./ci-build.sh | /usr/bin/ts -s' cirunner diff --git a/tests/platform-ci/ci-build.sh b/tests/platform-ci/ci-build.sh index af1dba0f..869e7517 100755 --- a/tests/platform-ci/ci-build.sh +++ b/tests/platform-ci/ci-build.sh @@ -15,75 +15,105 @@ # * ssh private key used to login to remove vm # * `SSH_PRIVATE_KEY` # -# Todo: -# - Running locally works fine, now use it in gitlab CI ( which ssh-key ? create cloud.json from env vars ) -# - Speed up vm boot if possible ( right now 3-4mins ) # exit if any commands returns non-zero status set -e +# because the ci-build is running in a pipe we need to also set the following +# so exit codes will be caught correctly. +set -o pipefail # leap_platform/tests/platform-ci # shellcheck disable=SC2086 ROOTDIR=$(readlink -f "$(dirname $0)") -# leap_platform/tests/platform-ci/provider -PROVIDERDIR="${ROOTDIR}/provider" - # leap_platform PLATFORMDIR=$(readlink -f "${ROOTDIR}/../..") -LEAP_CMD="/usr/local/bin/bundle exec leap -v2 --yes" +LEAP_CMD() { + /usr/local/bin/bundle exec leap -v2 --yes "$@" +} + +deploy() { + LEAP_CMD deploy "$TAG" +} + +test() { + LEAP_CMD test "$TAG" +} + +build_from_scratch() { + # leap_platform/tests/platform-ci/provider + PROVIDERDIR="${ROOTDIR}/provider" + /bin/echo "Provider directory: ${PROVIDERDIR}" + cd "$PROVIDERDIR" -# create node(s) with unique id so we can run tests in parallel -NAME="citest${CI_BUILD_ID}" -# when using gitlab-runner locally, CI_BUILD_ID is always 1 which -# will conflict with running/terminating AWS instances in subsequent runs -# therefore we pick a random number in this case -[ "$CI_BUILD_ID" -eq "1" ] && NAME+="000${RANDOM}" + # Create cloud.json needed for `leap vm` commands using AWS credentials + which jq || ( apt-get update -y && apt-get install jq -y ) + /usr/bin/jq ".platform_ci.auth |= .+ {\"aws_access_key_id\":\"$AWS_ACCESS_KEY\", \"aws_secret_access_key\":\"$AWS_SECRET_KEY\"}" < cloud.json.template > cloud.json -TAG='single' -SERVICES='couchdb,soledad,mx,webapp,tor,monitor' -SEEDS='' + [ -d "./tags" ] || mkdir "./tags" + /bin/echo "{\"environment\": \"$TAG\"}" | /usr/bin/json_pp > "${PROVIDERDIR}/tags/${TAG}.json" + + pwd + LEAP_CMD vm status "$TAG" + # shellcheck disable=SC2086 + LEAP_CMD vm add "$NAME" services:"$SERVICES" tags:"$TAG" $SEEDS + LEAP_CMD compile "$TAG" + LEAP_CMD vm status "$TAG" + + LEAP_CMD node init "$TAG" + LEAP_CMD info "${TAG}" +} # # Main # - /bin/echo "CI directory: ${ROOTDIR}" -/bin/echo "Provider directory: ${PROVIDERDIR}" /bin/echo "Platform directory: ${PLATFORMDIR}" -cd "$PROVIDERDIR" # Ensure we don't output secret stuff to console even when running in verbose mode with -x set +x -# Create cloud.json needed for `leap vm` commands using AWS credentials -which jq || ( apt-get update -y && apt-get install jq -y ) -/usr/bin/jq ".platform_ci.auth |= .+ {\"aws_access_key_id\":\"$AWS_ACCESS_KEY\", \"aws_secret_access_key\":\"$AWS_SECRET_KEY\"}" < cloud.json.template > cloud.json - # Configure ssh keypair [ -d ~/.ssh ] || /bin/mkdir ~/.ssh /bin/echo "$SSH_PRIVATE_KEY" > ~/.ssh/id_rsa /bin/chmod 600 ~/.ssh/id_rsa -/bin/cp users/gitlab-runner/gitlab-runner_ssh.pub ~/.ssh/id_rsa.pub - -[ -d "./tags" ] || mkdir "./tags" -/bin/echo "{\"environment\": \"$TAG\"}" | /usr/bin/json_pp > "${PROVIDERDIR}/tags/${TAG}.json" - -$LEAP_CMD vm status "$TAG" -# shellcheck disable=SC2086 -$LEAP_CMD vm add "$NAME" services:"$SERVICES" tags:"$TAG" $SEEDS -$LEAP_CMD compile "$TAG" -$LEAP_CMD vm status "$TAG" - -$LEAP_CMD node init "$TAG" - -# Deploy and test -$LEAP_CMD deploy "$TAG" -$LEAP_CMD info "${TAG}" -$LEAP_CMD test "$TAG" - -# if everything succeeds, destroy the vm -$LEAP_CMD vm rm "${TAG}" -[ -f "nodes/${NAME}.json" ] && /bin/rm "nodes/${NAME}.json" +/bin/cp "${ROOTDIR}/provider/users/gitlab-runner/gitlab-runner_ssh.pub" ~/.ssh/id_rsa.pub + +case "$CI_BUILD_STAGE" in + build) + # create node(s) with unique id so we can run tests in parallel + NAME="citest${CI_BUILD_ID}" + # when using gitlab-runner locally, CI_BUILD_ID is always 1 which + # will conflict with running/terminating AWS instances in subsequent runs + # therefore we pick a random number in this case + [ "$CI_BUILD_ID" -eq "1" ] && NAME+="000${RANDOM}" + + TAG='single' + SERVICES='couchdb,soledad,mx,webapp,tor,monitor' + SEEDS='' + build_from_scratch + # Deploy and test + deploy + test + # if everything succeeds, destroy the vm + LEAP_CMD vm rm "${TAG}" + [ -f "nodes/${NAME}.json" ] && /bin/rm "nodes/${NAME}.json" + ;; + latest) + TAG='latest' + echo "Cloning ibex provider..." + git clone -q --depth 1 ssh://gitolite@leap.se/ibex + cd ibex + git rev-parse HEAD + echo -n "Operating in the ibex directory: " + pwd + echo "Listing current node information..." + LEAP_CMD list + echo "Attempting a deploy..." + deploy + echo "Attempting to run tests..." + test + ;; +esac -- cgit v1.2.3 From 9ab23ac448d629a362bdba142b685217b2103f07 Mon Sep 17 00:00:00 2001 From: Micah Anderson Date: Thu, 20 Apr 2017 14:59:51 -0400 Subject: switch to using CI_ENVIRONMENT_NAME and defaulting to the basic deployment --- tests/platform-ci/ci-build.sh | 34 +++++++++++++++++----------------- 1 file changed, 17 insertions(+), 17 deletions(-) diff --git a/tests/platform-ci/ci-build.sh b/tests/platform-ci/ci-build.sh index 869e7517..a9731fca 100755 --- a/tests/platform-ci/ci-build.sh +++ b/tests/platform-ci/ci-build.sh @@ -81,8 +81,23 @@ set +x /bin/chmod 600 ~/.ssh/id_rsa /bin/cp "${ROOTDIR}/provider/users/gitlab-runner/gitlab-runner_ssh.pub" ~/.ssh/id_rsa.pub -case "$CI_BUILD_STAGE" in - build) +case "$CI_ENVIRONMENT_NAME" in + latest) + TAG='latest' + echo "Cloning ibex provider..." + git clone -q --depth 1 ssh://gitolite@leap.se/ibex + cd ibex + git rev-parse HEAD + echo -n "Operating in the ibex directory: " + pwd + echo "Listing current node information..." + LEAP_CMD list + echo "Attempting a deploy..." + deploy + echo "Attempting to run tests..." + test + ;; + *) # create node(s) with unique id so we can run tests in parallel NAME="citest${CI_BUILD_ID}" # when using gitlab-runner locally, CI_BUILD_ID is always 1 which @@ -101,19 +116,4 @@ case "$CI_BUILD_STAGE" in LEAP_CMD vm rm "${TAG}" [ -f "nodes/${NAME}.json" ] && /bin/rm "nodes/${NAME}.json" ;; - latest) - TAG='latest' - echo "Cloning ibex provider..." - git clone -q --depth 1 ssh://gitolite@leap.se/ibex - cd ibex - git rev-parse HEAD - echo -n "Operating in the ibex directory: " - pwd - echo "Listing current node information..." - LEAP_CMD list - echo "Attempting a deploy..." - deploy - echo "Attempting to run tests..." - test - ;; esac -- cgit v1.2.3 From 076f81ef624e5dc83dfebecf76496b90a87ff8ef Mon Sep 17 00:00:00 2001 From: Micah Anderson Date: Sun, 23 Apr 2017 12:17:06 -0400 Subject: CI: deploy_test should run for MRs, but not when merged into master --- .gitlab-ci.yml | 10 ++++++++-- 1 file changed, 8 insertions(+), 2 deletions(-) diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml index 8d3afaa5..a85d6483 100644 --- a/.gitlab-ci.yml +++ b/.gitlab-ci.yml @@ -51,8 +51,14 @@ catalog: # script: # - /usr/local/bin/bundle exec rake spec -build: +# The deploy_test job is run on any merge request. This is used to ensure that +# the merge request will deploy and test properly. It is not run when the merge +# request is accepted into master, instead the 'latest' job below is run +# instead. +deploy_test: stage: deploy + except: + - master script: - su -c '/usr/bin/unbuffer ./ci-build.sh | /usr/bin/ts -s' cirunner @@ -64,7 +70,7 @@ build: latest: stage: deploy environment: - name: latest + name: staging only: - master script: -- cgit v1.2.3 From 64d0b3f97c02dc502a4d6e44b62379170792de8e Mon Sep 17 00:00:00 2001 From: Micah Anderson Date: Tue, 25 Apr 2017 09:20:01 -0400 Subject: Switch to using new docker location for ruby image --- .gitlab-ci.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml index a85d6483..2df6eb5c 100644 --- a/.gitlab-ci.yml +++ b/.gitlab-ci.yml @@ -1,4 +1,4 @@ -image: 0xacab.org:4567/leap/gitlab-buildpackage:ruby +image: 0xacab.org:4567/leap-docker/ruby:latest # This is for caching the gems not only between the stages, but also persistent # on the gitlab-runner so we don't need to install from scratch on every pipeline -- cgit v1.2.3 From f5d1850e7b3831b1ee6374627bb403aa3858a320 Mon Sep 17 00:00:00 2001 From: Micah Anderson Date: Tue, 25 Apr 2017 13:42:14 -0400 Subject: Fix the pipefail by putting ts inside of ci-build.sh --- .gitlab-ci.yml | 4 ++-- tests/platform-ci/ci-build.sh | 6 ++++++ 2 files changed, 8 insertions(+), 2 deletions(-) diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml index 2df6eb5c..5654238e 100644 --- a/.gitlab-ci.yml +++ b/.gitlab-ci.yml @@ -60,7 +60,7 @@ deploy_test: except: - master script: - - su -c '/usr/bin/unbuffer ./ci-build.sh | /usr/bin/ts -s' cirunner + - su -c '/usr/bin/unbuffer ./ci-build.sh' cirunner # Latest job will only run on the master branch, which means all merge requests # that are created from branches don't get to deploy to the latest-ci server. @@ -74,4 +74,4 @@ latest: only: - master script: - - su -c '/usr/bin/unbuffer ./ci-build.sh | /usr/bin/ts -s' cirunner + - su -c '/usr/bin/unbuffer ./ci-build.sh' cirunner diff --git a/tests/platform-ci/ci-build.sh b/tests/platform-ci/ci-build.sh index a9731fca..b39f6874 100755 --- a/tests/platform-ci/ci-build.sh +++ b/tests/platform-ci/ci-build.sh @@ -22,6 +22,11 @@ set -e # so exit codes will be caught correctly. set -o pipefail +# we wrap the whole script in curly braces so we can pipe it all through ts to +# get timestamps. If we put it outside of the script, then we can't get proper +# pipefail results. + +{ # leap_platform/tests/platform-ci # shellcheck disable=SC2086 ROOTDIR=$(readlink -f "$(dirname $0)") @@ -117,3 +122,4 @@ case "$CI_ENVIRONMENT_NAME" in [ -f "nodes/${NAME}.json" ] && /bin/rm "nodes/${NAME}.json" ;; esac +} | /usr/bin/ts -s -- cgit v1.2.3 From 10d96b990af1d680e31c291a15c7b66a6522de89 Mon Sep 17 00:00:00 2001 From: Micah Anderson Date: Mon, 24 Apr 2017 12:04:55 -0400 Subject: git subrepo pull (merge) puppet/modules/tor subrepo: subdir: "puppet/modules/tor" merged: "5ef29012" upstream: origin: "https://leap.se/git/puppet_tor" branch: "master" commit: "5ef29012" git-subrepo: version: "0.4.0" origin: "https://github.com/ingydotnet/git-subrepo" commit: "2e78d5d" --- puppet/modules/tor/.gitrepo | 6 +++--- puppet/modules/tor/README | 2 +- puppet/modules/tor/manifests/daemon/base.pp | 14 ++++---------- puppet/modules/tor/manifests/daemon/bridge.pp | 3 --- puppet/modules/tor/manifests/daemon/control.pp | 18 +++++++++--------- puppet/modules/tor/manifests/daemon/directory.pp | 3 --- puppet/modules/tor/manifests/daemon/dns.pp | 3 --- puppet/modules/tor/manifests/daemon/exit_policy.pp | 3 --- puppet/modules/tor/manifests/daemon/hidden_service.pp | 18 +++++++++++------- puppet/modules/tor/manifests/daemon/map_address.pp | 3 --- puppet/modules/tor/manifests/daemon/relay.pp | 3 --- puppet/modules/tor/manifests/daemon/snippet.pp | 3 --- puppet/modules/tor/manifests/daemon/socks.pp | 3 --- puppet/modules/tor/manifests/daemon/transparent.pp | 3 --- puppet/modules/tor/manifests/munin.pp | 2 +- puppet/modules/tor/manifests/repo.pp | 3 ++- puppet/modules/tor/manifests/repo/debian.pp | 2 +- puppet/modules/tor/templates/torrc.directory.erb | 4 ++-- puppet/modules/tor/templates/torrc.global.erb | 4 ++-- puppet/modules/tor/templates/torrc.hidden_service.erb | 6 ++++++ 20 files changed, 42 insertions(+), 64 deletions(-) diff --git a/puppet/modules/tor/.gitrepo b/puppet/modules/tor/.gitrepo index dfc1b3d9..5e3e3c1f 100644 --- a/puppet/modules/tor/.gitrepo +++ b/puppet/modules/tor/.gitrepo @@ -6,6 +6,6 @@ [subrepo] remote = https://leap.se/git/puppet_tor branch = master - commit = 9981a70f7ba1f9e4fe33e4eb46654295287c1fc1 - parent = 26aac7ccf240b06d65616bdd00ae472d980aaea9 - cmdver = 0.3.0 + commit = 5ef29012dccc90e68afc215be9521629a0903bc6 + parent = 747d3e9b55c8b7b7d98a63474b6de82d7114c389 + cmdver = 0.4.0 diff --git a/puppet/modules/tor/README b/puppet/modules/tor/README index 7777438a..188accac 100644 --- a/puppet/modules/tor/README +++ b/puppet/modules/tor/README @@ -113,7 +113,7 @@ Installing torsocks To install torsocks, simply include the 'torsocks' class in your manifests: - class { 'torsocks': } + class { 'tor::torsocks': } You can specify the $ensure_version class parameter to get a specific version installed. diff --git a/puppet/modules/tor/manifests/daemon/base.pp b/puppet/modules/tor/manifests/daemon/base.pp index 63d7bc4d..c0c82ac6 100644 --- a/puppet/modules/tor/manifests/daemon/base.pp +++ b/puppet/modules/tor/manifests/daemon/base.pp @@ -2,7 +2,7 @@ class tor::daemon::base inherits tor::base { # packages, user, group Service['tor'] { - subscribe => File[$tor::daemon::config_file], + subscribe => Concat[$tor::daemon::config_file], } Package[ 'tor' ] { @@ -49,18 +49,15 @@ class tor::daemon::base inherits tor::base { # tor configuration file concat { $tor::daemon::config_file: - mode => '0600', - owner => 'debian-tor', - group => 'debian-tor', + mode => '0600', + owner => 'debian-tor', + group => 'debian-tor', } # config file headers concat::fragment { '00.header': ensure => present, content => template('tor/torrc.header.erb'), - owner => 'debian-tor', - group => 'debian-tor', - mode => '0644', order => 00, target => $tor::daemon::config_file, } @@ -68,9 +65,6 @@ class tor::daemon::base inherits tor::base { # global configurations concat::fragment { '01.global': content => template('tor/torrc.global.erb'), - owner => 'debian-tor', - group => 'debian-tor', - mode => '0644', order => 01, target => $tor::daemon::config_file, } diff --git a/puppet/modules/tor/manifests/daemon/bridge.pp b/puppet/modules/tor/manifests/daemon/bridge.pp index 063f5656..83d74e07 100644 --- a/puppet/modules/tor/manifests/daemon/bridge.pp +++ b/puppet/modules/tor/manifests/daemon/bridge.pp @@ -8,9 +8,6 @@ define tor::daemon::bridge( concat::fragment { "10.bridge.${name}": ensure => $ensure, content => template('tor/torrc.bridge.erb'), - owner => 'debian-tor', - group => 'debian-tor', - mode => '0644', order => 10, target => $tor::daemon::config_file, } diff --git a/puppet/modules/tor/manifests/daemon/control.pp b/puppet/modules/tor/manifests/daemon/control.pp index 01726562..ee425f33 100644 --- a/puppet/modules/tor/manifests/daemon/control.pp +++ b/puppet/modules/tor/manifests/daemon/control.pp @@ -7,20 +7,20 @@ define tor::daemon::control( $cookie_auth_file_group_readable = '', $ensure = present ) { - if $cookie_authentication == '0' and $hashed_control_password == '' and $ensure != 'absent' { - fail('You need to define the tor control password') - } + if $cookie_authentication == '0' + and $hashed_control_password == '' + and $ensure != 'absent' { + fail('You need to define the tor control password') + } - if $cookie_authentication == 0 and ($cookie_auth_file != '' or $cookie_auth_file_group_readable != '') { - notice('You set a tor cookie authentication option, but do not have cookie_authentication on') - } + if $cookie_authentication == 0 + and ($cookie_auth_file != '' or $cookie_auth_file_group_readable != '') { + notice('You set a tor cookie authentication option, but do not have cookie_authentication on') # lint:ignore:80chars + } concat::fragment { '04.control': ensure => $ensure, content => template('tor/torrc.control.erb'), - owner => 'debian-tor', - group => 'debian-tor', - mode => '0600', order => 04, target => $tor::daemon::config_file, } diff --git a/puppet/modules/tor/manifests/daemon/directory.pp b/puppet/modules/tor/manifests/daemon/directory.pp index d877a861..e2e405da 100644 --- a/puppet/modules/tor/manifests/daemon/directory.pp +++ b/puppet/modules/tor/manifests/daemon/directory.pp @@ -8,9 +8,6 @@ define tor::daemon::directory ( concat::fragment { '06.directory': ensure => $ensure, content => template('tor/torrc.directory.erb'), - owner => 'debian-tor', - group => 'debian-tor', - mode => '0644', order => 06, target => $tor::daemon::config_file, } diff --git a/puppet/modules/tor/manifests/daemon/dns.pp b/puppet/modules/tor/manifests/daemon/dns.pp index 4677f24d..e8d4fc88 100644 --- a/puppet/modules/tor/manifests/daemon/dns.pp +++ b/puppet/modules/tor/manifests/daemon/dns.pp @@ -7,9 +7,6 @@ define tor::daemon::dns( concat::fragment { "08.dns.${name}": ensure => $ensure, content => template('tor/torrc.dns.erb'), - owner => 'debian-tor', - group => 'debian-tor', - mode => '0644', order => '08', target => $tor::daemon::config_file, } diff --git a/puppet/modules/tor/manifests/daemon/exit_policy.pp b/puppet/modules/tor/manifests/daemon/exit_policy.pp index f459ece7..df0fb999 100644 --- a/puppet/modules/tor/manifests/daemon/exit_policy.pp +++ b/puppet/modules/tor/manifests/daemon/exit_policy.pp @@ -8,9 +8,6 @@ define tor::daemon::exit_policy( concat::fragment { "07.exit_policy.${name}": ensure => $ensure, content => template('tor/torrc.exit_policy.erb'), - owner => 'debian-tor', - group => 'debian-tor', - mode => '0644', order => 07, target => $tor::daemon::config_file, } diff --git a/puppet/modules/tor/manifests/daemon/hidden_service.pp b/puppet/modules/tor/manifests/daemon/hidden_service.pp index c8272116..07121bd6 100644 --- a/puppet/modules/tor/manifests/daemon/hidden_service.pp +++ b/puppet/modules/tor/manifests/daemon/hidden_service.pp @@ -1,17 +1,21 @@ # hidden services definition define tor::daemon::hidden_service( - $ports = [], - $data_dir = $tor::daemon::data_dir, - $ensure = present ) { + $ports = [], + $single_hop = false, + $data_dir = $tor::daemon::data_dir, + $ensure = present ) { + + + if $single_hop { + file { "${$data_dir}/${$name}/onion_service_non_anonymous": + ensure => 'present', + } + } concat::fragment { "05.hidden_service.${name}": ensure => $ensure, content => template('tor/torrc.hidden_service.erb'), - owner => 'debian-tor', - group => 'debian-tor', - mode => '0644', order => 05, target => $tor::daemon::config_file, } } - diff --git a/puppet/modules/tor/manifests/daemon/map_address.pp b/puppet/modules/tor/manifests/daemon/map_address.pp index 270eac21..ac624a0a 100644 --- a/puppet/modules/tor/manifests/daemon/map_address.pp +++ b/puppet/modules/tor/manifests/daemon/map_address.pp @@ -7,9 +7,6 @@ define tor::daemon::map_address( concat::fragment { "08.map_address.${name}": ensure => $ensure, content => template('tor/torrc.map_address.erb'), - owner => 'debian-tor', - group => 'debian-tor', - mode => '0644', order => '08', target => $tor::daemon::config_file, } diff --git a/puppet/modules/tor/manifests/daemon/relay.pp b/puppet/modules/tor/manifests/daemon/relay.pp index ff528937..555587cd 100644 --- a/puppet/modules/tor/manifests/daemon/relay.pp +++ b/puppet/modules/tor/manifests/daemon/relay.pp @@ -33,9 +33,6 @@ define tor::daemon::relay( concat::fragment { '03.relay': ensure => $ensure, content => template('tor/torrc.relay.erb'), - owner => 'debian-tor', - group => 'debian-tor', - mode => '0644', order => 03, target => $tor::daemon::config_file, } diff --git a/puppet/modules/tor/manifests/daemon/snippet.pp b/puppet/modules/tor/manifests/daemon/snippet.pp index b9089b40..7e1494c5 100644 --- a/puppet/modules/tor/manifests/daemon/snippet.pp +++ b/puppet/modules/tor/manifests/daemon/snippet.pp @@ -6,9 +6,6 @@ define tor::daemon::snippet( concat::fragment { "99.snippet.${name}": ensure => $ensure, content => $content, - owner => 'debian-tor', - group => 'debian-tor', - mode => '0644', order => 99, target => $tor::daemon::config_file, } diff --git a/puppet/modules/tor/manifests/daemon/socks.pp b/puppet/modules/tor/manifests/daemon/socks.pp index 910461c9..54c8b6a2 100644 --- a/puppet/modules/tor/manifests/daemon/socks.pp +++ b/puppet/modules/tor/manifests/daemon/socks.pp @@ -6,9 +6,6 @@ define tor::daemon::socks( concat::fragment { '02.socks': content => template('tor/torrc.socks.erb'), - owner => 'debian-tor', - group => 'debian-tor', - mode => '0644', order => 02, target => $tor::daemon::config_file, } diff --git a/puppet/modules/tor/manifests/daemon/transparent.pp b/puppet/modules/tor/manifests/daemon/transparent.pp index 65d744f4..6ac7b44c 100644 --- a/puppet/modules/tor/manifests/daemon/transparent.pp +++ b/puppet/modules/tor/manifests/daemon/transparent.pp @@ -7,9 +7,6 @@ define tor::daemon::transparent( concat::fragment { "09.transparent.${name}": ensure => $ensure, content => template('tor/torrc.transparent.erb'), - owner => 'debian-tor', - group => 'debian-tor', - mode => '0644', order => '09', target => $tor::daemon::config_file, } diff --git a/puppet/modules/tor/manifests/munin.pp b/puppet/modules/tor/manifests/munin.pp index 4412337a..2a01175c 100644 --- a/puppet/modules/tor/manifests/munin.pp +++ b/puppet/modules/tor/manifests/munin.pp @@ -8,7 +8,7 @@ class tor::munin { } Munin::Plugin::Deploy { - config => "user debian-tor\n env.cookiefile /var/run/tor/control.authcookie\n env.port 19051" + config => "user debian-tor\n env.cookiefile /var/run/tor/control.authcookie\n env.port 19051" # lint:ignore:80chars } munin::plugin::deploy { 'tor_connections': diff --git a/puppet/modules/tor/manifests/repo.pp b/puppet/modules/tor/manifests/repo.pp index f6255995..95492191 100644 --- a/puppet/modules/tor/manifests/repo.pp +++ b/puppet/modules/tor/manifests/repo.pp @@ -1,3 +1,4 @@ +# setup repository for tor class tor::repo ( $ensure = present, $source_name = 'torproject.org', @@ -10,7 +11,7 @@ class tor::repo ( class { 'tor::repo::debian': } } default: { - fail("Unsupported managed repository for osfamily: ${::osfamily}, operatingsystem: ${::operatingsystem}, module ${module_name} currently only supports managing repos for osfamily Debian and Ubuntu") + fail("Unsupported managed repository for osfamily: ${::osfamily}, operatingsystem: ${::operatingsystem}, module ${module_name} currently only supports managing repos for osfamily Debian and Ubuntu") # lint:ignore:80chars } } } diff --git a/puppet/modules/tor/manifests/repo/debian.pp b/puppet/modules/tor/manifests/repo/debian.pp index 174c3310..81976a2e 100644 --- a/puppet/modules/tor/manifests/repo/debian.pp +++ b/puppet/modules/tor/manifests/repo/debian.pp @@ -1,6 +1,6 @@ # PRIVATE CLASS: do not use directly class tor::repo::debian inherits tor::repo { - apt::source { $source_name: + apt::source { $tor::repo::source_name: ensure => $::tor::repo::ensure, location => $::tor::repo::location, key => $::tor::repo::key, diff --git a/puppet/modules/tor/templates/torrc.directory.erb b/puppet/modules/tor/templates/torrc.directory.erb index 1af9f40f..c7dc4ab5 100644 --- a/puppet/modules/tor/templates/torrc.directory.erb +++ b/puppet/modules/tor/templates/torrc.directory.erb @@ -1,11 +1,11 @@ # directory listing -<% if port != '0' -%> +<% if @port != '0' -%> DirPort <%= @port %> <% end -%> <% listen_addresses.each do |listen_address| -%> DirListenAddress <%= listen_address %> <% end -%> <% if @port_front_page != '' -%> -DirPortFrontPage <%= port_front_page %> +DirPortFrontPage <%= @port_front_page %> <%- end -%> diff --git a/puppet/modules/tor/templates/torrc.global.erb b/puppet/modules/tor/templates/torrc.global.erb index f577673d..a02afc8e 100644 --- a/puppet/modules/tor/templates/torrc.global.erb +++ b/puppet/modules/tor/templates/torrc.global.erb @@ -12,8 +12,8 @@ Log notice syslog Log <%= log_rule %> <% end -%> <% end -%> -<%- if @safe_logging != 1 then -%> -SafeLogging <%= @safe_logging %> +<%- if (v=scope.lookupvar('tor::daemon::safe_logging')) != '1' then -%> +SafeLogging <%= v %> <%- end -%> <% if (v=scope.lookupvar('tor::daemon::automap_hosts_on_resolve')) != '0' -%> diff --git a/puppet/modules/tor/templates/torrc.hidden_service.erb b/puppet/modules/tor/templates/torrc.hidden_service.erb index 4dec0b25..5b6afe1c 100644 --- a/puppet/modules/tor/templates/torrc.hidden_service.erb +++ b/puppet/modules/tor/templates/torrc.hidden_service.erb @@ -1,3 +1,9 @@ +<% if @single_hop != false %> +HiddenServiceSingleHopMode 1 +HiddenServiceNonAnonymousMode 1 +SOCKSPort 0 +<% end %> + # hidden service <%= @name %> HiddenServiceDir <%= @data_dir %>/<%= @name %> <% @ports.each do |port| -%> -- cgit v1.2.3 From c393af8fd5321b8ddf547aed22f833899e56e20e Mon Sep 17 00:00:00 2001 From: Micah Anderson Date: Mon, 24 Apr 2017 12:08:10 -0400 Subject: Lint --- .../modules/site_static/manifests/hidden_service.pp | 20 ++++++++++---------- .../modules/site_webapp/manifests/hidden_service.pp | 20 ++++++++++---------- 2 files changed, 20 insertions(+), 20 deletions(-) diff --git a/puppet/modules/site_static/manifests/hidden_service.pp b/puppet/modules/site_static/manifests/hidden_service.pp index f1f15f8e..8a10398a 100644 --- a/puppet/modules/site_static/manifests/hidden_service.pp +++ b/puppet/modules/site_static/manifests/hidden_service.pp @@ -5,18 +5,18 @@ class site_static::hidden_service { tor::daemon::hidden_service { 'static': ports => [ '80 127.0.0.1:80'] } file { '/var/lib/tor/webapp/': - ensure => directory, - owner => 'debian-tor', - group => 'debian-tor', - mode => '2700'; + ensure => directory, + owner => 'debian-tor', + group => 'debian-tor', + mode => '2700'; '/var/lib/tor/static/private_key': - ensure => present, - source => "/srv/leap/files/nodes/${::hostname}/tor.key", - owner => 'debian-tor', - group => 'debian-tor', - mode => '0600', - notify => Service['tor']; + ensure => present, + source => "/srv/leap/files/nodes/${::hostname}/tor.key", + owner => 'debian-tor', + group => 'debian-tor', + mode => '0600', + notify => Service['tor']; '/var/lib/tor/static/hostname': ensure => present, diff --git a/puppet/modules/site_webapp/manifests/hidden_service.pp b/puppet/modules/site_webapp/manifests/hidden_service.pp index d2662b65..81d431cd 100644 --- a/puppet/modules/site_webapp/manifests/hidden_service.pp +++ b/puppet/modules/site_webapp/manifests/hidden_service.pp @@ -15,18 +15,18 @@ class site_webapp::hidden_service { file { '/var/lib/tor/webapp/': - ensure => directory, - owner => 'debian-tor', - group => 'debian-tor', - mode => '2700'; + ensure => directory, + owner => 'debian-tor', + group => 'debian-tor', + mode => '2700'; '/var/lib/tor/webapp/private_key': - ensure => present, - source => "/srv/leap/files/nodes/${::hostname}/tor.key", - owner => 'debian-tor', - group => 'debian-tor', - mode => '0600', - notify => Service['tor']; + ensure => present, + source => "/srv/leap/files/nodes/${::hostname}/tor.key", + owner => 'debian-tor', + group => 'debian-tor', + mode => '0600', + notify => Service['tor']; '/var/lib/tor/webapp/hostname': ensure => present, -- cgit v1.2.3 From ada9645de11d75701db8202f34de5c26a2b749c2 Mon Sep 17 00:00:00 2001 From: Micah Anderson Date: Mon, 24 Apr 2017 14:38:32 -0400 Subject: Add single-hop hidden service capability. This cuts the number of hops for a tor onion service from 6 to 3, speeding it up considerably. This removes the anonymity aspect of the service, so it must be enabled intentionally, knowing that the server's location no longer is hidden. --- provider_base/services/tor.json | 3 ++- puppet/modules/site_static/manifests/hidden_service.pp | 7 +++++-- puppet/modules/site_static/manifests/init.pp | 3 +-- puppet/modules/site_tor/manifests/init.pp | 2 +- puppet/modules/site_webapp/manifests/hidden_service.pp | 5 ++++- 5 files changed, 13 insertions(+), 7 deletions(-) diff --git a/provider_base/services/tor.json b/provider_base/services/tor.json index e80310fe..a0d44fef 100644 --- a/provider_base/services/tor.json +++ b/provider_base/services/tor.json @@ -9,7 +9,8 @@ "key_type": "RSA", "public_key": "= tor_public_key_path(:node_tor_pub_key, tor.hidden_service.key_type) if tor.hidden_service.active", "private_key": "= tor_private_key_path(:node_tor_priv_key, tor.hidden_service.key_type) if tor.hidden_service.active", - "address": "=> tor.hidden_service.active && onion_address(:node_tor_pub_key)" + "address": "=> tor.hidden_service.active && onion_address(:node_tor_pub_key)", + "single_hop": false } } } diff --git a/puppet/modules/site_static/manifests/hidden_service.pp b/puppet/modules/site_static/manifests/hidden_service.pp index 8a10398a..b64a35bc 100644 --- a/puppet/modules/site_static/manifests/hidden_service.pp +++ b/puppet/modules/site_static/manifests/hidden_service.pp @@ -1,8 +1,11 @@ # create hidden service for static sites -class site_static::hidden_service { +class site_static::hidden_service ( $single_hop = false ) { include tor::daemon - tor::daemon::hidden_service { 'static': ports => [ '80 127.0.0.1:80'] } + tor::daemon::hidden_service { 'static': + ports => [ '80 127.0.0.1:80'], + single_hop => $single_hop + } file { '/var/lib/tor/webapp/': ensure => directory, diff --git a/puppet/modules/site_static/manifests/init.pp b/puppet/modules/site_static/manifests/init.pp index dd3f912d..8be791e5 100644 --- a/puppet/modules/site_static/manifests/init.pp +++ b/puppet/modules/site_static/manifests/init.pp @@ -74,8 +74,7 @@ class site_static { if $tor { $hidden_service = $tor['hidden_service'] $tor_domain = "${hidden_service['address']}.onion" - if $hidden_service['active'] { - include site_static::hidden_service + class { 'site_static::hidden_service': single_hop => $hidden_service['single_hop'] } # Currently, we only support a single hidden service address per server. # So if there is more than one domain configured, then we need to make sure diff --git a/puppet/modules/site_tor/manifests/init.pp b/puppet/modules/site_tor/manifests/init.pp index 2207a5a9..8a92a944 100644 --- a/puppet/modules/site_tor/manifests/init.pp +++ b/puppet/modules/site_tor/manifests/init.pp @@ -20,7 +20,7 @@ class site_tor { } include site_config::default - include tor::daemon + class { 'tor::daemon': ensure_version => latest } tor::daemon::relay { $nickname: port => 9001, address => $address, diff --git a/puppet/modules/site_webapp/manifests/hidden_service.pp b/puppet/modules/site_webapp/manifests/hidden_service.pp index 81d431cd..6651df86 100644 --- a/puppet/modules/site_webapp/manifests/hidden_service.pp +++ b/puppet/modules/site_webapp/manifests/hidden_service.pp @@ -11,7 +11,10 @@ class site_webapp::hidden_service { include apache::module::removeip include tor::daemon - tor::daemon::hidden_service { 'webapp': ports => [ '80 127.0.0.1:80'] } + tor::daemon::hidden_service { 'webapp': + ports => [ '80 127.0.0.1:80'], + single_hop => $hidden_service['single_hop'] + } file { '/var/lib/tor/webapp/': -- cgit v1.2.3 From 9d096ace3692f67fe82a97d648c930c2da19a830 Mon Sep 17 00:00:00 2001 From: Micah Anderson Date: Tue, 25 Apr 2017 19:40:28 -0400 Subject: Add a production environment for demovpn, demomail Pull duplicated bits into a function --- .gitlab-ci.yml | 10 ++++++++++ tests/platform-ci/ci-build.sh | 36 ++++++++++++++++++++++++------------ 2 files changed, 34 insertions(+), 12 deletions(-) diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml index 5654238e..2cef28e0 100644 --- a/.gitlab-ci.yml +++ b/.gitlab-ci.yml @@ -75,3 +75,13 @@ latest: - master script: - su -c '/usr/bin/unbuffer ./ci-build.sh' cirunner + +production: + stage: deploy + environment: + name: production + only: + - master + when: manual + script: + - su -c '/usr/bin/unbuffer ./ci-build.sh' cirunner diff --git a/tests/platform-ci/ci-build.sh b/tests/platform-ci/ci-build.sh index b39f6874..e25b8096 100755 --- a/tests/platform-ci/ci-build.sh +++ b/tests/platform-ci/ci-build.sh @@ -70,6 +70,21 @@ build_from_scratch() { LEAP_CMD info "${TAG}" } +run() { + echo "Cloning $1 repo: $2" + git clone -q --depth 1 "$2" + cd "$1" + git rev-parse HEAD + echo -n "Operating in the $1 directory: " + pwd + echo "Listing current node information..." + LEAP_CMD list + echo "Attempting a deploy..." + deploy + echo "Attempting to run tests..." + test +} + # # Main # @@ -89,18 +104,15 @@ set +x case "$CI_ENVIRONMENT_NAME" in latest) TAG='latest' - echo "Cloning ibex provider..." - git clone -q --depth 1 ssh://gitolite@leap.se/ibex - cd ibex - git rev-parse HEAD - echo -n "Operating in the ibex directory: " - pwd - echo "Listing current node information..." - LEAP_CMD list - echo "Attempting a deploy..." - deploy - echo "Attempting to run tests..." - test + run ibex ssh://gitolite@leap.se/ibex + ;; + production/mail) + TAG='demomail' + run bitmask ssh://gitolite@leap.se/bitmask + ;; + production/vpn) + TAG='demovpn' + run bitmask ssh://gitolite@leap.se/bitmask ;; *) # create node(s) with unique id so we can run tests in parallel -- cgit v1.2.3 From 0b3aef03cb113e997c2a654ef2f7b1674a0a8877 Mon Sep 17 00:00:00 2001 From: elijah Date: Tue, 25 Apr 2017 12:18:04 -0700 Subject: bugfix: ensure that nodes only have one environment specified (closes #8778) --- lib/leap_cli/config/manager.rb | 15 +++++++++++++-- 1 file changed, 13 insertions(+), 2 deletions(-) diff --git a/lib/leap_cli/config/manager.rb b/lib/leap_cli/config/manager.rb index d69a5808..a9f1a85f 100644 --- a/lib/leap_cli/config/manager.rb +++ b/lib/leap_cli/config/manager.rb @@ -342,14 +342,25 @@ module LeapCli if node.vagrant? return self.env("local") else - environment = self.env(default_environment) + environment = nil if node['tags'] node['tags'].to_a.each do |tag| if self.environment_names.include?(tag) - environment = self.env(tag) + if environment.nil? + environment = self.env(tag) + else + LeapCli::Util.bail! do + LeapCli.log( + :error, + "The node '%s' is invalid, because it cannot have two environments ('%s' and '%s')." % + [node.name, environment.name, tag] + ) + end + end end end end + environment ||= self.env(default_environment) return environment end end -- cgit v1.2.3 From a577fbf20357ae1bc611da975cde001ef9dfa310 Mon Sep 17 00:00:00 2001 From: varac Date: Wed, 26 Apr 2017 11:59:57 +0200 Subject: Improve ci-build.sh (Closes #8771) * Change environment names for clarity: . Use staging for deploying to latest . Use production environments to deploy to demo: production/vpn production/mail * Install leap_cli if not present and define default values * Remove old nodes from cached runs * Remove no longer used SEEDS variable * Debugging improvements: . Hide secrets when calling ci-build.sh with xtrace enabled . Use unbuffer to we can add debug output locally . Add debugging to build_from_scratch() --- .gitlab-ci.yml | 20 +++++++++---- tests/platform-ci/ci-build.sh | 66 +++++++++++++++++++++++++++++++++---------- 2 files changed, 66 insertions(+), 20 deletions(-) diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml index 2cef28e0..7b0f8852 100644 --- a/.gitlab-ci.yml +++ b/.gitlab-ci.yml @@ -60,26 +60,36 @@ deploy_test: except: - master script: - - su -c '/usr/bin/unbuffer ./ci-build.sh' cirunner + - su -c '/usr/bin/unbuffer bash -o pipefail ./ci-build.sh | /usr/bin/ts' cirunner # Latest job will only run on the master branch, which means all merge requests # that are created from branches don't get to deploy to the latest-ci server. # When a merge request is merged, then the latest job will deploy the code to # the latest provider, and the deployment will be recorded in an environment # named 'latest' -latest: +ci.leap.se: stage: deploy environment: name: staging only: - - master + - master + script: + - su -c '/usr/bin/unbuffer ./ci-build.sh' cirunner + +demo.bitmask.net: + stage: deploy + environment: + name: production/vpn + only: + - master + when: manual script: - su -c '/usr/bin/unbuffer ./ci-build.sh' cirunner -production: +mail.bitmask.net: stage: deploy environment: - name: production + name: production/mail only: - master when: manual diff --git a/tests/platform-ci/ci-build.sh b/tests/platform-ci/ci-build.sh index e25b8096..747e09a7 100755 --- a/tests/platform-ci/ci-build.sh +++ b/tests/platform-ci/ci-build.sh @@ -22,11 +22,16 @@ set -e # so exit codes will be caught correctly. set -o pipefail -# we wrap the whole script in curly braces so we can pipe it all through ts to -# get timestamps. If we put it outside of the script, then we can't get proper -# pipefail results. +# Check if scipt is run in debug mode so we can hide secrets +if [[ "$-" =~ 'x' ]] +then + echo 'Running with xtrace enabled!' + xtrace=true +else + echo 'Running with xtrace disabled!' + xtrace=false +fi -{ # leap_platform/tests/platform-ci # shellcheck disable=SC2086 ROOTDIR=$(readlink -f "$(dirname $0)") @@ -34,9 +39,20 @@ ROOTDIR=$(readlink -f "$(dirname $0)") # leap_platform PLATFORMDIR=$(readlink -f "${ROOTDIR}/../..") -LEAP_CMD() { - /usr/local/bin/bundle exec leap -v2 --yes "$@" -} +# In the gitlab CI pipeline leap is installed in a different +# stage by bundle. To debug you can run a single CI job locally +# so we install leap_cli as gem here. +if /usr/local/bin/bundle exec leap >/dev/null 2>&1 +then + LEAP_CMD() { + /usr/local/bin/bundle exec leap -v2 --yes "$@" + } +else + sudo gem install leap_cli + LEAP_CMD() { + leap -v2 --yes "$@" + } +fi deploy() { LEAP_CMD deploy "$TAG" @@ -54,19 +70,38 @@ build_from_scratch() { # Create cloud.json needed for `leap vm` commands using AWS credentials which jq || ( apt-get update -y && apt-get install jq -y ) + + # Dsiable xtrace + set +x /usr/bin/jq ".platform_ci.auth |= .+ {\"aws_access_key_id\":\"$AWS_ACCESS_KEY\", \"aws_secret_access_key\":\"$AWS_SECRET_KEY\"}" < cloud.json.template > cloud.json + # Enable xtrace again only if it was set at beginning of script + [[ $xtrace == true ]] && set -x [ -d "./tags" ] || mkdir "./tags" /bin/echo "{\"environment\": \"$TAG\"}" | /usr/bin/json_pp > "${PROVIDERDIR}/tags/${TAG}.json" pwd + +# remove old cached nodes + echo "Removing old cached nodes..." + find nodes -name 'citest*' -exec rm {} \; + + echo "Listing current VM status..." LEAP_CMD vm status "$TAG" # shellcheck disable=SC2086 - LEAP_CMD vm add "$NAME" services:"$SERVICES" tags:"$TAG" $SEEDS + echo "Adding VM $NAME with the services: $SERVICES and the tags: $TAG" + LEAP_CMD vm add "$NAME" services:"$SERVICES" tags:"$TAG" + echo "Compiling $TAG..." LEAP_CMD compile "$TAG" + echo "Listing current VM status for TAG: $TAG..." LEAP_CMD vm status "$TAG" + echo "Running leap list..." + LEAP_CMD list + + echo "Running leap node init on TAG: $TAG" LEAP_CMD node init "$TAG" + echo "Running leap info on $TAG" LEAP_CMD info "${TAG}" } @@ -101,30 +136,32 @@ set +x /bin/chmod 600 ~/.ssh/id_rsa /bin/cp "${ROOTDIR}/provider/users/gitlab-runner/gitlab-runner_ssh.pub" ~/.ssh/id_rsa.pub +# Enable xtrace again only if it was set at beginning of script +[[ $xtrace == true ]] && set -x + case "$CI_ENVIRONMENT_NAME" in - latest) + staging) TAG='latest' run ibex ssh://gitolite@leap.se/ibex ;; - production/mail) + demo/mail) TAG='demomail' run bitmask ssh://gitolite@leap.se/bitmask ;; - production/vpn) + demo/vpn) TAG='demovpn' run bitmask ssh://gitolite@leap.se/bitmask ;; *) # create node(s) with unique id so we can run tests in parallel - NAME="citest${CI_BUILD_ID}" + NAME="citest${CI_BUILD_ID:-0}" # when using gitlab-runner locally, CI_BUILD_ID is always 1 which # will conflict with running/terminating AWS instances in subsequent runs # therefore we pick a random number in this case - [ "$CI_BUILD_ID" -eq "1" ] && NAME+="000${RANDOM}" + [ "${CI_BUILD_ID:-0}" -eq "1" ] && NAME+="000${RANDOM}" TAG='single' SERVICES='couchdb,soledad,mx,webapp,tor,monitor' - SEEDS='' build_from_scratch # Deploy and test deploy @@ -134,4 +171,3 @@ case "$CI_ENVIRONMENT_NAME" in [ -f "nodes/${NAME}.json" ] && /bin/rm "nodes/${NAME}.json" ;; esac -} | /usr/bin/ts -s -- cgit v1.2.3 From 8bc60685875e2eb289d0d860ebe7ba7839eb20e2 Mon Sep 17 00:00:00 2001 From: Micah Anderson Date: Thu, 27 Apr 2017 13:41:22 -0400 Subject: change environment names to match ci-build.sh --- .gitlab-ci.yml | 4 ++-- tests/platform-ci/ci-build.sh | 4 ++-- 2 files changed, 4 insertions(+), 4 deletions(-) diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml index 7b0f8852..a1ad49a0 100644 --- a/.gitlab-ci.yml +++ b/.gitlab-ci.yml @@ -79,7 +79,7 @@ ci.leap.se: demo.bitmask.net: stage: deploy environment: - name: production/vpn + name: production/demo/vpn only: - master when: manual @@ -89,7 +89,7 @@ demo.bitmask.net: mail.bitmask.net: stage: deploy environment: - name: production/mail + name: production/demo/mail only: - master when: manual diff --git a/tests/platform-ci/ci-build.sh b/tests/platform-ci/ci-build.sh index 747e09a7..34876a73 100755 --- a/tests/platform-ci/ci-build.sh +++ b/tests/platform-ci/ci-build.sh @@ -144,11 +144,11 @@ case "$CI_ENVIRONMENT_NAME" in TAG='latest' run ibex ssh://gitolite@leap.se/ibex ;; - demo/mail) + production/demo/mail) TAG='demomail' run bitmask ssh://gitolite@leap.se/bitmask ;; - demo/vpn) + production/demo/vpn) TAG='demovpn' run bitmask ssh://gitolite@leap.se/bitmask ;; -- cgit v1.2.3 From 22c947c33a452e912859832c78bd3660b6734cc6 Mon Sep 17 00:00:00 2001 From: Micah Anderson Date: Tue, 2 May 2017 12:32:05 -0400 Subject: Add signed-by option to sources.list (Closes: #8425) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit This gets us a simple apt repository privilege separation: (a) our key can't be used to forge other repos (b) other keys can't be used to forge our repo. From sources.list(5): · Signed-By (signed-by) is either an absolute path to a keyring file (has to be accessible and readable for the _apt user, so ensure everyone has read-permissions on the file) or one or more fingerprints of keys either in the trusted.gpg keyring or in the keyrings in the trusted.gpg.d/ directory (see apt-key fingerprint). If the option is set, only the key(s) in this keyring or only the keys with these fingerprints are used for the apt-secure(8) verification of this repository. Defaults to the value of the option with the same name if set in the previously acquired Release file. Otherwise all keys in the trusted keyrings are considered valid signers for this repository. --- puppet/modules/site_apt/manifests/leap_repo.pp | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/puppet/modules/site_apt/manifests/leap_repo.pp b/puppet/modules/site_apt/manifests/leap_repo.pp index 3d95d8b6..7c6c49c5 100644 --- a/puppet/modules/site_apt/manifests/leap_repo.pp +++ b/puppet/modules/site_apt/manifests/leap_repo.pp @@ -4,8 +4,14 @@ class site_apt::leap_repo { $platform = hiera_hash('platform') $major_version = $platform['major_version'] + if $::site_apt::apt_url_platform_basic =~ /.*experimental.*/ { + $archive_key = '/usr/share/keyrings/leap-experimental-archive.gpg' + } else { + $archive_key = '/usr/share/keyrings/leap-archive.gpg' + } + apt::sources_list { 'leap.list': - content => "deb ${::site_apt::apt_url_platform_basic} ${::site_apt::apt_platform_codename} ${::site_apt::apt_platform_component}\n", + content => "deb [signed-by=${archive_key}] ${::site_apt::apt_url_platform_basic} ${::site_apt::apt_platform_codename} ${::site_apt::apt_platform_component}\n", before => Exec[refresh_apt] } -- cgit v1.2.3 From f1e63b2de0f18523bda56e41e76ab137add8c3ba Mon Sep 17 00:00:00 2001 From: Micah Anderson Date: Tue, 2 May 2017 15:39:56 -0400 Subject: Limit ci.leap.se deployment to leap/master (Closes #8782) --- .gitlab-ci.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml index a1ad49a0..6da735d4 100644 --- a/.gitlab-ci.yml +++ b/.gitlab-ci.yml @@ -72,7 +72,7 @@ ci.leap.se: environment: name: staging only: - - master + - master@leap/platform script: - su -c '/usr/bin/unbuffer ./ci-build.sh' cirunner -- cgit v1.2.3 From 68e9a28da2db4cb494bc19a1aeaa0663cb286414 Mon Sep 17 00:00:00 2001 From: Micah Anderson Date: Tue, 2 May 2017 16:23:20 -0400 Subject: Restructure site_tor to be more clear and re-usable (fixes #8784). This makes a more clear site_tor::relay class that the leap service includes, and a more generic site_tor class that other classes can depend on for setting up the initial install. --- puppet/manifests/site.pp | 2 +- .../site_static/manifests/hidden_service.pp | 2 +- puppet/modules/site_tor/manifests/init.pp | 41 +------------------- puppet/modules/site_tor/manifests/relay.pp | 45 ++++++++++++++++++++++ .../site_webapp/manifests/hidden_service.pp | 2 +- 5 files changed, 49 insertions(+), 43 deletions(-) create mode 100644 puppet/modules/site_tor/manifests/relay.pp diff --git a/puppet/manifests/site.pp b/puppet/manifests/site.pp index 3bf6a5c1..e243c5df 100644 --- a/puppet/manifests/site.pp +++ b/puppet/manifests/site.pp @@ -45,7 +45,7 @@ node default { } if member($services, 'tor') { - include site_tor + include site_tor::relay } if member($services, 'mx') { diff --git a/puppet/modules/site_static/manifests/hidden_service.pp b/puppet/modules/site_static/manifests/hidden_service.pp index b64a35bc..31cf328e 100644 --- a/puppet/modules/site_static/manifests/hidden_service.pp +++ b/puppet/modules/site_static/manifests/hidden_service.pp @@ -1,7 +1,7 @@ # create hidden service for static sites class site_static::hidden_service ( $single_hop = false ) { - include tor::daemon + include site_tor tor::daemon::hidden_service { 'static': ports => [ '80 127.0.0.1:80'], single_hop => $single_hop diff --git a/puppet/modules/site_tor/manifests/init.pp b/puppet/modules/site_tor/manifests/init.pp index 8a92a944..356053c1 100644 --- a/puppet/modules/site_tor/manifests/init.pp +++ b/puppet/modules/site_tor/manifests/init.pp @@ -1,45 +1,6 @@ +# generic configuration needed for tor class site_tor { - tag 'leap_service' - Class['site_config::default'] -> Class['site_tor'] - $tor = hiera('tor') - $bandwidth_rate = $tor['bandwidth_rate'] - $tor_type = $tor['type'] - $nickname = $tor['nickname'] - $contact_emails = join($tor['contacts'],', ') - $family = $tor['family'] - - $address = hiera('ip_address') - - $openvpn = hiera('openvpn', undef) - if $openvpn { - $openvpn_ports = $openvpn['ports'] - } - else { - $openvpn_ports = [] - } - - include site_config::default class { 'tor::daemon': ensure_version => latest } - tor::daemon::relay { $nickname: - port => 9001, - address => $address, - contact_info => obfuscate_email($contact_emails), - bandwidth_rate => $bandwidth_rate, - my_family => $family - } - - if ( $tor_type == 'exit'){ - # Only enable the daemon directory if the node isn't also a webapp node - # or running openvpn on port 80 - if ! member($::services, 'webapp') and ! member($openvpn_ports, '80') { - tor::daemon::directory { $::hostname: port => 80 } - } - } - else { - include site_tor::disable_exit - } - - include site_shorewall::tor } diff --git a/puppet/modules/site_tor/manifests/relay.pp b/puppet/modules/site_tor/manifests/relay.pp new file mode 100644 index 00000000..fcb83bc1 --- /dev/null +++ b/puppet/modules/site_tor/manifests/relay.pp @@ -0,0 +1,45 @@ +class site_tor::relay { + tag 'leap_service' + Class['site_config::default'] -> Class['site_tor::relay'] + + $tor = hiera('tor') + $bandwidth_rate = $tor['bandwidth_rate'] + $tor_type = $tor['type'] + $nickname = $tor['nickname'] + $contact_emails = join($tor['contacts'],', ') + $family = $tor['family'] + + $address = hiera('ip_address') + + $openvpn = hiera('openvpn', undef) + if $openvpn { + $openvpn_ports = $openvpn['ports'] + } + else { + $openvpn_ports = [] + } + + include site_config::default + include site_tor + + tor::daemon::relay { $nickname: + port => 9001, + address => $address, + contact_info => obfuscate_email($contact_emails), + bandwidth_rate => $bandwidth_rate, + my_family => $family + } + + if ( $tor_type == 'exit'){ + # Only enable the daemon directory if the node isn't also a webapp node + # or running openvpn on port 80 + if ! member($::services, 'webapp') and ! member($openvpn_ports, '80') { + tor::daemon::directory { $::hostname: port => 80 } + } + } + else { + include site_tor::disable_exit + } + + include site_shorewall::tor +} diff --git a/puppet/modules/site_webapp/manifests/hidden_service.pp b/puppet/modules/site_webapp/manifests/hidden_service.pp index 6651df86..3f3f1d0c 100644 --- a/puppet/modules/site_webapp/manifests/hidden_service.pp +++ b/puppet/modules/site_webapp/manifests/hidden_service.pp @@ -10,7 +10,7 @@ class site_webapp::hidden_service { include apache::module::expires include apache::module::removeip - include tor::daemon + include site_tor tor::daemon::hidden_service { 'webapp': ports => [ '80 127.0.0.1:80'], single_hop => $hidden_service['single_hop'] -- cgit v1.2.3 From 449d6169b8d0c7f31279b445f5dd103b244d7382 Mon Sep 17 00:00:00 2001 From: Micah Anderson Date: Tue, 2 May 2017 16:24:15 -0400 Subject: Install tor from backports (fixes #8783). The newer version is needed for the single-hop functionality. --- puppet/modules/site_tor/manifests/init.pp | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/puppet/modules/site_tor/manifests/init.pp b/puppet/modules/site_tor/manifests/init.pp index 356053c1..5e209ba8 100644 --- a/puppet/modules/site_tor/manifests/init.pp +++ b/puppet/modules/site_tor/manifests/init.pp @@ -1,6 +1,14 @@ # generic configuration needed for tor class site_tor { + # Ensure the tor version is the latest from backports + # see https://0xacab.org/leap/platform/issues/8783 + apt::preferences_snippet { 'tor': + package => 'tor', + release => "${::lsbdistcodename}-backports", + priority => 999, + before => Class['tor::daemon'] } + class { 'tor::daemon': ensure_version => latest } } -- cgit v1.2.3 From 2af7329fb6966d172c368bbd0ceb5e8c07bef710 Mon Sep 17 00:00:00 2001 From: varac Date: Wed, 3 May 2017 10:58:14 +0200 Subject: Add timestamps to all platform deploys Resolves: #8791 --- .gitlab-ci.yml | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml index 6da735d4..97eda43c 100644 --- a/.gitlab-ci.yml +++ b/.gitlab-ci.yml @@ -60,7 +60,7 @@ deploy_test: except: - master script: - - su -c '/usr/bin/unbuffer bash -o pipefail ./ci-build.sh | /usr/bin/ts' cirunner + - su -c '/usr/bin/unbuffer /bin/bash -o pipefail ./ci-build.sh | /usr/bin/ts' cirunner # Latest job will only run on the master branch, which means all merge requests # that are created from branches don't get to deploy to the latest-ci server. @@ -74,7 +74,7 @@ ci.leap.se: only: - master@leap/platform script: - - su -c '/usr/bin/unbuffer ./ci-build.sh' cirunner + - su -c '/usr/bin/unbuffer /bin/bash -o pipefail ./ci-build.sh | /usr/bin/ts' cirunner demo.bitmask.net: stage: deploy @@ -84,7 +84,7 @@ demo.bitmask.net: - master when: manual script: - - su -c '/usr/bin/unbuffer ./ci-build.sh' cirunner + - su -c '/usr/bin/unbuffer /bin/bash -o pipefail ./ci-build.sh | /usr/bin/ts' cirunner mail.bitmask.net: stage: deploy @@ -94,4 +94,4 @@ mail.bitmask.net: - master when: manual script: - - su -c '/usr/bin/unbuffer ./ci-build.sh' cirunner + - su -c '/usr/bin/unbuffer /bin/bash -o pipefail ./ci-build.sh | /usr/bin/ts' cirunner -- cgit v1.2.3 From a49126a0587323f8f8fb5f11d22c87b824e0201a Mon Sep 17 00:00:00 2001 From: varac Date: Wed, 3 May 2017 11:00:10 +0200 Subject: Ignore rbenv files --- .gitignore | 3 +++ 1 file changed, 3 insertions(+) diff --git a/.gitignore b/.gitignore index d80ef422..5c9d135a 100644 --- a/.gitignore +++ b/.gitignore @@ -24,3 +24,6 @@ /tests/platform-ci/provider/test /builds + +/.ruby-version +/.rbenv-gemsets -- cgit v1.2.3 From 65ca1b572024d30dfc12e9fc927099de41c30bd4 Mon Sep 17 00:00:00 2001 From: varac Date: Wed, 10 May 2017 16:33:40 +0200 Subject: Depend soledad-server on ssl-cert package We should include this in soledad-server package as dependency but until we sorted out this, we depend soledad-server on ssl-cert in the platform. see https://0xacab.org/leap/soledad/issues/8849 for --- puppet/modules/soledad/manifests/server.pp | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/puppet/modules/soledad/manifests/server.pp b/puppet/modules/soledad/manifests/server.pp index 81f51188..3b6a2314 100644 --- a/puppet/modules/soledad/manifests/server.pp +++ b/puppet/modules/soledad/manifests/server.pp @@ -54,7 +54,10 @@ class soledad::server { package { $sources['soledad']['package']: ensure => $sources['soledad']['revision'], - require => Class['site_apt::leap_repo']; + require => [ + Class['site_apt::leap_repo'], + Package['ssl-cert'] + ]; } file { '/etc/default/soledad': -- cgit v1.2.3 From c2ea863eb2cf301d8c0bf3e2cc52f548f7e86016 Mon Sep 17 00:00:00 2001 From: Micah Anderson Date: Wed, 10 May 2017 11:07:37 -0400 Subject: fix CI image location --- .gitlab-ci.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml index 97eda43c..f515337f 100644 --- a/.gitlab-ci.yml +++ b/.gitlab-ci.yml @@ -1,4 +1,4 @@ -image: 0xacab.org:4567/leap-docker/ruby:latest +image: 0xacab.org:4567/leap/docker/ruby:latest # This is for caching the gems not only between the stages, but also persistent # on the gitlab-runner so we don't need to install from scratch on every pipeline -- cgit v1.2.3 From 04a1c0bc3ed7173836ed790776be1a13437310a7 Mon Sep 17 00:00:00 2001 From: Tulio Casagrande Date: Wed, 10 May 2017 14:43:46 -0300 Subject: Remove pixelated submodule from the example provider --- tests/example-provider/vagrant/add-pixelated.sh | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/tests/example-provider/vagrant/add-pixelated.sh b/tests/example-provider/vagrant/add-pixelated.sh index f9908947..38a7ea47 100755 --- a/tests/example-provider/vagrant/add-pixelated.sh +++ b/tests/example-provider/vagrant/add-pixelated.sh @@ -6,8 +6,8 @@ cd "$PROVIDERDIR" -if ! git submodule status files/puppet/modules/pixelated > /dev/null 2>&1; then - git submodule add https://github.com/pixelated/puppet-pixelated.git files/puppet/modules/pixelated +if ! [ -d files/puppet/modules/pixelated ]; then + git clone https://github.com/pixelated/puppet-pixelated.git files/puppet/modules/pixelated fi echo '{}' > services/pixelated.json @@ -26,7 +26,7 @@ $LEAP $OPTS -v 2 test --continue echo -e '\n===========================================================================================================\n\n' echo -e 'You are now ready to use your vagrant Pixelated provider.\n' -echo -e 'The LEAP webapp is available at https://localhost:4443. Use it to register an account before using the Pixelated Useragent.\n' -echo -e 'The Pixelated Useragent is available at https://localhost:8080\n' +echo -e 'The LEAP webapp is available at https://localhost:4443. Use it to register an account before using the Pixelated User Agent.\n' +echo -e 'The Pixelated User Agent is available at https://localhost:8080\n' echo -e 'Please add an exception for both sites in your browser dialog to allow the self-signed certificate.\n' -- cgit v1.2.3 From 82a9d9cd3b11b5278a1e06c61c0e0b548f533593 Mon Sep 17 00:00:00 2001 From: varac Date: Wed, 10 May 2017 20:46:38 +0200 Subject: Increase Vagrant defaut mem to 2gb --- lib/leap_cli/commands/vagrant.rb | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/lib/leap_cli/commands/vagrant.rb b/lib/leap_cli/commands/vagrant.rb index f8a75b61..78b2fede 100644 --- a/lib/leap_cli/commands/vagrant.rb +++ b/lib/leap_cli/commands/vagrant.rb @@ -132,10 +132,10 @@ module LeapCli; module Commands lines << %[ config.vm.provider "virtualbox" do |v|] lines << %[ v.customize ["modifyvm", :id, "--natdnshostresolver1", "on"]] lines << %[ v.name = "#{node.name}"] - lines << %[ v.memory = 1536] + lines << %[ v.memory = 2048] lines << %[ end] lines << %[ config.vm.provider "libvirt" do |v|] - lines << %[ v.memory = 1536] + lines << %[ v.memory = 2048] lines << %[ end] lines << %[ #{leapfile.custom_vagrant_vm_line}] if leapfile.custom_vagrant_vm_line lines << %[ end] -- cgit v1.2.3 From 746601becc47fcee9fc85700f175a5b33b310979 Mon Sep 17 00:00:00 2001 From: varac Date: Wed, 10 May 2017 20:29:17 +0200 Subject: Nickserver direct access to couchdb on same node Depending whether couchdb is running on the same node as nickserver, couchdb is available on localhost: - When couchdb is running on a different node: Via stunnel, which is bound to 4000. - When couchdb is running on the same node: On port 5984 Resolves: #8793 --- provider_base/services/webapp.json | 1 + puppet/modules/site_nickserver/manifests/init.pp | 6 ++++-- 2 files changed, 5 insertions(+), 2 deletions(-) diff --git a/provider_base/services/webapp.json b/provider_base/services/webapp.json index ede3bf66..36f161b1 100644 --- a/provider_base/services/webapp.json +++ b/provider_base/services/webapp.json @@ -63,6 +63,7 @@ }, "nickserver": { "domain": "= 'nicknym.' + domain.full_suffix", + "couchdb_port": "= couchdb_port", "couchdb_nickserver_user": { "username": "= global.services[:couchdb].couch.users[:nickserver].username", "password": "= secret :couch_nickserver_password", diff --git a/puppet/modules/site_nickserver/manifests/init.pp b/puppet/modules/site_nickserver/manifests/init.pp index cab13522..48f7b73d 100644 --- a/puppet/modules/site_nickserver/manifests/init.pp +++ b/puppet/modules/site_nickserver/manifests/init.pp @@ -28,9 +28,11 @@ class site_nickserver { # the port that nickserver is actually running on $nickserver_local_port = '64250' - # couchdb is available on localhost via stunnel, which is bound to 4000. + # couchdb is available on localhost: + # - When couchdb is running on a different node: Via stunnel, which is bound to 4000. + # - When couchdb is running on the same node: On port 5984 $couchdb_host = 'localhost' - $couchdb_port = '4000' + $couchdb_port = $nickserver['couchdb_port'] $sources = hiera('sources') -- cgit v1.2.3 From 26be1f84de9efabf5fec1278401ae0c5538454d9 Mon Sep 17 00:00:00 2001 From: kwadronaut Date: Fri, 19 May 2017 23:41:57 +0200 Subject: makes sure locales packages is installed before locale-gen fixes #8649 --- bin/node_init | 10 ++++++---- 1 file changed, 6 insertions(+), 4 deletions(-) diff --git a/bin/node_init b/bin/node_init index 148ecc34..e86374b6 100755 --- a/bin/node_init +++ b/bin/node_init @@ -22,10 +22,6 @@ if ! egrep -q "$DEBIAN_VERSION" /etc/debian_version; then exit 1 fi mkdir -p $LEAP_DIR -if ! grep -q -e '^en_US.UTF-8' /etc/locale.gen 2> /dev/null; then - echo "en_US.UTF-8 UTF-8" >> /etc/locale.gen - /usr/sbin/locale-gen -fi # # UPDATE PACKAGES @@ -83,6 +79,12 @@ if [[ $exit_code -ne 0 ]]; then exit $exit_code fi +# need to have the locales package from above +if ! grep -q -e '^en_US.UTF-8' /etc/locale.gen 2> /dev/null; then + echo "en_US.UTF-8 UTF-8" >> /etc/locale.gen + /usr/sbin/locale-gen +fi + # # FINALIZE # -- cgit v1.2.3 From f93953299044ffa0154c08368ecb91b7a3f08a93 Mon Sep 17 00:00:00 2001 From: kwadronaut Date: Sat, 20 May 2017 00:20:08 +0200 Subject: generate missing ssh host keys on node init (closes #8790) closes #8414 as well --- bin/node_init | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/bin/node_init b/bin/node_init index 148ecc34..66dc17e7 100755 --- a/bin/node_init +++ b/bin/node_init @@ -89,3 +89,8 @@ fi mkdir -p $HIERA_DIR chmod 0755 $HIERA_DIR touch $INIT_FILE + +# Sometimes not all keys are already generated, happens more often +# with VMs +# that would give us errors in the get_ssh_keys_cmd during node init +/usr/bin/ssh-keygen -A -- cgit v1.2.3 From 61252fe74c8ec3668af551fb0b0b91f1bfa4705a Mon Sep 17 00:00:00 2001 From: varac Date: Sun, 21 May 2017 23:28:20 +0200 Subject: [vagrant] Use eth1 on vagrant if present Virtualbox adds eth1 as second interface when private networking is enabled. - Related: #7769 --- puppet/modules/site_config/lib/facter/vagrant.rb | 8 ++++++++ puppet/modules/site_config/manifests/params.pp | 13 +++++++++++-- 2 files changed, 19 insertions(+), 2 deletions(-) create mode 100644 puppet/modules/site_config/lib/facter/vagrant.rb diff --git a/puppet/modules/site_config/lib/facter/vagrant.rb b/puppet/modules/site_config/lib/facter/vagrant.rb new file mode 100644 index 00000000..29a218dd --- /dev/null +++ b/puppet/modules/site_config/lib/facter/vagrant.rb @@ -0,0 +1,8 @@ +# Checks if systems runs inside vagrant +require 'facter' + +Facter.add(:vagrant) do + setcode do + FileTest.exists?('/vagrant') + end +end diff --git a/puppet/modules/site_config/manifests/params.pp b/puppet/modules/site_config/manifests/params.pp index 012b3ce0..4627515a 100644 --- a/puppet/modules/site_config/manifests/params.pp +++ b/puppet/modules/site_config/manifests/params.pp @@ -1,3 +1,4 @@ +# Default parameters class site_config::params { $ip_address = hiera('ip_address') @@ -6,8 +7,16 @@ class site_config::params { $environment = hiera('environment', undef) - if $environment == 'local' { - $interface = 'eth1' + if $::vagrant { + # Depending on the backend hypervisor networking is setup differently. + if $::interfaces =~ /eth1/ { + # Virtualbox: Private networking creates a second interface eth1 + $interface = 'eth1' + } + else { + # KVM/Libvirt: Private networking is done by defauly on first interface + $interface = 'eth0' + } include site_config::packages::build_essential } elsif hiera('interface','') != '' { -- cgit v1.2.3 From 9be35a2aaee59f8d78b620a5b1f02ea08ec3bc78 Mon Sep 17 00:00:00 2001 From: varac Date: Sun, 21 May 2017 23:29:34 +0200 Subject: [vagrant] Use private networking for direct acces Without private networking, the box cannot get directly accessed, only via port forwardings. https://www.vagrantup.com/docs/networking/private_network.html - Resolves: #7769 --- tests/example-provider/Vagrantfile | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/tests/example-provider/Vagrantfile b/tests/example-provider/Vagrantfile index 1e410f5e..e909e79b 100644 --- a/tests/example-provider/Vagrantfile +++ b/tests/example-provider/Vagrantfile @@ -42,6 +42,10 @@ Vagrant.configure("2") do |config| config.ssh.username = "vagrant" + # Enable private networking so the box can be accessed directly, + # not only via port forwaring + config.vm.network "private_network", type: "dhcp" + # forward leap_web ports config.vm.network "forwarded_port", guest: 443, host:4443 # forward pixelated ports -- cgit v1.2.3 From 10164a2651cb9ca07442e8382b7e238c8a8939c1 Mon Sep 17 00:00:00 2001 From: varac Date: Tue, 23 May 2017 12:07:44 +0200 Subject: Lint configure-leap.sh --- tests/example-provider/vagrant/configure-leap.sh | 54 ++++++++++++------------ 1 file changed, 27 insertions(+), 27 deletions(-) diff --git a/tests/example-provider/vagrant/configure-leap.sh b/tests/example-provider/vagrant/configure-leap.sh index 8bd591e0..a8c0ff20 100755 --- a/tests/example-provider/vagrant/configure-leap.sh +++ b/tests/example-provider/vagrant/configure-leap.sh @@ -1,41 +1,41 @@ -#!/bin/bash - +#!/bin/sh +# shellcheck disable=SC1091 . /vagrant/vagrant/vagrant.config echo '===============================================' -echo 'configuring leap' +echo "Configuring LEAP in ${PROVIDERDIR}" echo '===============================================' # purge $PROVIDERDIR so this script can be run multiple times -[ -e $PROVIDERDIR ] && rm -rf $PROVIDERDIR +[ -e "$PROVIDERDIR" ] && rm -rf "$PROVIDERDIR" -mkdir -p $PROVIDERDIR -chown ${USER}:${USER} ${PROVIDERDIR} -cd $PROVIDERDIR +mkdir -p "$PROVIDERDIR" +chown "${USER}:${USER}" "${PROVIDERDIR}" +cd "$PROVIDERDIR" || exit -$LEAP $OPTS new --contacts "$contacts" --domain "$provider_domain" --name "$provider_name" --platform="$PLATFORMDIR" . -echo -e '\n@log = "./deploy.log"' >> Leapfile +$LEAP "$OPTS" new --contacts "${contacts:?}" --domain "${provider_domain:?}" --name "${provider_name:?}" --platform="$PLATFORMDIR" . +printf '\n@log = "./deploy.log"' >> Leapfile -if [ ! -e /home/${USER}/.ssh/id_rsa ]; then - $SUDO ssh-keygen -f /home/${USER}/.ssh/id_rsa -P '' +if [ ! -e "/home/${USER}/.ssh/id_rsa" ]; then + $SUDO ssh-keygen -f "/home/${USER}/.ssh/id_rsa" -P '' [ -d /root/.ssh ] || mkdir /root/.ssh - cat /home/${USER}/.ssh/id_rsa.pub >> /root/.ssh/authorized_keys + cat "/home/${USER}/.ssh/id_rsa.pub" >> /root/.ssh/authorized_keys fi -$SUDO mkdir -p ${PROVIDERDIR}/files/nodes/${NODE} +$SUDO mkdir -p "${PROVIDERDIR}/files/nodes/${NODE}" sh -c "cat /etc/ssh/ssh_host_rsa_key.pub | cut -d' ' -f1,2 >> $PROVIDERDIR/files/nodes/$NODE/${NODE}_ssh.pub" -chown ${USER}:${USER} ${PROVIDERDIR}/files/nodes/${NODE}/${NODE}_ssh.pub +chown "${USER}:${USER}" "${PROVIDERDIR}/files/nodes/${NODE}/${NODE}_ssh.pub" -$LEAP $OPTS add-user --self -$LEAP $OPTS cert ca -$LEAP $OPTS cert csr -$LEAP $OPTS node add $NODE ip_address:"$(facter ipaddress)" couch.mode:plain services:"$services" tags:production +$LEAP "$OPTS" add-user --self +$LEAP "$OPTS" cert ca +$LEAP "$OPTS" cert csr +$LEAP "$OPTS" node add "$NODE" ip_address:"$(facter ipaddress)" couch.mode:plain services:"${services:?}" tags:production echo '{ "webapp": { "admins": ["testadmin"] } }' > services/webapp.json -$LEAP $OPTS compile +$LEAP "$OPTS" compile -$LEAP $OPTS node init $NODE +$LEAP "$OPTS" node init "$NODE" if [ $? -eq 1 ]; then echo 'node init failed' exit 1 @@ -46,7 +46,7 @@ fi # workaround is to install rake as gem gem install rake -$LEAP $OPTS -v 2 deploy +$LEAP "$OPTS" -v 2 deploy # Vagrant: leap_mx fails to start on jessie # https://leap.se/code/issues/7755 @@ -62,7 +62,7 @@ echo '===============================================' echo 'testing the platform' echo '===============================================' -$LEAP $OPTS -v 2 test --continue +$LEAP "$OPTS" -v 2 test --continue echo '===============================================' echo 'setting node to demo-mode' @@ -73,13 +73,13 @@ postconf -e default_transport='error: in demo mode' curl -s -k https://localhost/1/users.json -d "user%5Blogin%5D=testuser&user%5Bpassword_salt%5D=7d4880237a038e0e&user%5Bpassword_verifier%5D=b98dc393afcd16e5a40fb57ce9cddfa6a978b84be326196627c111d426cada898cdaf3a6427e98b27daf4b0ed61d278bc856515aeceb2312e50c8f816659fcaa4460d839a1e2d7ffb867d32ac869962061368141c7571a53443d58dc84ca1fca34776894414c1090a93e296db6cef12c2cc3f7a991b05d49728ed358fd868286" curl -s -k https://localhost/1/users.json -d "user%5Blogin%5D=testadmin&user%5Bpassword_salt%5D=ece1c457014d8282&user%5Bpassword_verifier%5D=9654d93ab409edf4ff1543d07e08f321107c3fd00de05c646c637866a94f28b3eb263ea9129dacebb7291b3374cc6f0bf88eb3d231eb3a76eed330a0e8fd2a5c477ed2693694efc1cc23ae83c2ae351a21139701983dd595b6c3225a1bebd2a4e6122f83df87606f1a41152d9890e5a11ac3749b3bfcf4407fc83ef60b4ced68" -echo -e '\n===========================================================================================================\n\n' -echo -e 'You are now ready to use your local LEAP provider.\n' +printf '\n===========================================================================================================\n\n' +printf 'You are now ready to use your local LEAP provider.\n' echo 'If you want to use the *Bitmask client* with your provider, please update your /etc/hosts with following dns overrides:' $LEAP list --print ip_address,domain.full,dns.aliases | sed 's/^.* //' | sed 's/, null//g' | tr -d '\]\[",' echo 'Please see https://leap.se/en/docs/platform/tutorials/vagrant#use-the-bitmask-client-to-do-an-initial-soledad-sync for more details how to use and test your LEAP provider.' -echo -e "\nIf you don't want to use the Bitmask client, please ignore the above instructions.\n" -echo -e 'The LEAP webapp is now available at https://localhost:4443\n' -echo -e 'Please add an exception in your browser dialog to allow the self-signed certificate.\n' +printf "\nIf you don't want to use the Bitmask client, please ignore the above instructions.\n" +printf 'The LEAP webapp is now available at https://localhost:4443\n' +printf 'Please add an exception in your browser dialog to allow the self-signed certificate.\n' -- cgit v1.2.3 From d2dc27738a26b824f42da3fdea527be072867678 Mon Sep 17 00:00:00 2001 From: varac Date: Tue, 23 May 2017 13:23:43 +0200 Subject: [vagrant] Move $OPTS to vagrant config --- tests/example-provider/vagrant/configure-leap.sh | 21 ++++++++++++--------- tests/example-provider/vagrant/vagrant.config | 2 +- 2 files changed, 13 insertions(+), 10 deletions(-) diff --git a/tests/example-provider/vagrant/configure-leap.sh b/tests/example-provider/vagrant/configure-leap.sh index a8c0ff20..2ea2f178 100755 --- a/tests/example-provider/vagrant/configure-leap.sh +++ b/tests/example-provider/vagrant/configure-leap.sh @@ -1,5 +1,8 @@ #!/bin/sh +# Exit on failure +set -e + # shellcheck disable=SC1091 . /vagrant/vagrant/vagrant.config @@ -14,7 +17,7 @@ mkdir -p "$PROVIDERDIR" chown "${USER}:${USER}" "${PROVIDERDIR}" cd "$PROVIDERDIR" || exit -$LEAP "$OPTS" new --contacts "${contacts:?}" --domain "${provider_domain:?}" --name "${provider_name:?}" --platform="$PLATFORMDIR" . +$LEAP new --contacts "${contacts:?}" --domain "${provider_domain:?}" --name "${provider_name:?}" --platform="$PLATFORMDIR" . printf '\n@log = "./deploy.log"' >> Leapfile if [ ! -e "/home/${USER}/.ssh/id_rsa" ]; then @@ -27,15 +30,15 @@ $SUDO mkdir -p "${PROVIDERDIR}/files/nodes/${NODE}" sh -c "cat /etc/ssh/ssh_host_rsa_key.pub | cut -d' ' -f1,2 >> $PROVIDERDIR/files/nodes/$NODE/${NODE}_ssh.pub" chown "${USER}:${USER}" "${PROVIDERDIR}/files/nodes/${NODE}/${NODE}_ssh.pub" -$LEAP "$OPTS" add-user --self -$LEAP "$OPTS" cert ca -$LEAP "$OPTS" cert csr -$LEAP "$OPTS" node add "$NODE" ip_address:"$(facter ipaddress)" couch.mode:plain services:"${services:?}" tags:production +$LEAP add-user --self +$LEAP cert ca +$LEAP cert csr +$LEAP node add "$NODE" ip_address:"$(facter ipaddress)" couch.mode:plain services:"${services:?}" tags:production echo '{ "webapp": { "admins": ["testadmin"] } }' > services/webapp.json -$LEAP "$OPTS" compile +$LEAP compile -$LEAP "$OPTS" node init "$NODE" +$LEAP node init "$NODE" if [ $? -eq 1 ]; then echo 'node init failed' exit 1 @@ -46,7 +49,7 @@ fi # workaround is to install rake as gem gem install rake -$LEAP "$OPTS" -v 2 deploy +$LEAP -v 2 deploy # Vagrant: leap_mx fails to start on jessie # https://leap.se/code/issues/7755 @@ -62,7 +65,7 @@ echo '===============================================' echo 'testing the platform' echo '===============================================' -$LEAP "$OPTS" -v 2 test --continue +$LEAP -v 2 test --continue echo '===============================================' echo 'setting node to demo-mode' diff --git a/tests/example-provider/vagrant/vagrant.config b/tests/example-provider/vagrant/vagrant.config index ff5dd38f..07222c3f 100644 --- a/tests/example-provider/vagrant/vagrant.config +++ b/tests/example-provider/vagrant/vagrant.config @@ -18,4 +18,4 @@ NODE='node1' SUDO="sudo -u ${USER}" PROVIDERDIR="/home/${USER}/leap/configuration" PLATFORMDIR="/srv/leap_platform" -LEAP="$SUDO /usr/local/bin/leap" +LEAP="$SUDO /usr/local/bin/leap $OPTS" -- cgit v1.2.3 From 85c0f8a67188902c36558d97651c2801849112f1 Mon Sep 17 00:00:00 2001 From: varac Date: Tue, 23 May 2017 13:28:54 +0200 Subject: [vagrant] Use private networking IP from eth1 if present --- tests/example-provider/vagrant/configure-leap.sh | 11 ++++++++++- 1 file changed, 10 insertions(+), 1 deletion(-) diff --git a/tests/example-provider/vagrant/configure-leap.sh b/tests/example-provider/vagrant/configure-leap.sh index 2ea2f178..7a1efc71 100755 --- a/tests/example-provider/vagrant/configure-leap.sh +++ b/tests/example-provider/vagrant/configure-leap.sh @@ -33,7 +33,16 @@ chown "${USER}:${USER}" "${PROVIDERDIR}/files/nodes/${NODE}/${NODE}_ssh.pub" $LEAP add-user --self $LEAP cert ca $LEAP cert csr -$LEAP node add "$NODE" ip_address:"$(facter ipaddress)" couch.mode:plain services:"${services:?}" tags:production + +# Try to see if there's a private IP for eth1 +# Otherwise take eth0 +# (virtualbox and libvirt backends behave differenently setting up +# direct accessible private networks. +# see https://www.vagrantup.com/docs/networking/private_network.html +IP="$(facter ipaddress_eth1)" +[ "$IP" = '' ] && IP="$(facter ipaddress_eth0)" +$LEAP node add "$NODE" ip_address:"${IP}" couch.mode:plain services:"${services:?}" tags:production + echo '{ "webapp": { "admins": ["testadmin"] } }' > services/webapp.json $LEAP compile -- cgit v1.2.3 From 0dec9e7305001353beca3b32e180bc9e707ce8b9 Mon Sep 17 00:00:00 2001 From: varac Date: Tue, 23 May 2017 13:48:15 +0200 Subject: [vagrant] Lint vagrant.pp --- puppet/modules/site_config/manifests/vagrant.pp | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/puppet/modules/site_config/manifests/vagrant.pp b/puppet/modules/site_config/manifests/vagrant.pp index 8f50b305..23ca4de1 100644 --- a/puppet/modules/site_config/manifests/vagrant.pp +++ b/puppet/modules/site_config/manifests/vagrant.pp @@ -1,11 +1,11 @@ +# Gets included on vagrant nodes class site_config::vagrant { - # class for vagrant nodes include site_shorewall::defaults - # eth0 on vagrant nodes is the uplink if + # eth0 on vagrant nodes is the uplink shorewall::interface { 'eth0': - zone => 'net', - options => 'tcpflags,blacklist,nosmurfs'; + zone => 'net', + options => 'tcpflags,blacklist,nosmurfs'; } } -- cgit v1.2.3 From d2824a6bc1178c6c2ce4923faacfde8e05f8389a Mon Sep 17 00:00:00 2001 From: varac Date: Tue, 23 May 2017 13:56:57 +0200 Subject: Include site_config::vagrant on vagrant nodes --- puppet/modules/site_config/manifests/setup.pp | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/puppet/modules/site_config/manifests/setup.pp b/puppet/modules/site_config/manifests/setup.pp index 82dfe76d..a96f87a6 100644 --- a/puppet/modules/site_config/manifests/setup.pp +++ b/puppet/modules/site_config/manifests/setup.pp @@ -37,7 +37,7 @@ class site_config::setup { # we need to include shorewall::interface{eth0} in setup.pp so # packages can be installed during main puppetrun, even before shorewall # is configured completly - if ( $::site_config::params::environment == 'local' ) { + if $::vagrant { include site_config::vagrant } -- cgit v1.2.3 From 40f7b49003594a1be8c0540a92292d7cfb63eb61 Mon Sep 17 00:00:00 2001 From: varac Date: Tue, 23 May 2017 13:58:38 +0200 Subject: [vagrant] Don't block eth0 if eth1 is configured Eth0 is vagrant's main interface to access the box --- puppet/modules/site_config/manifests/vagrant.pp | 12 ++++++++---- 1 file changed, 8 insertions(+), 4 deletions(-) diff --git a/puppet/modules/site_config/manifests/vagrant.pp b/puppet/modules/site_config/manifests/vagrant.pp index 23ca4de1..1682de8b 100644 --- a/puppet/modules/site_config/manifests/vagrant.pp +++ b/puppet/modules/site_config/manifests/vagrant.pp @@ -2,10 +2,14 @@ class site_config::vagrant { include site_shorewall::defaults - # eth0 on vagrant nodes is the uplink - shorewall::interface { 'eth0': - zone => 'net', - options => 'tcpflags,blacklist,nosmurfs'; + + if ( $::site_config::params::interface == 'eth1' ) { + # Don't block eth0 even if eth1 is configured, because + # it's vagrant's main interface to access the box + shorewall::interface { 'eth0': + zone => 'net', + options => 'tcpflags,blacklist,nosmurfs'; + } } } -- cgit v1.2.3 From 1e463c6638a05a237d660f458f5a147353be3fc1 Mon Sep 17 00:00:00 2001 From: elijah Date: Fri, 26 May 2017 16:41:51 -0700 Subject: static - support for renewing certs with let's encrypt for static sites --- lib/leap_cli/commands/cert.rb | 54 +++++++++++++--------- provider_base/common.json | 3 +- provider_base/services/static.rb | 2 + provider_base/services/webapp.json | 3 -- provider_base/services/webapp.rb | 2 + puppet/modules/site_static/manifests/domain.pp | 13 ++++-- .../modules/site_static/templates/apache.conf.erb | 10 +++- 7 files changed, 55 insertions(+), 32 deletions(-) create mode 100644 provider_base/services/static.rb create mode 100644 provider_base/services/webapp.rb diff --git a/lib/leap_cli/commands/cert.rb b/lib/leap_cli/commands/cert.rb index 1c67ae67..81f45eb5 100644 --- a/lib/leap_cli/commands/cert.rb +++ b/lib/leap_cli/commands/cert.rb @@ -337,31 +337,41 @@ module LeapCli; module Commands # This method will bail if any checks fail. # def domain_ready_for_acme!(domain) - begin - uri = URI("https://#{domain}/.well-known/acme-challenge/ok") - options = { - use_ssl: true, - open_timeout: 5, - verify_mode: OpenSSL::SSL::VERIFY_NONE - } - Net::HTTP.start(uri.host, uri.port, options) do |http| - http.request(Net::HTTP::Get.new(uri)) do |response| - if !response.is_a?(Net::HTTPSuccess) - bail!(:error, "Could not GET %s" % uri) do - log "%s %s" % [response.code, response.message] - log "You may need to run `leap deploy`" - end + uri = URI("https://#{domain}/.well-known/acme-challenge/ok") + options = { + use_ssl: true, + open_timeout: 5, + verify_mode: OpenSSL::SSL::VERIFY_NONE + } + http_get(uri, options) + end + + private + + def http_get(uri, options, limit = 10) + raise ArgumentError, "HTTP redirect too deep (#{uri})" if limit == 0 + Net::HTTP.start(uri.host, uri.port, options) do |http| + http.request(Net::HTTP::Get.new(uri)) do |response| + case response + when Net::HTTPSuccess then + return response + when Net::HTTPRedirection then + return http_get(URI(response['location']), options, limit - 1) + else + bail!(:error, "Could not GET %s" % uri) do + log "%s %s" % [response.code, response.message] + log "You may need to run `leap deploy`" end end end - rescue Errno::ETIMEDOUT, Net::OpenTimeout - bail! :error, "Connection attempt timed out: %s" % uri - rescue Interrupt - bail! - rescue StandardError => exc - bail!(:error, "Could not GET %s" % uri) do - log exc.to_s - end + end + rescue Errno::ETIMEDOUT, Net::OpenTimeout + bail! :error, "Connection attempt timed out: %s" % uri + rescue Interrupt + bail! + rescue StandardError => exc + bail!(:error, "Could not GET %s" % uri) do + log exc.to_s end end diff --git a/provider_base/common.json b/provider_base/common.json index 41e1daa3..97519950 100644 --- a/provider_base/common.json +++ b/provider_base/common.json @@ -12,7 +12,8 @@ "name": "= node.name + '.' + (dns.public ? domain.full_suffix : domain.internal_suffix)" }, "dns": { - "public": "= service_type != 'internal_service'" + "public": "= service_type != 'internal_service'", + "aliases": [] }, "ssh": { "authorized_keys": "= authorized_keys", diff --git a/provider_base/services/static.rb b/provider_base/services/static.rb new file mode 100644 index 00000000..d020ba26 --- /dev/null +++ b/provider_base/services/static.rb @@ -0,0 +1,2 @@ +self['dns']['aliases'] += self.static.domains.keys +self['dns']['aliases'].uniq! diff --git a/provider_base/services/webapp.json b/provider_base/services/webapp.json index 36f161b1..ac58ac12 100644 --- a/provider_base/services/webapp.json +++ b/provider_base/services/webapp.json @@ -71,9 +71,6 @@ }, "port": 6425 }, - "dns": { - "aliases": "= [domain.full, webapp.domain, api.domain, nickserver.domain]" - }, "x509": { "use": true, "use_commercial": true, diff --git a/provider_base/services/webapp.rb b/provider_base/services/webapp.rb new file mode 100644 index 00000000..a5f10a2d --- /dev/null +++ b/provider_base/services/webapp.rb @@ -0,0 +1,2 @@ +self['dns']['aliases'] += [domain.full, webapp.domain, api.domain, nickserver.domain] +self['dns']['aliases'].uniq! diff --git a/puppet/modules/site_static/manifests/domain.pp b/puppet/modules/site_static/manifests/domain.pp index 6cf2c653..e456c94e 100644 --- a/puppet/modules/site_static/manifests/domain.pp +++ b/puppet/modules/site_static/manifests/domain.pp @@ -1,25 +1,30 @@ # configure static service for domain define site_static::domain ( - $ca_cert, + $ca_cert=undef, $key, $cert, $tls_only=true, $use_hidden_service=false, $locations=undef, $aliases=undef, - $apache_config=undef) { + $apache_config=undef, + $www_alias=false) { $domain = $name $base_dir = '/srv/static' - $cafile = "${cert}\n${ca_cert}" + if ($ca_cert) { + $certfile = "${cert}\n${ca_cert}" + } else { + $certfile = $cert + } if is_hash($locations) { create_resources(site_static::location, $locations) } x509::cert { $domain: - content => $cafile, + content => $certfile, notify => Service[apache] } x509::key { $domain: diff --git a/puppet/modules/site_static/templates/apache.conf.erb b/puppet/modules/site_static/templates/apache.conf.erb index dd04ca43..eb21e4c9 100644 --- a/puppet/modules/site_static/templates/apache.conf.erb +++ b/puppet/modules/site_static/templates/apache.conf.erb @@ -80,7 +80,9 @@ ## ServerName <%= @tor_domain %> +<%- if @www_alias -%> ServerAlias www.<%= @tor_domain %> +<%- end -%> Header set X-Frame-Options "deny" @@ -102,7 +104,9 @@ ## ServerName <%= @domain %> - ServerAlias www.<%= @domain %> +<%- if @www_alias -%> + ServerAlias www.<%= @tor_domain %> +<%- end -%> <%- @aliases && @aliases.each do |domain_alias| -%> ServerAlias <%= domain_alias %> <%- end -%> @@ -122,7 +126,9 @@ ## ServerName <%= @domain %> - ServerAlias www.<%= @domain %> +<%- if @www_alias -%> + ServerAlias www.<%= @tor_domain %> +<%- end -%> <%- @aliases && @aliases.each do |domain_alias| -%> ServerAlias <%= domain_alias %> <%- end -%> -- cgit v1.2.3 From 69b70494c96c7db97cf5b535e4a049606aeafa2b Mon Sep 17 00:00:00 2001 From: varac Date: Wed, 31 May 2017 16:10:22 +0200 Subject: Remove .mailmap, dont leak email addresses --- .mailmap | 8 -------- 1 file changed, 8 deletions(-) delete mode 100644 .mailmap diff --git a/.mailmap b/.mailmap deleted file mode 100644 index aee70b0a..00000000 --- a/.mailmap +++ /dev/null @@ -1,8 +0,0 @@ -Varac -Micah Anderson Micah Anderson -Micah Anderson micah -Kwadronaut -Elijah elijah -Elijah elijah -Leap Admins root - -- cgit v1.2.3 From f89abb94469717e7c35ba01b30b17a320eba4c72 Mon Sep 17 00:00:00 2001 From: Varac Date: Fri, 16 Jun 2017 13:49:35 +0200 Subject: [CI] Use master branch of leap_cli We moved from develop to master some time ago so we should use master for CI testing as well. --- tests/platform-ci/Gemfile | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tests/platform-ci/Gemfile b/tests/platform-ci/Gemfile index 36f556e5..6ec76b2e 100644 --- a/tests/platform-ci/Gemfile +++ b/tests/platform-ci/Gemfile @@ -13,5 +13,5 @@ group :test do # Use puppet-catalog-test from git because last released gem 0.4.2 gives a deprecation # warning: "[DEPRECATION] `last_comment` is deprecated. Please use `last_description` instead." gem "puppet-catalog-test", :git => 'https://github.com/invadersmustdie/puppet-catalog-test.git' - gem "leap_cli", :git => 'https://leap.se/git/leap_cli.git', :branch => 'develop' + gem "leap_cli", :git => 'https://leap.se/git/leap_cli.git' end -- cgit v1.2.3 From c6ffb01f98f5c78bf25634d57cd05fb57f25c085 Mon Sep 17 00:00:00 2001 From: Varac Date: Fri, 16 Jun 2017 13:44:08 +0200 Subject: [CI] Use older commit for puppet-catalog-test After `puppet-catalog-test` has been recently updated it failed in our CI with: File[/etc/apt/sources.list] has notify relationship to invalid resource Exec[apt_updated] See #8814 for more details. Resolves: #8814 https://github.com/invadersmustdie/puppet-catalog-test/commit/ac386793c2c456d2071dd0adda716224128f0bb3 --- tests/platform-ci/Gemfile | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tests/platform-ci/Gemfile b/tests/platform-ci/Gemfile index 6ec76b2e..51e2c17b 100644 --- a/tests/platform-ci/Gemfile +++ b/tests/platform-ci/Gemfile @@ -12,6 +12,6 @@ group :test do gem "mocha" # Use puppet-catalog-test from git because last released gem 0.4.2 gives a deprecation # warning: "[DEPRECATION] `last_comment` is deprecated. Please use `last_description` instead." - gem "puppet-catalog-test", :git => 'https://github.com/invadersmustdie/puppet-catalog-test.git' + gem "puppet-catalog-test", :git => 'https://github.com/invadersmustdie/puppet-catalog-test.git', :ref => 'ac386793c2c456d2071dd0adda716224128f0bb3' gem "leap_cli", :git => 'https://leap.se/git/leap_cli.git' end -- cgit v1.2.3 From eb904e99e5f2a8ec77feca52f2a076bec1d0383a Mon Sep 17 00:00:00 2001 From: Varac Date: Sat, 17 Jun 2017 18:05:58 +0200 Subject: Renewed commercial cert for platform CI --- .../provider/files/cert/example.org.crt | 58 ++++++------- .../provider/files/cert/example.org.csr | 46 +++++----- .../provider/files/cert/example.org.key | 98 +++++++++++----------- 3 files changed, 101 insertions(+), 101 deletions(-) diff --git a/tests/platform-ci/provider/files/cert/example.org.crt b/tests/platform-ci/provider/files/cert/example.org.crt index 7de2982d..174cac51 100644 --- a/tests/platform-ci/provider/files/cert/example.org.crt +++ b/tests/platform-ci/provider/files/cert/example.org.crt @@ -1,31 +1,31 @@ -----BEGIN CERTIFICATE----- -MIIFbDCCA1SgAwIBAgIRAJW2X9xbiBvmbN1kMlRVKtQwDQYJKoZIhvcNAQELBQAw -SjEQMA4GA1UECgwHRXhhbXBsZTEcMBoGA1UECwwTaHR0cHM6Ly9leGFtcGxlLm9y -ZzEYMBYGA1UEAwwPRXhhbXBsZSBSb290IENBMB4XDTE2MDYxMTAwMDAwMFoXDTE3 -MDYxMTAwMDAwMFowKDEQMA4GA1UECgwHRXhhbXBsZTEUMBIGA1UEAwwLZXhhbXBs -ZS5vcmcwggIiMA0GCSqGSIb3DQEBAQUAA4ICDwAwggIKAoICAQDFuKIL//hf5cjU -m18q5fSUyvwtmWREJPaVp+CiWiGJHmxFAiWMGuAFRRChhZ4SYmnEscNda0f6ntPz -rO+XjhQeA05bIYD9JcFT25Jg4kSX4pQ0+pK2vuHqk4ascZgOOaq4fN8SXD6ZiL3m -CONDRzbnZVR2LqsdCbEqIuHlo7VK7MO8/9A+rF7wKLVatBtk25uSWMQPt0Q41gw6 -YTV447SltFH3fgUZnNR6p7Oxpsi3qEWlt2vZMIa5xdq4ge2dx1GgC8oSBx1XT/Yd -qu//GECAH5XsZsAaPXDuor1iTbWELzHyGrQ7V80e67lE2lxoaHxRCOE/NDUU6UXm -CqXwhdBHarHehOCGSDXvHEwAH5zpV77XOm2bIoZmCjM1fRk5p2S3GmXteCdvCxBP -+2wECnRXuwN2aICrBk7sZ9FieRsYao8GZN/A7ZY24pf7CMEBsgjYktTjAwUb21m6 -vmmzt93dEVJgkd8LASFmoXn+YAIGF0/fD5ZutlsAsBfodoCH9JKBi25nVVTEQW8g -TzUegTC3PUqnathWv4gZIYDG1ZUDxjk30beNmXV2XudASmP7NG4uSlQwGAEWn+cc -dzOnRxR0BQpkMMNEV/HmJVuSV5Ak4DkruSXGjLpzi30BjJ8obx85YAusIrhWRUrR -2oz6gqDUnwq3Nkr3Nk45iOEDC0cZnwIDAQABo28wbTAdBgNVHQ4EFgQUS7rm3WfC -psxoh4i7q0YbTbMZWuIwCwYDVR0PBAQDAgWgMBMGA1UdJQQMMAoGCCsGAQUFBwMB -MAkGA1UdEwQCMAAwHwYDVR0jBBgwFoAUlhC2wfrVFzGrtuzcA0mkO+yn9bgwDQYJ -KoZIhvcNAQELBQADggIBAKxeVSMEpUOdBO1zmwd5NtugOlYV3/Gu9GqmUQdlB4FF -Wt6sKJmYYByNquKT79oJLb9dgUPw8qQiHCB+MAsjB4PpHvMRlpgrcDGsI8+esnfG -dJny+82aRIFZ2KnNbH8FchcCh4bviaY+DE9kyJNHILk0ujICXabR0G6ArVISTbyB -C+6BdFyKTT5zj9mtkiTgvZchlKCmOmvh/HeCONu6MGYbqcqp41RA3g1eEjFoROKO -wmf65VvfOBeb9VydOTICh/bJWRSmAMJqWxbOiV8+Ldufi0vXMcOhEfsyo316xxRq -1GMb5xVihtCxj/+qBKNoun4k9LTmUvComuPakbtEPT2QbxiTvqCbXsWHPoRwCKEj -RcFPsxWAnUslzqSl1b0oLaE1zNjBmB/Zd82i2MC4PncLC2hLHtAU1imRZKP6rnHx -cb1NyFLS0FmIPqZUz9qcY2Tj3GbjqYqRi/sXNKrR2axAUx+jGI/Ie7Zsqa4VZA0A -ZsiF0BGN3RTCYHuoJbXfEVFQ3o97JGNC3t07u9XhVuC0fjCiQu5PBbMRHSSvtBdN -+LSrhR5j4aiCmppgQSeTtoKSIS3EiOzDtawdewxhffK+co0pGnO3nox+iINvSIQ5 -IevAREmZ2ytjFDU/kVFFlINesFsLRouO37DUf2Kjxaa0RgkCBHpOnTAAD7bXiSaJ +MIIFazCCA1OgAwIBAgIQbk9ZqiquHG3E0IiLALe2pjANBgkqhkiG9w0BAQsFADBK +MRAwDgYDVQQKDAdFeGFtcGxlMRwwGgYDVQQLDBNodHRwczovL2V4YW1wbGUub3Jn +MRgwFgYDVQQDDA9FeGFtcGxlIFJvb3QgQ0EwHhcNMTcwNjE3MDAwMDAwWhcNMTgw +NjE3MDAwMDAwWjAoMRAwDgYDVQQKDAdFeGFtcGxlMRQwEgYDVQQDDAtleGFtcGxl +Lm9yZzCCAiIwDQYJKoZIhvcNAQEBBQADggIPADCCAgoCggIBAL3GmkcYpqzivoha +Fc1G2DgRpm658t27ELp6qk4iwjZ8IZMnQl3N9ZMmbU+jFWg+ZvNETZoi91bOgnPh +PtXnGEP71Cgt47+rJ5WvwcvlAvizlvMf7mqrDhFzAqKX7kLdP+akFfC/Wjf9wh3M +FvTxZOb3uhNHUesPMe/OORU0i2r3QzHwCTXc99c8FR0xYRdX9Cr6ig4irdbgmwbd +BKRE3q23dVJXUTqNzHc46a3j9KlfE9k1hJy0k0gpuDOliZslicIG/BljbeJqH2tW +GeJrovnO6THOft7SaUV1hJQtIxAj0YY2a0MMUbKitvmRZ+oPZusGQFl+RA7b0vxe +UK3hTO6oSGGVqBhSe4T52EBJ74c0FOJdG1Hp65t6DN/HS0rO8cQRA7EVFxwMnnvo +tf9E1O7Zv6URY+lU8Pr3aBIzYVE1XIvmZIj60Bst87mRz1tGsWf99hDxWyFaesVq +g5EI5G9rKf1CZyQyTTwhfLbr4wsJo+S3Y4zSP0xyOypaTiZwydWVb/4J5MBlaNs7 +ZMU3gugp9hEn4BzYzk8lxcqChKMl3rFdjWPkSl2/tP39WvVo/T088vKS6v/1Rq4n +0Iz33d3FbayBNlHMZjHPmwgbvL5YKjnEtkQyi+6isBt8g/4KLxZZbN5udyrCJW7n +3xGj3DYokhOZVt7rpiMAgqHAEB0VAgMBAAGjbzBtMB0GA1UdDgQWBBSvA0QXoIqX +2Pf+Gb+va4aEdTxTsTALBgNVHQ8EBAMCBaAwEwYDVR0lBAwwCgYIKwYBBQUHAwEw +CQYDVR0TBAIwADAfBgNVHSMEGDAWgBSWELbB+tUXMau27NwDSaQ77Kf1uDANBgkq +hkiG9w0BAQsFAAOCAgEAAfI9WvIw+HdtcNtnmpIqNP9o2S+QfVd75jEnslV/NDIS +chZOeS2OnGLsV2oxh4EypRnAAIUTRF49k9gE6i9g+E4YGEOCmLzwHHm8VAiz1UNP +Gubn1vNUfwk5Ct4lZWgb4QxrJQ+7AjIX+742D9fvfb3aP8sEWxmq4kfTZDxbjtGz +mIUQ9Dtd5Ck00LU/eJv4dcsTf6vSjkc/QL+aan3o2J/6wUXWLH00kOHnXex4eofd +DJok7rL9E5HbFlCup2bmUgeQEx8keegG2TEQgt61JMrwQjh9EsuA69AMLiDSlfhv +WOreUjPZCHRDtLhZJD54AvMuDWo7p8lcKyVBo4jkwx80qkfnZgh81lWIuYxIOB+y +VfyEy0+jSCc9Jub/57HADMj29hxkpT6FYPzMPk+tInxL8Z5iwb0UQ/fVNX2mgXwp +U5E+M4OYaNi9xo2992xd6rok8Lnkaq27tUVV0ZiMLfe8Njv1UQLV2hTiPzSntwsK +SYHtN8mavrl6tdqU5F5V+GydAXfcmEKQnV3AxOYTFsfjotSbipBRGfAoRksmRGiH +8dT5jWpi/Nvta+azn55029ej1AVuD7nWMhT+7mYvfmiSqzItorI28+g0P2jSBOTf +t4MfXLecy4mnqIWM4ciHAVF6UMWJbnTSc0DwEQAgpdw21CoPp6/BERLI9frvU7o= -----END CERTIFICATE----- diff --git a/tests/platform-ci/provider/files/cert/example.org.csr b/tests/platform-ci/provider/files/cert/example.org.csr index 95e8b65d..5d1dfc77 100644 --- a/tests/platform-ci/provider/files/cert/example.org.csr +++ b/tests/platform-ci/provider/files/cert/example.org.csr @@ -1,27 +1,27 @@ -----BEGIN CERTIFICATE REQUEST----- MIIEqzCCApMCAQAwKDEQMA4GA1UECgwHRXhhbXBsZTEUMBIGA1UEAwwLZXhhbXBs -ZS5vcmcwggIiMA0GCSqGSIb3DQEBAQUAA4ICDwAwggIKAoICAQDFuKIL//hf5cjU -m18q5fSUyvwtmWREJPaVp+CiWiGJHmxFAiWMGuAFRRChhZ4SYmnEscNda0f6ntPz -rO+XjhQeA05bIYD9JcFT25Jg4kSX4pQ0+pK2vuHqk4ascZgOOaq4fN8SXD6ZiL3m -CONDRzbnZVR2LqsdCbEqIuHlo7VK7MO8/9A+rF7wKLVatBtk25uSWMQPt0Q41gw6 -YTV447SltFH3fgUZnNR6p7Oxpsi3qEWlt2vZMIa5xdq4ge2dx1GgC8oSBx1XT/Yd -qu//GECAH5XsZsAaPXDuor1iTbWELzHyGrQ7V80e67lE2lxoaHxRCOE/NDUU6UXm -CqXwhdBHarHehOCGSDXvHEwAH5zpV77XOm2bIoZmCjM1fRk5p2S3GmXteCdvCxBP -+2wECnRXuwN2aICrBk7sZ9FieRsYao8GZN/A7ZY24pf7CMEBsgjYktTjAwUb21m6 -vmmzt93dEVJgkd8LASFmoXn+YAIGF0/fD5ZutlsAsBfodoCH9JKBi25nVVTEQW8g -TzUegTC3PUqnathWv4gZIYDG1ZUDxjk30beNmXV2XudASmP7NG4uSlQwGAEWn+cc -dzOnRxR0BQpkMMNEV/HmJVuSV5Ak4DkruSXGjLpzi30BjJ8obx85YAusIrhWRUrR -2oz6gqDUnwq3Nkr3Nk45iOEDC0cZnwIDAQABoD4wPAYJKoZIhvcNAQkOMS8wLTAJ +ZS5vcmcwggIiMA0GCSqGSIb3DQEBAQUAA4ICDwAwggIKAoICAQC9xppHGKas4r6I +WhXNRtg4EaZuufLduxC6eqpOIsI2fCGTJ0JdzfWTJm1PoxVoPmbzRE2aIvdWzoJz +4T7V5xhD+9QoLeO/qyeVr8HL5QL4s5bzH+5qqw4RcwKil+5C3T/mpBXwv1o3/cId +zBb08WTm97oTR1HrDzHvzjkVNItq90Mx8Ak13PfXPBUdMWEXV/Qq+ooOIq3W4JsG +3QSkRN6tt3VSV1E6jcx3OOmt4/SpXxPZNYSctJNIKbgzpYmbJYnCBvwZY23iah9r +Vhnia6L5zukxzn7e0mlFdYSULSMQI9GGNmtDDFGyorb5kWfqD2brBkBZfkQO29L8 +XlCt4UzuqEhhlagYUnuE+dhASe+HNBTiXRtR6eubegzfx0tKzvHEEQOxFRccDJ57 +6LX/RNTu2b+lEWPpVPD692gSM2FRNVyL5mSI+tAbLfO5kc9bRrFn/fYQ8VshWnrF +aoORCORvayn9QmckMk08IXy26+MLCaPkt2OM0j9McjsqWk4mcMnVlW/+CeTAZWjb +O2TFN4LoKfYRJ+Ac2M5PJcXKgoSjJd6xXY1j5Epdv7T9/Vr1aP09PPLykur/9Uau +J9CM993dxW2sgTZRzGYxz5sIG7y+WCo5xLZEMovuorAbfIP+Ci8WWWzebncqwiVu +598Ro9w2KJITmVbe66YjAIKhwBAdFQIDAQABoD4wPAYJKoZIhvcNAQkOMS8wLTAJ BgNVHRMEAjAAMAsGA1UdDwQEAwIFoDATBgNVHSUEDDAKBggrBgEFBQcDATANBgkq -hkiG9w0BAQsFAAOCAgEAG0IpXLHZpgXtBZHEnGBghrucWnAuhRf0sXauboBVWnwA -5noESIIX/hNq9DdaBba684u1Qga+lZcFsO1Zh/K1Guu74FTNxV2jCLKcX1T+Ymx4 -uRJ1jcdCc+YB/f+ce+pAhFJei/6sKP//MtYIBHlbe8aGQx1yVPJ5oSb4yS9Hloe4 -DuM0bp6ZXhXFv4YxxxDbaTMs9D46AKnqXV0rLe8WwHH1Mbdxl0bi7roZ3/1NPYsg -diUMWQlnrR1d1xxUG7x+PJRpPcN3GmZQ0WyZoNrIQA7OLEg6nM8T4sQX5OZFdQrQ -KQJyX8+Cc8j/UtPrPIPgch6iYX32e+1wTAP82npw1KMELxRsxjX6ERl65apkADFa -w6LrCFtUQApWY/vZPz88udzSxVytJL4ZrHJxuZEG1WFE3kPY2Ak5LYw/IVxCDFsL -GVfhb92zkn5iUkULXbwjcTytK3IqXZHl05PW+etGtqbkdh99m8eH1HxolKEgtehm -l7FMD/JrC0GJWhI4Dl0CpvhAsV61pa8f1KmfGFTt+zpS4epSIItWTuSd4tzaXwNq -3K1zJaKHs16VWBFuhH5kle4QGRIuDRPHchBQQg0wgy/sfHuzqbcVNotGZ7qzvnRL -x5eXmWm1HaVKl1NpxbntMY4o9u0WgyzmU0VVsv+oWJj6J88T97rqTNg1Q1Uj8ic= +hkiG9w0BAQsFAAOCAgEAif8FifpW2oup62+Wq/bYozwotreb7UOkaCMo+6ZsTyY3 +XcLoUa/7lM4dX7ZDPujOB3MyvF8mw2fsl7DJO605bjERbjzwyKXKPn4TFpHopPKm +qMJzZT7KHqIdkVUnitsOKysLoIHYV7fTdjspnt4UlyYoJji6lCtAmsVt4mxO3jmf +6EVVyR5fr+cz0n6qwmhjRXGNDHA3CkAV77rLtiK3oYQ7fQ4VY2/3uZpu62nP4u77 +HAuEj4mBsPq5I0+oxFPoM6Yx5dyVrHlPsfl8SzrFbXBCrSQkvEEbbsE3MAuQ+SJS +MVwngXxnpViNHYYl97Lmxc4c2ZxG9eyusFIamDN62adGUkkZqvg0ZQzyVzMnIVYC +jc8h8MxxeQNHSrcPu8wziavA8xRMj0c51Q09gvlVBQmDNxZqib9MSH6U464zilZZ +E/1tEAoNWqmDq0mzzdgY8ShRBOcVfaU+S/K0nRJWKLF/g84d7n5j+P2eJXE7aDkw +6dzqPVjbZk/0BrkTuh895BgjzqjAVvKONdFpZ4ugcYMZX75IqE7ta4DskXD2Fd6v +OQdDqRFhHLS/xVPiLRr981qT3NZMDaezLU/VfJFAsTf8wmgGqtBQqM/N1JWrUU/N +b0ouLa3CohZoM7dY44a137r5/rDTpsv33gES2BVumY8XZzaN+RLJ45I7BqMpLOo= -----END CERTIFICATE REQUEST----- diff --git a/tests/platform-ci/provider/files/cert/example.org.key b/tests/platform-ci/provider/files/cert/example.org.key index 7ca1c512..13adeb7c 100644 --- a/tests/platform-ci/provider/files/cert/example.org.key +++ b/tests/platform-ci/provider/files/cert/example.org.key @@ -1,51 +1,51 @@ -----BEGIN RSA PRIVATE KEY----- -MIIJKQIBAAKCAgEAxbiiC//4X+XI1JtfKuX0lMr8LZlkRCT2lafgolohiR5sRQIl -jBrgBUUQoYWeEmJpxLHDXWtH+p7T86zvl44UHgNOWyGA/SXBU9uSYOJEl+KUNPqS -tr7h6pOGrHGYDjmquHzfElw+mYi95gjjQ0c252VUdi6rHQmxKiLh5aO1SuzDvP/Q -Pqxe8Ci1WrQbZNubkljED7dEONYMOmE1eOO0pbRR934FGZzUeqezsabIt6hFpbdr -2TCGucXauIHtncdRoAvKEgcdV0/2Harv/xhAgB+V7GbAGj1w7qK9Yk21hC8x8hq0 -O1fNHuu5RNpcaGh8UQjhPzQ1FOlF5gql8IXQR2qx3oTghkg17xxMAB+c6Ve+1zpt -myKGZgozNX0ZOadktxpl7XgnbwsQT/tsBAp0V7sDdmiAqwZO7GfRYnkbGGqPBmTf -wO2WNuKX+wjBAbII2JLU4wMFG9tZur5ps7fd3RFSYJHfCwEhZqF5/mACBhdP3w+W -brZbALAX6HaAh/SSgYtuZ1VUxEFvIE81HoEwtz1Kp2rYVr+IGSGAxtWVA8Y5N9G3 -jZl1dl7nQEpj+zRuLkpUMBgBFp/nHHczp0cUdAUKZDDDRFfx5iVbkleQJOA5K7kl -xoy6c4t9AYyfKG8fOWALrCK4VkVK0dqM+oKg1J8KtzZK9zZOOYjhAwtHGZ8CAwEA -AQKCAgAht6KquTP55o2g8/3+qshSt2rZu9bFaChEzSQZi5U8dNuxyPPuOIcLXwO/ -B7I1IGM5D7dpLupPatZqL4uMJMZ5d8bc85GzmcSmMEN+EhfwbssnXbO3RkXwYsgM -kDKF+n+KhoDj+KcUN6VqnQlkZ7iNLVKB9ONpSEXWEazEJG6+IDIhAN7aUTq/abHD -jgM959VX15tXssEHkDj1m64qt2oO9/kiY3MrMvtpD0Atg2unJiL6Z5UUrJnNBFiQ -Llf/GAZrbJdBC8WNJi2qUYQr1E7rindeoQcRcnjXuRjisq3JpOK3jqY9mHN6Wmh1 -vWcUxvysNP90b8q9jipFWHuD0M37kq+BLn5Bub0ypiIkId0CUnAB9MBYcBJlYhai -ZwI1fe0uGFD7XlJbHexTgnLreDAo3FR9CIUDo2HUWqmUNWadAl/rPNRe6+QDDvmP -5v4HiFmSuCjZJOu9x2z/ly1JzM+iCUp+q6BxYYYW/5tDLYAw7sl1uaiLTzZuhrrM -PlO6DNLAQhMn29jeszPHt7iXHdHAHAuYSeHpfeqnAV1qB+6x7UFVZjbDxXkt/Sn0 -+LvCzJUQOwQNlnnzIwVdn8phS3r9TN2rI3dtlvPMWJqgBiheJ9qn2tHjjoPETt9I -hfvw949Gi65D+AFSzowjNUFwDXzphOwETv5tpKCRROhdBRBdwQKCAQEA+L9RsVqT -F+7HyGza+F53mgED5SQoS52vRA2OiAbCgiNjY7JH7bqIpuO4RlgqKo+GKoboCP6D -1CmVGUm/Z8wYspzQs15O/jUO1bZ8KFREt8TquxFtikwyvIXQhUdJhZYUnhfMzV3O -sH1blWhJnSX21rxJWlrkN0I8Zdkl6mjvFa97Kr9UA/pdZd0qgIw5Vi7MFLPC7j2Q -YmTPhNsb0oZMJHGvwENUmuCQDhGiRhQV06R963mTMvxY7LWqUVf6dr7xg89Qt5Yo -AdSHllOxHOMTAa+kZNF1N8UM9S2iJSn6ZeUEOXOJEuosghpE/QIuvo81Txm63G7e -BjU3H7cFqDetfQKCAQEAy3xy2cQ/+GlSIbwXrzBr483Z0jXnvknlCJMh+NCTXObk -idOhhnIuZu+JoAovv2AfKNPvYXotmb1xxmws5RSrlZDGiQQzEwvJPeLN2DnUGqzc -ZPenu64Je6v9L35iRMF8vyx3xf27FC4zmR6nLuZbgfEfQdModqCbTpzh23Cl3mkM -IZFYPhhfnh/pcwccuqfOn0Adt+1X3jvp3QzCh1jkEjhaRB5qjt58nlmxA2EKYv1w -OzSTH9owqsCMmdrqzR7iKh59LrfOfggJbhHCyrORZ/S8h5lwqIk3+zLMrwGSvkXL -tuKLXtkX/Xy98cbHwk5M/bf3hH6I5njlsssFsS8+SwKCAQEAxCzu2raaJ1fUDAd9 -sj+eh8ChN8gKV4hmv38Jl9Hs+QG70ta5z407VJNns2K47pP+te9rdBx2D48z3ZvB -7rSSDduK5MtN9UIXDwk6Zfv/rgcJMLuP7nAl23SVfWc5Xrd8TypqBNUkuyBCaFS1 -KdDVGYmpOC9SqRn91D0rn/FeDXY15wK52eFMY5fHe1YbqhKCNRmIdKftBQyIdTjw -elocFunqN/Fh+jt8oPvbRPV2OVITVPCu3JkT8KtdRYXjLF9uzgtkl0U/DCJ3RGGA -301eogfJ2REwJumrTHnO1QyERHQXns+1nUs+CuV43ykngHYlDts1+b8eLzss3EBV -n9M5aQKCAQEArqKmmtg/on0ZPNSFaxfecEq5lxwmQHyAsMQ9UqIG5qNOHi9fn9gc -lMEdVxmG8vKWq16AQiMuQZSBsa4jNZNw0tLGYM8W2lCyLIea6+htbVtPZuPYs0zg -3J+1ke4gfiukWRnbzTM+PEqOg+n3x1txy2pZzg9f2bdqsqQXflIGOIPlImXv2pLm -dPmkS9Edyd+8h5XqK3DpiVPYGJsb1Dbove5ZIb8M6oJtZyVIssK0vFIP4O/1GFAU -lmbcBCsKenH33ff+rXqYIDfbh/h8OaS0tQgoSSPZuPrS7aYiXku2Wc/izplMzWD5 -otZM2dQkmlDC6LjbF33VFh9J2xE8WF1YUwKCAQAeJYro7nBxM0eOmof1ty24UPfg -jx72sH/FpgKIyvZ4yQoreNUc4TVsy5QMIVd0G966CRgvzaE0vcBHm//7YCXHtIa9 -ihqmYDo7SoaF7nZNjxJIxyQVPY0+Kntkwz0XAX0IbJ0nMx+3x6d5UhbQbxFVKe7X -5WmOMb0ro9NLaCvh5IUxSHsG/a8hYRqoX3tZbPRvTJMZMTMxWslsscWINNu/80KS -ggpD9Uu9hdVwT7yavl6JKC3ypRdBzmpKZfiLt5CTFex+XGIgKLHVqbHxXu487YsL -AlexBvk1/RKMTHIgUl7uMmaJsUSD+ME4SWuU9cW115kwp+JBMXES4ZfWnRHZ +MIIJKAIBAAKCAgEAvcaaRximrOK+iFoVzUbYOBGmbrny3bsQunqqTiLCNnwhkydC +Xc31kyZtT6MVaD5m80RNmiL3Vs6Cc+E+1ecYQ/vUKC3jv6snla/By+UC+LOW8x/u +aqsOEXMCopfuQt0/5qQV8L9aN/3CHcwW9PFk5ve6E0dR6w8x7845FTSLavdDMfAJ +Ndz31zwVHTFhF1f0KvqKDiKt1uCbBt0EpETerbd1UldROo3MdzjpreP0qV8T2TWE +nLSTSCm4M6WJmyWJwgb8GWNt4mofa1YZ4mui+c7pMc5+3tJpRXWElC0jECPRhjZr +QwxRsqK2+ZFn6g9m6wZAWX5EDtvS/F5QreFM7qhIYZWoGFJ7hPnYQEnvhzQU4l0b +Uenrm3oM38dLSs7xxBEDsRUXHAyee+i1/0TU7tm/pRFj6VTw+vdoEjNhUTVci+Zk +iPrQGy3zuZHPW0axZ/32EPFbIVp6xWqDkQjkb2sp/UJnJDJNPCF8tuvjCwmj5Ldj +jNI/THI7KlpOJnDJ1ZVv/gnkwGVo2ztkxTeC6Cn2ESfgHNjOTyXFyoKEoyXesV2N +Y+RKXb+0/f1a9Wj9PTzy8pLq//VGrifQjPfd3cVtrIE2UcxmMc+bCBu8vlgqOcS2 +RDKL7qKwG3yD/govFlls3m53KsIlbuffEaPcNiiSE5lW3uumIwCCocAQHRUCAwEA +AQKCAgAq/Vhpjp4DQAIlZTLXI5tLaEQphRoNPJkXhT4bISiZqxj3+sa/9S4SPXw5 +tBnfWXN83BYwOoeJNJK8qWPQlN8cV2nCCFM1UhJPiFcAV0qLCDPDs5IQu9sd/M22 +A3DH2NLm7njB5rcLRAK7OUZiCmTvJWMThu55ryGCz1aDBTon6wdfwp5zgzDpowSt +bkguP5BiRAsOFLEIoiy8K0kn5SoEdDCxjHIsL38H6u6Uo7UCwTT/2W3HCejrSfge +SvuXF7PTLj3hinKT/bJMYF7Quc1bhZGx5LM8yrFqOKrl3iu5NkP6gU1fDOVc8Bs/ +Ab9meEK6LtpY64+DRizPWpYwLX99lMlGS1v8GsLuAPCbQ4RGJLRl2nUsrowk8xI1 +BSVEv+QcYB0Tp5CeOa+ejsI23HczmSBW6j4sqx1/vTu5tKxtvxppKOjkV4rrbAnv +mK6NPpUo3bVtUJIejVALE2PlXLM6XlZJinLPKFfOGRQ9VTdgVsQS6oHfoxknzxCy +MHtzxNt/ZLCzWa0uszta6kjta0qburn/q9DAGwC65W/BeSzc79qaEOf2bdNSHxeD +1ENnKGN3G3PUGCH9/Fs6bEX6/bX0WihfOq2XTxBNs0jcjCycHXvOGTp7jRFjbaIs +a87b6whFLL52oKEqpq6o7JqXa8I/Pn0eBp07Soos5WWfHtLsgQKCAQEA56U80Jjt +iAZMoxLDRNY0WsYavvLlarZISbHy51eYl54c1YNvnmPheSddRoRW+AWQQ1iQ710z +p/KMm7D81PArUplIe8GutbqqvmezsIQqvCUbWC672KiA7xH6zGKlnRmqryefare6 +nvR48sEcsfwywoW7v6yg2ZzY9uz6FcHuN27pHk+b2oMd1qGeSSVuhqZ1XUPFqAF2 +IcxVMd4upsFLCVz94kqOcyYHAUNHZ8ANaT6EATF4bRbccmGzjo8PiS0mhUCL8nVh +2tcB6fsT1i0sJ19e1cPBgmxy2pOc25D8aldt6Bds0lQl0Zhfl63/R1o00S0z4Mgt +9/LycTKuL0fTZQKCAQEA0bpwQHNLF7nwtfsLNZifXApHNNmiiyBvK/SQqqn2wdA3 +l8Ol6hcborYfuwvL+5dTi79DIROF9RqpWoRwlMq0XIAldW4H46KUWuZrtfDD0T2B +VFB4nlihMCZT2lOsDrNMcykZGLAb49u0GKeyNE1z2RiLR/zTQvD2mVXSt7UVNsgA +xnInCF3wN1oMZOu2N5paXAfiw7L91ZVpacV8LtZhGaby6Y0hjy9Oh1/DjjQb+aFH +r2A7AIMFtUtKIpy4ymBBcJWH952fVdImAZWcYUTMILj/eOML9KIc/iAdaLZzgPA+ +cged1qGIEDjq1pjyhgB83NHPjQwN9+qnuJ6KD5d/8QKCAQBCKQGLmoINHa95y/wV +hxCA9J/i9cDsj49p7Pcxd+VPOIuHt/iDRoe5cLLFVzr3r68abQtIMlh0mWPre8ta +jhXxi3IWVqUtpljp9Phva/BAuGBs+TwRj17z6TC2e15N1mzXhfuUIVGd4LlCrEEj +3KIqyMNs5UMzpAabz+dFm5QMJRNT4uziphw5lwN8ZMKClrxinN6xM2moEZIroge0 +/PQPgo3Iuf6hiLTreWnFiibKbl6JcLWygpsH2DvOdbG8PjcIS/0sjLYcKNiVGW76 +9v/rZuPRLg3w0ESlhQ+B7pEJVls/tKESGZKLoHd8/OcyW+NF8yEkDEiEztDL3+u7 ++Um5AoIBAEQZZZmH1jo+BgsRGPKuPBCGxdhDUqQaQ/7hypef2J/WHDcLMDRyjGao +/GSfz/xBhr85u3JiBH0xywu5NVUa/LWMZp1avPUNNV7OsurA4tRuRDA8cO35mV0p +FNEvmg6r6Yb7MUry5Bt4m2HmEdcpzxQPBCq7zV1PIMNpOqKsCddf2eCV8FanKfjP +JHcgkmo7lFuNs/QXipRvj2ro1QngUmch7n4ndV4o3jbWwYjLIspLtBseBAqGgLP9 +XnUermIIHzePVjbw12vmcLDTA5QR4rY9W7bFAnzMpt1dUC9QDvEvERe1oWqvyJ57 +3MURpK4eOmz5M4t4/pAhgZRQ5kNpzgECggEBANT1KAU4EvXxYXpYaq9O9YTQnSbH +8HBxsN2TeKEb29EyFHRhd5TxdV1/wDcRY4uqopIN8/xsLFqoOWyPLn/23bwpAuU9 +YskLKnQspN4w2Eyn0b9K4cF3SigwmlEEqDk8sxTtZ5iv7olE5GaeoZiQQ8yS3oZc +QWVe6gierK0XdwnuPvE4Yio3gMFxuy1PrMPJj9JyKjWIvT6ODF7nWYRdnwPTfqKA +cxXoPKajyk2LVAXt9qQao/o6/AWeyZhgPhFiT9NUEFiBcmnUTq8xblOXpbZev6eC +nUKF5Oy9r4aI9eafeLWhVqTmXLUtExLjfL6rmLu+HlOdd9SYpYEIdCzMIe8= -----END RSA PRIVATE KEY----- -- cgit v1.2.3 From a5ac9fa3c8c7910997fc1b0ee305f729f1bf061a Mon Sep 17 00:00:00 2001 From: Varac Date: Thu, 15 Jun 2017 15:48:32 +0200 Subject: Stop sending mails for nagios alerts It's just too much mail... And there are other tools like nagstamon that are better suited to get an overview what's failing. Resolves: #8772 --- puppet/modules/site_nagios/files/configs/Debian/nagios.cfg | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/puppet/modules/site_nagios/files/configs/Debian/nagios.cfg b/puppet/modules/site_nagios/files/configs/Debian/nagios.cfg index 62f26f2c..1a9f266e 100644 --- a/puppet/modules/site_nagios/files/configs/Debian/nagios.cfg +++ b/puppet/modules/site_nagios/files/configs/Debian/nagios.cfg @@ -773,7 +773,7 @@ accept_passive_host_checks=1 # service notifications when it is initially (re)started. # Values: 1 = enable notifications, 0 = disable notifications -enable_notifications=1 +enable_notifications=0 @@ -1299,4 +1299,3 @@ host_perfdata_file_template=DATATYPE::HOSTPERFDATA\tTIMET::$TIMET$\tHOSTNAME::$H host_perfdata_file_mode=a host_perfdata_file_processing_interval=15 host_perfdata_file_processing_command=process-host-perfdata-file-pnp4nagios-bulk-npcd - -- cgit v1.2.3 From 5a0d716c70c5e62edb83df91b084e1c9b89a33f6 Mon Sep 17 00:00:00 2001 From: Varac Date: Wed, 21 Jun 2017 13:16:36 +0200 Subject: Use apt master component for LEAP packages Currently, the platform configures the `snapshots` component in /etc/apt/sources.list.d/leap.list. `snapshots` contains packages uploaded by feature branches and merge requests so we change to `master` (which contains packges built from changes to the master branches. Resolves: #8828 --- provider_base/common.json | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/provider_base/common.json b/provider_base/common.json index 97519950..346e86a6 100644 --- a/provider_base/common.json +++ b/provider_base/common.json @@ -77,7 +77,7 @@ "platform": { "apt": { "basic": "http://deb.leap.se/platform", - "component": "snapshots" + "component": "master" } }, "soledad": { -- cgit v1.2.3 From f3bafe6b2cb55305bba588fdca4f4e77e3ef2026 Mon Sep 17 00:00:00 2001 From: Varac Date: Thu, 22 Jun 2017 11:23:43 +0200 Subject: Delay hard state of the nagios APT check Delay a hard state of the APT check for 1 day so unattended_upgrades has time to upgrade packages. Resolves: #8748 --- puppet/modules/site_check_mk/files/extra_service_conf.mk | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/puppet/modules/site_check_mk/files/extra_service_conf.mk b/puppet/modules/site_check_mk/files/extra_service_conf.mk index c7120a96..9212af95 100644 --- a/puppet/modules/site_check_mk/files/extra_service_conf.mk +++ b/puppet/modules/site_check_mk/files/extra_service_conf.mk @@ -1,6 +1,9 @@ # retry 3 times before setting a service into a hard state -# and send out notification +# Delay a hard state of the APT check for 1 day +# so unattended_upgrades has time to upgrade packages. +# extra_service_conf["max_check_attempts"] = [ + ("360", ALL_HOSTS , ["APT"] ), ("4", ALL_HOSTS , ALL_SERVICES ) ] @@ -11,4 +14,3 @@ extra_service_conf["max_check_attempts"] = [ extra_service_conf["normal_check_interval"] = [ ("4", ALL_HOSTS , "Check_MK" ) ] - -- cgit v1.2.3 From 16c8f00a6a6e388f8a8f8b831d7e507f8cb67c78 Mon Sep 17 00:00:00 2001 From: Varac Date: Fri, 23 Jun 2017 20:13:18 +0200 Subject: Use default apt component for CI Resolves: #8828 --- tests/platform-ci/provider/common.json | 10 ---------- 1 file changed, 10 deletions(-) delete mode 100644 tests/platform-ci/provider/common.json diff --git a/tests/platform-ci/provider/common.json b/tests/platform-ci/provider/common.json deleted file mode 100644 index e5096c47..00000000 --- a/tests/platform-ci/provider/common.json +++ /dev/null @@ -1,10 +0,0 @@ -{ - "sources": { - "platform": { - "apt": { - "basic": "http://deb.leap.se/platform", - "component": "snapshots" - } - } - } -} -- cgit v1.2.3 From d0f016cf8f3130c769a1de8be8d89790c0d8e511 Mon Sep 17 00:00:00 2001 From: Varac Date: Fri, 23 Jun 2017 20:25:06 +0200 Subject: Use stdbuf instead of unbuffer for CI timestamping There are different reasons for this: - Using `stdbuf` will correctly return and non-zero exit code so when something breaks during CI `gitlab-runner` will mark the build as failed (Resolves: #8821). - `stdbuf` is already installed by the `coreutils` package and thus saves diskspace --- .gitlab-ci.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml index f515337f..3d622d1c 100644 --- a/.gitlab-ci.yml +++ b/.gitlab-ci.yml @@ -60,7 +60,7 @@ deploy_test: except: - master script: - - su -c '/usr/bin/unbuffer /bin/bash -o pipefail ./ci-build.sh | /usr/bin/ts' cirunner + - su -c 'set -o pipefail; stdbuf -oL -eL ./ci-build.sh | ts' cirunner # Latest job will only run on the master branch, which means all merge requests # that are created from branches don't get to deploy to the latest-ci server. -- cgit v1.2.3 From fc907004a75ab2f8f8302706150b68a9cdd6baf0 Mon Sep 17 00:00:00 2001 From: Varac Date: Sat, 24 Jun 2017 12:14:10 +0200 Subject: Add configured apt component to the unattended-upgrades whitelist Resolves: #8792 --- puppet/modules/site_apt/files/Debian/51unattended-upgrades-leap | 6 ------ puppet/modules/site_apt/manifests/unattended_upgrades.pp | 3 +-- puppet/modules/site_apt/templates/51unattended-upgrades-leap | 5 +++++ 3 files changed, 6 insertions(+), 8 deletions(-) delete mode 100644 puppet/modules/site_apt/files/Debian/51unattended-upgrades-leap create mode 100644 puppet/modules/site_apt/templates/51unattended-upgrades-leap diff --git a/puppet/modules/site_apt/files/Debian/51unattended-upgrades-leap b/puppet/modules/site_apt/files/Debian/51unattended-upgrades-leap deleted file mode 100644 index bbaac6a2..00000000 --- a/puppet/modules/site_apt/files/Debian/51unattended-upgrades-leap +++ /dev/null @@ -1,6 +0,0 @@ -// this file is managed by puppet ! - -Unattended-Upgrade::Allowed-Origins { - "leap.se:stable"; -} - diff --git a/puppet/modules/site_apt/manifests/unattended_upgrades.pp b/puppet/modules/site_apt/manifests/unattended_upgrades.pp index 42f1f4c6..ddadd35a 100644 --- a/puppet/modules/site_apt/manifests/unattended_upgrades.pp +++ b/puppet/modules/site_apt/manifests/unattended_upgrades.pp @@ -11,8 +11,7 @@ class site_apt::unattended_upgrades { # configure LEAP upgrades apt::apt_conf { '51unattended-upgrades-leap': - source => [ - "puppet:///modules/site_apt/${::lsbdistid}/51unattended-upgrades-leap"], + content => template('site_apt/51unattended-upgrades-leap'), require => Package['unattended-upgrades'], refresh_apt => false, } diff --git a/puppet/modules/site_apt/templates/51unattended-upgrades-leap b/puppet/modules/site_apt/templates/51unattended-upgrades-leap new file mode 100644 index 00000000..3e28531f --- /dev/null +++ b/puppet/modules/site_apt/templates/51unattended-upgrades-leap @@ -0,0 +1,5 @@ +// this file is managed by puppet ! + +Unattended-Upgrade::Origins-Pattern { + "site=deb.leap.se,component=<%= @apt_platform_component %>"; +} -- cgit v1.2.3 From 1c00a62a89301866cd68900e25e1a45a108f67d4 Mon Sep 17 00:00:00 2001 From: Varac Date: Mon, 26 Jun 2017 21:10:52 +0200 Subject: Lint .gitlab-ci.yml --- .gitlab-ci.yml | 20 ++++++++++++++------ 1 file changed, 14 insertions(+), 6 deletions(-) diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml index 3d622d1c..c4de7086 100644 --- a/.gitlab-ci.yml +++ b/.gitlab-ci.yml @@ -1,7 +1,9 @@ +--- image: 0xacab.org:4567/leap/docker/ruby:latest -# This is for caching the gems not only between the stages, but also persistent -# on the gitlab-runner so we don't need to install from scratch on every pipeline +# This is for caching the gems not only between the stages, but also +# persistent on the gitlab-runner so we don't need to install from +# scratch on every pipeline cache: key: "$CI_BUILD_REF_NAME" untracked: true @@ -46,7 +48,7 @@ catalog: script: - su -c '/usr/local/bin/bundle exec rake catalog' cirunner -#rspec: +# rspec: # stage: rspec # script: # - /usr/local/bin/bundle exec rake spec @@ -74,7 +76,9 @@ ci.leap.se: only: - master@leap/platform script: - - su -c '/usr/bin/unbuffer /bin/bash -o pipefail ./ci-build.sh | /usr/bin/ts' cirunner + - > + su -c '/usr/bin/unbuffer /bin/bash -o pipefail ./ci-build.sh | + /usr/bin/ts' cirunner demo.bitmask.net: stage: deploy @@ -84,7 +88,9 @@ demo.bitmask.net: - master when: manual script: - - su -c '/usr/bin/unbuffer /bin/bash -o pipefail ./ci-build.sh | /usr/bin/ts' cirunner + - > + su -c '/usr/bin/unbuffer /bin/bash -o pipefail ./ci-build.sh | + /usr/bin/ts' cirunner mail.bitmask.net: stage: deploy @@ -94,4 +100,6 @@ mail.bitmask.net: - master when: manual script: - - su -c '/usr/bin/unbuffer /bin/bash -o pipefail ./ci-build.sh | /usr/bin/ts' cirunner + - > + su -c '/usr/bin/unbuffer /bin/bash -o pipefail ./ci-build.sh | + /usr/bin/ts' cirunner -- cgit v1.2.3 From e24292f13a6bef60ee7b8f2aa0ac2174ad96397e Mon Sep 17 00:00:00 2001 From: Varac Date: Mon, 26 Jun 2017 21:31:07 +0200 Subject: Add manual deploy test for master branch --- .gitlab-ci.yml | 11 +++++++++++ 1 file changed, 11 insertions(+) diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml index c4de7086..21ea1f73 100644 --- a/.gitlab-ci.yml +++ b/.gitlab-ci.yml @@ -64,6 +64,17 @@ deploy_test: script: - su -c 'set -o pipefail; stdbuf -oL -eL ./ci-build.sh | ts' cirunner +# However, sometimes it's important to have a way of triggering a deploy +# from scratch manually even from the master branch, when i.e. new packages +# got uploaded to the master component of the platform deb repo. +deploy_test:manual: + stage: deploy + only: + - master + when: manual + script: + - su -c 'set -o pipefail; stdbuf -oL -eL ./ci-build.sh | ts' cirunner + # Latest job will only run on the master branch, which means all merge requests # that are created from branches don't get to deploy to the latest-ci server. # When a merge request is merged, then the latest job will deploy the code to -- cgit v1.2.3 From b04a1a8e7f4e4f53d1e57fd4868f6741d9420fe1 Mon Sep 17 00:00:00 2001 From: Varac Date: Tue, 27 Jun 2017 14:19:24 +0200 Subject: Don't depend on leap-keymanager anymore leap-mx is now independent of leap-keymanager and we can remove this dependency now. see https://0xacab.org/leap/leap_mx/issues/8558 --- puppet/modules/leap_mx/manifests/init.pp | 14 +++++--------- 1 file changed, 5 insertions(+), 9 deletions(-) diff --git a/puppet/modules/leap_mx/manifests/init.pp b/puppet/modules/leap_mx/manifests/init.pp index 558b5404..f26448e2 100644 --- a/puppet/modules/leap_mx/manifests/init.pp +++ b/puppet/modules/leap_mx/manifests/init.pp @@ -94,15 +94,11 @@ class leap_mx { # LEAP-MX CODE AND DEPENDENCIES # - package { - $sources['leap-mx']['package']: - ensure => $sources['leap-mx']['revision'], - require => [ - Class['site_apt::leap_repo'], - User['leap-mx'] ]; - - 'leap-keymanager': - ensure => latest; + package { $sources['leap-mx']['package']: + ensure => $sources['leap-mx']['revision'], + require => [ + Class['site_apt::leap_repo'], + User['leap-mx'] ]; } # -- cgit v1.2.3 From 98b6713afff0eec77fdbfe5d1a079607e6ed5b2c Mon Sep 17 00:00:00 2001 From: Varac Date: Tue, 27 Jun 2017 16:12:37 +0200 Subject: Install python-treq from strech on jessie nodes New soledad-common depends on `python-treq`, which is only available in debian stretch. We pin all stretch packages to 1 (same as for sid), which means (from `man apt_preferences`): "causes a version to be installed only if there is no installed version of the package" - Resolves: #8836 --- puppet/modules/site_apt/manifests/init.pp | 22 +++++++++++++--------- .../templates/Debian/preferences_jessie.erb | 19 +++++++++++++++++++ 2 files changed, 32 insertions(+), 9 deletions(-) create mode 100644 puppet/modules/site_apt/templates/Debian/preferences_jessie.erb diff --git a/puppet/modules/site_apt/manifests/init.pp b/puppet/modules/site_apt/manifests/init.pp index 798d0b84..60fe0483 100644 --- a/puppet/modules/site_apt/manifests/init.pp +++ b/puppet/modules/site_apt/manifests/init.pp @@ -20,20 +20,24 @@ class site_apt { $apt_platform_codename = $platform_sources['apt']['codename'] } - # needed on jessie hosts for getting pnp4nagios from testing + # needed on jessie hosts for getting python-treq from stretch + # see https://0xacab.org/leap/platform/issues/8836 if ( $::operatingsystemmajrelease == '8' ) { - $use_next_release = true + $use_next_release = true + $custom_preferences = template("site_apt/${::operatingsystem}/preferences_jessie.erb") } else { - $use_next_release = false + $use_next_release = false + $custom_preferences = '' } class { 'apt': - custom_key_dir => 'puppet:///modules/site_apt/keys', - debian_url => $apt_url_basic, - security_url => $apt_url_security, - backports_url => $apt_url_backports, - use_next_release => $use_next_release, - repos => 'main' + custom_key_dir => 'puppet:///modules/site_apt/keys', + debian_url => $apt_url_basic, + security_url => $apt_url_security, + backports_url => $apt_url_backports, + use_next_release => $use_next_release, + custom_preferences => $custom_preferences, + repos => 'main' } # enable http://deb.leap.se debian package repository diff --git a/puppet/modules/site_apt/templates/Debian/preferences_jessie.erb b/puppet/modules/site_apt/templates/Debian/preferences_jessie.erb new file mode 100644 index 00000000..879885dd --- /dev/null +++ b/puppet/modules/site_apt/templates/Debian/preferences_jessie.erb @@ -0,0 +1,19 @@ +Explanation: Debian jessie +Package: * +Pin: release o=Debian,n=jessie +Pin-Priority: 990 + +Explanation: Debian stretch +Package: * +Pin: release o=Debian,n=stretch +Pin-Priority: 1 + +Explanation: Debian sid +Package: * +Pin: release o=Debian,n=sid +Pin-Priority: 1 + +Explanation: Debian fallback +Package: * +Pin: release o=Debian +Pin-Priority: -10 -- cgit v1.2.3 From 03ff5b1d22f4487d97818da3693e9a33ba1421a3 Mon Sep 17 00:00:00 2001 From: Varac Date: Tue, 27 Jun 2017 20:26:04 +0200 Subject: Pin python-cryptography to jessie-backports Needed to satisfy leap-mx dependency (>=17.0) - Resolves: #8837 --- .../site_apt/manifests/preferences/python_cryptography.pp | 12 ++++++++++++ puppet/modules/site_mx/manifests/init.pp | 2 ++ 2 files changed, 14 insertions(+) create mode 100644 puppet/modules/site_apt/manifests/preferences/python_cryptography.pp diff --git a/puppet/modules/site_apt/manifests/preferences/python_cryptography.pp b/puppet/modules/site_apt/manifests/preferences/python_cryptography.pp new file mode 100644 index 00000000..d725c1af --- /dev/null +++ b/puppet/modules/site_apt/manifests/preferences/python_cryptography.pp @@ -0,0 +1,12 @@ +# Pin python-cryptography to jessie-backports in order to +# satisfy leap-mx dependency (>=17.0) +# see https://0xacab.org/leap/platform/issues/8837 +class site_apt::preferences::python_cryptography { + + apt::preferences_snippet { 'python_cryptography': + package => 'python-cryptography python-openssl python-pyasn1 python-setuptools python-pkg-resources python-cffi', + release => "${::lsbdistcodename}-backports", + priority => 999; + } + +} diff --git a/puppet/modules/site_mx/manifests/init.pp b/puppet/modules/site_mx/manifests/init.pp index 5876e555..28a01d4a 100644 --- a/puppet/modules/site_mx/manifests/init.pp +++ b/puppet/modules/site_mx/manifests/init.pp @@ -19,4 +19,6 @@ class site_mx { include ::site_check_mk::agent::mx # install twisted from jessie backports include ::site_apt::preferences::twisted + # install python-cryptography from jessie backports + include ::site_apt::preferences::python_cryptography } -- cgit v1.2.3 From de892c3821b2053f23ca6a91bf097b05a150ff41 Mon Sep 17 00:00:00 2001 From: elijah Date: Wed, 28 Jun 2017 13:23:02 -0700 Subject: static - gracefully handle incorrect static site configs --- provider_base/services/static.rb | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/provider_base/services/static.rb b/provider_base/services/static.rb index d020ba26..4c7d2e59 100644 --- a/provider_base/services/static.rb +++ b/provider_base/services/static.rb @@ -1,2 +1,4 @@ -self['dns']['aliases'] += self.static.domains.keys -self['dns']['aliases'].uniq! +if self['static'] && self['static']['domains'] + self['dns']['aliases'] += self['static']['domains'].keys + self['dns']['aliases'].uniq! +end \ No newline at end of file -- cgit v1.2.3 From ef455a41d6529898a2d8848e65e464705729e661 Mon Sep 17 00:00:00 2001 From: elijah Date: Wed, 28 Jun 2017 15:26:11 -0700 Subject: platform test - pin ruby version to 2.1.10 when using rbenv, since that is the latests that will work. --- tests/platform-ci/.ruby-version | 1 + 1 file changed, 1 insertion(+) create mode 100644 tests/platform-ci/.ruby-version diff --git a/tests/platform-ci/.ruby-version b/tests/platform-ci/.ruby-version new file mode 100644 index 00000000..8dbb0f26 --- /dev/null +++ b/tests/platform-ci/.ruby-version @@ -0,0 +1 @@ +2.1.10 -- cgit v1.2.3 From 30a4bc4eba4654de8e5bd56a083a78b474a2a6ff Mon Sep 17 00:00:00 2001 From: Varac Date: Thu, 29 Jun 2017 15:30:52 +0200 Subject: [CI] Pin leap_cli so fog-aws is installed --- tests/platform-ci/Gemfile | 2 ++ 1 file changed, 2 insertions(+) diff --git a/tests/platform-ci/Gemfile b/tests/platform-ci/Gemfile index 51e2c17b..44a8637d 100644 --- a/tests/platform-ci/Gemfile +++ b/tests/platform-ci/Gemfile @@ -13,5 +13,7 @@ group :test do # Use puppet-catalog-test from git because last released gem 0.4.2 gives a deprecation # warning: "[DEPRECATION] `last_comment` is deprecated. Please use `last_description` instead." gem "puppet-catalog-test", :git => 'https://github.com/invadersmustdie/puppet-catalog-test.git', :ref => 'ac386793c2c456d2071dd0adda716224128f0bb3' + # Install fog-aws because the gem dependency of leap_cli is now optional + gem "fog-aws" gem "leap_cli", :git => 'https://leap.se/git/leap_cli.git' end -- cgit v1.2.3 From 9f315d37968f0135deb3fadcdf6cf54278153de0 Mon Sep 17 00:00:00 2001 From: Varac Date: Thu, 29 Jun 2017 16:20:30 +0200 Subject: [CI] Check for mandatory env variables on platform builds --- tests/platform-ci/ci-build.sh | 10 ++++++++++ 1 file changed, 10 insertions(+) diff --git a/tests/platform-ci/ci-build.sh b/tests/platform-ci/ci-build.sh index 34876a73..e2485c0a 100755 --- a/tests/platform-ci/ci-build.sh +++ b/tests/platform-ci/ci-build.sh @@ -54,6 +54,11 @@ else } fi +fail() { + echo "$*" + exit 1 +} + deploy() { LEAP_CMD deploy "$TAG" } @@ -73,6 +78,11 @@ build_from_scratch() { # Dsiable xtrace set +x + + [ -z "$AWS_ACCESS_KEY" ] && fail "\$AWS_ACCESS_KEY is not set - please provide it as env variable." + [ -z "$AWS_SECRET_KEY" ] && fail "\$AWS_SECRET_KEY is not set - please provide it as env variable." + [ -z "$SSH_PRIVATE_KEY" ] && fail "\$SSH_PRIVATE_KEY is not set - please provide it as env variable." + /usr/bin/jq ".platform_ci.auth |= .+ {\"aws_access_key_id\":\"$AWS_ACCESS_KEY\", \"aws_secret_access_key\":\"$AWS_SECRET_KEY\"}" < cloud.json.template > cloud.json # Enable xtrace again only if it was set at beginning of script [[ $xtrace == true ]] && set -x -- cgit v1.2.3 From 3f61221b48730be56ddb19d31d79fc225c8626c7 Mon Sep 17 00:00:00 2001 From: Varac Date: Thu, 29 Jun 2017 16:25:22 +0200 Subject: [CI] Run setup.sh when running CI tests locally --- .gitlab-ci.yml | 3 +++ 1 file changed, 3 insertions(+) diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml index 21ea1f73..0e90ce75 100644 --- a/.gitlab-ci.yml +++ b/.gitlab-ci.yml @@ -12,6 +12,9 @@ cache: before_script: - cd tests/platform-ci + # Check if running locally ($CI_PROJECT_ID is 0 then) and run setup.sh in + # this case + - if [ "$CI_PROJECT_ID" = "0" ]; then ./setup.sh; fi stages: - setup -- cgit v1.2.3 From c2cc39e63b7afbc755c81e94f34791c605fb092b Mon Sep 17 00:00:00 2001 From: Varac Date: Thu, 29 Jun 2017 16:25:32 +0200 Subject: [CI] Add upgrade platform CI test Resolves: #8541 --- .gitlab-ci.yml | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml index 0e90ce75..5933331b 100644 --- a/.gitlab-ci.yml +++ b/.gitlab-ci.yml @@ -78,6 +78,14 @@ deploy_test:manual: script: - su -c 'set -o pipefail; stdbuf -oL -eL ./ci-build.sh | ts' cirunner +# Test upgrades from the latetest release to latest HEAD +upgrade_test: + stage: deploy + script: + # Allow unpriviledged user to checkout last release of leap_platform + - chown cirunner:cirunner -R ../.. + - su -c 'set -o pipefail; stdbuf -oL -eL ./ci-build.sh | ts' cirunner + # Latest job will only run on the master branch, which means all merge requests # that are created from branches don't get to deploy to the latest-ci server. # When a merge request is merged, then the latest job will deploy the code to -- cgit v1.2.3 From a3fc434020da4ea8fb447536fd37906ca0a8d890 Mon Sep 17 00:00:00 2001 From: Varac Date: Thu, 29 Jun 2017 16:40:57 +0200 Subject: [CI] Use CI_JOB_NAME to determine what action to take --- tests/platform-ci/ci-build.sh | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/tests/platform-ci/ci-build.sh b/tests/platform-ci/ci-build.sh index e2485c0a..b554b1d3 100755 --- a/tests/platform-ci/ci-build.sh +++ b/tests/platform-ci/ci-build.sh @@ -92,7 +92,7 @@ build_from_scratch() { pwd -# remove old cached nodes + # remove old cached nodes echo "Removing old cached nodes..." find nodes -name 'citest*' -exec rm {} \; @@ -149,16 +149,16 @@ set +x # Enable xtrace again only if it was set at beginning of script [[ $xtrace == true ]] && set -x -case "$CI_ENVIRONMENT_NAME" in - staging) +case "$CI_JOB_NAME" in + ci.leap.se) TAG='latest' run ibex ssh://gitolite@leap.se/ibex ;; - production/demo/mail) + mail.bitmask.net) TAG='demomail' run bitmask ssh://gitolite@leap.se/bitmask ;; - production/demo/vpn) + demo.bitmask.net) TAG='demovpn' run bitmask ssh://gitolite@leap.se/bitmask ;; -- cgit v1.2.3 From 6e7b18f8732e79957aaa2e536b368aef8a9d6ab9 Mon Sep 17 00:00:00 2001 From: Varac Date: Thu, 29 Jun 2017 16:47:54 +0200 Subject: [CI] Move more commands into functions --- tests/platform-ci/ci-build.sh | 30 +++++++++++++++++------------- 1 file changed, 17 insertions(+), 13 deletions(-) diff --git a/tests/platform-ci/ci-build.sh b/tests/platform-ci/ci-build.sh index b554b1d3..abc5fec8 100755 --- a/tests/platform-ci/ci-build.sh +++ b/tests/platform-ci/ci-build.sh @@ -68,6 +68,16 @@ test() { } build_from_scratch() { + # create node(s) with unique id so we can run tests in parallel + NAME="citest${CI_BUILD_ID:-0}" + # when using gitlab-runner locally, CI_BUILD_ID is always 1 which + # will conflict with running/terminating AWS instances in subsequent runs + # therefore we pick a random number in this case + [ "${CI_BUILD_ID:-0}" -eq "1" ] && NAME+="000${RANDOM}" + + TAG='single' + SERVICES='couchdb,soledad,mx,webapp,tor,monitor' + # leap_platform/tests/platform-ci/provider PROVIDERDIR="${ROOTDIR}/provider" /bin/echo "Provider directory: ${PROVIDERDIR}" @@ -130,6 +140,12 @@ run() { test } +cleanup() { + # if everything succeeds, destroy the vm + LEAP_CMD vm rm "${TAG}" + [ -f "nodes/${NAME}.json" ] && /bin/rm "nodes/${NAME}.json" +} + # # Main # @@ -163,21 +179,9 @@ case "$CI_JOB_NAME" in run bitmask ssh://gitolite@leap.se/bitmask ;; *) - # create node(s) with unique id so we can run tests in parallel - NAME="citest${CI_BUILD_ID:-0}" - # when using gitlab-runner locally, CI_BUILD_ID is always 1 which - # will conflict with running/terminating AWS instances in subsequent runs - # therefore we pick a random number in this case - [ "${CI_BUILD_ID:-0}" -eq "1" ] && NAME+="000${RANDOM}" - - TAG='single' - SERVICES='couchdb,soledad,mx,webapp,tor,monitor' build_from_scratch - # Deploy and test deploy test - # if everything succeeds, destroy the vm - LEAP_CMD vm rm "${TAG}" - [ -f "nodes/${NAME}.json" ] && /bin/rm "nodes/${NAME}.json" + cleanup ;; esac -- cgit v1.2.3 From 1bcd200ec20a28a1c360afdf097dc4fcaa9c00a8 Mon Sep 17 00:00:00 2001 From: Varac Date: Thu, 29 Jun 2017 16:51:58 +0200 Subject: [CI] Fail when CI_JOB_NAME is not recognized --- tests/platform-ci/ci-build.sh | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/tests/platform-ci/ci-build.sh b/tests/platform-ci/ci-build.sh index abc5fec8..b02fe89d 100755 --- a/tests/platform-ci/ci-build.sh +++ b/tests/platform-ci/ci-build.sh @@ -178,10 +178,13 @@ case "$CI_JOB_NAME" in TAG='demovpn' run bitmask ssh://gitolite@leap.se/bitmask ;; - *) + deploy_test*) build_from_scratch deploy test cleanup ;; + *) + fail "Don't know what to do for \$CI_JOB_NAME \"$CI_JOB_NAME\"!" + ;; esac -- cgit v1.2.3 From 5816661ab20f2b2641bc3c19dc495b28e531213e Mon Sep 17 00:00:00 2001 From: Varac Date: Thu, 29 Jun 2017 17:14:45 +0200 Subject: [CI] Add upgrade tests to ci-build.sh --- tests/platform-ci/ci-build.sh | 26 ++++++++++++++++++++++++++ 1 file changed, 26 insertions(+) diff --git a/tests/platform-ci/ci-build.sh b/tests/platform-ci/ci-build.sh index b02fe89d..256164ac 100755 --- a/tests/platform-ci/ci-build.sh +++ b/tests/platform-ci/ci-build.sh @@ -140,6 +140,29 @@ run() { test } +upgrade_test() { + # Checkout stable branch containing last release + # and deploy this + cd "$PLATFORMDIR" + git remote add leap https://leap.se/git/leap_platform + git fetch leap + git checkout -b leap_stable remotes/leap/stable + cd "$PROVIDERDIR" + build_from_scratch + deploy + test + + # Checkout HEAD of current branch and re-deploy + cd "$PLATFORMDIR" + git checkout "$CI_COMMIT_REF" + cd "$PROVIDERDIR" + deploy + test + + cleanup + +} + cleanup() { # if everything succeeds, destroy the vm LEAP_CMD vm rm "${TAG}" @@ -184,6 +207,9 @@ case "$CI_JOB_NAME" in test cleanup ;; + upgrade_test) + upgrade_test + ;; *) fail "Don't know what to do for \$CI_JOB_NAME \"$CI_JOB_NAME\"!" ;; -- cgit v1.2.3 From c232b863172a4f5b511e64f02882e5a4804c337e Mon Sep 17 00:00:00 2001 From: Varac Date: Thu, 29 Jun 2017 17:37:02 +0200 Subject: [CI] Run bundle install after checking out different platform branch --- tests/platform-ci/ci-build.sh | 11 +++++++++++ 1 file changed, 11 insertions(+) diff --git a/tests/platform-ci/ci-build.sh b/tests/platform-ci/ci-build.sh index 256164ac..88856511 100755 --- a/tests/platform-ci/ci-build.sh +++ b/tests/platform-ci/ci-build.sh @@ -147,7 +147,13 @@ upgrade_test() { git remote add leap https://leap.se/git/leap_platform git fetch leap git checkout -b leap_stable remotes/leap/stable + # After checking out a different platform branch + # bundle install is needed again + cd "$ROOTDIR" + /usr/local/bin/bundle install + cd "$PROVIDERDIR" + build_from_scratch deploy test @@ -155,6 +161,11 @@ upgrade_test() { # Checkout HEAD of current branch and re-deploy cd "$PLATFORMDIR" git checkout "$CI_COMMIT_REF" + # After checking out a different platform branch + # bundle install is needed again + cd "$ROOTDIR" + /usr/local/bin/bundle install + cd "$PROVIDERDIR" deploy test -- cgit v1.2.3 From 0d304e582d643893f5e139eb5126c793bc82ae6d Mon Sep 17 00:00:00 2001 From: Varac Date: Thu, 29 Jun 2017 20:05:17 +0200 Subject: [CI] Fix node name for local tests --- tests/platform-ci/ci-build.sh | 10 ++++++---- 1 file changed, 6 insertions(+), 4 deletions(-) diff --git a/tests/platform-ci/ci-build.sh b/tests/platform-ci/ci-build.sh index 88856511..9332c12c 100755 --- a/tests/platform-ci/ci-build.sh +++ b/tests/platform-ci/ci-build.sh @@ -68,12 +68,14 @@ test() { } build_from_scratch() { - # create node(s) with unique id so we can run tests in parallel - NAME="citest${CI_BUILD_ID:-0}" - # when using gitlab-runner locally, CI_BUILD_ID is always 1 which + # when using gitlab-runner locally, CI_JOB_ID is always 1 which # will conflict with running/terminating AWS instances in subsequent runs # therefore we pick a random number in this case - [ "${CI_BUILD_ID:-0}" -eq "1" ] && NAME+="000${RANDOM}" + [ "${CI_JOB_ID}" == "1" ] && CI_JOB_ID="000${RANDOM}" + + # create node(s) with unique id so we can run tests in parallel + NAME="citest${CI_JOB_ID:-0}" + TAG='single' SERVICES='couchdb,soledad,mx,webapp,tor,monitor' -- cgit v1.2.3 From f365b914662491ab33e6af18e1b02046f6b99538 Mon Sep 17 00:00:00 2001 From: elijah Date: Wed, 28 Jun 2017 13:24:39 -0700 Subject: leap_cli - make fog gem optional --- lib/leap_cli/cloud.rb | 3 +-- lib/leap_cli/cloud/dependencies.rb | 47 ++++++++++++++++++-------------------- lib/leap_cli/commands/vm.rb | 5 +++- tests/platform-ci/Gemfile | 1 + 4 files changed, 28 insertions(+), 28 deletions(-) diff --git a/lib/leap_cli/cloud.rb b/lib/leap_cli/cloud.rb index 268cea38..b8e45b3b 100644 --- a/lib/leap_cli/cloud.rb +++ b/lib/leap_cli/cloud.rb @@ -1,4 +1,3 @@ - -require 'fog/aws' +require_relative 'cloud/dependencies.rb' require_relative 'cloud/cloud.rb' require_relative 'cloud/image.rb' diff --git a/lib/leap_cli/cloud/dependencies.rb b/lib/leap_cli/cloud/dependencies.rb index fd690e59..670d6134 100644 --- a/lib/leap_cli/cloud/dependencies.rb +++ b/lib/leap_cli/cloud/dependencies.rb @@ -1,40 +1,37 @@ # -# I am not sure this is a good idea, but it might be. Tricky, so disabled for now +# Ensure that the needed fog gems are installed # - -=begin module LeapCli class Cloud - def self.check_required_gems - begin - require "fog" - rescue LoadError - bail! do - log :error, "The 'vm' command requires the gem 'fog-core'. Please run `gem install fog-core` and try again." - end - end + SUPPORTED = { + 'aws' => {require: 'fog/aws', gem: 'fog-aws'} + }.freeze - fog_gems = @cloud.required_gems - if !options[:mock] && fog_gems.empty? - bail! do - log :warning, "no vm providers are configured in cloud.json." - log "You must have credentials for one of: #{@cloud.possible_apis.join(', ')}." + def self.check_dependencies!(config) + required_gem = map_api_to_gem(config['api']) + if required_gem.nil? + Util.bail! do + Util.log :error, "The API '#{config['api']}' specified in cloud.json is not one that I know how to speak. Try one of #{supported_list}." end end - fog_gems.each do |name, gem_name| - begin - require gem_name.sub('-','/') - rescue LoadError - bail! do - log :error, "The 'vm' command requires the gem '#{gem_name}' (because of what is configured in cloud.json)." - log "Please run `sudo gem install #{gem_name}` and try again." - end + begin + require required_gem[:require] + rescue LoadError + Util.bail! do + Util.log :error, "The 'vm' command requires the gem '#{required_gem[:gem]}'. Please run `gem install #{required_gem[:gem]}` and try again." + Util.log "(make sure you install the gem in the ruby #{RUBY_VERSION} environment)" end end end + def self.supported_list + SUPPORTED.keys.join(', ') + end + + def self.map_api_to_gem(api) + SUPPORTED[api] + end end end -=end \ No newline at end of file diff --git a/lib/leap_cli/commands/vm.rb b/lib/leap_cli/commands/vm.rb index 790774f1..6f97dbce 100644 --- a/lib/leap_cli/commands/vm.rb +++ b/lib/leap_cli/commands/vm.rb @@ -415,7 +415,6 @@ module LeapCli; module Commands config = manager.env.cloud name = nil if options[:mock] - Fog.mock! name = 'mock_aws' config['mock_aws'] = { "api" => "aws", @@ -451,6 +450,10 @@ module LeapCli; module Commands assert! entry['api'] == 'aws', "cloud.json: currently, only 'aws' is supported for `api`." assert! entry['vendor'] == 'aws', "cloud.json: currently, only 'aws' is supported for `vendor`." + LeapCli::Cloud::check_dependencies!(entry) + if options[:mock] + Fog.mock! + end return LeapCli::Cloud.new(name, entry, node) end diff --git a/tests/platform-ci/Gemfile b/tests/platform-ci/Gemfile index 44a8637d..4cf14e43 100644 --- a/tests/platform-ci/Gemfile +++ b/tests/platform-ci/Gemfile @@ -16,4 +16,5 @@ group :test do # Install fog-aws because the gem dependency of leap_cli is now optional gem "fog-aws" gem "leap_cli", :git => 'https://leap.se/git/leap_cli.git' + gem "fog-aws" end -- cgit v1.2.3 From 9adaa316050e3eed971fe316eb545ba74c562cca Mon Sep 17 00:00:00 2001 From: Micah Anderson Date: Tue, 4 Jul 2017 12:00:20 -0700 Subject: Fix upgrade_test failure. See https://0xacab.org/leap/platform/-/jobs/14029 for an example. --- tests/platform-ci/ci-build.sh | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/tests/platform-ci/ci-build.sh b/tests/platform-ci/ci-build.sh index 9332c12c..1c9cc416 100755 --- a/tests/platform-ci/ci-build.sh +++ b/tests/platform-ci/ci-build.sh @@ -146,7 +146,8 @@ upgrade_test() { # Checkout stable branch containing last release # and deploy this cd "$PLATFORMDIR" - git remote add leap https://leap.se/git/leap_platform + # due to cache, this remote is sometimes already added + git remote add leap https://leap.se/git/leap_platform || true git fetch leap git checkout -b leap_stable remotes/leap/stable # After checking out a different platform branch -- cgit v1.2.3 From b9f562813b5005577bed3f2a40c5eb147696e18c Mon Sep 17 00:00:00 2001 From: Micah Anderson Date: Tue, 4 Jul 2017 13:54:43 -0700 Subject: Additional fix for CI cache failure --- tests/platform-ci/ci-build.sh | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tests/platform-ci/ci-build.sh b/tests/platform-ci/ci-build.sh index 1c9cc416..5c21bfb7 100755 --- a/tests/platform-ci/ci-build.sh +++ b/tests/platform-ci/ci-build.sh @@ -149,7 +149,7 @@ upgrade_test() { # due to cache, this remote is sometimes already added git remote add leap https://leap.se/git/leap_platform || true git fetch leap - git checkout -b leap_stable remotes/leap/stable + git checkout -b leap_stable remotes/leap/stable || true # After checking out a different platform branch # bundle install is needed again cd "$ROOTDIR" -- cgit v1.2.3 From ff3878a70235206d182116c74c4ac7b3cc1a478f Mon Sep 17 00:00:00 2001 From: Micah Anderson Date: Tue, 4 Jul 2017 14:31:38 -0700 Subject: Fix non-existent CI variable CI_COMMIT_REF (#8844) --- tests/platform-ci/ci-build.sh | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/tests/platform-ci/ci-build.sh b/tests/platform-ci/ci-build.sh index 5c21bfb7..1445f562 100755 --- a/tests/platform-ci/ci-build.sh +++ b/tests/platform-ci/ci-build.sh @@ -149,7 +149,10 @@ upgrade_test() { # due to cache, this remote is sometimes already added git remote add leap https://leap.se/git/leap_platform || true git fetch leap + echo "Checking out leap/stable" git checkout -b leap_stable remotes/leap/stable || true + echo -n "Current version: " + git rev-parse HEAD # After checking out a different platform branch # bundle install is needed again cd "$ROOTDIR" @@ -163,7 +166,10 @@ upgrade_test() { # Checkout HEAD of current branch and re-deploy cd "$PLATFORMDIR" - git checkout "$CI_COMMIT_REF" + echo "Checking out: $CI_COMMIT_SHA" + git checkout "$CI_COMMIT_SHA" + echo -n "Current version: " + git rev-parse HEAD # After checking out a different platform branch # bundle install is needed again cd "$ROOTDIR" -- cgit v1.2.3 From 9365cd2c8cdf70ea24b382cd547fb94f8d403b2f Mon Sep 17 00:00:00 2001 From: kwadronaut Date: Tue, 11 Jul 2017 21:29:55 +0200 Subject: Style: updates a git commit template A nice readable git history is always appreciated. By nudging people in using our template, we hope to achieve that. Through the inclusion of a CONTRIBUTING.md we hope to make that easier. - Fixes #8845 --- CONTRIBUTING.md | 13 +++++++++++++ contrib/README.md | 9 --------- contrib/commit-template.txt | 7 ------- contrib/leap-commit-template | 20 ++++++++++++++++++++ 4 files changed, 33 insertions(+), 16 deletions(-) create mode 100644 CONTRIBUTING.md delete mode 100644 contrib/README.md delete mode 100644 contrib/commit-template.txt create mode 100644 contrib/leap-commit-template diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md new file mode 100644 index 00000000..6ae7f6ca --- /dev/null +++ b/CONTRIBUTING.md @@ -0,0 +1,13 @@ +# Contributed Files + +## Commit Template + +To install our commit template, use following command (use --global to use it in your global .gitconfig): + + git config [--global] commit.template "~/path_to_leap_platform/contrib/leap-commit-template" + +## Signing commits + +We very much appreciate signed commits, you can stop forgetting it like this: + + git config [--global] commit.gpgSign diff --git a/contrib/README.md b/contrib/README.md deleted file mode 100644 index e836bc7e..00000000 --- a/contrib/README.md +++ /dev/null @@ -1,9 +0,0 @@ -# Contributed Files - -## Commit Template - -to install this commit template, use following cmd (use --global to use it in your global .gitconfig): - - git config [--global] commit.template "~/path_to_leap_platform/contrib/commit-template.txt" - - diff --git a/contrib/commit-template.txt b/contrib/commit-template.txt deleted file mode 100644 index 9a1fa81b..00000000 --- a/contrib/commit-template.txt +++ /dev/null @@ -1,7 +0,0 @@ -#[bug|feat|docs|style|refactor|test|pkg|i18n] - -#- Tested: [local singlenode|local multinode|citest|unstable.bitmask.net] -#- Resolves: #XYZ -#- Related: #XYZ -#- Documentation: #XYZ -#- Releases: XYZ diff --git a/contrib/leap-commit-template b/contrib/leap-commit-template new file mode 100644 index 00000000..afc56b2b --- /dev/null +++ b/contrib/leap-commit-template @@ -0,0 +1,20 @@ +# Bug|Feat|CI|Docs|Style|Tests|I18n|Vagrant|Lint: [style: https://github.com/m1foley/fit-commit] + + +# [Story about your commit.] + +# Use one or more of these things: +# - Fixes|Closes #3210 +# - Related: #1234 + +#example: +#     +#   Bug: fix apache systemd insanity +#     +#   Apache2 systemd autorestart was failing because of #2314432 +#   and it was caused by a gremlin that was entering through a wormhole +#   caused by #985843, so this commit resolves this wormhole +#     +#    # Use one of these things: +#    [- Fixes|Closes #5234] +#    [- Related: #1234] -- cgit v1.2.3 From a38a421a17b199e2207bc0009cba6869d17d4c21 Mon Sep 17 00:00:00 2001 From: Micah Anderson Date: Tue, 4 Jul 2017 17:56:25 -0700 Subject: Ensure directory has proper owner/group (#8841) --- puppet/modules/site_apache/manifests/common/autorestart.pp | 2 ++ 1 file changed, 2 insertions(+) diff --git a/puppet/modules/site_apache/manifests/common/autorestart.pp b/puppet/modules/site_apache/manifests/common/autorestart.pp index 0273f272..6d8c4c3a 100644 --- a/puppet/modules/site_apache/manifests/common/autorestart.pp +++ b/puppet/modules/site_apache/manifests/common/autorestart.pp @@ -5,6 +5,8 @@ class site_apache::common::autorestart { file { '/etc/systemd/system/apache2.service.d': ensure => directory, + owner => root, + group => root, mode => '0755', } -- cgit v1.2.3 From f9ff6afebde2fe037fc9a90928501ff1a8d4ccde Mon Sep 17 00:00:00 2001 From: Micah Anderson Date: Wed, 12 Jul 2017 13:29:06 -0700 Subject: bug: Set .placeholder to fix removal Add a .placeholder file so the directory doesn't get removed by deb-systemd-helper when a package runs a purge in its postrm. This is a work-around and fixes #8841. It probably wont be needed post-jessie. --- .../site_apache/manifests/common/autorestart.pp | 20 +++++++++++++++----- 1 file changed, 15 insertions(+), 5 deletions(-) diff --git a/puppet/modules/site_apache/manifests/common/autorestart.pp b/puppet/modules/site_apache/manifests/common/autorestart.pp index 6d8c4c3a..8b7b590d 100644 --- a/puppet/modules/site_apache/manifests/common/autorestart.pp +++ b/puppet/modules/site_apache/manifests/common/autorestart.pp @@ -3,11 +3,21 @@ # class site_apache::common::autorestart { - file { '/etc/systemd/system/apache2.service.d': - ensure => directory, - owner => root, - group => root, - mode => '0755', + file { + '/etc/systemd/system/apache2.service.d': + ensure => directory, + owner => 'root', + group => 'root', + mode => '0755'; + + # Add .placeholder file so directory doesn't get removed by + # deb-systemd-helper in a package removal postrm, see + # issue #8841 for more details. + '/etc/systemd/system/apache2.service.d/.placeholder': + ensure => file, + owner => 'root', + group => 'root', + mode => '0755'; } ::systemd::unit_file { 'apache2.service.d/autorestart.conf': -- cgit v1.2.3 From d3c88e6c1dfca8c5e5804a1b543d50b9f45c05a8 Mon Sep 17 00:00:00 2001 From: Varac Date: Fri, 23 Jun 2017 16:33:18 +0200 Subject: Use unqualified path to bundler So users can run it locally when they have `bundler` installed as debian package. --- tests/platform-ci/setup.sh | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/tests/platform-ci/setup.sh b/tests/platform-ci/setup.sh index e92dddc7..c10cb4c8 100755 --- a/tests/platform-ci/setup.sh +++ b/tests/platform-ci/setup.sh @@ -1,5 +1,5 @@ #!/bin/sh which bundle || /usr/bin/apt install bundle -/usr/local/bin/bundle install --binstubs --path=vendor --with=test --jobs "$(nproc)" -/usr/local/bin/bundle exec leap -v2 --yes help +bundle install --binstubs --path=vendor --with=test --jobs "$(nproc)" +bundle exec leap -v2 --yes help -- cgit v1.2.3 From a34d1b14a9a0874bfd6a0c84fa6ac64f2bc3f55a Mon Sep 17 00:00:00 2001 From: Varac Date: Fri, 23 Jun 2017 19:35:52 +0200 Subject: Update tests/platform-ci/README.md how to run platform tests --- tests/platform-ci/README.md | 48 ++++++++++++++++++++++++++++++++++++++++----- 1 file changed, 43 insertions(+), 5 deletions(-) diff --git a/tests/platform-ci/README.md b/tests/platform-ci/README.md index 60c17e41..7a44a2fe 100644 --- a/tests/platform-ci/README.md +++ b/tests/platform-ci/README.md @@ -1,15 +1,53 @@ -Continuous integration tests for the leap_platform code. +# Continuous integration tests for the leap_platform code -Usage: +# Setup + cd tests/platform-ci ./setup.sh + +# Run syntax checks and test if catalog compiles + bin/rake test:syntax - bin/rake test:catalog + bin/rake catalog For a list of all tasks: bin/rake -T -To create a virtual provider, run tests on it, then tear it down: +# Full integration test + +You can create a virtual provider using AWS, run tests on it, then tear it down +when the tests succeed. +In order to do so, you need to set your AWS credentials as environment variables: + + export AWS_ACCESS_KEY='...' + export AWS_SECRET_KEY='...' + +If you want to login to this machine during or after the deploy you need to + + export SSH_PRIVATE_KEY=$(cat ~/.ssh/id_rsa) + +then start the deply test with + + ./ci-build.sh + +# Running tests with docker and gitlab-runner + +Another possibility to run the platform tests is to use [gitlab-runner](https://docs.gitlab.com/runner/) +together with [Docker](https://www.docker.com/). + +Export `AWS_ACCESS_KEY`, `AWS_SECRET_KEY` and `SSH_PRIVATE_KEY` as shown above. +From the root dir of this repo run: + + gitlab-runner exec docker --env AWS_ACCESS_KEY="$AWS_ACCESS_KEY" --env AWS_SECRET_KEY="$AWS_SECRET_KEY" --env SSH_PRIVATE_KEY="$SSH_PRIVATE_KEY" deploy_test + +See `.gitlab-ci.yml` for all the different test jobs. + +To ssh into the VM you first need to enter the docker container: + + docker exec -u cirunner -it $(docker ps --latest -q) bash + +From there you can access the test provider config directory and ssh into the VM: - ./ci-build.sh + cd /builds/project-0/tests/platform-ci/provider/ + leap ssh citest0 -- cgit v1.2.3 From c0ddb0da43910e9a064e08acf424b2f2a0ccdd88 Mon Sep 17 00:00:00 2001 From: elijah Date: Fri, 30 Jun 2017 00:24:38 -0700 Subject: by default, new providers will now require invites. requires leap_cli 4173154a177b00c11a36b3168b1ce12af59f04af or later (>1.9.2). resolves #8474. create new invites with `leap run invite` --- lib/leap_cli/commands/run.rb | 53 +++++++++++++++++++++++++++++++++-- lib/leap_cli/config/object_list.rb | 4 +++ provider_base/templates/provider.json | 19 +++++++++++++ 3 files changed, 73 insertions(+), 3 deletions(-) create mode 100644 provider_base/templates/provider.json diff --git a/lib/leap_cli/commands/run.rb b/lib/leap_cli/commands/run.rb index cad9b7a0..9149d594 100644 --- a/lib/leap_cli/commands/run.rb +++ b/lib/leap_cli/commands/run.rb @@ -3,13 +3,27 @@ module LeapCli; module Commands desc 'Run a shell command remotely' long_desc "Runs the specified command COMMAND on each node in the FILTER set. " + "For example, `leap run 'uname -a' webapp`" - arg_name 'COMMAND FILTER' command :run do |c| c.switch 'stream', :default => false, :desc => 'If set, stream the output as it arrives. (default: --stream for a single node, --no-stream for multiple nodes)' c.flag 'port', :arg_name => 'SSH_PORT', :desc => 'Override default SSH port used when trying to connect to the server.' - c.action do |global, options, args| - run_shell_command(global, options, args) + + c.desc 'Run an arbitrary shell command.' + c.arg_name 'FILTER', optional: true + c.command :command do |command| + command.action do |global, options, args| + run_shell_command(global, options, args) + end + end + + c.desc 'Generate one or more new invite codes.' + c.arg_name '[COUNT] [ENVIRONMENT]' + c.command :invite do |invite| + invite.action do |global_options,options,args| + run_new_invites(global_options, options, args) + end end + + c.default_command :command end private @@ -27,6 +41,39 @@ module LeapCli; module Commands end end + CMD_NEW_INVITES="cd /srv/leap/webapp; RAILS_ENV=production bundle exec rake \"generate_invites[NUM,USES]\"" + + def run_new_invites(global, options, args) + require 'leap_cli/ssh' + count = 1 + uses = 1 + env = nil + arg1 = args.shift + arg2 = args.shift + if arg1 && arg2 + env = manager.env(arg2) + count = arg1 + elsif arg1 + env = manager.env(arg1) + else + env = manager.env(nil) + end + unless env + bail! "Environment name you specified does not match one that is available. See `leap env ls` for the available names" + end + + env_name = env.name == 'default' ? nil : env.name + webapp_nodes = env.nodes[:environment => env_name][:services => 'webapp'].first + if webapp_nodes.empty? + bail! "Could not find a webapp node for the specified environment" + end + stream_command( + webapp_nodes, + CMD_NEW_INVITES.sub('NUM', count.to_s).sub('USES', uses.to_s), + options + ) + end + def capture_command(nodes, cmd, options) SSH.remote_command(nodes, options) do |ssh, host| output = ssh.capture(cmd, :log_output => false) diff --git a/lib/leap_cli/config/object_list.rb b/lib/leap_cli/config/object_list.rb index 80f89d92..815864e4 100644 --- a/lib/leap_cli/config/object_list.rb +++ b/lib/leap_cli/config/object_list.rb @@ -49,6 +49,10 @@ module LeapCli end end + def first + ObjectList.new(self.values.first) + end + def exclude(node) list = self.dup list.delete(node.name) diff --git a/provider_base/templates/provider.json b/provider_base/templates/provider.json new file mode 100644 index 00000000..297327d1 --- /dev/null +++ b/provider_base/templates/provider.json @@ -0,0 +1,19 @@ +// +// This file defines global aspects of your service provider +// See https://leap.se/provider-configuration +// +{ + "domain": "<%= domain %>", + "name": { + "en": "<%= name %>" + }, + "description": { + "en": "You really should change this text" + }, + "contacts": { + "default": "<%= contacts %>" + }, + "languages": ["en"], + "default_language": "en", + "enrollment_policy": "invite" +} -- cgit v1.2.3 From 7dedd106737effd6c1ba849983e01c0b2866e7f2 Mon Sep 17 00:00:00 2001 From: Micah Anderson Date: Tue, 25 Jul 2017 16:01:37 -0700 Subject: Style: remove tabs, fix language to fit template The example had many tabs causing my eyes to bleed. It also didn't use the template properly to fix an issue. --- contrib/leap-commit-template | 20 ++++++++++---------- 1 file changed, 10 insertions(+), 10 deletions(-) diff --git a/contrib/leap-commit-template b/contrib/leap-commit-template index afc56b2b..73cc6436 100644 --- a/contrib/leap-commit-template +++ b/contrib/leap-commit-template @@ -8,13 +8,13 @@ # - Related: #1234 #example: -#     -#   Bug: fix apache systemd insanity -#     -#   Apache2 systemd autorestart was failing because of #2314432 -#   and it was caused by a gremlin that was entering through a wormhole -#   caused by #985843, so this commit resolves this wormhole -#     -#    # Use one of these things: -#    [- Fixes|Closes #5234] -#    [- Related: #1234] +# +# Bug: fix apache systemd insanity +# +# Apache2 systemd autorestart was failing because of a gremlin that was entering +# through a wormhole. This fixes #985843 by closing the wormhole so no further +# gremlins can come through. +# +# # Use one of these things: +# [- Fixes|Closes #5234] +# [- Related: #1234] -- cgit v1.2.3 From 55b784f2ccd6336db4bab9157a8498cb87c562ff Mon Sep 17 00:00:00 2001 From: Micah Anderson Date: Tue, 25 Jul 2017 15:49:46 -0700 Subject: CI: Cleanup and enhance to specify platform branch Fix indentation; setup some more clear variables. Add a third variable to ensure the proper platform branch is checked out. This is necessary because otherwise environment deploys get stuck because the platform directory is in a detached state and then the environment deploys will not proceed because the branch check fails. This will fix #8843. --- tests/platform-ci/ci-build.sh | 42 +++++++++++++++++++++++++++++------------- 1 file changed, 29 insertions(+), 13 deletions(-) diff --git a/tests/platform-ci/ci-build.sh b/tests/platform-ci/ci-build.sh index 1445f562..57b874f9 100755 --- a/tests/platform-ci/ci-build.sh +++ b/tests/platform-ci/ci-build.sh @@ -128,18 +128,34 @@ build_from_scratch() { } run() { - echo "Cloning $1 repo: $2" - git clone -q --depth 1 "$2" - cd "$1" - git rev-parse HEAD - echo -n "Operating in the $1 directory: " - pwd - echo "Listing current node information..." - LEAP_CMD list - echo "Attempting a deploy..." - deploy - echo "Attempting to run tests..." - test + provider_name=$1 + provider_URI=$2 + platform_branch=$3 + + # If the third argument is set make sure we are on that platform branch + if [[ -n $platform_branch ]] + then + echo "Checking out $platform_branch branch of platform" + cd "$PLATFORMDIR" + git checkout -B "$platform_branch" + fi + + # Setup the provider repository + echo "Setting up the provider repository: $provider_name by cloning $provider_URI" + git clone -q --depth 1 "$provider_URI" "$ROOTDIR" + cd "$provider_name" + echo -n "$provider_name repo at revision: " + git rev-parse HEAD + echo -n "Operating in the $provider_name directory: " + pwd + echo "Listing current node information..." + LEAP_CMD list + + # Do the deployment + echo "Attempting a deploy..." + deploy + echo "Attempting to run tests..." + test } upgrade_test() { @@ -215,7 +231,7 @@ case "$CI_JOB_NAME" in ;; mail.bitmask.net) TAG='demomail' - run bitmask ssh://gitolite@leap.se/bitmask + run bitmask ssh://gitolite@leap.se/bitmask master ;; demo.bitmask.net) TAG='demovpn' -- cgit v1.2.3 From 33b56edf683b82acb3f3d077c1b2e907a1dc02dd Mon Sep 17 00:00:00 2001 From: Micah Anderson Date: Thu, 27 Jul 2017 14:18:46 -0700 Subject: CI: fix provider checkout Provider checkout was being done to a pre-existing directory, which resulted in an error about the directory already existing (see https://0xacab.org/leap/platform/-/jobs/15730), this should fix that problem. --- tests/platform-ci/ci-build.sh | 19 +++++++++++-------- 1 file changed, 11 insertions(+), 8 deletions(-) diff --git a/tests/platform-ci/ci-build.sh b/tests/platform-ci/ci-build.sh index 57b874f9..459264d5 100755 --- a/tests/platform-ci/ci-build.sh +++ b/tests/platform-ci/ci-build.sh @@ -132,6 +132,16 @@ run() { provider_URI=$2 platform_branch=$3 + # Setup the provider repository + echo "Setting up the provider repository: $provider_name by cloning $provider_URI" + git clone -q --depth 1 "$provider_URI" + cd "$provider_name" + echo -n "$provider_name repo at revision: " + git rev-parse HEAD + echo -n "Operating in the $provider_name directory: " + pwd + + # If the third argument is set make sure we are on that platform branch if [[ -n $platform_branch ]] then @@ -140,14 +150,7 @@ run() { git checkout -B "$platform_branch" fi - # Setup the provider repository - echo "Setting up the provider repository: $provider_name by cloning $provider_URI" - git clone -q --depth 1 "$provider_URI" "$ROOTDIR" - cd "$provider_name" - echo -n "$provider_name repo at revision: " - git rev-parse HEAD - echo -n "Operating in the $provider_name directory: " - pwd + cd "${ROOTDIR}/${provider_name}" echo "Listing current node information..." LEAP_CMD list -- cgit v1.2.3 From c7657bcba40c936a9b1676374c1489c2f7c25907 Mon Sep 17 00:00:00 2001 From: Micah Anderson Date: Thu, 27 Jul 2017 15:47:25 -0700 Subject: Bug: allow old client to connect to VPN The old client is compatible, just the version check did not allow it. People are still relying on the old client for a while, and this prevents people from upgrading. This fixes #8850. --- provider_base/provider.json | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/provider_base/provider.json b/provider_base/provider.json index 521c682f..a9980bf8 100644 --- a/provider_base/provider.json +++ b/provider_base/provider.json @@ -58,7 +58,7 @@ } }, "client_version": { - "min": "0.9.4", + "min": "0.9.2", "max": null } } -- cgit v1.2.3 From 61d8d9e0e35dc9759ec93b517b0a67df1c3506d3 Mon Sep 17 00:00:00 2001 From: elijah Date: Fri, 18 Aug 2017 16:05:19 -0700 Subject: Bug: allow `leap test --continue` to run on additional nodes if there was an ssh error. closes #8811 --- lib/leap_cli/commands/test.rb | 2 +- lib/leap_cli/ssh/backend.rb | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/lib/leap_cli/commands/test.rb b/lib/leap_cli/commands/test.rb index 70eb00fd..e2815aae 100644 --- a/lib/leap_cli/commands/test.rb +++ b/lib/leap_cli/commands/test.rb @@ -35,7 +35,7 @@ module LeapCli; module Commands SSH::remote_command(node, options) do |ssh, host| ssh.stream(test_cmd(options), :raise_error => true, :log_wrap => true) end - rescue LeapCli::SSH::ExecuteError + rescue LeapCli::SSH::TimeoutError, SSHKit::Runner::ExecuteError, SSHKit::Command::Failed if options[:continue] exit_status(1) else diff --git a/lib/leap_cli/ssh/backend.rb b/lib/leap_cli/ssh/backend.rb index 3894d815..599fc9a0 100644 --- a/lib/leap_cli/ssh/backend.rb +++ b/lib/leap_cli/ssh/backend.rb @@ -178,7 +178,7 @@ module LeapCli rescue StandardError => exc if exc.is_a?(SSHKit::Command::Failed) || exc.is_a?(SSHKit::Runner::ExecuteError) if @options[:raise_error] - raise LeapCli::SSH::ExecuteError, exc.to_s + raise exc elsif @options[:fail_msg] @logger.log(@options[:fail_msg], host: @host.hostname, :color => :red) else -- cgit v1.2.3 From 9679c7e1cd5d7b5824fa99b070dc0899779c92ec Mon Sep 17 00:00:00 2001 From: elijah Date: Sat, 19 Aug 2017 13:42:39 -0700 Subject: leap_cli: minor help wording correction --- lib/leap_cli/commands/compile.rb | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/lib/leap_cli/commands/compile.rb b/lib/leap_cli/commands/compile.rb index 92c879d7..16dff3df 100644 --- a/lib/leap_cli/commands/compile.rb +++ b/lib/leap_cli/commands/compile.rb @@ -155,7 +155,7 @@ module LeapCli buffer = StringIO.new keys = Dir.glob(path([:user_ssh, '*'])) if keys.empty? - bail! "You must have at least one public SSH user key configured in order to proceed. See `leap help add-user`." + bail! "You must have at least one public SSH user key configured in order to proceed. See `leap help user add`." end if file_exists?(path(:monitor_pub_key)) keys << path(:monitor_pub_key) -- cgit v1.2.3 From 804e022221bfb0b5200282e556d75e601271dac5 Mon Sep 17 00:00:00 2001 From: elijah Date: Tue, 15 Aug 2017 17:35:55 -0700 Subject: Bug: fix hidden service for static hidden service should be activated iff tor is among the active services and tor.hidden_service.active == true --- puppet/modules/site_static/manifests/init.pp | 13 ++++++++++--- puppet/modules/site_static/templates/apache.conf.erb | 2 +- 2 files changed, 11 insertions(+), 4 deletions(-) diff --git a/puppet/modules/site_static/manifests/init.pp b/puppet/modules/site_static/manifests/init.pp index 8be791e5..96d92f74 100644 --- a/puppet/modules/site_static/manifests/init.pp +++ b/puppet/modules/site_static/manifests/init.pp @@ -12,6 +12,11 @@ class site_static { $formats = $static['formats'] $bootstrap = $static['bootstrap_files'] $tor = hiera('tor', false) + if $tor and member($services, 'tor') and $tor['hidden_service']['active'] == true { + $tor_active = true + } else { + $tor_active = false + } file { '/srv/static/': @@ -67,15 +72,17 @@ class site_static { } package { 'zlib1g-dev': - ensure => installed + ensure => installed } } - if $tor { + if $tor_active { $hidden_service = $tor['hidden_service'] $tor_domain = "${hidden_service['address']}.onion" - class { 'site_static::hidden_service': single_hop => $hidden_service['single_hop'] + class { 'site_static::hidden_service': + single_hop => $hidden_service['single_hop'] } + # Currently, we only support a single hidden service address per server. # So if there is more than one domain configured, then we need to make sure # we don't enable the hidden service for every domain. diff --git a/puppet/modules/site_static/templates/apache.conf.erb b/puppet/modules/site_static/templates/apache.conf.erb index eb21e4c9..75d834e7 100644 --- a/puppet/modules/site_static/templates/apache.conf.erb +++ b/puppet/modules/site_static/templates/apache.conf.erb @@ -74,7 +74,7 @@ Require all granted -<%- if @tor && (@always_use_hidden_service || @use_hidden_service) -%> +<%- if @tor_active && (@always_use_hidden_service || @use_hidden_service) -%> ## ## Tor ## -- cgit v1.2.3 From 437f28b2cbfedfc7d119dcf4e228c5626bb8a152 Mon Sep 17 00:00:00 2001 From: elijah Date: Sun, 27 Aug 2017 23:51:14 -0700 Subject: bugfix: fix `leap test init` --- lib/leap_cli/commands/cert.rb | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/lib/leap_cli/commands/cert.rb b/lib/leap_cli/commands/cert.rb index 81f45eb5..68fa9444 100644 --- a/lib/leap_cli/commands/cert.rb +++ b/lib/leap_cli/commands/cert.rb @@ -102,13 +102,13 @@ module LeapCli; module Commands def generate_test_client_cert(prefix=nil) require 'leap_cli/x509' cert = CertificateAuthority::Certificate.new - cert.serial_number.number = cert_serial_number(provider.domain) - cert.subject.common_name = [prefix, random_common_name(provider.domain)].join + cert.serial_number.number = X509.cert_serial_number(provider.domain) + cert.subject.common_name = [prefix, X509.random_common_name(provider.domain)].join cert.not_before = X509.yesterday cert.not_after = X509.yesterday.advance(:years => 1) cert.key_material.generate_key(1024) # just for testing, remember! - cert.parent = client_ca_root - cert.sign! client_test_signing_profile + cert.parent = X509.client_ca_root + cert.sign! X509.client_test_signing_profile yield cert.key_material.private_key.to_pem, cert.to_pem end -- cgit v1.2.3 From 6482a4ccb3d72773cc6d00d5fa7933fa83c4cafe Mon Sep 17 00:00:00 2001 From: elijah Date: Tue, 5 Sep 2017 18:24:31 -0700 Subject: Bug: fix vpn network problem caused by vagrant fact Boolean facts must be escaped with str2bool. This commit includes new tests to catch VPN problems like this in the future. --- puppet/modules/site_config/manifests/params.pp | 3 +- puppet/modules/site_config/manifests/setup.pp | 2 +- puppet/modules/site_openvpn/manifests/init.pp | 2 +- .../site_openvpn/templates/add_gateway_ips.sh.erb | 14 +++++++-- tests/server-tests/white-box/openvpn.rb | 36 ++++++++++++++++++++++ 5 files changed, 51 insertions(+), 6 deletions(-) diff --git a/puppet/modules/site_config/manifests/params.pp b/puppet/modules/site_config/manifests/params.pp index 4627515a..2c9687a3 100644 --- a/puppet/modules/site_config/manifests/params.pp +++ b/puppet/modules/site_config/manifests/params.pp @@ -6,8 +6,7 @@ class site_config::params { $ec2_local_ipv4_interface = getvar("interface_${::ec2_local_ipv4}") $environment = hiera('environment', undef) - - if $::vagrant { + if str2bool("$::vagrant") { # Depending on the backend hypervisor networking is setup differently. if $::interfaces =~ /eth1/ { # Virtualbox: Private networking creates a second interface eth1 diff --git a/puppet/modules/site_config/manifests/setup.pp b/puppet/modules/site_config/manifests/setup.pp index a96f87a6..bd3097fa 100644 --- a/puppet/modules/site_config/manifests/setup.pp +++ b/puppet/modules/site_config/manifests/setup.pp @@ -37,7 +37,7 @@ class site_config::setup { # we need to include shorewall::interface{eth0} in setup.pp so # packages can be installed during main puppetrun, even before shorewall # is configured completly - if $::vagrant { + if str2bool("$::vagrant") { include site_config::vagrant } diff --git a/puppet/modules/site_openvpn/manifests/init.pp b/puppet/modules/site_openvpn/manifests/init.pp index f1ecefb9..ee7d6840 100644 --- a/puppet/modules/site_openvpn/manifests/init.pp +++ b/puppet/modules/site_openvpn/manifests/init.pp @@ -68,7 +68,7 @@ class site_openvpn { # find out the netmask in cidr format of the primary IF # thx to https://blog.kumina.nl/tag/puppet-tips-and-tricks/ # we can do this using an inline_template: - $factname_primary_netmask = "netmask_cidr_${::site_config::params::interface}" + $factname_primary_netmask = "netmask_${::site_config::params::interface}" $primary_netmask = inline_template('<%= scope.lookupvar(@factname_primary_netmask) %>') # deploy dh keys diff --git a/puppet/modules/site_openvpn/templates/add_gateway_ips.sh.erb b/puppet/modules/site_openvpn/templates/add_gateway_ips.sh.erb index e76b756b..f2d2bc70 100644 --- a/puppet/modules/site_openvpn/templates/add_gateway_ips.sh.erb +++ b/puppet/modules/site_openvpn/templates/add_gateway_ips.sh.erb @@ -1,11 +1,21 @@ #!/bin/sh -ip addr show dev <%= scope.lookupvar('site_config::params::interface') %> | grep -q <%= @openvpn_gateway_address %>/<%= @primary_netmask %> || +ip addr show dev <%= scope.lookupvar('site_config::params::interface') %> | grep -q "inet <%= @openvpn_gateway_address %>/" || ip addr add <%= @openvpn_gateway_address %>/<%= @primary_netmask %> dev <%= scope.lookupvar('site_config::params::interface') %> +EXITCODE=$? +if [ $EXITCODE != 0 ]; then + exit $EXITCODE +fi + <% if @openvpn_second_gateway_address %> -ip addr show dev <%= scope.lookupvar('site_config::params::interface') %> | grep -q <%= @openvpn_second_gateway_address %>/<%= @primary_netmask %> || +ip addr show dev <%= scope.lookupvar('site_config::params::interface') %> | grep -q "<%= @openvpn_second_gateway_address %>/" || ip addr add <%= @openvpn_second_gateway_address %>/<%= @primary_netmask %> dev <%= scope.lookupvar('site_config::params::interface') %> + +EXITCODE=$? +if [ $EXITCODE != 0 ]; then + exit $EXITCODE +fi <% end %> /bin/echo 1 > /proc/sys/net/ipv4/ip_forward diff --git a/tests/server-tests/white-box/openvpn.rb b/tests/server-tests/white-box/openvpn.rb index 4eed7eb9..adda34a9 100644 --- a/tests/server-tests/white-box/openvpn.rb +++ b/tests/server-tests/white-box/openvpn.rb @@ -13,4 +13,40 @@ class OpenVPN < LeapTest pass end + def test_02_Can_connect_to_openvpn? + # because of the way the firewall rules are currently set up, you can only + # connect to the standard 1194 openvpn port when you are connecting + # from the same host as openvpn is running on. + # + # so, this is disabled for now: + # $node['openvpn']['ports'].each {|port| ...} + # + + $node['openvpn']['protocols'].each do |protocol| + assert_openvpn_is_bound_to_port($node['openvpn']['gateway_address'], protocol, 1194) + end + pass + end + + private + + # + # asserting succeeds if openvpn appears to be correctly bound and we can + # connect to it. we don't actually try to establish a vpn connection in this + # test, we just check to see that it sort of looks like it is openvpn running + # on the port. + # + def assert_openvpn_is_bound_to_port(ip_address, protocol, port) + protocol = protocol.downcase + if protocol == 'udp' + # this sends a magic string to openvpn to attempt to start the protocol. + nc_output = `/bin/echo -e "\\x38\\x01\\x00\\x00\\x00\\x00\\x00\\x00\\x00" | timeout 0.5 nc -u #{ip_address} #{port}`.strip + assert !nc_output.empty?, "Could not connect to OpenVPN daemon at #{ip_address} on port #{port} (#{protocol})." + elsif protocol == 'tcp' + assert system("openssl s_client -connect #{ip_address}:#{port} 2>&1 | grep -q CONNECTED"), + "Could not connect to OpenVPN daemon at #{ip_address} on port #{port} (#{protocol})." + else + assert false, "invalid openvpn protocol #{protocol}" + end + end end -- cgit v1.2.3 From 6757651f24375124149465fd14cd7d674895c27b Mon Sep 17 00:00:00 2001 From: elijah Date: Tue, 5 Sep 2017 21:10:36 -0700 Subject: Docs: fix instructions for signing git commits --- CONTRIBUTING.md | 12 ++++++++++-- 1 file changed, 10 insertions(+), 2 deletions(-) diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 6ae7f6ca..1eb6d3fd 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -4,10 +4,18 @@ To install our commit template, use following command (use --global to use it in your global .gitconfig): - git config [--global] commit.template "~/path_to_leap_platform/contrib/leap-commit-template" + git config commit.template "~/path_to_leap_platform/contrib/leap-commit-template" + +To use for all projects: + + git config --global commit.template "~/path_to_leap_platform/contrib/leap-commit-template" ## Signing commits We very much appreciate signed commits, you can stop forgetting it like this: - git config [--global] commit.gpgSign + git config commit.gpgsign true + +To enable for all projects: + + git config --global commit.gpgsign true -- cgit v1.2.3 From aaf1cbe103d17820feffa09c8dcb6e1fef12e236 Mon Sep 17 00:00:00 2001 From: kwadronaut Date: Thu, 27 Jul 2017 09:12:15 +0200 Subject: Bug: remove shared couchdb design docs Soledad is now taking care of the design of said database. Closes #8428 --- .../modules/site_couchdb/files/designs/shared/docs.json | 8 -------- .../site_couchdb/files/designs/shared/syncs.json | 11 ----------- .../site_couchdb/files/designs/shared/transactions.json | 13 ------------- puppet/modules/site_couchdb/manifests/designs.pp | 17 ++++++++--------- 4 files changed, 8 insertions(+), 41 deletions(-) delete mode 100644 puppet/modules/site_couchdb/files/designs/shared/docs.json delete mode 100644 puppet/modules/site_couchdb/files/designs/shared/syncs.json delete mode 100644 puppet/modules/site_couchdb/files/designs/shared/transactions.json diff --git a/puppet/modules/site_couchdb/files/designs/shared/docs.json b/puppet/modules/site_couchdb/files/designs/shared/docs.json deleted file mode 100644 index 004180cd..00000000 --- a/puppet/modules/site_couchdb/files/designs/shared/docs.json +++ /dev/null @@ -1,8 +0,0 @@ -{ - "_id": "_design/docs", - "views": { - "get": { - "map": "function(doc) {\n if (doc.u1db_rev) {\n var is_tombstone = true;\n var has_conflicts = false;\n if (doc._attachments) {\n if (doc._attachments.u1db_content)\n is_tombstone = false;\n if (doc._attachments.u1db_conflicts)\n has_conflicts = true;\n }\n emit(doc._id,\n {\n \"couch_rev\": doc._rev,\n \"u1db_rev\": doc.u1db_rev,\n \"is_tombstone\": is_tombstone,\n \"has_conflicts\": has_conflicts,\n }\n );\n }\n}\n" - } - } -} \ No newline at end of file diff --git a/puppet/modules/site_couchdb/files/designs/shared/syncs.json b/puppet/modules/site_couchdb/files/designs/shared/syncs.json deleted file mode 100644 index bab5622f..00000000 --- a/puppet/modules/site_couchdb/files/designs/shared/syncs.json +++ /dev/null @@ -1,11 +0,0 @@ -{ - "_id": "_design/syncs", - "updates": { - "put": "function(doc, req){\n if (!doc) {\n doc = {}\n doc['_id'] = 'u1db_sync_log';\n doc['syncs'] = [];\n }\n body = JSON.parse(req.body);\n // remove outdated info\n doc['syncs'] = doc['syncs'].filter(\n function (entry) {\n return entry[0] != body['other_replica_uid'];\n }\n );\n // store u1db rev\n doc['syncs'].push([\n body['other_replica_uid'],\n body['other_generation'],\n body['other_transaction_id']\n ]);\n return [doc, 'ok'];\n}\n\n" - }, - "views": { - "log": { - "map": "function(doc) {\n if (doc._id == 'u1db_sync_log') {\n if (doc.syncs)\n doc.syncs.forEach(function (entry) {\n emit(entry[0],\n {\n 'known_generation': entry[1],\n 'known_transaction_id': entry[2]\n });\n });\n }\n}\n" - } - } -} \ No newline at end of file diff --git a/puppet/modules/site_couchdb/files/designs/shared/transactions.json b/puppet/modules/site_couchdb/files/designs/shared/transactions.json deleted file mode 100644 index 106ad46c..00000000 --- a/puppet/modules/site_couchdb/files/designs/shared/transactions.json +++ /dev/null @@ -1,13 +0,0 @@ -{ - "_id": "_design/transactions", - "lists": { - "generation": "function(head, req) {\n var row;\n var rows=[];\n // fetch all rows\n while(row = getRow()) {\n rows.push(row);\n }\n if (rows.length > 0)\n send(JSON.stringify({\n \"generation\": rows.length,\n \"doc_id\": rows[rows.length-1]['id'],\n \"transaction_id\": rows[rows.length-1]['value']\n }));\n else\n send(JSON.stringify({\n \"generation\": 0,\n \"doc_id\": \"\",\n \"transaction_id\": \"\",\n }));\n}\n", - "trans_id_for_gen": "function(head, req) {\n var row;\n var rows=[];\n var i = 1;\n var gen = 1;\n if (req.query.gen)\n gen = parseInt(req.query['gen']);\n // fetch all rows\n while(row = getRow())\n rows.push(row);\n if (gen <= rows.length)\n send(JSON.stringify({\n \"generation\": gen,\n \"doc_id\": rows[gen-1]['id'],\n \"transaction_id\": rows[gen-1]['value'],\n }));\n else\n send('{}');\n}\n", - "whats_changed": "function(head, req) {\n var row;\n var gen = 1;\n var old_gen = 0;\n if (req.query.old_gen)\n old_gen = parseInt(req.query['old_gen']);\n send('{\"transactions\":[\\n');\n // fetch all rows\n while(row = getRow()) {\n if (gen > old_gen) {\n if (gen > old_gen+1)\n send(',\\n');\n send(JSON.stringify({\n \"generation\": gen,\n \"doc_id\": row[\"id\"],\n \"transaction_id\": row[\"value\"]\n }));\n }\n gen++;\n }\n send('\\n]}');\n}\n" - }, - "views": { - "log": { - "map": "function(doc) {\n if (doc.u1db_transactions)\n doc.u1db_transactions.forEach(function(t) {\n emit(t[0], // use timestamp as key so the results are ordered\n t[1]); // value is the transaction_id\n });\n}\n" - } - } -} \ No newline at end of file diff --git a/puppet/modules/site_couchdb/manifests/designs.pp b/puppet/modules/site_couchdb/manifests/designs.pp index e5fd94c6..33687999 100644 --- a/puppet/modules/site_couchdb/manifests/designs.pp +++ b/puppet/modules/site_couchdb/manifests/designs.pp @@ -11,6 +11,14 @@ class site_couchdb::designs { mode => '0755' } + #cleanup leftovers from before soledad created its db + file { + '/srv/leap/couchdb/designs/shared/': + ensure => absent, + recurse => true, + force => true, + } + site_couchdb::upload_design { 'customers': design => 'customers/Customer.json'; 'identities': design => 'identities/Identity.json'; @@ -19,15 +27,6 @@ class site_couchdb::designs { 'users': design => 'users/User.json'; 'tmp_users': design => 'users/User.json'; 'invite_codes': design => 'invite_codes/InviteCode.json'; - 'shared_docs': - db => 'shared', - design => 'shared/docs.json'; - 'shared_syncs': - db => 'shared', - design => 'shared/syncs.json'; - 'shared_transactions': - db => 'shared', - design => 'shared/transactions.json'; } $sessions_db = rotated_db_name('sessions', 'monthly') -- cgit v1.2.3 From d66bbeb065c2f8f38b946e45e77607629a96f2dc Mon Sep 17 00:00:00 2001 From: Varac Date: Wed, 13 Sep 2017 22:53:53 +0200 Subject: CI: Use master branch for demo.bitmask.net deploy Commit 55b784f2 fixed this for mail.bitmask.net, but not for demo.bitmask.net. See https://0xacab.org/leap/platform/commit/55b784f2ccd6336db4bab9157a8498cb87c562ff This fixes #8843. --- tests/platform-ci/ci-build.sh | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tests/platform-ci/ci-build.sh b/tests/platform-ci/ci-build.sh index 459264d5..58e2bcc5 100755 --- a/tests/platform-ci/ci-build.sh +++ b/tests/platform-ci/ci-build.sh @@ -238,7 +238,7 @@ case "$CI_JOB_NAME" in ;; demo.bitmask.net) TAG='demovpn' - run bitmask ssh://gitolite@leap.se/bitmask + run bitmask ssh://gitolite@leap.se/bitmask master ;; deploy_test*) build_from_scratch -- cgit v1.2.3 From e2e3fa4ea0c26b4ccf226f4a3b692ce0f78f5bf5 Mon Sep 17 00:00:00 2001 From: Varac Date: Tue, 26 Sep 2017 20:36:06 +0200 Subject: Add cert renewal to production deployments --- tests/platform-ci/ci-build.sh | 1 + 1 file changed, 1 insertion(+) diff --git a/tests/platform-ci/ci-build.sh b/tests/platform-ci/ci-build.sh index 58e2bcc5..3c6a1ff4 100755 --- a/tests/platform-ci/ci-build.sh +++ b/tests/platform-ci/ci-build.sh @@ -156,6 +156,7 @@ run() { # Do the deployment echo "Attempting a deploy..." + LEAP_CMD cert renew "$provider_name" deploy echo "Attempting to run tests..." test -- cgit v1.2.3 From b566104d04ebe89d724c089491d9ba478b20d1a6 Mon Sep 17 00:00:00 2001 From: Varac Date: Tue, 26 Sep 2017 20:44:41 +0200 Subject: Use right domain name to renew certs --- tests/platform-ci/ci-build.sh | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tests/platform-ci/ci-build.sh b/tests/platform-ci/ci-build.sh index 3c6a1ff4..120e2858 100755 --- a/tests/platform-ci/ci-build.sh +++ b/tests/platform-ci/ci-build.sh @@ -156,7 +156,7 @@ run() { # Do the deployment echo "Attempting a deploy..." - LEAP_CMD cert renew "$provider_name" + LEAP_CMD cert renew "$CI_JOB_NAME" deploy echo "Attempting to run tests..." test -- cgit v1.2.3 From 0a4a99a0b289402cf527702b91b287e9c6a3cc7c Mon Sep 17 00:00:00 2001 From: Varac Date: Thu, 28 Sep 2017 14:45:39 +0200 Subject: Lint: site_config/manifests/setup.pp --- puppet/modules/site_config/manifests/setup.pp | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/puppet/modules/site_config/manifests/setup.pp b/puppet/modules/site_config/manifests/setup.pp index bd3097fa..ce0f91d4 100644 --- a/puppet/modules/site_config/manifests/setup.pp +++ b/puppet/modules/site_config/manifests/setup.pp @@ -37,7 +37,7 @@ class site_config::setup { # we need to include shorewall::interface{eth0} in setup.pp so # packages can be installed during main puppetrun, even before shorewall # is configured completly - if str2bool("$::vagrant") { + if str2bool($::vagrant) { include site_config::vagrant } -- cgit v1.2.3 From 1848c4c1f580ede26695a4731e280bc40d88f9a3 Mon Sep 17 00:00:00 2001 From: Varac Date: Thu, 28 Sep 2017 12:01:22 +0200 Subject: CI: Fix caching between jobs fog-aws gem was not installed so Job #19895 failed for b566104d The reason was that caching was configured wrongly so files got cached vertically for each job. After re-configuring the gitlab-runner cache on beluga, caches were lost and resulted in failing builds. This commit configures caching in a way that we define one global cache for all platform pipelines. Resolves: #8872 --- .gitlab-ci.yml | 12 ++---------- 1 file changed, 2 insertions(+), 10 deletions(-) diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml index 5933331b..00dcb2b5 100644 --- a/.gitlab-ci.yml +++ b/.gitlab-ci.yml @@ -5,7 +5,7 @@ image: 0xacab.org:4567/leap/docker/ruby:latest # persistent on the gitlab-runner so we don't need to install from # scratch on every pipeline cache: - key: "$CI_BUILD_REF_NAME" + key: "global_platform_cache_between_pipelines" untracked: true paths: - tests/platform-ci/vendor/ @@ -51,15 +51,6 @@ catalog: script: - su -c '/usr/local/bin/bundle exec rake catalog' cirunner -# rspec: -# stage: rspec -# script: -# - /usr/local/bin/bundle exec rake spec - -# The deploy_test job is run on any merge request. This is used to ensure that -# the merge request will deploy and test properly. It is not run when the merge -# request is accepted into master, instead the 'latest' job below is run -# instead. deploy_test: stage: deploy except: @@ -75,6 +66,7 @@ deploy_test:manual: only: - master when: manual + allow_failure: false script: - su -c 'set -o pipefail; stdbuf -oL -eL ./ci-build.sh | ts' cirunner -- cgit v1.2.3 From d9d38bb283ff1c94cbf4bd488175cb77ae3fa3a4 Mon Sep 17 00:00:00 2001 From: Varac Date: Thu, 28 Sep 2017 20:12:08 +0200 Subject: CI: Test staging deb repo component Resolves: #8871 --- .gitlab-ci.yml | 9 ++++++++- tests/platform-ci/ci-build.sh | 8 ++++++-- 2 files changed, 14 insertions(+), 3 deletions(-) diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml index 00dcb2b5..a7a79d11 100644 --- a/.gitlab-ci.yml +++ b/.gitlab-ci.yml @@ -51,13 +51,20 @@ catalog: script: - su -c '/usr/local/bin/bundle exec rake catalog' cirunner -deploy_test: +deploy_test:master: stage: deploy except: - master script: - su -c 'set -o pipefail; stdbuf -oL -eL ./ci-build.sh | ts' cirunner +deploy_test:staging: + stage: deploy + variables: + COMPONENT: "staging" + script: + - su -c 'set -o pipefail; stdbuf -oL -eL ./ci-build.sh | ts' cirunner + # However, sometimes it's important to have a way of triggering a deploy # from scratch manually even from the master branch, when i.e. new packages # got uploaded to the master component of the platform deb repo. diff --git a/tests/platform-ci/ci-build.sh b/tests/platform-ci/ci-build.sh index 120e2858..4710bc88 100755 --- a/tests/platform-ci/ci-build.sh +++ b/tests/platform-ci/ci-build.sh @@ -39,6 +39,9 @@ ROOTDIR=$(readlink -f "$(dirname $0)") # leap_platform PLATFORMDIR=$(readlink -f "${ROOTDIR}/../..") +# deb repo component to configure +COMPONENT=${COMPONENT:-"master"} + # In the gitlab CI pipeline leap is installed in a different # stage by bundle. To debug you can run a single CI job locally # so we install leap_cli as gem here. @@ -88,7 +91,7 @@ build_from_scratch() { # Create cloud.json needed for `leap vm` commands using AWS credentials which jq || ( apt-get update -y && apt-get install jq -y ) - # Dsiable xtrace + # Disable xtrace set +x [ -z "$AWS_ACCESS_KEY" ] && fail "\$AWS_ACCESS_KEY is not set - please provide it as env variable." @@ -102,7 +105,8 @@ build_from_scratch() { [ -d "./tags" ] || mkdir "./tags" /bin/echo "{\"environment\": \"$TAG\"}" | /usr/bin/json_pp > "${PROVIDERDIR}/tags/${TAG}.json" - pwd + # configure deb repo component + echo '{}' | jq ".sources.platform.apt |= { \"source\": \"http://deb.leap.se/platform\", \"component\": \"${COMPONENT}\" }" > common.json # remove old cached nodes echo "Removing old cached nodes..." -- cgit v1.2.3 From 258a7ecfa9e6ac3d32ad5280e856265c5b463bd7 Mon Sep 17 00:00:00 2001 From: kwadronaut Date: Tue, 26 Sep 2017 10:54:27 +0200 Subject: Bug: jessie apt keys must be in /etc/apt/trusted.gpg.d MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit For newer than jessie the 'old' code was enough. This bug didn't show up because our testing images had the keys and sources lines already included within /etc/apt… solves #8862 --- puppet/modules/site_apt/manifests/leap_repo.pp | 19 +++++++++++++++---- 1 file changed, 15 insertions(+), 4 deletions(-) diff --git a/puppet/modules/site_apt/manifests/leap_repo.pp b/puppet/modules/site_apt/manifests/leap_repo.pp index 7c6c49c5..08c3d0e6 100644 --- a/puppet/modules/site_apt/manifests/leap_repo.pp +++ b/puppet/modules/site_apt/manifests/leap_repo.pp @@ -4,10 +4,21 @@ class site_apt::leap_repo { $platform = hiera_hash('platform') $major_version = $platform['major_version'] - if $::site_apt::apt_url_platform_basic =~ /.*experimental.*/ { - $archive_key = '/usr/share/keyrings/leap-experimental-archive.gpg' - } else { - $archive_key = '/usr/share/keyrings/leap-archive.gpg' + # on jessie, keys need to be in /etc/apt/... + # see https://0xacab.org/leap/platform/issues/8862 + if ( $::operatingsystemmajrelease == '8' ) { + if $::site_apt::apt_url_platform_basic =~ /.*experimental.*/ { + $archive_key = 'CE433F407BAB443AFEA196C1837C1AD5367429D9' + } else { + $archive_key = '1E453B2CE87BEE2F7DFE99661E34A1828E207901' + } + } + if ( $::operatingsystemmajrelease != '8' ) { + if $::site_apt::apt_url_platform_basic =~ /.*experimental.*/ { + $archive_key = '/usr/share/keyrings/leap-experimental-archive.gpg' + } else { + $archive_key = '/usr/share/keyrings/leap-archive.gpg' + } } apt::sources_list { 'leap.list': -- cgit v1.2.3 From 18db08c95b0de9cf1ad511fa1dbb20f5eda8bbac Mon Sep 17 00:00:00 2001 From: Varac Date: Thu, 28 Sep 2017 22:27:51 +0200 Subject: Feat: Use version branches for webapp + nickserver We'll release soon so we pin both git repos to there release version branches instead of pulling from master. --- provider_base/common.json | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/provider_base/common.json b/provider_base/common.json index 346e86a6..7b412fe6 100644 --- a/provider_base/common.json +++ b/provider_base/common.json @@ -72,7 +72,7 @@ "nickserver": { "type": "git", "source": "https://leap.se/git/nickserver", - "revision": "origin/master" + "revision": "origin/version/0.10" }, "platform": { "apt": { @@ -88,7 +88,7 @@ "webapp": { "type": "git", "source": "https://leap.se/git/leap_web", - "revision": "origin/master" + "revision": "origin/version/0.9" } } } -- cgit v1.2.3 From aac73fbe01660f5a231ab891c967c16b635fc78d Mon Sep 17 00:00:00 2001 From: kwadronaut Date: Tue, 3 Oct 2017 22:52:25 +0200 Subject: Bug: jessie apt keys stable/experimental/staging The apt sources lines for people using more experimental software was wrong, we abolished the 'experimental' repository some time ago and develoment happens now in the master branch. solves #8862, #8876 --- puppet/modules/site_apt/manifests/leap_repo.pp | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/puppet/modules/site_apt/manifests/leap_repo.pp b/puppet/modules/site_apt/manifests/leap_repo.pp index 08c3d0e6..8b688cfb 100644 --- a/puppet/modules/site_apt/manifests/leap_repo.pp +++ b/puppet/modules/site_apt/manifests/leap_repo.pp @@ -7,14 +7,14 @@ class site_apt::leap_repo { # on jessie, keys need to be in /etc/apt/... # see https://0xacab.org/leap/platform/issues/8862 if ( $::operatingsystemmajrelease == '8' ) { - if $::site_apt::apt_url_platform_basic =~ /.*experimental.*/ { + if $::site_apt::apt_platform_component =~ /.*(staging|master).*/ { $archive_key = 'CE433F407BAB443AFEA196C1837C1AD5367429D9' } else { $archive_key = '1E453B2CE87BEE2F7DFE99661E34A1828E207901' } } if ( $::operatingsystemmajrelease != '8' ) { - if $::site_apt::apt_url_platform_basic =~ /.*experimental.*/ { + if $::site_apt::apt_platform_component =~ /.*(staging|master).*/ { $archive_key = '/usr/share/keyrings/leap-experimental-archive.gpg' } else { $archive_key = '/usr/share/keyrings/leap-archive.gpg' -- cgit v1.2.3 From 96f8af37b4a3bbd9a15651e27f588073c0601299 Mon Sep 17 00:00:00 2001 From: elijah Date: Tue, 19 Sep 2017 11:54:27 -0700 Subject: Feat: split tor service into three The 'tor' service is now three separate services, 'tor_exit', 'tor_relay', or 'hidden_service'. --- provider_base/services/_tor_common.json | 8 ++++++++ provider_base/services/hidden_service.json | 11 +++++++++++ provider_base/services/hidden_service.rb | 4 ++++ provider_base/services/tor_exit.json | 5 +++++ provider_base/services/tor_exit.rb | 6 ++++++ provider_base/services/tor_relay.json | 5 +++++ provider_base/services/tor_relay.rb | 6 ++++++ puppet/manifests/site.pp | 10 +++++++++- .../site_apache/templates/vhosts.d/hidden_service.conf.erb | 2 +- puppet/modules/site_static/manifests/hidden_service.pp | 2 +- puppet/modules/site_static/manifests/init.pp | 10 +++++----- puppet/modules/site_static/templates/apache.conf.erb | 12 ++++++------ puppet/modules/site_webapp/manifests/hidden_service.pp | 4 ++-- puppet/modules/site_webapp/manifests/init.pp | 6 ++---- 14 files changed, 71 insertions(+), 20 deletions(-) create mode 100644 provider_base/services/_tor_common.json create mode 100644 provider_base/services/hidden_service.json create mode 100644 provider_base/services/hidden_service.rb create mode 100644 provider_base/services/tor_exit.json create mode 100644 provider_base/services/tor_exit.rb create mode 100644 provider_base/services/tor_relay.json create mode 100644 provider_base/services/tor_relay.rb diff --git a/provider_base/services/_tor_common.json b/provider_base/services/_tor_common.json new file mode 100644 index 00000000..461232dc --- /dev/null +++ b/provider_base/services/_tor_common.json @@ -0,0 +1,8 @@ +{ + "tor": { + "type": "disabled", + "contacts": "= [provider.contacts['tor'] || provider.contacts.default].flatten", + "nickname": "= (self.name + secret(:tor_family)).sub('_','')[0..18]", + "family": "= nodes[:services => 'tor'][:environment => '!local'].field('tor.nickname').join(',')" + } +} diff --git a/provider_base/services/hidden_service.json b/provider_base/services/hidden_service.json new file mode 100644 index 00000000..137932fa --- /dev/null +++ b/provider_base/services/hidden_service.json @@ -0,0 +1,11 @@ +{ + "tor": { + "hidden_service": { + "key_type": "RSA", + "public_key": "= tor_public_key_path(:node_tor_pub_key, tor.hidden_service.key_type)", + "private_key": "= tor_private_key_path(:node_tor_priv_key, tor.hidden_service.key_type)", + "address": "=> onion_address(:node_tor_pub_key)", + "single_hop": false + } + } +} diff --git a/provider_base/services/hidden_service.rb b/provider_base/services/hidden_service.rb new file mode 100644 index 00000000..50701681 --- /dev/null +++ b/provider_base/services/hidden_service.rb @@ -0,0 +1,4 @@ +if self.services.include?("tor_exit") || self.services.include?("tor_relay") + LeapCli.log :error, "service `hidden_service` is not compatible with tor_exit or tor_relay (node #{self.name})." +end +self.tor['type'] = "hidden_service" \ No newline at end of file diff --git a/provider_base/services/tor_exit.json b/provider_base/services/tor_exit.json new file mode 100644 index 00000000..dab3b76f --- /dev/null +++ b/provider_base/services/tor_exit.json @@ -0,0 +1,5 @@ +{ + "tor": { + "bandwidth_rate": 6550 + } +} diff --git a/provider_base/services/tor_exit.rb b/provider_base/services/tor_exit.rb new file mode 100644 index 00000000..05c67438 --- /dev/null +++ b/provider_base/services/tor_exit.rb @@ -0,0 +1,6 @@ +if self.services.include?("hidden_service") || self.services.include?("tor_relay") + LeapCli.log :error, "service `tor_exit` is not compatible with tor_relay or hidden_service (node #{self.name})." + exit(1) +end +apply_partial("_tor_common") +self.tor['type'] = "exit" diff --git a/provider_base/services/tor_relay.json b/provider_base/services/tor_relay.json new file mode 100644 index 00000000..dab3b76f --- /dev/null +++ b/provider_base/services/tor_relay.json @@ -0,0 +1,5 @@ +{ + "tor": { + "bandwidth_rate": 6550 + } +} diff --git a/provider_base/services/tor_relay.rb b/provider_base/services/tor_relay.rb new file mode 100644 index 00000000..42bafb94 --- /dev/null +++ b/provider_base/services/tor_relay.rb @@ -0,0 +1,6 @@ + +if self.services.include?("tor_exit") || self.services.include?("hidden_service") + LeapCli.log :error, "service `tor_relay` is not compatible with tor_exit or hidden_service (node #{self.name})." +end +apply_partial("_tor_common") +self.tor['type'] = "relay" diff --git a/puppet/manifests/site.pp b/puppet/manifests/site.pp index e243c5df..f3e752cc 100644 --- a/puppet/manifests/site.pp +++ b/puppet/manifests/site.pp @@ -44,10 +44,18 @@ node default { include site_nagios } - if member($services, 'tor') { + if member($services, 'tor_relay') { include site_tor::relay } + if member($services, 'tor_exit') { + include site_tor::relay + } + + if member($services, 'hidden_service') { + include site_tor::hidden_service + } + if member($services, 'mx') { include site_mx } diff --git a/puppet/modules/site_apache/templates/vhosts.d/hidden_service.conf.erb b/puppet/modules/site_apache/templates/vhosts.d/hidden_service.conf.erb index 1d19094e..ddf69a42 100644 --- a/puppet/modules/site_apache/templates/vhosts.d/hidden_service.conf.erb +++ b/puppet/modules/site_apache/templates/vhosts.d/hidden_service.conf.erb @@ -1,5 +1,5 @@ - ServerName <%= @tor_domain %> + ServerName <%= @onion_domain %> Header always unset X-Powered-By diff --git a/puppet/modules/site_static/manifests/hidden_service.pp b/puppet/modules/site_static/manifests/hidden_service.pp index 31cf328e..dcf3785e 100644 --- a/puppet/modules/site_static/manifests/hidden_service.pp +++ b/puppet/modules/site_static/manifests/hidden_service.pp @@ -23,7 +23,7 @@ class site_static::hidden_service ( $single_hop = false ) { '/var/lib/tor/static/hostname': ensure => present, - content => "${::site_static::tor_domain}\n", + content => "${::site_static::onion_domain}\n", owner => 'debian-tor', group => 'debian-tor', mode => '0600', diff --git a/puppet/modules/site_static/manifests/init.pp b/puppet/modules/site_static/manifests/init.pp index 96d92f74..4ddce5ed 100644 --- a/puppet/modules/site_static/manifests/init.pp +++ b/puppet/modules/site_static/manifests/init.pp @@ -12,10 +12,10 @@ class site_static { $formats = $static['formats'] $bootstrap = $static['bootstrap_files'] $tor = hiera('tor', false) - if $tor and member($services, 'tor') and $tor['hidden_service']['active'] == true { - $tor_active = true + if $tor and member($services, 'hidden_service') { + $onion_active = true } else { - $tor_active = false + $onion_active = false } file { @@ -76,9 +76,9 @@ class site_static { } } - if $tor_active { + if $onion_active { $hidden_service = $tor['hidden_service'] - $tor_domain = "${hidden_service['address']}.onion" + $onion_domain = "${hidden_service['address']}.onion" class { 'site_static::hidden_service': single_hop => $hidden_service['single_hop'] } diff --git a/puppet/modules/site_static/templates/apache.conf.erb b/puppet/modules/site_static/templates/apache.conf.erb index 75d834e7..716df437 100644 --- a/puppet/modules/site_static/templates/apache.conf.erb +++ b/puppet/modules/site_static/templates/apache.conf.erb @@ -74,14 +74,14 @@ Require all granted -<%- if @tor_active && (@always_use_hidden_service || @use_hidden_service) -%> +<%- if @onion_active && (@always_use_hidden_service || @use_hidden_service) -%> ## -## Tor +## Hidden Service ## - ServerName <%= @tor_domain %> + ServerName <%= @onion_domain %> <%- if @www_alias -%> - ServerAlias www.<%= @tor_domain %> + ServerAlias www.<%= @onion_domain %> <%- end -%> @@ -105,7 +105,7 @@ ServerName <%= @domain %> <%- if @www_alias -%> - ServerAlias www.<%= @tor_domain %> + ServerAlias www.<%= @domain %> <%- end -%> <%- @aliases && @aliases.each do |domain_alias| -%> ServerAlias <%= domain_alias %> @@ -127,7 +127,7 @@ ServerName <%= @domain %> <%- if @www_alias -%> - ServerAlias www.<%= @tor_domain %> + ServerAlias www.<%= @domain %> <%- end -%> <%- @aliases && @aliases.each do |domain_alias| -%> ServerAlias <%= domain_alias %> diff --git a/puppet/modules/site_webapp/manifests/hidden_service.pp b/puppet/modules/site_webapp/manifests/hidden_service.pp index 3f3f1d0c..658d62f9 100644 --- a/puppet/modules/site_webapp/manifests/hidden_service.pp +++ b/puppet/modules/site_webapp/manifests/hidden_service.pp @@ -2,7 +2,7 @@ class site_webapp::hidden_service { $tor = hiera('tor') $hidden_service = $tor['hidden_service'] - $tor_domain = "${hidden_service['address']}.onion" + $onion_domain = "${hidden_service['address']}.onion" include site_apache::common include apache::module::headers @@ -33,7 +33,7 @@ class site_webapp::hidden_service { '/var/lib/tor/webapp/hostname': ensure => present, - content => "${tor_domain}\n", + content => "${onion_domain}\n", owner => 'debian-tor', group => 'debian-tor', mode => '0600', diff --git a/puppet/modules/site_webapp/manifests/init.pp b/puppet/modules/site_webapp/manifests/init.pp index deb8e8c8..968859bf 100644 --- a/puppet/modules/site_webapp/manifests/init.pp +++ b/puppet/modules/site_webapp/manifests/init.pp @@ -177,11 +177,9 @@ class site_webapp { notify => Service['apache']; } - if $tor { + if $tor and member($services, 'hidden_service') { $hidden_service = $tor['hidden_service'] - if $hidden_service['active'] { - include ::site_webapp::hidden_service - } + include ::site_webapp::hidden_service } -- cgit v1.2.3 From fdb58381afa317ab9639dffa59f4155395b68718 Mon Sep 17 00:00:00 2001 From: Micah Anderson Date: Thu, 14 Sep 2017 10:33:41 -0400 Subject: Bug: Ensure tor exit is disabled properly Simply disabling exit policies is not enough to disable an exit node, it also needs to be explicitly disabled. This may change in future versions of tor, but for now, explicitly adding 'ExitRelay 0' to the configuration is needed. This fixes #8863. --- puppet/modules/site_tor/manifests/disable_exit.pp | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/puppet/modules/site_tor/manifests/disable_exit.pp b/puppet/modules/site_tor/manifests/disable_exit.pp index 078f80ae..85c24bfc 100644 --- a/puppet/modules/site_tor/manifests/disable_exit.pp +++ b/puppet/modules/site_tor/manifests/disable_exit.pp @@ -1,7 +1,13 @@ +# ensure that the tor relay is not configured as an exit node class site_tor::disable_exit { tor::daemon::exit_policy { 'no_exit_at_all': reject => [ '*:*' ]; } +# In a future version of Tor, ExitRelay 0 may become the default when no ExitPolicy is given. + tor::daemon::snippet { + 'disable_exit': + content => 'ExitRelay 0'; + } } -- cgit v1.2.3 From 5b10def43d134e5735bfcec1237c04cf66e8610b Mon Sep 17 00:00:00 2001 From: Micah Anderson Date: Tue, 19 Sep 2017 15:36:06 -0400 Subject: Feat: Refactor tor services In order to refactor the tor services, we need to split them out into three different services. This adds the hidden service class that is necessary to support the previous commits. Fixes #8864. --- provider_base/services/hidden_service.json | 11 ----------- provider_base/services/hidden_service.rb | 4 ---- provider_base/services/tor_exit.rb | 4 ++-- provider_base/services/tor_hidden_service.json | 11 +++++++++++ provider_base/services/tor_hidden_service.rb | 4 ++++ provider_base/services/tor_relay.rb | 4 ++-- puppet/manifests/site.pp | 2 +- puppet/modules/site_static/manifests/hidden_service.pp | 6 ++++-- puppet/modules/site_static/manifests/init.pp | 13 +++++++------ puppet/modules/site_tor/manifests/hidden_service.pp | 13 +++++++++++++ puppet/modules/site_webapp/manifests/hidden_service.pp | 3 ++- puppet/modules/site_webapp/manifests/init.pp | 3 ++- tests/platform-ci/ci-build.sh | 17 +++++++++++++---- tests/platform-ci/provider/nodes/catalogtest.json | 2 +- 14 files changed, 62 insertions(+), 35 deletions(-) delete mode 100644 provider_base/services/hidden_service.json delete mode 100644 provider_base/services/hidden_service.rb create mode 100644 provider_base/services/tor_hidden_service.json create mode 100644 provider_base/services/tor_hidden_service.rb create mode 100644 puppet/modules/site_tor/manifests/hidden_service.pp diff --git a/provider_base/services/hidden_service.json b/provider_base/services/hidden_service.json deleted file mode 100644 index 137932fa..00000000 --- a/provider_base/services/hidden_service.json +++ /dev/null @@ -1,11 +0,0 @@ -{ - "tor": { - "hidden_service": { - "key_type": "RSA", - "public_key": "= tor_public_key_path(:node_tor_pub_key, tor.hidden_service.key_type)", - "private_key": "= tor_private_key_path(:node_tor_priv_key, tor.hidden_service.key_type)", - "address": "=> onion_address(:node_tor_pub_key)", - "single_hop": false - } - } -} diff --git a/provider_base/services/hidden_service.rb b/provider_base/services/hidden_service.rb deleted file mode 100644 index 50701681..00000000 --- a/provider_base/services/hidden_service.rb +++ /dev/null @@ -1,4 +0,0 @@ -if self.services.include?("tor_exit") || self.services.include?("tor_relay") - LeapCli.log :error, "service `hidden_service` is not compatible with tor_exit or tor_relay (node #{self.name})." -end -self.tor['type'] = "hidden_service" \ No newline at end of file diff --git a/provider_base/services/tor_exit.rb b/provider_base/services/tor_exit.rb index 05c67438..bd801a3d 100644 --- a/provider_base/services/tor_exit.rb +++ b/provider_base/services/tor_exit.rb @@ -1,5 +1,5 @@ -if self.services.include?("hidden_service") || self.services.include?("tor_relay") - LeapCli.log :error, "service `tor_exit` is not compatible with tor_relay or hidden_service (node #{self.name})." +if self.services.include?("tor_hidden_service") || self.services.include?("tor_relay") + LeapCli.log :error, "service `tor_exit` is not compatible with tor_relay or tor_hidden_service (node #{self.name})." exit(1) end apply_partial("_tor_common") diff --git a/provider_base/services/tor_hidden_service.json b/provider_base/services/tor_hidden_service.json new file mode 100644 index 00000000..137932fa --- /dev/null +++ b/provider_base/services/tor_hidden_service.json @@ -0,0 +1,11 @@ +{ + "tor": { + "hidden_service": { + "key_type": "RSA", + "public_key": "= tor_public_key_path(:node_tor_pub_key, tor.hidden_service.key_type)", + "private_key": "= tor_private_key_path(:node_tor_priv_key, tor.hidden_service.key_type)", + "address": "=> onion_address(:node_tor_pub_key)", + "single_hop": false + } + } +} diff --git a/provider_base/services/tor_hidden_service.rb b/provider_base/services/tor_hidden_service.rb new file mode 100644 index 00000000..8b8eb24d --- /dev/null +++ b/provider_base/services/tor_hidden_service.rb @@ -0,0 +1,4 @@ +if self.services.include?("tor_exit") || self.services.include?("tor_relay") + LeapCli.log :error, "service `tor_hidden_service` is not compatible with tor_exit or tor_relay (node #{self.name})." +end +self.tor['type'] = "hidden_service" diff --git a/provider_base/services/tor_relay.rb b/provider_base/services/tor_relay.rb index 42bafb94..7fce6ae4 100644 --- a/provider_base/services/tor_relay.rb +++ b/provider_base/services/tor_relay.rb @@ -1,6 +1,6 @@ -if self.services.include?("tor_exit") || self.services.include?("hidden_service") - LeapCli.log :error, "service `tor_relay` is not compatible with tor_exit or hidden_service (node #{self.name})." +if self.services.include?("tor_exit") || self.services.include?("tor_hidden_service") + LeapCli.log :error, "service `tor_relay` is not compatible with tor_exit or tor_hidden_service (node #{self.name})." end apply_partial("_tor_common") self.tor['type'] = "relay" diff --git a/puppet/manifests/site.pp b/puppet/manifests/site.pp index f3e752cc..1f80c47c 100644 --- a/puppet/manifests/site.pp +++ b/puppet/manifests/site.pp @@ -52,7 +52,7 @@ node default { include site_tor::relay } - if member($services, 'hidden_service') { + if member($services, 'tor_hidden_service') { include site_tor::hidden_service } diff --git a/puppet/modules/site_static/manifests/hidden_service.pp b/puppet/modules/site_static/manifests/hidden_service.pp index dcf3785e..f23727f7 100644 --- a/puppet/modules/site_static/manifests/hidden_service.pp +++ b/puppet/modules/site_static/manifests/hidden_service.pp @@ -1,13 +1,15 @@ # create hidden service for static sites class site_static::hidden_service ( $single_hop = false ) { + Class['site_tor::hidden_service'] -> Class['site_static::hidden_service'] + include site_tor::hidden_service - include site_tor tor::daemon::hidden_service { 'static': ports => [ '80 127.0.0.1:80'], single_hop => $single_hop } + file { - '/var/lib/tor/webapp/': + '/var/lib/tor/static/': ensure => directory, owner => 'debian-tor', group => 'debian-tor', diff --git a/puppet/modules/site_static/manifests/init.pp b/puppet/modules/site_static/manifests/init.pp index 4ddce5ed..40c6a28b 100644 --- a/puppet/modules/site_static/manifests/init.pp +++ b/puppet/modules/site_static/manifests/init.pp @@ -7,12 +7,13 @@ class site_static { include site_config::x509::key include site_config::x509::ca_bundle - $static = hiera('static') - $domains = $static['domains'] - $formats = $static['formats'] - $bootstrap = $static['bootstrap_files'] - $tor = hiera('tor', false) - if $tor and member($services, 'hidden_service') { + $services = hiera('services', []) + $static = hiera('static') + $domains = $static['domains'] + $formats = $static['formats'] + $bootstrap = $static['bootstrap_files'] + $tor = hiera('tor', false) + if $tor and member($services, 'tor_hidden_service') { $onion_active = true } else { $onion_active = false diff --git a/puppet/modules/site_tor/manifests/hidden_service.pp b/puppet/modules/site_tor/manifests/hidden_service.pp new file mode 100644 index 00000000..87a7b696 --- /dev/null +++ b/puppet/modules/site_tor/manifests/hidden_service.pp @@ -0,0 +1,13 @@ +# This class simply makes sure a base tor is installed and configured +# It doesn't configure any specific hidden service functionality, +# instead that is configured in site_webapp::hidden_service and +# site_static::hidden_service. +# +# Those could be factored out to make them more generic. +class site_tor::hidden_service { + tag 'leap_service' + Class['site_config::default'] -> Class['site_tor::hidden_service'] + + include site_config::default + include site_tor +} diff --git a/puppet/modules/site_webapp/manifests/hidden_service.pp b/puppet/modules/site_webapp/manifests/hidden_service.pp index 658d62f9..1f87da6b 100644 --- a/puppet/modules/site_webapp/manifests/hidden_service.pp +++ b/puppet/modules/site_webapp/manifests/hidden_service.pp @@ -1,5 +1,7 @@ # Configure tor hidden service for webapp class site_webapp::hidden_service { + Class['site_tor::hidden_service'] -> Class['site_webapp::hidden_service'] + include site_tor::hidden_service $tor = hiera('tor') $hidden_service = $tor['hidden_service'] $onion_domain = "${hidden_service['address']}.onion" @@ -10,7 +12,6 @@ class site_webapp::hidden_service { include apache::module::expires include apache::module::removeip - include site_tor tor::daemon::hidden_service { 'webapp': ports => [ '80 127.0.0.1:80'], single_hop => $hidden_service['single_hop'] diff --git a/puppet/modules/site_webapp/manifests/init.pp b/puppet/modules/site_webapp/manifests/init.pp index 968859bf..605d71b3 100644 --- a/puppet/modules/site_webapp/manifests/init.pp +++ b/puppet/modules/site_webapp/manifests/init.pp @@ -1,6 +1,7 @@ # configure webapp service class site_webapp { tag 'leap_service' + $services = hiera('services', []) $definition_files = hiera('definition_files') $provider = $definition_files['provider'] $eip_service = $definition_files['eip_service'] @@ -177,7 +178,7 @@ class site_webapp { notify => Service['apache']; } - if $tor and member($services, 'hidden_service') { + if $tor and member($services, 'tor_hidden_service') { $hidden_service = $tor['hidden_service'] include ::site_webapp::hidden_service } diff --git a/tests/platform-ci/ci-build.sh b/tests/platform-ci/ci-build.sh index 4710bc88..06af59ca 100755 --- a/tests/platform-ci/ci-build.sh +++ b/tests/platform-ci/ci-build.sh @@ -71,6 +71,13 @@ test() { } build_from_scratch() { + # allow passing into the function the services, use a default set if empty + SERVICES=$1 + if [ -z "$SERVICES" ] + then + SERVICES='couchdb,soledad,mx,webapp,tor_relay,monitor' + fi + # when using gitlab-runner locally, CI_JOB_ID is always 1 which # will conflict with running/terminating AWS instances in subsequent runs # therefore we pick a random number in this case @@ -78,10 +85,7 @@ build_from_scratch() { # create node(s) with unique id so we can run tests in parallel NAME="citest${CI_JOB_ID:-0}" - - TAG='single' - SERVICES='couchdb,soledad,mx,webapp,tor,monitor' # leap_platform/tests/platform-ci/provider PROVIDERDIR="${ROOTDIR}/provider" @@ -184,7 +188,7 @@ upgrade_test() { cd "$PROVIDERDIR" - build_from_scratch + build_from_scratch 'couchdb,soledad,mx,webapp,tor,monitor' deploy test @@ -200,6 +204,11 @@ upgrade_test() { /usr/local/bin/bundle install cd "$PROVIDERDIR" + + # due to the 'tor' service no longer being valid in 0.10, we need to change + # that service to 'tor_relay'. This is done by changing the services array + # with jq to be set to the full correct list of services + jq '.services = ["couchdb","soledad","mx","webapp","tor_relay","monitor"]' < nodes/${NAME}.json deploy test diff --git a/tests/platform-ci/provider/nodes/catalogtest.json b/tests/platform-ci/provider/nodes/catalogtest.json index 05703666..bbf79d9e 100644 --- a/tests/platform-ci/provider/nodes/catalogtest.json +++ b/tests/platform-ci/provider/nodes/catalogtest.json @@ -10,7 +10,7 @@ "webapp", "monitor", "openvpn", - "tor", + "tor_relay", "obfsproxy", "static" ], -- cgit v1.2.3 From ccfe0c38e532cf68d5485a6e4656b80100f1c1b9 Mon Sep 17 00:00:00 2001 From: Micah Anderson Date: Fri, 6 Oct 2017 16:01:37 -0400 Subject: Bug: Remove unused class 0255d8a42fc2c37cfaa660a43936ae546b6178ef removed this class, but it still was being referenced. Since it is not needed, we can just remove the reference. Fixes: #8878 --- puppet/modules/site_static/manifests/init.pp | 2 -- 1 file changed, 2 deletions(-) diff --git a/puppet/modules/site_static/manifests/init.pp b/puppet/modules/site_static/manifests/init.pp index 40c6a28b..1a92c29e 100644 --- a/puppet/modules/site_static/manifests/init.pp +++ b/puppet/modules/site_static/manifests/init.pp @@ -60,10 +60,8 @@ class site_static { include site_config::ruby::dev if (member($formats, 'rack')) { - include site_apt::preferences::passenger class { 'passenger': manage_munin => false, - require => Class['site_apt::preferences::passenger'] } } -- cgit v1.2.3 From 414e36cf11364a9e581eb260b3267078b6cdda44 Mon Sep 17 00:00:00 2001 From: Micah Anderson Date: Sat, 7 Oct 2017 13:50:55 -0400 Subject: feat: add v3 tor hidden service support Resolves: #8879 --- provider_base/services/tor_hidden_service.json | 3 ++- puppet/modules/site_static/manifests/hidden_service.pp | 5 +++-- puppet/modules/site_static/manifests/init.pp | 5 ++--- puppet/modules/site_webapp/manifests/hidden_service.pp | 3 ++- 4 files changed, 9 insertions(+), 7 deletions(-) diff --git a/provider_base/services/tor_hidden_service.json b/provider_base/services/tor_hidden_service.json index 137932fa..d7f3ec27 100644 --- a/provider_base/services/tor_hidden_service.json +++ b/provider_base/services/tor_hidden_service.json @@ -5,7 +5,8 @@ "public_key": "= tor_public_key_path(:node_tor_pub_key, tor.hidden_service.key_type)", "private_key": "= tor_private_key_path(:node_tor_priv_key, tor.hidden_service.key_type)", "address": "=> onion_address(:node_tor_pub_key)", - "single_hop": false + "single_hop": false, + "v3": false } } } diff --git a/puppet/modules/site_static/manifests/hidden_service.pp b/puppet/modules/site_static/manifests/hidden_service.pp index f23727f7..c5d12c34 100644 --- a/puppet/modules/site_static/manifests/hidden_service.pp +++ b/puppet/modules/site_static/manifests/hidden_service.pp @@ -1,11 +1,12 @@ # create hidden service for static sites -class site_static::hidden_service ( $single_hop = false ) { +class site_static::hidden_service ( $single_hop = false, $v3 = false ) { Class['site_tor::hidden_service'] -> Class['site_static::hidden_service'] include site_tor::hidden_service tor::daemon::hidden_service { 'static': ports => [ '80 127.0.0.1:80'], - single_hop => $single_hop + single_hop => $single_hop, + v3 => $v3 } file { diff --git a/puppet/modules/site_static/manifests/init.pp b/puppet/modules/site_static/manifests/init.pp index 40c6a28b..fdc5782f 100644 --- a/puppet/modules/site_static/manifests/init.pp +++ b/puppet/modules/site_static/manifests/init.pp @@ -60,10 +60,8 @@ class site_static { include site_config::ruby::dev if (member($formats, 'rack')) { - include site_apt::preferences::passenger class { 'passenger': manage_munin => false, - require => Class['site_apt::preferences::passenger'] } } @@ -81,7 +79,8 @@ class site_static { $hidden_service = $tor['hidden_service'] $onion_domain = "${hidden_service['address']}.onion" class { 'site_static::hidden_service': - single_hop => $hidden_service['single_hop'] + single_hop => $hidden_service['single_hop'], + v3 => $hidden_service['v3'] } # Currently, we only support a single hidden service address per server. diff --git a/puppet/modules/site_webapp/manifests/hidden_service.pp b/puppet/modules/site_webapp/manifests/hidden_service.pp index 1f87da6b..290f9665 100644 --- a/puppet/modules/site_webapp/manifests/hidden_service.pp +++ b/puppet/modules/site_webapp/manifests/hidden_service.pp @@ -14,7 +14,8 @@ class site_webapp::hidden_service { tor::daemon::hidden_service { 'webapp': ports => [ '80 127.0.0.1:80'], - single_hop => $hidden_service['single_hop'] + single_hop => $hidden_service['single_hop'], + v3 => $hidden_service['v3'] } file { -- cgit v1.2.3 From 3478b7b46087971f4396de3ea370741694963ca9 Mon Sep 17 00:00:00 2001 From: Micah Anderson Date: Sat, 7 Oct 2017 14:04:02 -0400 Subject: git subrepo pull puppet/modules/tor subrepo: subdir: "puppet/modules/tor" merged: "4380e2ea" upstream: origin: "https://leap.se/git/puppet_tor" branch: "master" commit: "4380e2ea" git-subrepo: version: "0.3.1" origin: "https://github.com/ingydotnet/git-subrepo" commit: "a7ee886" --- puppet/modules/tor/.gitrepo | 6 +++--- puppet/modules/tor/manifests/daemon/hidden_service.pp | 1 + puppet/modules/tor/templates/torrc.directory.erb | 2 +- puppet/modules/tor/templates/torrc.hidden_service.erb | 8 ++++++++ 4 files changed, 13 insertions(+), 4 deletions(-) diff --git a/puppet/modules/tor/.gitrepo b/puppet/modules/tor/.gitrepo index 5e3e3c1f..ea3c1495 100644 --- a/puppet/modules/tor/.gitrepo +++ b/puppet/modules/tor/.gitrepo @@ -6,6 +6,6 @@ [subrepo] remote = https://leap.se/git/puppet_tor branch = master - commit = 5ef29012dccc90e68afc215be9521629a0903bc6 - parent = 747d3e9b55c8b7b7d98a63474b6de82d7114c389 - cmdver = 0.4.0 + commit = 4380e2eabd94d8f0df7f63c642dd46ec4783ef07 + parent = be4182d7227d57b4da20d088b4750c756f759888 + cmdver = 0.3.1 diff --git a/puppet/modules/tor/manifests/daemon/hidden_service.pp b/puppet/modules/tor/manifests/daemon/hidden_service.pp index 07121bd6..d91bdc89 100644 --- a/puppet/modules/tor/manifests/daemon/hidden_service.pp +++ b/puppet/modules/tor/manifests/daemon/hidden_service.pp @@ -2,6 +2,7 @@ define tor::daemon::hidden_service( $ports = [], $single_hop = false, + $v3 = false, $data_dir = $tor::daemon::data_dir, $ensure = present ) { diff --git a/puppet/modules/tor/templates/torrc.directory.erb b/puppet/modules/tor/templates/torrc.directory.erb index c7dc4ab5..23ed3392 100644 --- a/puppet/modules/tor/templates/torrc.directory.erb +++ b/puppet/modules/tor/templates/torrc.directory.erb @@ -2,7 +2,7 @@ <% if @port != '0' -%> DirPort <%= @port %> <% end -%> -<% listen_addresses.each do |listen_address| -%> +<% @listen_addresses.each do |listen_address| -%> DirListenAddress <%= listen_address %> <% end -%> <% if @port_front_page != '' -%> diff --git a/puppet/modules/tor/templates/torrc.hidden_service.erb b/puppet/modules/tor/templates/torrc.hidden_service.erb index 5b6afe1c..8a691c6b 100644 --- a/puppet/modules/tor/templates/torrc.hidden_service.erb +++ b/puppet/modules/tor/templates/torrc.hidden_service.erb @@ -10,3 +10,11 @@ HiddenServiceDir <%= @data_dir %>/<%= @name %> HiddenServicePort <%= port %> <% end -%> +<% if @v3 != false %> +# hidden service v3 static +HiddenServiceDir <%= @data_dir %>/<%= @name -%>3 +HiddenServiceVersion 3 +<% @ports.each do |port| -%> +HiddenServicePort <%= port %> +<% end -%> +<% end -%> -- cgit v1.2.3 From 6998301b6fa9485b940e23a9c753d32a76a98fc5 Mon Sep 17 00:00:00 2001 From: Micah Anderson Date: Tue, 17 Oct 2017 16:28:17 -0400 Subject: Bug: replace single provider key with an unique one For the CI, we were using the gitlab-runner ssh key for all provider builds, this replaces it with an unique one for each provider. --- .gitignore | 9 ++++++--- tests/platform-ci/ci-build.sh | 21 +++++++++++---------- .../gitlab-runner-bitmask_ssh.pub | 1 + .../gitlab-runner-ibex/gitlab-runner-ibex_ssh.pub | 1 + .../users/gitlab-runner/gitlab-runner_ssh.pub | 1 - 5 files changed, 19 insertions(+), 14 deletions(-) create mode 100644 tests/platform-ci/provider/users/gitlab-runner-bitmask/gitlab-runner-bitmask_ssh.pub create mode 100644 tests/platform-ci/provider/users/gitlab-runner-ibex/gitlab-runner-ibex_ssh.pub delete mode 100644 tests/platform-ci/provider/users/gitlab-runner/gitlab-runner_ssh.pub diff --git a/.gitignore b/.gitignore index 5c9d135a..47c6a61a 100644 --- a/.gitignore +++ b/.gitignore @@ -18,9 +18,12 @@ /tests/platform-ci/provider/tags/* !/tests/platform-ci/provider/tags/catalogtest.json /tests/platform-ci/provider/users/* -!/tests/platform-ci/provider/users/gitlab-runner -/tests/platform-ci/provider/users/gitlab-runner/* -!/tests/platform-ci/provider/users/gitlab-runner/gitlab-runner_ssh.pub +!tests/platform-ci/provider/users/gitlab-runner-bitmask +tests/platform-ci/provider/users/gitlab-runner-bitmask/* +!tests/platform-ci/provider/users/gitlab-runner-bitmask/gitlab-runner-bitmask_ssh.pub +!tests/platform-ci/provider/users/gitlab-runner-ibex +tests/platform-ci/provider/users/gitlab-runner-ibex/* +!tests/platform-ci/provider/users/gitlab-runner-ibex/gitlab-runner-ibex_ssh.pub /tests/platform-ci/provider/test /builds diff --git a/tests/platform-ci/ci-build.sh b/tests/platform-ci/ci-build.sh index 06af59ca..39fc513b 100755 --- a/tests/platform-ci/ci-build.sh +++ b/tests/platform-ci/ci-build.sh @@ -12,9 +12,9 @@ # * AWS credentials as environment variables: # * `AWS_ACCESS_KEY` # * `AWS_SECRET_KEY` -# * ssh private key used to login to remove vm -# * `SSH_PRIVATE_KEY` -# +# * ssh private keys used to clone providers: +# * `BITMASK_PROVIDER_SSH_PRIVATE_KEY` +# * `IBEX_PROVIDER_SSH_PRIVATE_KEY` # exit if any commands returns non-zero status set -e @@ -100,7 +100,8 @@ build_from_scratch() { [ -z "$AWS_ACCESS_KEY" ] && fail "\$AWS_ACCESS_KEY is not set - please provide it as env variable." [ -z "$AWS_SECRET_KEY" ] && fail "\$AWS_SECRET_KEY is not set - please provide it as env variable." - [ -z "$SSH_PRIVATE_KEY" ] && fail "\$SSH_PRIVATE_KEY is not set - please provide it as env variable." + [ -z "$BITMASK_PROVIDER_SSH_PRIVATE_KEY" ] && fail "\$BITMASK_PROVIDER_SSH_PRIVATE_KEY is not set - please provide it as env variable." + [ -z "$IBEX_PROVIDER_SSH_PRIVATE_KEY" ] && fail "\$IBEX_PROVIDER_SSH_PRIVATE_KEY is not set - please provide it as env variable." /usr/bin/jq ".platform_ci.auth |= .+ {\"aws_access_key_id\":\"$AWS_ACCESS_KEY\", \"aws_secret_access_key\":\"$AWS_SECRET_KEY\"}" < cloud.json.template > cloud.json # Enable xtrace again only if it was set at beginning of script @@ -140,6 +141,12 @@ run() { provider_URI=$2 platform_branch=$3 + # Configure ssh keypair + [ -d ~/.ssh ] || /bin/mkdir ~/.ssh + /bin/echo "${provider_name}_PROVIDER_SSH_PRIVATE_KEY" > ~/.ssh/id_rsa + /bin/chmod 600 ~/.ssh/id_rsa + /bin/cp "${ROOTDIR}/provider/users/gitlab-runner-${provider_name}/gitlab-runner-${provider_name}_ssh.pub" ~/.ssh/id_rsa.pub + # Setup the provider repository echo "Setting up the provider repository: $provider_name by cloning $provider_URI" git clone -q --depth 1 "$provider_URI" @@ -232,12 +239,6 @@ cleanup() { # Ensure we don't output secret stuff to console even when running in verbose mode with -x set +x -# Configure ssh keypair -[ -d ~/.ssh ] || /bin/mkdir ~/.ssh -/bin/echo "$SSH_PRIVATE_KEY" > ~/.ssh/id_rsa -/bin/chmod 600 ~/.ssh/id_rsa -/bin/cp "${ROOTDIR}/provider/users/gitlab-runner/gitlab-runner_ssh.pub" ~/.ssh/id_rsa.pub - # Enable xtrace again only if it was set at beginning of script [[ $xtrace == true ]] && set -x diff --git a/tests/platform-ci/provider/users/gitlab-runner-bitmask/gitlab-runner-bitmask_ssh.pub b/tests/platform-ci/provider/users/gitlab-runner-bitmask/gitlab-runner-bitmask_ssh.pub new file mode 100644 index 00000000..eb206639 --- /dev/null +++ b/tests/platform-ci/provider/users/gitlab-runner-bitmask/gitlab-runner-bitmask_ssh.pub @@ -0,0 +1 @@ +ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC8ICt9oOuuuP7Rt1nIy1qcUV/xW7mDmCb0fcKkFDeAo+7UerEMcA+68oDNw+crc1nfoaW++lnDRIYnyJY43hX0P72u8mzIbt7YB0XgrQiofoygp5c72jQGbeV/59HoKiHI/PUsAG8Sy1oynBpzSd9OWi+h9dBdGq/Wisjdw1/0cILCmNZp0bKDnYfAgEUNmtxd6FFs+dx9x9hHBlquXYzOnMq0XBZiKvxdsnK1gFkNp34y3id7flXyOD5ecTNZJlhPwLTo2z22Re2GCqCh2og8tE58eIQXDeKNyyvmslgyJr2GxKpnjWOlNXW+SCX+bCx02GFuAiww5CcDWu1QuowL micah@muck diff --git a/tests/platform-ci/provider/users/gitlab-runner-ibex/gitlab-runner-ibex_ssh.pub b/tests/platform-ci/provider/users/gitlab-runner-ibex/gitlab-runner-ibex_ssh.pub new file mode 100644 index 00000000..25f085d2 --- /dev/null +++ b/tests/platform-ci/provider/users/gitlab-runner-ibex/gitlab-runner-ibex_ssh.pub @@ -0,0 +1 @@ +ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC1PAycLznUiMoWyEnb3e4AXT8EaAMW+K3of8EA1+NLQMYots35hmCFG/T9fUMV/j3pf0afG8A6uaJc00gc7otph2DWMboVB9cjvqgSQSZ9VZEy7aoc85jX0RyeKONE1N/aIWu2/8vsFInx4iBS+Sh0/H3nQEMxekOTSeyz3tWN4gLZK5n8i65PpwzlpynGyILq0pdMobfPPuRiCT9Xx4/2NbIYgKZJDYcZeis5FyY6M/TO2u4StDBin8+056NLpS4Q9z5/8K9oT0b7pzx66ebd33Yon4pP2I7Bm2cW+2h4F5bJ7gM8WaiZzuPhEZGzmFiD7XLTvGaoR43jdw6cJP05 micah@muck diff --git a/tests/platform-ci/provider/users/gitlab-runner/gitlab-runner_ssh.pub b/tests/platform-ci/provider/users/gitlab-runner/gitlab-runner_ssh.pub deleted file mode 100644 index 3e72b70f..00000000 --- a/tests/platform-ci/provider/users/gitlab-runner/gitlab-runner_ssh.pub +++ /dev/null @@ -1 +0,0 @@ -ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEtniDgIYEm4WtGgiQsZKBpY8x3tbzDBIoMLbZT496juCu4c3f+F5KkMPLmYRPcAupF8tVf+j7Fns7z69PuTjdGfe/cA9CTw/4sNAu3iLpunGR0d2Wtctez5mwz13bKRu9fck3H9p2F9Z47vMKtRTJJ6iIgaUVWU/eFd/MSMJeUVd2ns4Wr7SkHCBB3PV+QL1xl4+AZsUtnGVQ5cE4MZZFia/g6SlrKQYFtLRVIIpDuuaDSvULg1BFMhSCBDNygts8dKTJsCEQYeGVvHZaDwtKTnMqEIwBP4TkIoP+YWnZTPrGywFEJOlZ8b+4HdgdUAFLcFCycWMM9nVcWX7P2lIN gitlab-runner_ssh -- cgit v1.2.3 From 2c50305985f171f80e406e5e430911cd3e9e0f07 Mon Sep 17 00:00:00 2001 From: Micah Anderson Date: Thu, 19 Oct 2017 10:00:45 -0400 Subject: CI: fix variable names, abstract ssh setup --- .gitignore | 3 ++ tests/platform-ci/ci-build.sh | 32 ++++++++++++++++------ .../gitlab-runner-platform_ssh.pub | 1 + 3 files changed, 27 insertions(+), 9 deletions(-) create mode 100644 tests/platform-ci/provider/users/gitlab-runner-platform/gitlab-runner-platform_ssh.pub diff --git a/.gitignore b/.gitignore index 47c6a61a..0dabe1db 100644 --- a/.gitignore +++ b/.gitignore @@ -24,6 +24,9 @@ tests/platform-ci/provider/users/gitlab-runner-bitmask/* !tests/platform-ci/provider/users/gitlab-runner-ibex tests/platform-ci/provider/users/gitlab-runner-ibex/* !tests/platform-ci/provider/users/gitlab-runner-ibex/gitlab-runner-ibex_ssh.pub +!tests/platform-ci/provider/users/gitlab-runner-platform +tests/platform-ci/provider/users/gitlab-runner-platform/* +!tests/platform-ci/provider/users/gitlab-runner-platform/gitlab-runner-platform_ssh.pub /tests/platform-ci/provider/test /builds diff --git a/tests/platform-ci/ci-build.sh b/tests/platform-ci/ci-build.sh index 39fc513b..843642a4 100755 --- a/tests/platform-ci/ci-build.sh +++ b/tests/platform-ci/ci-build.sh @@ -13,8 +13,8 @@ # * `AWS_ACCESS_KEY` # * `AWS_SECRET_KEY` # * ssh private keys used to clone providers: -# * `BITMASK_PROVIDER_SSH_PRIVATE_KEY` -# * `IBEX_PROVIDER_SSH_PRIVATE_KEY` +# * `bitmask_PROVIDER_SSH_PRIVATE_KEY` +# * `ibex_PROVIDER_SSH_PRIVATE_KEY` # exit if any commands returns non-zero status set -e @@ -70,7 +70,26 @@ test() { LEAP_CMD test "$TAG" } +ssh_setup() { + # set the provider name from the first argument passed to the function + provider_name=$1 + # set CI_SSH_SECRET_PRIVATE_KEY to the variable name keyed off of the provider_name + CI_SSH_SECRET_PRIVATE_KEY=${provider_name}_PROVIDER_SSH_PRIVATE_KEY + # Set the SSH_PRIVATE_KEY to the value provided in the CI runner secret variable setting in gitlab + SSH_PRIVATE_KEY=${!CI_SSH_SECRET_PRIVATE_KEY} + echo "Working with provider: $provider_name" + [ -z "$SSH_PRIVATE_KEY" ] && fail "${provider_name}_PROVIDER_SSH_PRIVATE_KEY is not set - please provide it as env variable." + # Configure ssh keypair + [ -d ~/.ssh ] || /bin/mkdir ~/.ssh + /bin/echo "$SSH_PRIVATE_KEY" > ~/.ssh/id_rsa + /bin/chmod 600 ~/.ssh/id_rsa + /bin/cp "${ROOTDIR}/provider/users/gitlab-runner-${provider_name}/gitlab-runner-${provider_name}_ssh.pub" ~/.ssh/id_rsa.pub +} + build_from_scratch() { + # setup ssh keys + ssh_setup platform + # allow passing into the function the services, use a default set if empty SERVICES=$1 if [ -z "$SERVICES" ] @@ -100,8 +119,6 @@ build_from_scratch() { [ -z "$AWS_ACCESS_KEY" ] && fail "\$AWS_ACCESS_KEY is not set - please provide it as env variable." [ -z "$AWS_SECRET_KEY" ] && fail "\$AWS_SECRET_KEY is not set - please provide it as env variable." - [ -z "$BITMASK_PROVIDER_SSH_PRIVATE_KEY" ] && fail "\$BITMASK_PROVIDER_SSH_PRIVATE_KEY is not set - please provide it as env variable." - [ -z "$IBEX_PROVIDER_SSH_PRIVATE_KEY" ] && fail "\$IBEX_PROVIDER_SSH_PRIVATE_KEY is not set - please provide it as env variable." /usr/bin/jq ".platform_ci.auth |= .+ {\"aws_access_key_id\":\"$AWS_ACCESS_KEY\", \"aws_secret_access_key\":\"$AWS_SECRET_KEY\"}" < cloud.json.template > cloud.json # Enable xtrace again only if it was set at beginning of script @@ -141,11 +158,8 @@ run() { provider_URI=$2 platform_branch=$3 - # Configure ssh keypair - [ -d ~/.ssh ] || /bin/mkdir ~/.ssh - /bin/echo "${provider_name}_PROVIDER_SSH_PRIVATE_KEY" > ~/.ssh/id_rsa - /bin/chmod 600 ~/.ssh/id_rsa - /bin/cp "${ROOTDIR}/provider/users/gitlab-runner-${provider_name}/gitlab-runner-${provider_name}_ssh.pub" ~/.ssh/id_rsa.pub + # setup ssh keys + ssh_setup "$provider_name" # Setup the provider repository echo "Setting up the provider repository: $provider_name by cloning $provider_URI" diff --git a/tests/platform-ci/provider/users/gitlab-runner-platform/gitlab-runner-platform_ssh.pub b/tests/platform-ci/provider/users/gitlab-runner-platform/gitlab-runner-platform_ssh.pub new file mode 100644 index 00000000..3347e621 --- /dev/null +++ b/tests/platform-ci/provider/users/gitlab-runner-platform/gitlab-runner-platform_ssh.pub @@ -0,0 +1 @@ +ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDCctVP6O1R6x0KnNpCyIJq7B/M5BWVeCNq1FexHqWBlOabJN+GeUKRkxAjIdPLf3J6Wki7q9hMyIAvKoqaIu3kQ9EHbRwc0znt/ofF9abZ8g+d3v0eg+WFVWopUktp97SfOfHkUUjlWJQUMh4HCl5SStaIBUgrB+l8FGmLYGZvGgoA86AWNo7Zr7D1RHfNYKYAC1uPa+RnxfzGgiy+hz8PmVjaRFmH7UcgsiwinTUSzDsEXVx8NXlEwv1NV86/RZ/EB2nOhYoKOn1WFXVVhtOtqlzoi0M4jLD3nylzyAnX4HdslTIuDB1aoawfTfvdcuqCzs6Z7dFAo0OqUGo7faO1 platform_provider -- cgit v1.2.3 From a96fefa7cf9f9583adf5c152c19f53c8bdeca167 Mon Sep 17 00:00:00 2001 From: Azul Date: Tue, 17 Oct 2017 14:08:31 +0200 Subject: webapp: update design docs for sorted invite codes webapp#8806 needs couch design docs that allow invite codes to be sorted by date. This updated needs to be deployed in sync with the new webapp version. --- .../files/designs/identities/Identity.json | 8 ++-- .../files/designs/invite_codes/InviteCode.json | 44 ++++++++++++---------- .../files/designs/messages/Message.json | 8 ++-- .../site_couchdb/files/designs/tickets/Ticket.json | 16 ++++---- 4 files changed, 40 insertions(+), 36 deletions(-) diff --git a/puppet/modules/site_couchdb/files/designs/identities/Identity.json b/puppet/modules/site_couchdb/files/designs/identities/Identity.json index b1c567c1..c099ae4a 100644 --- a/puppet/modules/site_couchdb/files/designs/identities/Identity.json +++ b/puppet/modules/site_couchdb/files/designs/identities/Identity.json @@ -9,14 +9,14 @@ "all": { "map": " function(doc) {\n if (doc['type'] == 'Identity') {\n emit(doc._id, null);\n }\n }\n" }, - "cert_fingerprints_by_expiry": { - "map": "function(doc) {\n if (doc.type != 'Identity') {\n return;\n }\n if (typeof doc.cert_fingerprints === \"object\") {\n for (fp in doc.cert_fingerprints) {\n if (doc.cert_fingerprints.hasOwnProperty(fp)) {\n emit(doc.cert_fingerprints[fp], fp);\n }\n }\n }\n}\n" + "disabled": { + "map": "function(doc) {\n if (doc.type != 'Identity') {\n return;\n }\n if (typeof doc.user_id === \"undefined\") {\n emit(doc._id, 1);\n }\n}\n" }, "cert_expiry_by_fingerprint": { "map": "function(doc) {\n if (doc.type != 'Identity') {\n return;\n }\n if (typeof doc.cert_fingerprints === \"object\") {\n for (fp in doc.cert_fingerprints) {\n if (doc.cert_fingerprints.hasOwnProperty(fp)) {\n emit(fp, doc.cert_fingerprints[fp]);\n }\n }\n }\n}\n" }, - "disabled": { - "map": "function(doc) {\n if (doc.type != 'Identity') {\n return;\n }\n if (typeof doc.user_id === \"undefined\") {\n emit(doc._id, 1);\n }\n}\n" + "cert_fingerprints_by_expiry": { + "map": "function(doc) {\n if (doc.type != 'Identity') {\n return;\n }\n if (typeof doc.cert_fingerprints === \"object\") {\n for (fp in doc.cert_fingerprints) {\n if (doc.cert_fingerprints.hasOwnProperty(fp)) {\n emit(doc.cert_fingerprints[fp], fp);\n }\n }\n }\n}\n" }, "pgp_key_by_email": { "map": "function(doc) {\n if (doc.type != 'Identity') {\n return;\n }\n if (typeof doc.keys === \"object\") {\n emit(doc.address, doc.keys[\"pgp\"]);\n }\n}\n" diff --git a/puppet/modules/site_couchdb/files/designs/invite_codes/InviteCode.json b/puppet/modules/site_couchdb/files/designs/invite_codes/InviteCode.json index 006c1ea1..d6e1e9d5 100644 --- a/puppet/modules/site_couchdb/files/designs/invite_codes/InviteCode.json +++ b/puppet/modules/site_couchdb/files/designs/invite_codes/InviteCode.json @@ -1,22 +1,26 @@ { - "_id": "_design/InviteCode", - "language": "javascript", - "views": { - "by__id": { - "map": " function(doc) {\n if ((doc['type'] == 'InviteCode') && (doc['_id'] != null)) {\n emit(doc['_id'], 1);\n }\n }\n", - "reduce": "_sum" - }, - "by_invite_code": { - "map": " function(doc) {\n if ((doc['type'] == 'InviteCode') && (doc['invite_code'] != null)) {\n emit(doc['invite_code'], 1);\n }\n }\n", - "reduce": "_sum" - }, - "by_invite_count": { - "map": " function(doc) {\n if ((doc['type'] == 'InviteCode') && (doc['invite_count'] != null)) {\n emit(doc['invite_count'], 1);\n }\n }\n", - "reduce": "_sum" - }, - "all": { - "map": " function(doc) {\n if (doc['type'] == 'InviteCode') {\n emit(doc._id, null);\n }\n }\n" - } - }, - "couchrest-hash": "83fb8f504520b4a9c7ddbb7928cd0ce3" + "_id": "_design/InviteCode", + "language": "javascript", + "views": { + "by_invite_code": { + "map": " function(doc) {\n if ((doc['type'] == 'InviteCode') && (doc['invite_code'] != null)) {\n emit(doc['invite_code'], 1);\n }\n }\n", + "reduce": "_sum" + }, + "by_invite_count": { + "map": " function(doc) {\n if ((doc['type'] == 'InviteCode') && (doc['invite_count'] != null)) {\n emit(doc['invite_count'], 1);\n }\n }\n", + "reduce": "_sum" + }, + "by_created_at": { + "map": " function(doc) {\n if ((doc['type'] == 'InviteCode') && (doc['created_at'] != null)) {\n emit(doc['created_at'], 1);\n }\n }\n", + "reduce": "_sum" + }, + "by_updated_at": { + "map": " function(doc) {\n if ((doc['type'] == 'InviteCode') && (doc['updated_at'] != null)) {\n emit(doc['updated_at'], 1);\n }\n }\n", + "reduce": "_sum" + }, + "all": { + "map": " function(doc) {\n if (doc['type'] == 'InviteCode') {\n emit(doc._id, null);\n }\n }\n" + } + }, + "couchrest-hash": "2d1883c83164a0be127c3a569d9c1902" } \ No newline at end of file diff --git a/puppet/modules/site_couchdb/files/designs/messages/Message.json b/puppet/modules/site_couchdb/files/designs/messages/Message.json index 6a48fc4d..2cb031c6 100644 --- a/puppet/modules/site_couchdb/files/designs/messages/Message.json +++ b/puppet/modules/site_couchdb/files/designs/messages/Message.json @@ -2,14 +2,14 @@ "_id": "_design/Message", "language": "javascript", "views": { - "by_user_ids_to_show": { - "map": "function (doc) {\n if (doc.type === 'Message' && doc.user_ids_to_show && Array.isArray(doc.user_ids_to_show)) {\n doc.user_ids_to_show.forEach(function (userId) {\n emit(userId, 1);\n });\n }\n}\n", - "reduce": " function(key, values, rereduce) {\n return sum(values);\n }\n" - }, "by_user_ids_to_show_and_created_at": { "map": "// not using at moment\n// call with something like Message.by_user_ids_to_show_and_created_at.startkey([user_id, start_date]).endkey([user_id,end_date])\nfunction (doc) {\n if (doc.type === 'Message' && doc.user_ids_to_show && Array.isArray(doc.user_ids_to_show)) {\n doc.user_ids_to_show.forEach(function (userId) {\n emit([userId, doc.created_at], 1);\n });\n }\n}\n", "reduce": " function(key, values, rereduce) {\n return sum(values);\n }\n" }, + "by_user_ids_to_show": { + "map": "function (doc) {\n if (doc.type === 'Message' && doc.user_ids_to_show && Array.isArray(doc.user_ids_to_show)) {\n doc.user_ids_to_show.forEach(function (userId) {\n emit(userId, 1);\n });\n }\n}\n", + "reduce": " function(key, values, rereduce) {\n return sum(values);\n }\n" + }, "all": { "map": " function(doc) {\n if (doc['type'] == 'Message') {\n emit(doc._id, null);\n }\n }\n" } diff --git a/puppet/modules/site_couchdb/files/designs/tickets/Ticket.json b/puppet/modules/site_couchdb/files/designs/tickets/Ticket.json index 578f632b..7ec24634 100644 --- a/puppet/modules/site_couchdb/files/designs/tickets/Ticket.json +++ b/puppet/modules/site_couchdb/files/designs/tickets/Ticket.json @@ -22,8 +22,12 @@ "map": " function(doc) {\n if ((doc['type'] == 'Ticket') && (doc['is_open'] != null) && (doc['updated_at'] != null)) {\n emit([doc['is_open'], doc['updated_at']], 1);\n }\n }\n", "reduce": "_sum" }, - "by_includes_post_by_and_is_open_and_created_at": { - "map": "function(doc) {\n var arr = {}\n if (doc['type'] == 'Ticket' && doc.comments) {\n doc.comments.forEach(function(comment){\n if (comment.posted_by && !arr[comment.posted_by]) {\n //don't add duplicates\n arr[comment.posted_by] = true;\n emit([comment.posted_by, doc.is_open, doc.created_at], 1);\n }\n });\n }\n}\n", + "by_includes_post_by_and_updated_at": { + "map": "function(doc) {\n var arr = {}\n if (doc['type'] == 'Ticket' && doc.comments) {\n doc.comments.forEach(function(comment){\n if (comment.posted_by && !arr[comment.posted_by]) {\n //don't add duplicates\n arr[comment.posted_by] = true;\n emit([comment.posted_by, doc.updated_at], 1);\n }\n });\n }\n}\n", + "reduce": " function(key, values, rereduce) {\n return sum(values);\n }\n" + }, + "by_includes_post_by_and_created_at": { + "map": "function(doc) {\n var arr = {}\n if (doc['type'] == 'Ticket' && doc.comments) {\n doc.comments.forEach(function(comment){\n if (comment.posted_by && !arr[comment.posted_by]) {\n //don't add duplicates\n arr[comment.posted_by] = true;\n emit([comment.posted_by, doc.created_at], 1);\n }\n });\n }\n}\n", "reduce": " function(key, values, rereduce) {\n return sum(values);\n }\n" }, "by_includes_post_by": { @@ -34,12 +38,8 @@ "map": "function(doc) {\n var arr = {}\n if (doc['type'] == 'Ticket' && doc.comments) {\n doc.comments.forEach(function(comment){\n if (comment.posted_by && !arr[comment.posted_by]) {\n //don't add duplicates\n arr[comment.posted_by] = true;\n emit([comment.posted_by, doc.is_open, doc.updated_at], 1);\n }\n });\n }\n}\n", "reduce": " function(key, values, rereduce) {\n return sum(values);\n }\n" }, - "by_includes_post_by_and_created_at": { - "map": "function(doc) {\n var arr = {}\n if (doc['type'] == 'Ticket' && doc.comments) {\n doc.comments.forEach(function(comment){\n if (comment.posted_by && !arr[comment.posted_by]) {\n //don't add duplicates\n arr[comment.posted_by] = true;\n emit([comment.posted_by, doc.created_at], 1);\n }\n });\n }\n}\n", - "reduce": " function(key, values, rereduce) {\n return sum(values);\n }\n" - }, - "by_includes_post_by_and_updated_at": { - "map": "function(doc) {\n var arr = {}\n if (doc['type'] == 'Ticket' && doc.comments) {\n doc.comments.forEach(function(comment){\n if (comment.posted_by && !arr[comment.posted_by]) {\n //don't add duplicates\n arr[comment.posted_by] = true;\n emit([comment.posted_by, doc.updated_at], 1);\n }\n });\n }\n}\n", + "by_includes_post_by_and_is_open_and_created_at": { + "map": "function(doc) {\n var arr = {}\n if (doc['type'] == 'Ticket' && doc.comments) {\n doc.comments.forEach(function(comment){\n if (comment.posted_by && !arr[comment.posted_by]) {\n //don't add duplicates\n arr[comment.posted_by] = true;\n emit([comment.posted_by, doc.is_open, doc.created_at], 1);\n }\n });\n }\n}\n", "reduce": " function(key, values, rereduce) {\n return sum(values);\n }\n" }, "all": { -- cgit v1.2.3 From 71d10f85c5c9cc75dfb3f032a09877102833f739 Mon Sep 17 00:00:00 2001 From: Micah Anderson Date: Tue, 31 Oct 2017 12:16:29 -0400 Subject: Bug: fix repository layout Fix the order of the leap repository so it matches the correct repository layout. Fixes #8888. --- puppet/modules/site_apt/manifests/leap_repo.pp | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/puppet/modules/site_apt/manifests/leap_repo.pp b/puppet/modules/site_apt/manifests/leap_repo.pp index 8b688cfb..1e18b441 100644 --- a/puppet/modules/site_apt/manifests/leap_repo.pp +++ b/puppet/modules/site_apt/manifests/leap_repo.pp @@ -22,7 +22,7 @@ class site_apt::leap_repo { } apt::sources_list { 'leap.list': - content => "deb [signed-by=${archive_key}] ${::site_apt::apt_url_platform_basic} ${::site_apt::apt_platform_codename} ${::site_apt::apt_platform_component}\n", + content => "deb [signed-by=${archive_key}] ${::site_apt::apt_url_platform_basic} ${::site_apt::apt_platform_component} ${::site_apt::apt_platform_codename}\n", before => Exec[refresh_apt] } -- cgit v1.2.3 From 57faf66d7b82cc1ce67cf2e39ba7293c5a9d4bfa Mon Sep 17 00:00:00 2001 From: Azul Date: Wed, 8 Nov 2017 09:13:46 +0100 Subject: webapp: alert on 409 responses They might be meaningful response codes for some scenarios. But so far we are not conciously sending them out. If they occur that is because we handed them down from couch. So we might want to fix the underlying issue. Couch 409s should be caught by the webapp and handled there. --- puppet/modules/site_check_mk/files/agent/logwatch/webapp.cfg | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/puppet/modules/site_check_mk/files/agent/logwatch/webapp.cfg b/puppet/modules/site_check_mk/files/agent/logwatch/webapp.cfg index 337d9ec6..a5375cc8 100644 --- a/puppet/modules/site_check_mk/files/agent/logwatch/webapp.cfg +++ b/puppet/modules/site_check_mk/files/agent/logwatch/webapp.cfg @@ -1,6 +1,10 @@ /var/log/leap/webapp.log # check for webapp errors C Completed 500 +# also alert conflicts. They might be meaningful response codes +# but so far we were just handing them on from couch and they +# indicated some actual problem. + C Completed 409 # couch connection issues C webapp.*Could not connect to couch database messages due to 401 Unauthorized: {"error":"unauthorized","reason":"You are not a server admin."} # ignore RoutingErrors that rails throw when it can't handle a url -- cgit v1.2.3 From a0eea43cbb93665d9d1ac96765d1abdf2a665d15 Mon Sep 17 00:00:00 2001 From: Micah Anderson Date: Tue, 7 Nov 2017 11:54:19 -0500 Subject: CI: do soledad migration during upgrade test Updating platfrom 0.9 (soledad 0.8.0) to 0.10 (soledad 0.10.3) requires a soledad-server migration. This integrates the migration in the CI upgrade_test. Fixes #8881 --- tests/platform-ci/ci-build.sh | 35 ++++++++++++++++++++++++++++++++--- 1 file changed, 32 insertions(+), 3 deletions(-) diff --git a/tests/platform-ci/ci-build.sh b/tests/platform-ci/ci-build.sh index 843642a4..4c9a516c 100755 --- a/tests/platform-ci/ci-build.sh +++ b/tests/platform-ci/ci-build.sh @@ -191,8 +191,32 @@ run() { test } +soledad_migration() { + # check the version of soledad installed + # if the version is not greater than 0.9, we need to do the migration + if ! LEAP_CMD run "dpkg --compare-versions \$(dpkg -l |grep soledad-server|grep ^ii|awk '{ print \$3}') gt 0.9" vm |grep -q oops + then + echo "Need to migrate from soledad 0.9!" + if ! LEAP_CMD run 'systemctl stop leap-mx' vm + then fail + fi + if ! LEAP_CMD run 'systemctl stop soledad-server' vm + then fail + fi + if ! LEAP_CMD run --stream '/usr/share/soledad-server/migration/0.9/migrate.py --verbose --log-file /var/log/leap/soledad_migration.log --do-migrate' vm + then fail + fi + if ! LEAP_CMD run 'systemctl start leap-mx' vm + then fail + fi + if ! LEAP_CMD run 'systemctl start soledad-server' vm + then fail + fi + fi +} + upgrade_test() { - # Checkout stable branch containing last release + # Checkout stable branch containing previous stable release # and deploy this cd "$PLATFORMDIR" # due to cache, this remote is sometimes already added @@ -208,7 +232,7 @@ upgrade_test() { /usr/local/bin/bundle install cd "$PROVIDERDIR" - + LEAP_CMD --version build_from_scratch 'couchdb,soledad,mx,webapp,tor,monitor' deploy test @@ -225,12 +249,17 @@ upgrade_test() { /usr/local/bin/bundle install cd "$PROVIDERDIR" - + LEAP_CMD --version + # due to the 'tor' service no longer being valid in 0.10, we need to change # that service to 'tor_relay'. This is done by changing the services array # with jq to be set to the full correct list of services jq '.services = ["couchdb","soledad","mx","webapp","tor_relay","monitor"]' < nodes/${NAME}.json deploy + + # check for soledad migration, and run it if necessary + soledad_migration + test cleanup -- cgit v1.2.3 From a2959d8696348220cc171da436c6703e1a2cb5fd Mon Sep 17 00:00:00 2001 From: Micah Anderson Date: Thu, 9 Nov 2017 10:43:21 -0500 Subject: CI: produce CI-useful output for soledad test --- tests/platform-ci/ci-build.sh | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/tests/platform-ci/ci-build.sh b/tests/platform-ci/ci-build.sh index 4c9a516c..2b665749 100755 --- a/tests/platform-ci/ci-build.sh +++ b/tests/platform-ci/ci-build.sh @@ -203,7 +203,7 @@ soledad_migration() { if ! LEAP_CMD run 'systemctl stop soledad-server' vm then fail fi - if ! LEAP_CMD run --stream '/usr/share/soledad-server/migration/0.9/migrate.py --verbose --log-file /var/log/leap/soledad_migration.log --do-migrate' vm + if ! LEAP_CMD run --stream '/usr/share/soledad-server/migration/0.9/migrate.py --log-file /dev/stdout --verbose --do-migrate | tee /var/log/leap/soledad_migration.log' vm then fail fi if ! LEAP_CMD run 'systemctl start leap-mx' vm @@ -257,9 +257,13 @@ upgrade_test() { jq '.services = ["couchdb","soledad","mx","webapp","tor_relay","monitor"]' < nodes/${NAME}.json deploy + # pre-migration test + test + # check for soledad migration, and run it if necessary soledad_migration + # run the test again, this should succeed test cleanup -- cgit v1.2.3 From 67b73969cbc5fc4d98f096d47e5c19169c60592e Mon Sep 17 00:00:00 2001 From: Varac Date: Thu, 16 Nov 2017 18:13:29 +0100 Subject: Docs: Fixed gitlab-runner ssh-key env variable --- tests/platform-ci/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tests/platform-ci/README.md b/tests/platform-ci/README.md index 7a44a2fe..2eae5ca0 100644 --- a/tests/platform-ci/README.md +++ b/tests/platform-ci/README.md @@ -39,7 +39,7 @@ together with [Docker](https://www.docker.com/). Export `AWS_ACCESS_KEY`, `AWS_SECRET_KEY` and `SSH_PRIVATE_KEY` as shown above. From the root dir of this repo run: - gitlab-runner exec docker --env AWS_ACCESS_KEY="$AWS_ACCESS_KEY" --env AWS_SECRET_KEY="$AWS_SECRET_KEY" --env SSH_PRIVATE_KEY="$SSH_PRIVATE_KEY" deploy_test + gitlab-runner exec docker --env AWS_ACCESS_KEY="$AWS_ACCESS_KEY" --env AWS_SECRET_KEY="$AWS_SECRET_KEY" --env platform_PROVIDER_SSH_PRIVATE_KEY="$SSH_PRIVATE_KEY" deploy_test See `.gitlab-ci.yml` for all the different test jobs. -- cgit v1.2.3 From 6cc86459acccc915cb3dd4c9700dd5b64aef50c2 Mon Sep 17 00:00:00 2001 From: Micah Anderson Date: Thu, 16 Nov 2017 13:10:41 -0500 Subject: CI: Change the soledad test to run before webapp It is more useful to check to see if Soledad is running before running the webapp tests that require soledad to be running. --- tests/server-tests/white-box/webapp.rb | 1 + 1 file changed, 1 insertion(+) diff --git a/tests/server-tests/white-box/webapp.rb b/tests/server-tests/white-box/webapp.rb index b1ceddb1..b0285e2e 100644 --- a/tests/server-tests/white-box/webapp.rb +++ b/tests/server-tests/white-box/webapp.rb @@ -4,6 +4,7 @@ require 'json' class Webapp < LeapTest depends_on "Network" + depends_on "Soledad" def setup end -- cgit v1.2.3 From 5a25afe09665693f08f74d139cd52a671d0d05b6 Mon Sep 17 00:00:00 2001 From: Micah Anderson Date: Thu, 16 Nov 2017 13:12:13 -0500 Subject: CI: fix soledad version number --- tests/platform-ci/ci-build.sh | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/tests/platform-ci/ci-build.sh b/tests/platform-ci/ci-build.sh index 2b665749..909ed1b1 100755 --- a/tests/platform-ci/ci-build.sh +++ b/tests/platform-ci/ci-build.sh @@ -194,9 +194,9 @@ run() { soledad_migration() { # check the version of soledad installed # if the version is not greater than 0.9, we need to do the migration - if ! LEAP_CMD run "dpkg --compare-versions \$(dpkg -l |grep soledad-server|grep ^ii|awk '{ print \$3}') gt 0.9" vm |grep -q oops + if ! LEAP_CMD run "dpkg --compare-versions \$(dpkg -l |grep soledad-server|grep ^ii|awk '{ print \$3}') gt 0.8" vm |grep -q oops then - echo "Need to migrate from soledad 0.9!" + echo "Need to migrate from soledad 0.8!" if ! LEAP_CMD run 'systemctl stop leap-mx' vm then fail fi -- cgit v1.2.3 From 1653a99d5f70a68f59fea2c1d2eea40447e8a466 Mon Sep 17 00:00:00 2001 From: Varac Date: Thu, 16 Nov 2017 18:30:09 +0100 Subject: CI: Show LEAP apt source --- bin/debug.sh | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/bin/debug.sh b/bin/debug.sh index 35bcfa3e..4f29f300 100755 --- a/bin/debug.sh +++ b/bin/debug.sh @@ -12,6 +12,10 @@ export FACTERLIB="/srv/leap/puppet/modules/apache/lib/facter:/srv/leap/puppet/mo facter 2>/dev/null | egrep -i "$facts" +# show leap debian repo used +echo -e '\n\n' +cat /etc/apt/sources.list.d/leap*.list + # query installed versions echo -e '\n\n' dpkg -l | egrep "$apps" -- cgit v1.2.3 From 464da0db5abe5008b281412548d4f85e1710ba43 Mon Sep 17 00:00:00 2001 From: Varac Date: Thu, 16 Nov 2017 18:41:58 +0100 Subject: CI: Run leap info,allow pre-migration test to fail --- tests/platform-ci/ci-build.sh | 16 ++++++++++++---- 1 file changed, 12 insertions(+), 4 deletions(-) diff --git a/tests/platform-ci/ci-build.sh b/tests/platform-ci/ci-build.sh index 909ed1b1..b2958f7c 100755 --- a/tests/platform-ci/ci-build.sh +++ b/tests/platform-ci/ci-build.sh @@ -66,6 +66,11 @@ deploy() { LEAP_CMD deploy "$TAG" } +leap_info() { + echo "Running leap info on $TAG" + LEAP_CMD info "${TAG}" +} + test() { LEAP_CMD test "$TAG" } @@ -149,8 +154,6 @@ build_from_scratch() { echo "Running leap node init on TAG: $TAG" LEAP_CMD node init "$TAG" - echo "Running leap info on $TAG" - LEAP_CMD info "${TAG}" } run() { @@ -235,6 +238,7 @@ upgrade_test() { LEAP_CMD --version build_from_scratch 'couchdb,soledad,mx,webapp,tor,monitor' deploy + leap_info test # Checkout HEAD of current branch and re-deploy @@ -250,7 +254,7 @@ upgrade_test() { cd "$PROVIDERDIR" LEAP_CMD --version - + # due to the 'tor' service no longer being valid in 0.10, we need to change # that service to 'tor_relay'. This is done by changing the services array # with jq to be set to the full correct list of services @@ -258,11 +262,14 @@ upgrade_test() { deploy # pre-migration test - test + # allowed to fail because when a migration is needed, soledad-server refuses to start + test || /bin/true # check for soledad migration, and run it if necessary soledad_migration + leap_info + # run the test again, this should succeed test @@ -305,6 +312,7 @@ case "$CI_JOB_NAME" in deploy_test*) build_from_scratch deploy + leap_info test cleanup ;; -- cgit v1.2.3 From b584e92a6b654eee9427cd56d7317aa0125c92a4 Mon Sep 17 00:00:00 2001 From: Varac Date: Fri, 17 Nov 2017 16:10:18 +0100 Subject: Bug: Fix unattended-upgrades for LEAP debs Resolves: #8891 --- puppet/modules/site_apt/templates/51unattended-upgrades-leap | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/puppet/modules/site_apt/templates/51unattended-upgrades-leap b/puppet/modules/site_apt/templates/51unattended-upgrades-leap index 3e28531f..2a3494ef 100644 --- a/puppet/modules/site_apt/templates/51unattended-upgrades-leap +++ b/puppet/modules/site_apt/templates/51unattended-upgrades-leap @@ -1,5 +1,5 @@ // this file is managed by puppet ! Unattended-Upgrade::Origins-Pattern { - "site=deb.leap.se,component=<%= @apt_platform_component %>"; + "site=deb.leap.se,codename=<%= @apt_platform_component %>"; } -- cgit v1.2.3 From 87896a7d79ecfe06d2538e719061d6e75e1d7952 Mon Sep 17 00:00:00 2001 From: Micah Anderson Date: Tue, 21 Nov 2017 10:05:14 -0500 Subject: Docs: Update docs to prepare for 0.10.0 release Fixes: #8427, #8812 --- CHANGES.md | 130 ++++++++++++++++++++++ README.md | 7 ++ docs/en/guide/keys-and-certificates.html | 89 +++++++++++++++ docs/en/guide/keys-and-certificates/index.html | 89 +++++++++++++++ docs/en/guide/virtual-machines.html | 6 + docs/en/guide/virtual-machines/index.html | 6 + docs/en/services.html | 2 +- docs/en/services/couchdb.html | 2 +- docs/en/services/couchdb/index.html | 2 +- docs/en/services/index.html | 2 +- docs/en/services/mx.html | 4 +- docs/en/services/mx/index.html | 4 +- docs/en/services/openvpn.html | 2 +- docs/en/services/openvpn/index.html | 2 +- docs/en/services/tor.html | 48 +++++--- docs/en/services/tor/index.html | 48 +++++--- docs/en/troubleshooting/known-issues.html | 2 + docs/en/troubleshooting/known-issues/index.html | 2 + docs/en/troubleshooting/where-to-look.html | 47 ++------ docs/en/troubleshooting/where-to-look/index.html | 47 ++------ docs/en/tutorials/quick-start.html | 58 ++++++++-- docs/en/tutorials/quick-start/guide/commands.html | 0 docs/en/tutorials/quick-start/index.html | 58 ++++++++-- docs/en/tutorials/quick-start/platform.html | 0 docs/en/tutorials/single-node-email.html | 4 +- docs/en/tutorials/single-node-email/index.html | 4 +- docs/en/tutorials/vagrant.html | 8 +- docs/en/tutorials/vagrant/index.html | 8 +- provider_base/common.json | 6 +- tests/platform-ci/ci-build.sh | 3 + 30 files changed, 549 insertions(+), 141 deletions(-) create mode 100644 docs/en/tutorials/quick-start/guide/commands.html create mode 100644 docs/en/tutorials/quick-start/platform.html diff --git a/CHANGES.md b/CHANGES.md index 3dc66746..41317b48 100644 --- a/CHANGES.md +++ b/CHANGES.md @@ -1,3 +1,133 @@ +Platform 0.10 +------------------------------------------------ + +The main focus for Platform 0.10 was to update of all client-side daemons to +newest releases, like Soledad and OpenVPN. This introduces a *compatibility +change*: by setting the platform version to 0.10, it also requires client 0.9.4 +or later. We also switched the development branch to the 'master' branch and are +creating a branch called 0.10.x to push hot-fixes during the 0.10 life-cycle. + +Note: This will be the last major release of the LEAP Platform for Debian +Jessie. We will continue to support 0.10 with minor releases with important +security and bug fixes, but the next major release will require an upgrade to +Stretch. + +New Features: + +* Tor single-hop onion service capability. +* `leap info` is now run after deploy +* Timestamps are added to deployments +* Missing ssh host keys are generated on node init +* Private networking support for local Vagrant development +* Static sites get lets encrypt support +* add command `leap node disable`, `leap node enable`, `leap ping` + +Notable Changes: + +* Removed haproxy because we don't support multi-node couchdb installations anymore (#8144). +* Disable nagios notification emails (#8772). +* Fix layout of apt repository (#8888) +* Limit what archive signing keys are accepted for the leap debian repository packages (#8425). +* Monitor the Webapp logs for errors (#5174). +* Moved development to the master branch. +* Rewrite leap_cli ssh code +* Debian wheezy was fully deprecated +* Restructure package archives to enable auto packaging, and CI testing +* Significant CI improvements +* Troubleshooting information added to `leap user ls` +* Couchdb service is no longer required on soledad nodes (#8693) +* Tor service refactored (#8864), and v3 hidden service support added (#8879) +* Fixed unattended-upgrades (#8891) +* Alert on 409 responses for webapp +* Many other issues resolved, full list: https://0xacab.org/groups/leap/milestones/platform-010?title=Platform+0.10 + +Upgrading: + +If you have a node with the service 'tor' defined, you will need to change it to +be either 'tor-relay', or 'tor-exit'. Look in your provider directory under the +nodes directory for any .json file that has a 'services' section with 'tor' +defined, change that to the correct tor service you are wanting to deploy. + +Make sure you have the correct version of leap_cli + + workstation$ sudo gem install leap_cli --version=1.9 + +If you are upgrading from a version previous to 0.9, please follow those upgrade +instructions before upgrading to 0.10. + +Prepare your platform source by checking out the 0.10.x branch: + + workstation$ cd leap_platform + workstation$ git fetch + workstation$ git checkout 0.10.x + +Then, deploy: + + workstation$ cd $PROVIDER_DIR + workstation$ leap deploy + workstation$ leap test + +After deployment, if the leap test does not succeed, you should +investigate. Please see below for some post-deployment upgrade steps that you +may need to perform. + +Starting with Soledad Server 0.9.0, the CouchDB database schema was changed to +improve speed of the server side storage backend. If you provided email, you +will need to run the migration script, otherwise it is unnecessary. Until you +migrate, soledad will refuse to start. + +To run the migration script, do the following (replacing $PROVIDER_DIR, +$COUCHDB_NODE, $MX_NODE, and $SOLEDAD_NODE with your values): + +First backup your couchdb databases, just to be safe. NOTE: This can take some +time and will place several hundred megabytes of data into +/var/backups/couchdb. The size and time depends on how many users there are on +your system. For example, 15k users took approximately 25 minutes and 308M of +space: + + workstation$ leap ssh $COUCHDB_NODE + server# cd /srv/leap/couchdb/scripts + server# ./cleanup-user-dbs + server# time ./couchdb_dumpall.sh + + Once that has finished, then its time to run the migration: + + workstation$ cd $PROVIDER_DIR + workstation$ leap run 'systemctl leap_mx stop' $MX_NODE + workstation$ leap run --stream '/usr/share/soledad-server/migration/0.9/migrate.py --log-file /var/log/leap/soledad_migration --verbose --do-migrate' $SOLEDAD_NODE + wait for it to finish (will print DONE) + rerun if interrupted + workstation$ leap deploy + workstation$ leap test + +Known Issues: + +If you have been deploying from our master branch (ie: unstable code), you might +end up with a broken sources line for apt. If you get the following: + WARNING: The following packages cannot be authenticated! + +Then you should remove the files on your nodes inside +/var/lib/puppet/modules/apt/keys and deploy again. (#8862, #8876) + +* When upgrading, sometimes systemd does not report the correct state of a + daemon. The daemon will be not running, but systemd thinks it is. The symptom + of this is that a deploy will succeed but `leap test` will fail. To fix, you + can run `systemctl stop DAEMON` and then `systemctl start DAEMON` on the + affected host (systemctl restart seems to work less reliably). + +Includes: + +* leap_web: 0.9.2 +* nickserver: 0.10.0 +* leap-mx: 0.10.1 +* soledad-server: 0.10.5 + +Commits: https://0xacab.org/groups/leap/milestones/platform-010?title=Platform+0.10 + +For details on about all the changes included in this release please consult the +[LEAP platform 0.10 milestone](https://0xacab.org/leap/platform/milestones/7 ). + + Platform 0.9 -------------------------------------- diff --git a/README.md b/README.md index 6e5cb68d..06a4ea0c 100644 --- a/README.md +++ b/README.md @@ -37,6 +37,13 @@ For a live deployment of the platform, the number of servers that is required depends on your needs and which services you want to deploy. At the moment, the LEAP Platform supports servers with a base Debian Jessie installation. +Upgrading +============================= + +If you are upgrading from a previous version of the LEAP Platform, take special +care to follow the instructions detailed in the CHANGES.md to move from one +release to the next. + Troubleshooting ============================= diff --git a/docs/en/guide/keys-and-certificates.html b/docs/en/guide/keys-and-certificates.html index f5f83066..95c08cb9 100644 --- a/docs/en/guide/keys-and-certificates.html +++ b/docs/en/guide/keys-and-certificates.html @@ -181,6 +181,25 @@ Keys and Certificates - LEAP Platform Documentation
  • Renewing a certificate
    • +
  • + Issues +
      +
    1. + Certs already expired +
        +
      1. + Install the official acme client +
        2. +
      2. + Fetch cert +
        3. +
      3. + Deploy the certs +
        4. +
      +
      2. +
    +
    • @@ -445,6 +464,76 @@ workstation$ leap deploy

    There is no need to create a new CSR: renewing will reuse the old private key and the old CSR. It is especially important to not create a new CSR if you have advertised public key pins using HPKP.

    +

    Issues

    + +

    Certs already expired

    + +

    When a cert is already expired, you can get into a possible deadlock situation on your servers which you can only resolve manually at the moment.

    + +

    Install the official acme client

    + +

    Log in to your webapp node and install the certbot package:

    + +
    server$ apt install -t jessie-backports certbot
+
    + +

    Fetch cert

    + +

    Stop apache so the letsencrypt client can bind to port 80:

    + +
    server$ systemctl stop apache2
+
    + +

    Fetch the certs

    + +
    server$ certbot certonly --standalone --email admin@$(hostname -d) -d $(hostname -d) -d api.$(hostname -d) -d $(hostname -f) -d nicknym.$(hostname -d)
+
    + +

    This will put the certs and keys into /etc/letsencrypt/live/DOMAIN/.

    + +

    Now, go to your workstation’s provider configuration directory and copy the newly created files from the server to your local config. You will override existing files so please make a backup before proceeding, or use a version control system to track changes.

    + +
    workstation$ cd PATH_TO_PROVIDER_CONFIG
+
    + +

    Copy the Certificate

    + +
    workstation$ scp 'root@SERVER:/etc/letsencrypt/live/$(hostname -d)/cert.pem' files/cert/DOMAIN.crt
+
    + +

    Copy the private key

    + +
    workstation$ scp 'root@SERVER:/etc/letsencrypt/live/$(hostname -d)/privkey.pem' files/cert/DOMAIN.key
+
    + +

    Copy the CA chain cert

    + +
    workstation$ scp 'root@SERVER:/etc/letsencrypt/live/$(hostname -d)/fullchain.pem' files/cert/commercial_ca.crt
+
    + +

    Deploy the certs

    + +

    Now you only need to deploy the certs

    + +
    workstation$ leap deploy
+
    + +

    This will put them into the right locations which are:

    + +
      +
    • /etc/x509/certs/leap_commercial.crt for the certificate
      • +
    • /etc/x509/./keys/leap_commercial.key for the private key
      • +
    • /usr/local/share/ca-certificates/leap_commercial_ca.crt for the CA chain cert.
      • +
    + + +

    Start apache2 again

    + +
    server$ systemctl start apache2
+
    + +

    Done! In the future please make sure to always renew letsencrypt certificates before they expire ;).

    + diff --git a/docs/en/guide/keys-and-certificates/index.html b/docs/en/guide/keys-and-certificates/index.html index 016a03a7..95279270 100644 --- a/docs/en/guide/keys-and-certificates/index.html +++ b/docs/en/guide/keys-and-certificates/index.html @@ -181,6 +181,25 @@ Keys and Certificates - LEAP Platform Documentation
  • Renewing a certificate
    • +
  • + Issues +
      +
    1. + Certs already expired +
        +
      1. + Install the official acme client +
        2. +
      2. + Fetch cert +
        3. +
      3. + Deploy the certs +
        4. +
      +
      2. +
    +
    • @@ -445,6 +464,76 @@ workstation$ leap deploy

    There is no need to create a new CSR: renewing will reuse the old private key and the old CSR. It is especially important to not create a new CSR if you have advertised public key pins using HPKP.

    +

    Issues

    + +

    Certs already expired

    + +

    When a cert is already expired, you can get into a possible deadlock situation on your servers which you can only resolve manually at the moment.

    + +

    Install the official acme client

    + +

    Log in to your webapp node and install the certbot package:

    + +
    server$ apt install -t jessie-backports certbot
+
    + +

    Fetch cert

    + +

    Stop apache so the letsencrypt client can bind to port 80:

    + +
    server$ systemctl stop apache2
+
    + +

    Fetch the certs

    + +
    server$ certbot certonly --standalone --email admin@$(hostname -d) -d $(hostname -d) -d api.$(hostname -d) -d $(hostname -f) -d nicknym.$(hostname -d)
+
    + +

    This will put the certs and keys into /etc/letsencrypt/live/DOMAIN/.

    + +

    Now, go to your workstation’s provider configuration directory and copy the newly created files from the server to your local config. You will override existing files so please make a backup before proceeding, or use a version control system to track changes.

    + +
    workstation$ cd PATH_TO_PROVIDER_CONFIG
+
    + +

    Copy the Certificate

    + +
    workstation$ scp 'root@SERVER:/etc/letsencrypt/live/$(hostname -d)/cert.pem' files/cert/DOMAIN.crt
+
    + +

    Copy the private key

    + +
    workstation$ scp 'root@SERVER:/etc/letsencrypt/live/$(hostname -d)/privkey.pem' files/cert/DOMAIN.key
+
    + +

    Copy the CA chain cert

    + +
    workstation$ scp 'root@SERVER:/etc/letsencrypt/live/$(hostname -d)/fullchain.pem' files/cert/commercial_ca.crt
+
    + +

    Deploy the certs

    + +

    Now you only need to deploy the certs

    + +
    workstation$ leap deploy
+
    + +

    This will put them into the right locations which are:

    + +
      +
    • /etc/x509/certs/leap_commercial.crt for the certificate
      • +
    • /etc/x509/./keys/leap_commercial.key for the private key
      • +
    • /usr/local/share/ca-certificates/leap_commercial_ca.crt for the CA chain cert.
      • +
    + + +

    Start apache2 again

    + +
    server$ systemctl start apache2
+
    + +

    Done! In the future please make sure to always renew letsencrypt certificates before they expire ;).

    + diff --git a/docs/en/guide/virtual-machines.html b/docs/en/guide/virtual-machines.html index c522c181..28be3211 100644 --- a/docs/en/guide/virtual-machines.html +++ b/docs/en/guide/virtual-machines.html @@ -220,6 +220,7 @@ Virtual Machines - LEAP Platform Documentation @@ -245,6 +246,11 @@ leap vm start mynode 
    leap vm add mynode services:webapp tags:seattle vm.options.InstanceType:t2.small
    +

    For an email provider installation, you should specify the following seeds:

    + +
    leap vm add mynode services:webapp,couchdb,soledad,mx
+
    +

    Check to see what the status is of all VMs:

    leap vm status
diff --git a/docs/en/guide/virtual-machines/index.html b/docs/en/guide/virtual-machines/index.html
index 4b2a2e0f..20d45a77 100644
--- a/docs/en/guide/virtual-machines/index.html
+++ b/docs/en/guide/virtual-machines/index.html
@@ -220,6 +220,7 @@ Virtual Machines - LEAP Platform Documentation
 
 
 
 
@@ -245,6 +246,11 @@ leap vm start mynode
 
    leap vm add mynode services:webapp tags:seattle vm.options.InstanceType:t2.small
 
    
 
+
    For an email provider installation, you should specify the following seeds:
    
+
+
    leap vm add mynode services:webapp,couchdb,soledad,mx
+
    
+
 
    Check to see what the status is of all VMs:
    
 
 
    leap vm status
diff --git a/docs/en/services.html b/docs/en/services.html
index 55211e64..8237b3e4 100644
--- a/docs/en/services.html
+++ b/docs/en/services.html
@@ -235,7 +235,7 @@ Services - LEAP Platform Documentation
 
    
 tor
 
    
-
    Tor exit node or hidden service
    
+
    Tor services: relay, exit node and hidden service
    
 
 
    
 
    
diff --git a/docs/en/services/couchdb.html b/docs/en/services/couchdb.html
index de50a692..43f7cfac 100644
--- a/docs/en/services/couchdb.html
+++ b/docs/en/services/couchdb.html
@@ -215,7 +215,7 @@ couchdb - LEAP Platform Documentation
 
 
      
 
    • search for the “user_id” field
      • 
-
    • in this example testuser@example.org uses the database user-665e004870ee17aa4c94331ff3cd59eb
      • 
+
    • in this example testuser@example.org uses the database user-665e004870ee17aa4c94331ff3cd59eb
      • 
 
    
 
 
diff --git a/docs/en/services/couchdb/index.html b/docs/en/services/couchdb/index.html
index 9eb7fcb8..b48c4eb7 100644
--- a/docs/en/services/couchdb/index.html
+++ b/docs/en/services/couchdb/index.html
@@ -215,7 +215,7 @@ couchdb - LEAP Platform Documentation
 
 
      
 
    • search for the “user_id” field
      • 
-
    • in this example testuser@example.org uses the database user-665e004870ee17aa4c94331ff3cd59eb
      • 
+
    • in this example testuser@example.org uses the database user-665e004870ee17aa4c94331ff3cd59eb
      • 
 
    
 
 
diff --git a/docs/en/services/index.html b/docs/en/services/index.html
index 6d5c68e1..261cd11b 100644
--- a/docs/en/services/index.html
+++ b/docs/en/services/index.html
@@ -235,7 +235,7 @@ Services - LEAP Platform Documentation
 
    
 tor
 
    
-
    Tor exit node or hidden service
    
+
    Tor services: relay, exit node and hidden service
    
 
    
 
    
 
    
diff --git a/docs/en/services/mx.html b/docs/en/services/mx.html
index 8f5a36da..aa41186a 100644
--- a/docs/en/services/mx.html
+++ b/docs/en/services/mx.html
@@ -156,8 +156,8 @@ mx - LEAP Platform Documentation
 
 
      
 
    1. alias lists: by specifying an array of destination addresses, as in the case of “flock”, the single email will get copied to each address.
      2. 
-
    2. chained resolution: alias resolution will recursively continue until there are no more matching aliases. For example, “flock” is resolved to “robin”, which then gets resolved to “robin@bird.org”.
      3. 
-
    3. virtual domains: by specifying the full domain, as in the case of “chickadee@avian.org”, the alias will work for any domain you want. Of course, the MX record for that domain must point to appropriate MX servers, but otherwise you don’t need to do any additional configuration.
      4. 
+
    4. chained resolution: alias resolution will recursively continue until there are no more matching aliases. For example, “flock” is resolved to “robin”, which then gets resolved to “robin@bird.org”.
      5. 
+
    5. virtual domains: by specifying the full domain, as in the case of “chickadee@avian.org”, the alias will work for any domain you want. Of course, the MX record for that domain must point to appropriate MX servers, but otherwise you don’t need to do any additional configuration.
      6. 
 
    6. local delivery: for testing purposes, it is often useful to copy all incoming mail for a particular address and send those copies to another address. You can do this by adding “@deliver.local” as one of the destination addresses. When “@local.delivery” is found, alias resolution stops and the mail is delivered to that username.
      7. 
 
    
 
diff --git a/docs/en/services/mx/index.html b/docs/en/services/mx/index.html
index e8e06e80..048f5198 100644
--- a/docs/en/services/mx/index.html
+++ b/docs/en/services/mx/index.html
@@ -156,8 +156,8 @@ mx - LEAP Platform Documentation
 
 
      
 
    1. alias lists: by specifying an array of destination addresses, as in the case of “flock”, the single email will get copied to each address.
      2. 
-
    2. chained resolution: alias resolution will recursively continue until there are no more matching aliases. For example, “flock” is resolved to “robin”, which then gets resolved to “robin@bird.org”.
      3. 
-
    3. virtual domains: by specifying the full domain, as in the case of “chickadee@avian.org”, the alias will work for any domain you want. Of course, the MX record for that domain must point to appropriate MX servers, but otherwise you don’t need to do any additional configuration.
      4. 
+
    4. chained resolution: alias resolution will recursively continue until there are no more matching aliases. For example, “flock” is resolved to “robin”, which then gets resolved to “robin@bird.org”.
      5. 
+
    5. virtual domains: by specifying the full domain, as in the case of “chickadee@avian.org”, the alias will work for any domain you want. Of course, the MX record for that domain must point to appropriate MX servers, but otherwise you don’t need to do any additional configuration.
      6. 
 
    6. local delivery: for testing purposes, it is often useful to copy all incoming mail for a particular address and send those copies to another address. You can do this by adding “@deliver.local” as one of the destination addresses. When “@local.delivery” is found, alias resolution stops and the mail is delivered to that username.
      7. 
 
    
 
diff --git a/docs/en/services/openvpn.html b/docs/en/services/openvpn.html
index e5fe1128..1a420e21 100644
--- a/docs/en/services/openvpn.html
+++ b/docs/en/services/openvpn.html
@@ -133,8 +133,8 @@ openvpn - LEAP Platform Documentation
 
    Essential configuration
    
 
 
      
-
    • openvpn.gateway_address: The address that OpenVPN daemon is bound to and that VPN clients connect to.
      • 
 
    • ip_address: The main IP of the server, and the egress address for outgoing traffic.
      • 
+
    • openvpn.gateway_address: A secondary address on the same machine (sharing the same interface, or on a separate interface). The OpenVPN daemon is bound to this address and VPN clients connect to it.
      • 
 
    
 
 
diff --git a/docs/en/services/openvpn/index.html b/docs/en/services/openvpn/index.html
index 4a9dc993..23866436 100644
--- a/docs/en/services/openvpn/index.html
+++ b/docs/en/services/openvpn/index.html
@@ -133,8 +133,8 @@ openvpn - LEAP Platform Documentation
 
    Essential configuration
    
 
 
      
-
    • openvpn.gateway_address: The address that OpenVPN daemon is bound to and that VPN clients connect to.
      • 
 
    • ip_address: The main IP of the server, and the egress address for outgoing traffic.
      • 
+
    • openvpn.gateway_address: A secondary address on the same machine (sharing the same interface, or on a separate interface). The OpenVPN daemon is bound to this address and VPN clients connect to it.
      • 
 
    
 
 
diff --git a/docs/en/services/tor.html b/docs/en/services/tor.html
index f649c086..1f6ce112 100644
--- a/docs/en/services/tor.html
+++ b/docs/en/services/tor.html
@@ -110,7 +110,7 @@ tor - LEAP Platform Documentation
 
    
 
    tor
    
 
-
    Tor exit node or hidden service
    
+
    Tor services: relay, exit node and hidden service
    
 
    
 
    
 
      
@@ -124,33 +124,53 @@ tor - LEAP Platform Documentation
 
 
      Topology
      
 
-
      Nodes with tor service will run a Tor exit or hidden service, depending on what other service it is paired with:
      
+
      Nodes with tor service will run a Tor relay with some pre-defined settings, which can be changed with some configuration (see Configuration below). You can enable an exit or a hidden service with additional configuration.
      
+
+
      Configuration
      
+
+
      By default, if a node has service ‘tor’ configured, it will run a tor relay (not an exit). The relay will be configured with bandwidth limitations, contacts, a nickname and a family. The defaults for these (shown below), can be overridden as desired.
      
 
 
        
-
      • tor + openvpn: when combined with openvpn nodes, tor will create a Tor exit node to provide extra cover traffic for the VPN. This can be especially useful if there are VPN gateways without much traffic.
        • 
-
      • tor + webapp: when combined with a webapp node, the tor service will make the webapp and the API available via .onion hidden service.
        • 
-
      • tor stand alone: a regular Tor exit node.
        • 
+
      • tor.bandwidth_rate: the max bandwidth allocated to Tor, in KB per second, when used as an exit node (default: 6550 KB/sec).
        • 
+
      • tor.type: what type of tor node to make, at this moment only ‘exit’ is supported. If not specified, acts as a relay.
        • 
+
      • tor.contacts: the contact information for the relay (default: the list of provider contacts)
        • 
+
      • tor.nickname: the nickname of the relay (default: a combination of the node name and a hash of the family)
        • 
+
      • tor.family: a list of the other nicknames that are part of the same provider
        • 
+
      • tor.hidden_service: to enable a hidden service, set ‘active’ to be true (see below for an example), do not configure “services”: [“tor”] for the node!
        • 
 
      
 
 
-
      If activated, you can list the hidden service .onion addresses this way:
      
+
      Examples:
      
 
-
         leap ls –print tor.hidden_service.address tor
      
+
      To add a relay to a node:
      
 
-
      Then just add ‘.onion’ to the end of the printed addresses.
      
+
      { 
+ "services": ["tor"]
+}
+
      
 
-
      Configuration
      
+
      To enable a hidden service, without a relay, do not specify the tor service (it is not considered secure to have a node configured as a relay and a hidden service at the same time, see: https://trac.torproject.org/8742), instead configure the node to have the following:
      
 
-
        
-
      • tor.bandwidth_rate: the max bandwidth allocated to Tor, in KB per second, when used as an exit node.
        • 
-
      
+
      {
+  "tor": {
+    "hidden_service": {
+    "active": true
+  }
+}
+
      
 
+
      If activated, you can list the hidden service .onion addresses this way:
      
+
+
         leap ls –print tor.hidden_service.address tor
      
+
+
      Then just add ‘.onion’ to the end of the printed addresses.
      
 
-
      For example:
      
+
      To enable a Tor exit node:
      
 
 
      {
   "tor": {
-    "bandwidth_rate": 6550
+    "bandwidth_rate": 6550,
+    "type": "exit"
   }
 }
 
      
diff --git a/docs/en/services/tor/index.html b/docs/en/services/tor/index.html
index 8fecf152..a6380d90 100644
--- a/docs/en/services/tor/index.html
+++ b/docs/en/services/tor/index.html
@@ -110,7 +110,7 @@ tor - LEAP Platform Documentation
 
      
 
      tor
      
 
-
      Tor exit node or hidden service
      
+
      Tor services: relay, exit node and hidden service
      
 
      
 
      
 
        
@@ -124,33 +124,53 @@ tor - LEAP Platform Documentation
 
 
        Topology
        
 
-
        Nodes with tor service will run a Tor exit or hidden service, depending on what other service it is paired with:
        
+
        Nodes with tor service will run a Tor relay with some pre-defined settings, which can be changed with some configuration (see Configuration below). You can enable an exit or a hidden service with additional configuration.
        
+
+
        Configuration
        
+
+
        By default, if a node has service ‘tor’ configured, it will run a tor relay (not an exit). The relay will be configured with bandwidth limitations, contacts, a nickname and a family. The defaults for these (shown below), can be overridden as desired.
        
 
 
          
-
        • tor + openvpn: when combined with openvpn nodes, tor will create a Tor exit node to provide extra cover traffic for the VPN. This can be especially useful if there are VPN gateways without much traffic.
          • 
-
        • tor + webapp: when combined with a webapp node, the tor service will make the webapp and the API available via .onion hidden service.
          • 
-
        • tor stand alone: a regular Tor exit node.
          • 
+
        • tor.bandwidth_rate: the max bandwidth allocated to Tor, in KB per second, when used as an exit node (default: 6550 KB/sec).
          • 
+
        • tor.type: what type of tor node to make, at this moment only ‘exit’ is supported. If not specified, acts as a relay.
          • 
+
        • tor.contacts: the contact information for the relay (default: the list of provider contacts)
          • 
+
        • tor.nickname: the nickname of the relay (default: a combination of the node name and a hash of the family)
          • 
+
        • tor.family: a list of the other nicknames that are part of the same provider
          • 
+
        • tor.hidden_service: to enable a hidden service, set ‘active’ to be true (see below for an example), do not configure “services”: [“tor”] for the node!
          • 
 
        
 
 
-
        If activated, you can list the hidden service .onion addresses this way:
        
+
        Examples:
        
 
-
           leap ls –print tor.hidden_service.address tor
        
+
        To add a relay to a node:
        
 
-
        Then just add ‘.onion’ to the end of the printed addresses.
        
+
        { 
+ "services": ["tor"]
+}
+
        
 
-
        Configuration
        
+
        To enable a hidden service, without a relay, do not specify the tor service (it is not considered secure to have a node configured as a relay and a hidden service at the same time, see: https://trac.torproject.org/8742), instead configure the node to have the following:
        
 
-
          
-
        • tor.bandwidth_rate: the max bandwidth allocated to Tor, in KB per second, when used as an exit node.
          • 
-
        
+
        {
+  "tor": {
+    "hidden_service": {
+    "active": true
+  }
+}
+
        
 
+
        If activated, you can list the hidden service .onion addresses this way:
        
+
+
           leap ls –print tor.hidden_service.address tor
        
+
+
        Then just add ‘.onion’ to the end of the printed addresses.
        
 
-
        For example:
        
+
        To enable a Tor exit node:
        
 
 
        {
   "tor": {
-    "bandwidth_rate": 6550
+    "bandwidth_rate": 6550,
+    "type": "exit"
   }
 }
 
        
diff --git a/docs/en/troubleshooting/known-issues.html b/docs/en/troubleshooting/known-issues.html
index 607970b1..72c64f96 100644
--- a/docs/en/troubleshooting/known-issues.html
+++ b/docs/en/troubleshooting/known-issues.html
@@ -232,6 +232,8 @@ users/userx/otherkey_ssh.pub
 
 
        It is not possible to actually use the EIP openvpn server on vagrant nodes (see: https://leap.se/code/issues/2401)
        
 
+
        Proxmox virtualization isn’t supported because it wants to overwrite our resolv.conf (see: https://leap.se/code/issues/8683)
        
+
 
      
 
      
 
diff --git a/docs/en/troubleshooting/known-issues/index.html b/docs/en/troubleshooting/known-issues/index.html
index eee3b120..3d3d05a8 100644
--- a/docs/en/troubleshooting/known-issues/index.html
+++ b/docs/en/troubleshooting/known-issues/index.html
@@ -232,6 +232,8 @@ users/userx/otherkey_ssh.pub
 
 
      It is not possible to actually use the EIP openvpn server on vagrant nodes (see: https://leap.se/code/issues/2401)
      
 
+
      Proxmox virtualization isn’t supported because it wants to overwrite our resolv.conf (see: https://leap.se/code/issues/8683)
      
+
 
    
 
    
 
diff --git a/docs/en/troubleshooting/where-to-look.html b/docs/en/troubleshooting/where-to-look.html
index a1207aca..93cbbe97 100644
--- a/docs/en/troubleshooting/where-to-look.html
+++ b/docs/en/troubleshooting/where-to-look.html
@@ -114,9 +114,6 @@ Where to look - LEAP Platform Documentation
       
  • 
         Places to look for errors
       
    • 
-      
  • 
-        Is haproxy ok ?
-      
    • 
       
  • 
         Is couchdb accessible through stunnel ?
       
    • 
@@ -216,22 +213,10 @@ Where to look - LEAP Platform Documentation
 
 
 
-
    Is haproxy ok ?
    
-
-
    curl -s -X  GET "http://127.0.0.1:4096"
-
    
-
 
    Is couchdb accessible through stunnel ?
    
 
-
      
-
    • Depending on how many couch nodes you have, increase the port for every test
-(see /etc/haproxy/haproxy.cfg for the server/port mapping):
      
-
-
        curl -s -X  GET “http://127.0.0.1:4000”
-  curl -s -X  GET “http://127.0.0.1:4001”
-  …
      • 
-
    
-
+
    curl -s -X  GET "http://127.0.0.1:4000"
+
    
 
 
    Check couchdb acl as admin
    
 
@@ -240,8 +225,8 @@ cat /srv/leap/webapp/config/couchdb.yml.admin  # see username and password
 echo "machine 127.0.0.1 login admin password <PASSWORD>" > /etc/couchdb/couchdb-admin.netrc
 chmod 600 /etc/couchdb/couchdb-admin.netrc
 
-curl -s --netrc-file /etc/couchdb/couchdb-admin.netrc -X GET "http://127.0.0.1:4096"
-curl -s --netrc-file /etc/couchdb/couchdb-admin.netrc -X GET "http://127.0.0.1:4096/_all_dbs"
+curl -s --netrc-file /etc/couchdb/couchdb-admin.netrc -X GET "http://127.0.0.1:4000"
+curl -s --netrc-file /etc/couchdb/couchdb-admin.netrc -X GET "http://127.0.0.1:4000/_all_dbs"
 
    
 
 
    Check couchdb acl as unpriviledged user
    
@@ -250,8 +235,8 @@ curl -s --netrc-file /etc/couchdb/couchdb-admin.netrc -X GET "http://127.0.0.1:4
 echo "machine 127.0.0.1 login webapp password <PASSWORD>" > /etc/couchdb/couchdb-webapp.netrc
 chmod 600 /etc/couchdb/couchdb-webapp.netrc
 
-curl -s --netrc-file /etc/couchdb/couchdb-webapp.netrc -X GET "http://127.0.0.1:4096"
-curl -s --netrc-file /etc/couchdb/couchdb-webapp.netrc -X GET "http://127.0.0.1:4096/_all_dbs"
+curl -s --netrc-file /etc/couchdb/couchdb-webapp.netrc -X GET "http://127.0.0.1:4000"
+curl -s --netrc-file /etc/couchdb/couchdb-webapp.netrc -X GET "http://127.0.0.1:4000/_all_dbs"

    All URLs accessible ?

    @@ -350,15 +335,8 @@ curl -s --netrc-file /etc/couchdb/couchdb-webapp.netrc -X GET "http://127.0.0.1:

    Is couchdb accessible through stunnel ?

    -
      -

    • Depending on how many couch nodes you have, increase the port for every test -(see /etc/haproxy/haproxy.cfg for the server/port mapping):

      - -

      curl -s -X GET “http://127.0.0.1:4000” - curl -s -X GET “http://127.0.0.1:4001” - …

      • -
    - +
    curl -s -X  GET "http://127.0.0.1:4000"
+

    Query leap-mx

    @@ -398,15 +376,10 @@ curl -s --netrc-file /etc/couchdb/couchdb-webapp.netrc -X GET "http://127.0.0.1: echo "machine 127.0.0.1 login leap_mx password <PASSWORD>" > /etc/couchdb/couchdb-leap_mx.netrc chmod 600 /etc/couchdb/couchdb-leap_mx.netrc -curl -s --netrc-file /etc/couchdb/couchdb-leap_mx.netrc -X GET "http://127.0.0.1:4096/_all_dbs" # pick one "user-<hash>" db -curl -s --netrc-file /etc/couchdb/couchdb-leap_mx.netrc -X GET "http://127.0.0.1:4096/user-de9c77a3d7efbc779c6c20da88e8fb9c" +curl -s --netrc-file /etc/couchdb/couchdb-leap_mx.netrc -X GET "http://127.0.0.1:4000/_all_dbs" # pick one "user-<hash>" db +curl -s --netrc-file /etc/couchdb/couchdb-leap_mx.netrc -X GET "http://127.0.0.1:4000/user-de9c77a3d7efbc779c6c20da88e8fb9c" -
      -
    • you may check multiple times, cause 127.0.0.1:4096 is haproxy load-balancing the different couchdb nodes
      • -
    - -

    Mailspool

      diff --git a/docs/en/troubleshooting/where-to-look/index.html b/docs/en/troubleshooting/where-to-look/index.html index ab3115af..63d05e45 100644 --- a/docs/en/troubleshooting/where-to-look/index.html +++ b/docs/en/troubleshooting/where-to-look/index.html @@ -114,9 +114,6 @@ Where to look - LEAP Platform Documentation
    • Places to look for errors
      • -
    • - Is haproxy ok ? -
    • Is couchdb accessible through stunnel ?
      • @@ -216,22 +213,10 @@ Where to look - LEAP Platform Documentation
    -

    Is haproxy ok ?

    - -
    curl -s -X  GET "http://127.0.0.1:4096"
-
    -

    Is couchdb accessible through stunnel ?

    -
      -

    • Depending on how many couch nodes you have, increase the port for every test -(see /etc/haproxy/haproxy.cfg for the server/port mapping):

      - -

      curl -s -X GET “http://127.0.0.1:4000” - curl -s -X GET “http://127.0.0.1:4001” - …

      • -
    - +
    curl -s -X  GET "http://127.0.0.1:4000"
+

    Check couchdb acl as admin

    @@ -240,8 +225,8 @@ cat /srv/leap/webapp/config/couchdb.yml.admin # see username and password echo "machine 127.0.0.1 login admin password <PASSWORD>" > /etc/couchdb/couchdb-admin.netrc chmod 600 /etc/couchdb/couchdb-admin.netrc -curl -s --netrc-file /etc/couchdb/couchdb-admin.netrc -X GET "http://127.0.0.1:4096" -curl -s --netrc-file /etc/couchdb/couchdb-admin.netrc -X GET "http://127.0.0.1:4096/_all_dbs" +curl -s --netrc-file /etc/couchdb/couchdb-admin.netrc -X GET "http://127.0.0.1:4000" +curl -s --netrc-file /etc/couchdb/couchdb-admin.netrc -X GET "http://127.0.0.1:4000/_all_dbs"

    Check couchdb acl as unpriviledged user

    @@ -250,8 +235,8 @@ curl -s --netrc-file /etc/couchdb/couchdb-admin.netrc -X GET "http://127.0.0.1:4 echo "machine 127.0.0.1 login webapp password <PASSWORD>" > /etc/couchdb/couchdb-webapp.netrc chmod 600 /etc/couchdb/couchdb-webapp.netrc -curl -s --netrc-file /etc/couchdb/couchdb-webapp.netrc -X GET "http://127.0.0.1:4096" -curl -s --netrc-file /etc/couchdb/couchdb-webapp.netrc -X GET "http://127.0.0.1:4096/_all_dbs" +curl -s --netrc-file /etc/couchdb/couchdb-webapp.netrc -X GET "http://127.0.0.1:4000" +curl -s --netrc-file /etc/couchdb/couchdb-webapp.netrc -X GET "http://127.0.0.1:4000/_all_dbs"

    All URLs accessible ?

    @@ -350,15 +335,8 @@ curl -s --netrc-file /etc/couchdb/couchdb-webapp.netrc -X GET "http://127.0.0.1:

    Is couchdb accessible through stunnel ?

    -
      -

    • Depending on how many couch nodes you have, increase the port for every test -(see /etc/haproxy/haproxy.cfg for the server/port mapping):

      - -

      curl -s -X GET “http://127.0.0.1:4000” - curl -s -X GET “http://127.0.0.1:4001” - …

      • -
    - +
    curl -s -X  GET "http://127.0.0.1:4000"
+

    Query leap-mx

    @@ -398,15 +376,10 @@ curl -s --netrc-file /etc/couchdb/couchdb-webapp.netrc -X GET "http://127.0.0.1: echo "machine 127.0.0.1 login leap_mx password <PASSWORD>" > /etc/couchdb/couchdb-leap_mx.netrc chmod 600 /etc/couchdb/couchdb-leap_mx.netrc -curl -s --netrc-file /etc/couchdb/couchdb-leap_mx.netrc -X GET "http://127.0.0.1:4096/_all_dbs" # pick one "user-<hash>" db -curl -s --netrc-file /etc/couchdb/couchdb-leap_mx.netrc -X GET "http://127.0.0.1:4096/user-de9c77a3d7efbc779c6c20da88e8fb9c" +curl -s --netrc-file /etc/couchdb/couchdb-leap_mx.netrc -X GET "http://127.0.0.1:4000/_all_dbs" # pick one "user-<hash>" db +curl -s --netrc-file /etc/couchdb/couchdb-leap_mx.netrc -X GET "http://127.0.0.1:4000/user-de9c77a3d7efbc779c6c20da88e8fb9c" -
      -
    • you may check multiple times, cause 127.0.0.1:4096 is haproxy load-balancing the different couchdb nodes
      • -
    - -

    Mailspool

      diff --git a/docs/en/tutorials/quick-start.html b/docs/en/tutorials/quick-start.html index d2670b30..d275a321 100644 --- a/docs/en/tutorials/quick-start.html +++ b/docs/en/tutorials/quick-start.html @@ -122,6 +122,9 @@ Quick Start Tutorial - LEAP Platform Documentation
    • Install pre-requisites
      • +
    • + The platform recipes +
    • Install the LEAP command-line utility
      • @@ -139,6 +142,9 @@ Quick Start Tutorial - LEAP Platform Documentation
    • Option B: Add a local node
      • +
    • + Option C: Add a virtual machine in the cloud +
    • @@ -197,7 +203,7 @@ Quick Start Tutorial - LEAP Platform Documentation
      1. A local Vagrant virtual machine: a Vagrant machine can only be useful for testing.
        2. -
      2. A real or paravirtualized server: The server must have Debian Jessie installed, and you must be able to SSH into the machine as root. Paravirtualization includes KVM, Xen, OpenStack, Amazon, but not VirtualBox or OpenVZ.
        3. +
      3. A real or paravirtualized server: The server must have Debian Jessie installed, and you must be able to SSH into the machine as root. Paravirtualization includes KVM, Xen, OpenStack, Amazon, but not VirtualBox or OpenVZ. Proxmox has an known issue when changing the resolver
      • @@ -214,15 +220,20 @@ Quick Start Tutorial - LEAP Platform Documentation

      Prepare your workstation

      -

      In order to be able to manage your servers, you need to install the leap command on your workstation:

      +

      In order to be able to manage your servers, you need to setup the LEAP Platform on your desktop. This consists of three parts: the platform recipes, the leap command, and your provider instance. We will go over these step-by-step below, you can find more details in the platform introduction.

      Install pre-requisites

      Install core prerequisites on your workstation.

      -

      Debian & Ubuntu

      +

      Debian Unstable (sid)

      + +
      workstation$ sudo apt-get install git rsync openssh-client openssl zlib1g-dev
+
      + +

      Other Debian & Ubuntu

      -
      workstation$ sudo apt-get install git ruby ruby-dev rsync openssh-client openssl rake make bzip2
+
      workstation$ sudo apt-get install git ruby ruby-dev rsync openssh-client openssl rake make bzip2 zlib1g-dev
 
      
 
 
      Mac OS
      
@@ -231,9 +242,38 @@ Quick Start Tutorial - LEAP Platform Documentation
 workstation$ ruby-install ruby
      +

      The platform recipes

      + +

      The LEAP platform recipes are a set modules designed to work together to provide you everything you need to manage your provider. You typically do not need to modify these, but do need them available for deploying your provider.

      + +

      To obtain the platform recipes, simply clone the git repository, and then check out the most recent stable release branch:

      + +
      workstation$ git clone -b version/0.9.x https://leap.se/git/leap_platform
+
      + +

      If you want to get the latest development branch (Beware: it could be unstable !) you could simply use the master branch instead by:

      + +
      workstation$ git clone https://leap.se/git/leap_platform
+
      +

      Install the LEAP command-line utility

      -

      Install the leap command system-wide:

      +

      The leap command line tool is what you use to manage everything about your provider.

      + +

      Keep these rules in mind:

      + +
        +
      • leap is run on your workstation: The leap command is always run locally on your workstation, never on a server you are deploying to.
        • +
      • leap is run from within a provider instance: The leap command requires that the current working directory is a valid provider instance, except when running leap new to create a new provider instance.
        • +
      + + +

      If on Debian Unstable (sid), simply do this:

      + +
      workstation$ sudo apt install leap-cli
+
      + +

      Otherwise, you will need to do this:

      workstation$ sudo gem install leap_cli
      @@ -340,14 +380,18 @@ workstation$ leap cert csr

      Option B: Add a local node

      -

      Create a node, with the services “webapp” and “couchdb”, and then start the local virtual machine:

      +

      Create a node, with the services “webapp”, “soledad” and “couchdb”, and then start the local virtual machine:

      -
      workstation$ leap node add --local wildebeest services:webapp,couchdb
+
      workstation$ leap node add --local wildebeest services:webapp,couchdb,soledad
 workstation$ leap local start wildebeest
 
      
 
 
      It will take a while to download the Virtualbox base box and create the virtual machine.
      
 
+
      Option C: Add a virtual machine in the cloud
      
+
+
      In order to create a provider using the cloud, please follow this instructions.
      
+
 
      Deploy your provider
      
 
 
      Initialize the node
      
diff --git a/docs/en/tutorials/quick-start/guide/commands.html b/docs/en/tutorials/quick-start/guide/commands.html
new file mode 100644
index 00000000..e69de29b
diff --git a/docs/en/tutorials/quick-start/index.html b/docs/en/tutorials/quick-start/index.html
index 27b21238..ae617e1b 100644
--- a/docs/en/tutorials/quick-start/index.html
+++ b/docs/en/tutorials/quick-start/index.html
@@ -122,6 +122,9 @@ Quick Start Tutorial - LEAP Platform Documentation
       
    • 
         Install pre-requisites
       
      • 
+      
    • 
+        The platform recipes
+      
      • 
       
    • 
         Install the LEAP command-line utility
       
      • 
@@ -139,6 +142,9 @@ Quick Start Tutorial - LEAP Platform Documentation
       
    • 
         Option B: Add a local node
       
      • 
+      
    • 
+        Option C: Add a virtual machine in the cloud
+      
      • 
     
   
   
    • 
@@ -197,7 +203,7 @@ Quick Start Tutorial - LEAP Platform Documentation
 
 
        
 
      1. A local Vagrant virtual machine: a Vagrant machine can only be useful for testing.
        2. 
-
      2. A real or paravirtualized server: The server must have Debian Jessie installed, and you must be able to SSH into the machine as root. Paravirtualization includes KVM, Xen, OpenStack, Amazon, but not VirtualBox or OpenVZ.
        3. 
+
      3. A real or paravirtualized server: The server must have Debian Jessie installed, and you must be able to SSH into the machine as root. Paravirtualization includes KVM, Xen, OpenStack, Amazon, but not VirtualBox or OpenVZ. Proxmox has an known issue when changing the resolver
        4. 
 
      
 
      • 
 
@@ -214,15 +220,20 @@ Quick Start Tutorial - LEAP Platform Documentation
 
 
      Prepare your workstation
      
 
-
      In order to be able to manage your servers, you need to install the leap command on your workstation:
      
+
      In order to be able to manage your servers, you need to setup the LEAP Platform on your desktop. This consists of three parts: the platform recipes, the leap command, and your provider instance. We will go over these step-by-step below, you can find more details in the platform introduction.
      
 
 
      Install pre-requisites
      
 
 
      Install core prerequisites on your workstation.
      
 
-
      Debian & Ubuntu
      
+
      Debian Unstable (sid)
      
+
+
      workstation$ sudo apt-get install git rsync openssh-client openssl zlib1g-dev
+
      
+
+
      Other Debian & Ubuntu
      
 
-
      workstation$ sudo apt-get install git ruby ruby-dev rsync openssh-client openssl rake make bzip2
+
      workstation$ sudo apt-get install git ruby ruby-dev rsync openssh-client openssl rake make bzip2 zlib1g-dev
 
      
 
 
      Mac OS
      
@@ -231,9 +242,38 @@ Quick Start Tutorial - LEAP Platform Documentation
 workstation$ ruby-install ruby
 
      
 
+
      The platform recipes
      
+
+
      The LEAP platform recipes are a set modules designed to work together to provide you everything you need to manage your provider. You typically do not need to modify these, but do need them available for deploying your provider.
      
+
+
      To obtain the platform recipes, simply clone the git repository, and then check out the most recent stable release branch:
      
+
+
      workstation$ git clone -b version/0.9.x https://leap.se/git/leap_platform
+
      
+
+
      If you want to get the latest development branch (Beware: it could be unstable !) you could simply use the master branch instead by:
      
+
+
      workstation$ git clone https://leap.se/git/leap_platform
+
      
+
 
      Install the LEAP command-line utility
      
 
-
      Install the leap command system-wide:
      
+
      The leap command line tool is what you use to manage everything about your provider.
      
+
+
      Keep these rules in mind:
      
+
+
        
+
      • leap is run on your workstation: The leap command is always run locally on your workstation, never on a server you are deploying to.
        • 
+
      • leap is run from within a provider instance: The leap command requires that the current working directory is a valid provider instance, except when running leap new to create a new provider instance.
        • 
+
      
+
+
+
      If on Debian Unstable (sid), simply do this:
      
+
+
      workstation$ sudo apt install leap-cli
+
      
+
+
      Otherwise, you will need to do this:
      
 
 
      workstation$ sudo gem install leap_cli
 
      
@@ -340,14 +380,18 @@ workstation$ leap cert csr
 
 
      Option B: Add a local node
      
 
-
      Create a node, with the services “webapp” and “couchdb”, and then start the local virtual machine:
      
+
      Create a node, with the services “webapp”, “soledad” and “couchdb”, and then start the local virtual machine:
      
 
-
      workstation$ leap node add --local wildebeest services:webapp,couchdb
+
      workstation$ leap node add --local wildebeest services:webapp,couchdb,soledad
 workstation$ leap local start wildebeest
 
      
 
 
      It will take a while to download the Virtualbox base box and create the virtual machine.
      
 
+
      Option C: Add a virtual machine in the cloud
      
+
+
      In order to create a provider using the cloud, please follow this instructions.
      
+
 
      Deploy your provider
      
 
 
      Initialize the node
      
diff --git a/docs/en/tutorials/quick-start/platform.html b/docs/en/tutorials/quick-start/platform.html
new file mode 100644
index 00000000..e69de29b
diff --git a/docs/en/tutorials/single-node-email.html b/docs/en/tutorials/single-node-email.html
index 6678fec3..d3372f91 100644
--- a/docs/en/tutorials/single-node-email.html
+++ b/docs/en/tutorials/single-node-email.html
@@ -144,11 +144,13 @@ Quick email - LEAP Platform Documentation
 
      In our example, we would edit nodes/wildebeest.json:
      
 
 
      {
-  "ip_address": "1.1.1.1",
+  "ip_address": "XXX.XXX.XXX.XXX",
   "services": ["couchdb", "webapp", "mx", "soledad"]
 }
 
      
 
+
      Where “XXX.XXX.XXX.XXX” should be replaced by your IP provider.
      
+
 
      Here, we added mx and soledad to the node’s services list. Briefly:
      
 
 
        
diff --git a/docs/en/tutorials/single-node-email/index.html b/docs/en/tutorials/single-node-email/index.html
index 45a1264f..fd501790 100644
--- a/docs/en/tutorials/single-node-email/index.html
+++ b/docs/en/tutorials/single-node-email/index.html
@@ -144,11 +144,13 @@ Quick email - LEAP Platform Documentation
 
        In our example, we would edit nodes/wildebeest.json:
        
 
 
        {
-  "ip_address": "1.1.1.1",
+  "ip_address": "XXX.XXX.XXX.XXX",
   "services": ["couchdb", "webapp", "mx", "soledad"]
 }
 
        
 
+
        Where “XXX.XXX.XXX.XXX” should be replaced by your IP provider.
        
+
 
        Here, we added mx and soledad to the node’s services list. Briefly:
        
 
 
          
diff --git a/docs/en/tutorials/vagrant.html b/docs/en/tutorials/vagrant.html
index 3d4f0520..e473ce82 100644
--- a/docs/en/tutorials/vagrant.html
+++ b/docs/en/tutorials/vagrant.html
@@ -437,12 +437,12 @@ $ leap local save web1
 
 
          Clone the platform with
          
 
-
          git clone --recursive -b develop https://github.com/leapcode/leap_platform.git
+
          git clone https://leap.se/git/leap_platform
 
          
 
 
          Start the vagrant box with
          
 
-
          cd leap_platform
+
          cd leap_platform/tests/example-provider
 vagrant up
 
          
 
@@ -531,7 +531,7 @@ started by the bitmask client:
          
 sudo apt-get install ruby-dev libxslt-dev libxml2-dev libvirt-dev
 
 # install the required plugins
-vagrant plugin install vagrant-libvirt fog fog-libvirt sahara
+vagrant plugin install vagrant-libvirt sahara
 
          
 
 
          Log out and then log back in.
          
@@ -585,8 +585,6 @@ virsh pool-autostart vagrant
 
        • Call to virConnectOpen failed: internal error: Unable to locate libvirtd daemon in /usr/sbin (to override, set $LIBVIRTD_PATH to the name of the libvirtd binary) - you don’t have the libvirtd daemon running or installed, be sure you installed the ‘libvirt-bin’ package and it is running
          • 
 
        • Call to virConnectOpen failed: Failed to connect socket to '/var/run/libvirt/libvirt-sock': Permission denied - you need to be in the libvirt group to access the socket, do ‘sudo adduser  libvirtd’ and then re-login to your session.
          • 
 
        • if each call to vagrant ends up with a segfault, it may be because you still have virtualbox around. if so, remove virtualbox to keep only libvirt + KVM. according to https://github.com/pradels/vagrant-libvirt/issues/75 having two virtualization engines installed simultaneously can lead to such weird issues.
          • 
-
        • see the vagrant-libvirt issue list on github
          • 
-
        • be sure to use vagrant-libvirt >= 0.0.11 and sahara >= 0.0.16 (which are the latest stable gems you would get with vagrant plugin install [vagrant-libvirt|sahara]) for proper libvirt support,
          • 
 
        
 
 
diff --git a/docs/en/tutorials/vagrant/index.html b/docs/en/tutorials/vagrant/index.html
index 95bd6b71..181a3ccf 100644
--- a/docs/en/tutorials/vagrant/index.html
+++ b/docs/en/tutorials/vagrant/index.html
@@ -437,12 +437,12 @@ $ leap local save web1
 
 
        Clone the platform with
        
 
-
        git clone --recursive -b develop https://github.com/leapcode/leap_platform.git
+
        git clone https://leap.se/git/leap_platform
 
        
 
 
        Start the vagrant box with
        
 
-
        cd leap_platform
+
        cd leap_platform/tests/example-provider
 vagrant up
 
        
 
@@ -531,7 +531,7 @@ started by the bitmask client:
        
 sudo apt-get install ruby-dev libxslt-dev libxml2-dev libvirt-dev
 
 # install the required plugins
-vagrant plugin install vagrant-libvirt fog fog-libvirt sahara
+vagrant plugin install vagrant-libvirt sahara
 
        
 
 
        Log out and then log back in.
        
@@ -585,8 +585,6 @@ virsh pool-autostart vagrant
 
      • Call to virConnectOpen failed: internal error: Unable to locate libvirtd daemon in /usr/sbin (to override, set $LIBVIRTD_PATH to the name of the libvirtd binary) - you don’t have the libvirtd daemon running or installed, be sure you installed the ‘libvirt-bin’ package and it is running
        • 
 
      • Call to virConnectOpen failed: Failed to connect socket to '/var/run/libvirt/libvirt-sock': Permission denied - you need to be in the libvirt group to access the socket, do ‘sudo adduser  libvirtd’ and then re-login to your session.
        • 
 
      • if each call to vagrant ends up with a segfault, it may be because you still have virtualbox around. if so, remove virtualbox to keep only libvirt + KVM. according to https://github.com/pradels/vagrant-libvirt/issues/75 having two virtualization engines installed simultaneously can lead to such weird issues.
        • 
-
      • see the vagrant-libvirt issue list on github
        • 
-
      • be sure to use vagrant-libvirt >= 0.0.11 and sahara >= 0.0.16 (which are the latest stable gems you would get with vagrant plugin install [vagrant-libvirt|sahara]) for proper libvirt support,
        • 
 
      
 
 
diff --git a/provider_base/common.json b/provider_base/common.json
index 7b412fe6..1052b8e1 100644
--- a/provider_base/common.json
+++ b/provider_base/common.json
@@ -72,12 +72,12 @@
     "nickserver": {
       "type": "git",
       "source": "https://leap.se/git/nickserver",
-      "revision": "origin/version/0.10"
+      "revision": "tags/0.10.0"
     },
     "platform": {
       "apt": {
         "basic": "http://deb.leap.se/platform",
-        "component": "master"
+        "component": "0.10"
       }
     },
     "soledad": {
@@ -88,7 +88,7 @@
     "webapp": {
       "type": "git",
       "source": "https://leap.se/git/leap_web",
-      "revision": "origin/version/0.9"
+      "revision": "tags/0.9.2"
     }
   }
 }
diff --git a/tests/platform-ci/ci-build.sh b/tests/platform-ci/ci-build.sh
index b2958f7c..9bdf75fb 100755
--- a/tests/platform-ci/ci-build.sh
+++ b/tests/platform-ci/ci-build.sh
@@ -239,6 +239,9 @@ upgrade_test() {
   build_from_scratch 'couchdb,soledad,mx,webapp,tor,monitor'
   deploy
   leap_info
+  # In 0.9 leap info did not output apt sources, so we do it manually
+  # but can remove it for next release
+  cat /etc/apt/sources.list.d/*
   test
 
   # Checkout HEAD of current branch and re-deploy
-- 
cgit v1.2.3


From 2d58cae32e87f94a1294265da547892fe21fe7a6 Mon Sep 17 00:00:00 2001
From: Micah Anderson 
Date: Thu, 23 Nov 2017 10:56:44 -0500
Subject: CI: set a contact email for CI tests

Because this is used in some places, such as tor configuration, it can be useful
to have this set to something real for contact purposes.
---
 tests/platform-ci/provider/provider.json | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/tests/platform-ci/provider/provider.json b/tests/platform-ci/provider/provider.json
index 218ff529..687f662a 100644
--- a/tests/platform-ci/provider/provider.json
+++ b/tests/platform-ci/provider/provider.json
@@ -10,7 +10,7 @@
     "en": "You really should change this text"
   },
   "contacts": {
-    "default": "root@example.org"
+    "default": "sysdevs@leap.se"
   },
   "languages": ["en"],
   "default_language": "en",
-- 
cgit v1.2.3


From 88a95ce8d6a3a151c20cbfa0ddfff7ca75f78751 Mon Sep 17 00:00:00 2001
From: elijah 
Date: Mon, 27 Nov 2017 10:52:45 -0800
Subject: fix tests - webapp test should only depend on soledad if soledad
 service is present

---
 tests/server-tests/white-box/webapp.rb | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/tests/server-tests/white-box/webapp.rb b/tests/server-tests/white-box/webapp.rb
index b0285e2e..42adf219 100644
--- a/tests/server-tests/white-box/webapp.rb
+++ b/tests/server-tests/white-box/webapp.rb
@@ -4,7 +4,7 @@ require 'json'
 
 class Webapp < LeapTest
   depends_on "Network"
-  depends_on "Soledad"
+  depends_on "Soledad" if service?(:soledad)
 
   def setup
   end
-- 
cgit v1.2.3


From bf6c56d86c7ba45e7ca766d990a9e9162025e5ac Mon Sep 17 00:00:00 2001
From: elijah 
Date: Mon, 27 Nov 2017 10:53:13 -0800
Subject: run_tests - use byebug instead of debugger

---
 bin/run_tests | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/bin/run_tests b/bin/run_tests
index 8450a9bf..241a3fa2 100755
--- a/bin/run_tests
+++ b/bin/run_tests
@@ -507,7 +507,7 @@ end
 
 if ARGV.include?('--debug') || ARGV.include?('-d')
   DEBUG=true
-  require 'debugger'
+  require 'byebug'
 else
   DEBUG=false
 end
-- 
cgit v1.2.3