diff options
110 files changed, 3308 insertions, 841 deletions
diff --git a/.gitmodules b/.gitmodules index 21966fc3..717ae5ed 100644 --- a/.gitmodules +++ b/.gitmodules @@ -39,7 +39,7 @@ url = git://code.leap.se/puppet_bundler [submodule "puppet/modules/vcsrepo"] path = puppet/modules/vcsrepo - url = git://github.com/puppetlabs/puppetlabs-vcsrepo.git + url = git://labs.riseup.net/module_vcs [submodule "puppet/modules/rubygems"] path = puppet/modules/rubygems url = git://code.leap.se/puppet_rubygems @@ -67,3 +67,9 @@ [submodule "puppet/modules/tor"] path = puppet/modules/tor url = git://labs.riseup.net/shared-tor +[submodule "puppet/modules/stunnel"] + path = puppet/modules/stunnel + url = git://code.leap.se/puppet_stunnel +[submodule "puppet/modules/haproxy"] + path = puppet/modules/haproxy + url = git://code.leap.se/puppet_haproxy @@ -5,44 +5,43 @@ Leap Platform What is it? =========== -The LEAP Provider Platform is the server-side part of the LEAP Encryption Access Project that is run by service providers. It consists of a set of complementary modules and recipes to automate the maintenance of LEAP services in a hardened GNU/Linux environment. LEAP makes it easy and straightforward for service providers and ISPs to deploy a secure communications platform for their users. +The LEAP Platform is set of complementary packages and server recipes to automate the maintenance of LEAP services in a hardened Debian environment. Its goal is to make it as painless as possible for sysadmins to deploy and maintain a service provider’s infrastructure for secure communication. These recipes define an abstract service provider. It is a set of Puppet modules designed to work together to provide to sysadmins everything they need to manage a service provider infrastructure that provides secure communication services. -The LEAP Platform is essentially a git repository of puppet recipes, with a few scripts to help with bootstrapping and deployment. A service provider who wants to deploy LEAP services will clone or fork this repository, edit the main configuration file to specify which services should run on which hosts, and run scripts to deploy this configuration. +As these recipes consist of abstract definitions, in order to configure settings for a particular service provider a system administrator has to obtain the leap command-line interface and create a provider instance. The details of how to get started are contained in the `Quick Start` documentation as detailed below. -Documentation -============= -Most of the current documentation can be found in Readme files of the different pieces. This will be consolidated on the website https://leap.se soon. +Getting started +=============== + +It is highly recommended that you start by reading the overview of the Leap Platform on the website (https://leap.se/docs/platform) and then begin with the `Quick Start` guide (https://leap.se/docs/platform/quick-start) to walk through a test environment setup to get familiar with how things work before deploying to live servers. + +An offline copy of this documentation is contained in the `doc` subdirectory. For more current updates to the documentation, visit the website. Requirements -============ +------------ -This highly depends on your (expected) user base. -For a minimal test or develop install we recommend a fairly recent computer x86_64 with hardware virtualization features (AMD-V or VT-x) with plenty of RAM. -You could use Vagrant or KVM to simulate a live deployment. +For a minimal test or develop install we recommend a fairly recent computer x86_64 with hardware virtualization features (AMD-V or VT-x) with plenty of RAM. If you follow the `Quick Start` documentation we will walk you through using Vagrant to setup a test deployment. -For a live deployment of the platform the amount of required (virtual) servers depends on your needs and which services you want to deploy. -In it's initial release you can deploy Tor, OpenVPN, CouchDB and a webapp to administer your users (billing, help tickets,...). -While you can deploy all services on one server, we stronly recommend to use seperate servers for better security. +For a live deployment of the platform the amount of required (virtual) servers depends on your needs and which services you want to deploy. At the moment, the Leap Platform supports servers with a base Debian Wheezy installation. +While you can deploy all services on one server, we stronly recommend to use seperate servers for better security. -Usage -===== -As mentioned above, Leap Platform are the server-side Puppet manifests, for deploying a service provider, you need the leap command line interface, -available here: https://github.com/leapcode/leap_cli +Troubleshooting +=============== -We strongly recommend to follow the `Quick Start` Documentaion which can be found on the website https://leap.se +If you have a problem, we are interested in fixing it! +If you have a problem, be sure to have a look at the Known Issues section of the documentation to see if your issue is detailed there. -Clone leap_platform and its submodules --------------------------------------- +If not, the best way for us to solve your problem is if you provide to us the complete log of what you did, and the output that was produced. Please don't cut out what appears to be useless information and only include the error that you received, instead copy and paste the complete log so that we can better determine the overall situation. If you can run the same command that produced the error with a raised verbosity level (such as -v2), that provides us with more useful debugging information. - git checkout develop +Visit https://leap.se/development for contact possibilities. -Initialize Submodules: +Known Issues +------------ - git submodule update --init +* Please read the section in the documentation about Known Issues (https://leap.se/docs/platform/known-issues) More Information @@ -50,23 +49,6 @@ More Information For more information about the LEAP Encryption Access Project, please visit the website https://leap.se which also lists contact data. - -Copyright/License ------------------ - -Read LICENSE - - -Known bugs ----------- - -* currently none known, there will probably be some around ! - -Troubleshooting ---------------- - -Visit https://leap.se/en/development for contact possibilities. - Changelog --------- @@ -81,3 +63,11 @@ See contributors: git shortlog -es --all + +Copyright/License +----------------- + +Read LICENSE + + + diff --git a/bin/puppet_command b/bin/puppet_command new file mode 100755 index 00000000..a6cd5a69 --- /dev/null +++ b/bin/puppet_command @@ -0,0 +1,191 @@ +#!/usr/bin/ruby + +# +# This is a wrapper script around the puppet command used by the LEAP platform. +# +# We do this in order to make it faster and easier to control puppet remotely +# (exit codes, lockfile, multiple manifests, etc) +# + +PUPPET_BIN = '/usr/bin/puppet' +PUPPET_DIRECTORY = '/srv/leap' +PUPPET_PARAMETERS = '--color=false --detailed-exitcodes --libdir=puppet/lib --confdir=puppet' +SITE_MANIFEST = 'puppet/manifests/site.pp' +SETUP_MANIFEST = 'puppet/manifests/setup.pp' +DEFAULT_TAGS = 'leap_base,leap_service' + +def main + process_command_line_arguments + with_lockfile do + @commands.each do |command| + self.send(command) + end + end +end + +def puts(str) + $stdout.puts str + $stdout.flush +end + +def process_command_line_arguments + @commands = [] + @verbosity = 1 + @tags = DEFAULT_TAGS + loop do + case ARGV[0] + when 'apply' then ARGV.shift; @commands << 'apply' + when 'set_hostname' then ARGV.shift; @commands << 'set_hostname' + when '--verbosity' then ARGV.shift; @verbosity = ARGV.shift.to_i + when '--force' then ARGV.shift; remove_lockfile + when '--tags' then ARGV.shift; @tags = ARGV.shift + when /^-/ then usage("Unknown option: #{ARGV[0].inspect}") + else break + end + end + usage("No command given") unless @commands.any? +end + +def apply + exit_code = puppet_apply do |line| + puts line + end + puts "Puppet apply complete (#{exitcode_description(exit_code)})." +end + +def set_hostname + exit_code = puppet_apply(:manifest => SETUP_MANIFEST, :tags => '') do |line| + # todo: replace setup.pp with https://github.com/lutter/ruby-augeas + # or try this: http://www.puppetcookbook.com/posts/override-a-facter-fact.html + if (line !~ /Finished catalog run/ || @verbosity > 2) && + (line !~ /dnsdomainname: Name or service not known/) && + (line !~ /warning: Could not retrieve fact fqdn/) + puts line + end + end + if exit_code == 2 + puts "Hostname updated." + elsif exit_code == 4 || exit_code == 6 + puts "ERROR: could not update hostname." + elsif exit_code == 0 && @verbosity > 1 + puts "No change to hostname." + end +end + +# +# each line of output is yielded. the exit code is returned. +# +def puppet_apply(options={}, &block) + options = {:verbosity => @verbosity, :tags => @tags}.merge(options) + manifest = options[:manifest] || SITE_MANIFEST + Dir.chdir(PUPPET_DIRECTORY) do + return run("#{PUPPET_BIN} apply #{custom_parameters(options)} #{PUPPET_PARAMETERS} #{manifest}", &block) + end +end + +def custom_parameters(options) + params = [] + if options[:tags] && options[:tags].chars.any? + params << "--tags #{options[:tags]}" + end + if options[:verbosity] + case options[:verbosity] + when 3 then params << '--verbose' + when 4 then params << '--verbose --debug' + when 5 then params << '--verbose --debug --trace' + end + end + params.join(' ') +end + +def exitcode_description(code) + case code + when 0 then "no changes" + when 1 then "failed" + when 2 then "changes made" + when 4 then "failed" + when 6 then "changes and failures" + else code + end +end + +def usage(s) + $stderr.puts(s) + $stderr.puts + $stderr.puts("Usage: #{File.basename($0)} COMMAND [OPTIONS]") + $stderr.puts + $stderr.puts("COMMAND may be one or more of: + set_hostname -- set the hostname of this server + apply -- apply puppet manifests") + $stderr.puts + $stderr.puts("OPTIONS may be one or more of: + --verbosity VERB -- set the verbosity level 0..5 + --tags TAGS -- set the tags to pass through to puppet + --force -- run even when lockfile is present") + exit(2) +end + +## +## Simple lock file +## + +require 'fileutils' +DEFAULT_LOCKFILE = '/tmp/puppet.lock' + +def remove_lockfile(lock_file_path=DEFAULT_LOCKFILE) + FileUtils.remove_file(lock_file_path, true) +end + +def with_lockfile(lock_file_path=DEFAULT_LOCKFILE) + begin + File.open(lock_file_path, File::CREAT | File::EXCL | File::WRONLY) do |o| + o.write(Process.pid) + end + yield + remove_lockfile + rescue Errno::EEXIST + puts("ERROR: the lock file '#{lock_file_path}' already exists. Wait a minute for the process to die, or run with --force to ignore. Bailing out.") + exit(1) + rescue IOError => exc + puts("ERROR: problem with lock file '#{lock_file_path}' (#{exc}). Bailing out.") + exit(1) + end +end + +## +## simple pass through process runner (to ensure output is not buffered and return exit code) +## this only works under ruby 1.9 +## + +require "pty" + +def run(cmd) + puts cmd if @verbosity >= 3 + PTY.spawn("#{cmd}") do |output, input, pid| + begin + while line = output.gets do + yield line + #$stdout.puts line + #$stdout.flush + end + rescue Errno::EIO + end + Process.wait(pid) # only works in ruby 1.9, required to capture the exit status. + end + status = $?.exitstatus + #yield status if block_given? + return status +rescue PTY::ChildExited +end + +## +## RUN MAIN +## + +Signal.trap("EXIT") do + remove_lockfile # clean up the lockfile when process is terminated. + # this will remove the lockfile if ^C killed the process + # but only after the child puppet process is also dead (I think). +end + +main()
\ No newline at end of file diff --git a/doc/commands.md b/doc/commands.md new file mode 100644 index 00000000..b176541f --- /dev/null +++ b/doc/commands.md @@ -0,0 +1,285 @@ +@title = 'Command Line Reference' + +The command "leap" can be used to manage a bevy of servers running the LEAP platform from the comfort of your own home. + + +# Global Options + +* `--log FILE` +Override default log file +Default Value: None + +* `-v|--verbose LEVEL` +Verbosity level 0..2 +Default Value: 1 + +* `--help` +Show this message + +* `--version` +Display version number and exit + +* `--yes` +Skip prompts and assume "yes" + + +# leap add-user USERNAME + +Adds a new trusted sysadmin + + + +**Options** + +* `--pgp-pub-key arg` +OpenPGP public key file for this new user +Default Value: None + +* `--ssh-pub-key arg` +SSH public key file for this new user +Default Value: None + +* `--self` +lets you choose among your public keys + + +# leap cert + +Manage X.509 certificates + + + +## leap cert ca + +Creates two Certificate Authorities (one for validating servers and one for validating clients). + +See see what values are used in the generation of the certificates (like name and key size), run `leap inspect provider` and look for the "ca" property. To see the details of the created certs, run `leap inspect <file>`. + +## leap cert csr + +Creates a CSR for use in buying a commercial X.509 certificate. + +The CSR created is for the for the provider's primary domain. The properties used for this CSR come from `provider.ca.server_certificates`. + +## leap cert dh + +Creates a Diffie-Hellman parameter file. + + + +## leap cert update <node-filter> + +Creates or renews a X.509 certificate/key pair for a single node or all nodes, but only if needed. + +This command will a generate new certificate for a node if some value in the node has changed that is included in the certificate (like hostname or IP address), or if the old certificate will be expiring soon. Sometimes, you might want to force the generation of a new certificate, such as in the cases where you have changed a CA parameter for server certificates, like bit size or digest hash. In this case, use --force. If <node-filter> is empty, this command will apply to all nodes. + +**Options** + +* `--force` +Always generate new certificates + + +# leap clean + +Removes all files generated with the "compile" command. + + + +# leap compile + +Compiles node configuration files into hiera files used for deployment. + + + +# leap deploy FILTER + +Apply recipes to a node or set of nodes. + +The FILTER can be the name of a node, service, or tag. + +**Options** + +* `--tags TAG[,TAG]` +Specify tags to pass through to puppet (overriding the default). +Default Value: leap_base,leap_service + +* `--fast` +Makes the deploy command faster by skipping some slow steps. A "fast" deploy can be used safely if you recently completed a normal deploy. + + +# leap help command + +Shows a list of commands or help for one command + +Gets help for the application or its commands. Can also list the commands in a way helpful to creating a bash-style completion function + +**Options** + +* `-c` +List commands one per line, to assist with shell completion + + +# leap inspect FILE + +Prints details about a file. Alternately, the argument FILE can be the name of a node, service or tag. + + + +# leap list [FILTER] + +List nodes and their classifications + +Prints out a listing of nodes, services, or tags. If present, the FILTER can be a list of names of nodes, services, or tags. If the name is prefixed with +, this acts like an AND condition. For example: + +`leap list node1 node2` matches all nodes named "node1" OR "node2" + +`leap list openvpn +local` matches all nodes with service "openvpn" AND tag "local" + +**Options** + +* `--print arg` +What attributes to print (optional) +Default Value: None + + +# leap local + +Manage local virtual machines. + +This command provides a convient way to manage Vagrant-based virtual machines. If FILTER argument is missing, the command runs on all local virtual machines. The Vagrantfile is automatically generated in 'test/Vagrantfile'. If you want to run vagrant commands manually, cd to 'test'. + +## leap local destroy [FILTER] + +Destroys the virtual machine(s), reclaiming the disk space + + + +## leap local reset [FILTER] + +Resets virtual machine(s) to the last saved snapshot + + + +## leap local save [FILTER] + +Saves the current state of the virtual machine as a new snapshot + + + +## leap local start [FILTER] + +Starts up the virtual machine(s) + + + +## leap local status [FILTER] + +Print the status of local virtual machine(s) + + + +## leap local stop [FILTER] + +Shuts down the virtual machine(s) + + + +# leap new DIRECTORY + +Creates a new provider instance in the specified directory, creating it if necessary. + + + +**Options** + +* `--contacts arg` +Default email address contacts. +Default Value: None + +* `--domain arg` +The primary domain of the provider. +Default Value: None + +* `--name arg` +The name of the provider. +Default Value: None + +* `--platform arg` +File path of the leap_platform directory. +Default Value: None + + +# leap node + +Node management + + + +## leap node add NAME [SEED] + +Create a new configuration file for a node named NAME. + +If specified, the optional argument SEED can be used to seed values in the node configuration file. + +The format is property_name:value. + +For example: `leap node add web1 ip_address:1.2.3.4 services:webapp`. + +To set nested properties, property name can contain '.', like so: `leap node add web1 ssh.port:44` + +Separeate multiple values for a single property with a comma, like so: `leap node add mynode services:webapp,dns` + +**Options** + +* `--local` +Make a local testing node (by automatically assigning the next available local IP address). Local nodes are run as virtual machines on your computer. + + +## leap node init FILTER + +Bootstraps a node or nodes, setting up SSH keys and installing prerequisite packages + +This command prepares a server to be used with the LEAP Platform by saving the server's SSH host key, copying the authorized_keys file, and installing packages that are required for deploying. Node init must be run before deploying to a server, and the server must be running and available via the network. This command only needs to be run once, but there is no harm in running it multiple times. + +**Options** + +* `--echo` +If set, passwords are visible as you type them (default is hidden) + + +## leap node mv OLD_NAME NEW_NAME + +Renames a node file, and all its related files. + + + +## leap node rm NAME + +Removes all the files related to the node named NAME. + + + +# leap ssh NAME + +Log in to the specified node with an interactive shell. + + + +# leap test + +Run tests. + + + +## leap test init + +Creates files needed to run tests. + + + +## leap test run + +Run tests. + + +Default Command: run diff --git a/doc/config.md b/doc/config.md new file mode 100644 index 00000000..d0b1f6a7 --- /dev/null +++ b/doc/config.md @@ -0,0 +1,229 @@ +@title = "Configuration Files" + +Leapfile +------------------------------------------- + +A `Leapfile` defines options for the `leap` command and lives at the root of your provider directory. `Leapfile` is evaluated as ruby, so you can include whatever weird logic you want in this file. In particular, there are several variables you can set that modify the behavior of leap. For example: + + @platform_directory_path = '../leap_platform' + @log = '/var/log/leap.log' + +Additionally, you can create a `~/.leaprc` file that is loaded after `Leapfile` and is evaluated the same way. + +Platform options: + +* `@platform_directory_path` (required). This must be set to the path where `leap_platform` lives. The path may be relative. +* `@platform_branch`. If set, a check is preformed before running any command to ensure that the currently checked out branch of `leap_platform` matches the value set for `@platform_branch`. This is useful if you have a stable branch of your provider that you want to ensure runs off the master branch of `leap_platform`. +* `@allow_production_deploy`. By default, you can only deploy to production nodes if the current branch is 'master' or if the provider directory is not a git repository. This option allows you to override this behavior. + +Vagrant options: + +* `@vagrant_network`. Allows you to override the default network used for local nodes. It should include a netmask like `@vagrant_network = '10.0.0.0/24'`. +* `@custom_vagrant_vm_line`. Insert arbitrary text into the auto-generated Vagrantfile. For example, `@custom_vagrant_vm_line = "config.vm.boot_mode = :gui"`. + +Logging options: + +* `@log`. If set, all command invocation and results are logged to the specified file. This is the same as the switch `--log FILE`, except that the command line switch will override the value in the Leapfile. + + +Configuration files +------------------------------------------- + +All configuration files, other than `Leapfile`, are in the JSON format. For example: + + { + "key1": "value1", + "key2": "value2" + } + +Keys should match `/[a-z0-9_]/` + +Unlike traditional JSON, comments are allowed. If the first non-whitespace characters are `//` then the line is treated as a comment. + + // this is a comment + { + // this is a comment + "key": "value" // this is an error + } + +Options in the configuration files might be nested hashes, arrays, numbers, strings, or boolean. Numbers and boolean values should **not** be quoted. For example: + + { + "openvpn": { + "ip_address": "1.1.1.1", + "protocols": ["tcp", "udp"], + "ports": [80, 53], + "options": { + "public_ip": false, + "adblock": true + } + } + } + +If the value string is prefixed with an '=' character, the result is evaluated as ruby. For example: + + { + "domain": { + "public": "domain.org" + } + "api_domain": "= 'api.' + domain.public" + } + +In this case, the property "api_domain" will be set to "api.domain.org". So long as you do not create unresolvable circular dependencies, you can reference other properties in evaluated ruby that are themselves evaluated ruby. + +See "Macros" below for information on the special macros available to the evaluated ruby. + +TIP: In rare cases, you might want to force the evaluation of a value to happen in a later pass after most of the other properties have been evaluated. To do this, prefix the value string with "=>" instead of "=". + +Node inheritance +---------------------------------------- + +Every node inherits from common.json and also any of the services or tags attached to the node. Additionally, the `leap_platform` contains a directory `provider_base` that defines the default values for tags, services and common.json. + +Suppose you have a node configuration for `bitmask/nodes/willamette.json` like so: + + { + "services": "webapp", + "tags": ["production", "northwest-us"], + "ip_address": "1.1.1.1" + } + +This node will have hostname "willamette" and it will inherit from the following files (in this order): + +1. common.json + - load defaults: `provider_base/common.json` + - load provider: `bitmask/common.json` +2. service "webapp" + - load defaults: `provider_base/services/webapp.json` + - load provider: `bitmask/services/webapp.json` +3. tag "production" + - load defaults: `provider_base/tags/production.json` + - load provider: `bitmask/tags/production.json` +4. tag "northwest-us" + - load: `bitmask/tags/northwest-us.json` +5. finally, load node "willamette" + - load: `bitmask/nodes/willamette.json` + +The `provider_base` directory is under the `leap_platform` specified in the file `Leapfile`. + +To see all the variables a node has inherited, you could run `leap inspect willamette`. + +Common configuration options +---------------------------------------- + +You can use the command `leap inspect` to see what options are available for a provider, node, service, or tag configuration. For example: + +* `leap inspect common` -- show the options inherited by all nodes. +* `leap inspect --base common` -- show the common.json from `provider_base` without the local `common.json` inheritance applied. +* `leap inspect webapp` -- show all the options available for the service `webapp`. + +Here are some of the more important options you should be aware of: + +* `ip_address` -- Required for all nodes, no default. +* `ssh.port` -- The SSH port you want the node's OpenSSH server to bind to. This is also the default when trying to connect to a node, but if the node currently has OpenSSH running on a different port then run deploy with `--port` to override the `ssh.port` configuration value. +* `mosh.enabled` -- If set to `true`, then mosh will be installed on the server. The default is `false`. + +Macros +---------------------------------------- + +When using evaluated ruby in a JSON configuration file, there are several special macros that are available. These are evaluated in the context of a node (available as the variable `self`). + +The following methods are available to the evaluated ruby: + +`variable.variable` + + > Any variable defined or inherited by a particular node configuration is available by just referencing it using either hash notation or object field notation (e.g. `['domain']['public']` or `domain.public`). Circular references are not allowed, but otherwise it is OK to nest evaluated values in other evaluated values. If a value has not been defined, the hash notation will return nil but the field notation will raise an exception. Properties of services, tags, and the global provider can all be referenced the same way. For example, `global.services['openvpn'].x509.dh`. + +`nodes` + + > A hash of all nodes. This list can be filtered. + +`nodes_like_me` + + > A hash of nodes that have the same deployment tags as the current node (e.g. 'production' or 'local'). + +`global.services` + + > A hash of all services, e.g. `global.services['openvpn']` would return the "openvpn" service. + +`global.tags` + + > A hash of all tags, e.g. `global.tags['production']` would return the "production" tag. + + `global.provider` + + > Can be used to access variables defined in `provider.json`, e.g. `global.provider.contacts.default`. + +`file(filename)` + + > Inserts the full contents of the file. If the file is an erb template, it is rendered. The filename can either be one of the pre-defined file symbols, or it can be a path relative to the "files" directory in your provider instance. E.g, `file :ca_cert` or `files 'ca/ca.crt'`. + +`file_path(filename)` + + > Ensures that the file will get rsynced to the node as an individual file. The value returned by `file_path` is the full path where this file will ultimately live when deploy to the node. e.g. `file_path :ca_cert` or `file_path 'branding/images/logo.png'`. + +`secret(:symbol)` + + > Returns the value of a secret in secrets.json (or creates it if necessary). E.g. `secret :couch_admin_password` + +`hosts_file` + + > Returns a data structure that puppet will use to generate /etc/hosts. Care is taken to use the local IP of other hosts when needed. + +`known_hosts_file` + + > Returns the lines needed in a SSH `known_hosts` file. + +`stunnel_client(node_list, port, options={})` + + > Returns a stunnel configuration data structure for the client side. Argument `node_list` is an `ObjectList` of nodes running stunnel servers. Argument `port` is the real port of the ultimate service running on the servers that the client wants to connect to. + +`stunnel_server(port)` + + > Generates a stunnel server entry. The `port` is the real port targeted service. + +Hash tables +----------------------------------------- + +The macros `nodes`, `nodes_like_me`, `global.services`, and `global.tags` all return a hash table of configuration objects (either nodes, services, or tags). There are several ways to filter and process these hash tables: + +Access an element by name: + + nodes['vpn1'] # returns node named 'vpn1' + global.services['openvpn'] # returns service named 'openvpn' + +Create a new hash table by applying filters: + + nodes[:public_dns => true] # all nodes where public_dns == true + nodes[:services => 'openvpn', :services => 'tor'] # openvpn OR tor + nodes[:services => 'openvpn'][:tags => 'production'] # openvpn AND production + nodes[:name => "!bob"] # all nodes that are NOT named "bob" + +Create an array of values by selecting a single field: + + nodes.field('location.name') + ==> ['seattle', 'istanbul'] + +Create an array of hashes by selecting multiple fields: + + nodes.fields('domain.full', 'ip_address') + ==> [ + {'domain_full' => 'red.bitmask.net', 'ip_address' => '1.1.1.1'}, + {'domain_full' => 'blue.bitmask.net', 'ip_address' => '1.1.1.2'}, + ] + +Create a new hash table of hashes, with only certain fields: + + nodes.pick_fields('domain.full', 'ip_address') + ==> { + "red" => {'domain_full' => 'red.bitmask.net', 'ip_address' => '1.1.1.1'}, + "blue => {'domain_full' => 'blue.bitmask.net', 'ip_address' => '1.1.1.2'}, + } + +With `pick_fields`, if there is only one field, it will generate a simple hash table: + + nodes.pick_fields('ip_address') + ==> { + "red" => '1.1.1.1', + "blue => '1.1.1.2', + } diff --git a/doc/en.md b/doc/en.md new file mode 100644 index 00000000..bdae4630 --- /dev/null +++ b/doc/en.md @@ -0,0 +1,77 @@ +@title = 'LEAP Platform for Service Providers' +@nav_title = 'Provider Platform' +@summary = 'Software platform to automate the process of running a communication service provider.' +@toc = true + +The *LEAP Platform* is set of complementary packages and server recipes to automate the maintenance of LEAP services in a hardened Debian environment. Its goal is to make it as painless as possible for sysadmins to deploy and maintain a service provider's infrastructure for secure communication. + +The LEAP Platform consists of three parts, detailed below: + +1. The platform recipes. +2. The provider instance. +3. The `leap` command line tool. + +The platform recipes +-------------------- + +The LEAP platform recipes define an abstract service provider. It is a set of [Puppet](https://puppetlabs.com/puppet/puppet-open-source/) modules designed to work together to provide to sysadmins everything they need to manage a service provider infrastructure that provides secure communication services. + +LEAP maintains a repository of platform recipes, which typically do not need to be modified, although it can be forked and merged as desired. Most service providers using the LEAP platform can use the same set of platform recipes. + +As these recipes consist in abstract definitions, in order to configure settings for a particular service provider a system administrator has to create a provider instance (see below). + +LEAP's platform recipes are distributed as a git repository: `git://leap.se/leap_platform.git` + +The provider instance +--------------------- + +A provider instance is a directory tree (typically tracked in git) containing all the configurations for a service provider's infrastructure. A provider instance primarily consists of: + +* A pointer to the platform recipes. +* A global configuration file for the provider. +* A configuration file for each server (node) in the provider's infrastructure. +* Additional files, such as certificates and keys. + +A minimal provider instance directory looks like this: + + └── bitmask # provider instance directory. + ├── Leapfile # settings for the `leap` command line tool. + ├── provider.json # global settings of the provider. + ├── common.json # settings common to all nodes. + ├── nodes/ # a directory for node configurations. + ├── files/ # keys, certificates, and other files. + └── users/ # public key information for privileged sysadmins. + + +A provider instance directory contains everything needed to manage all the servers that compose a provider's infrastructure. Because of this, any versioning tool and development work-flow can be used to manage your provider instance. + +The `leap` command line tool +---------------------------- + +The `leap` [command line tool](commands) is used by sysadmins to manage everything about a service provider's infrastructure. Except when creating an new provider instance, `leap` is run from within the directory tree of a provider instance. + +The `leap` command line has many capabilities, including: + +* Create, initialize, and deploy nodes. +* Manage keys and certificates. +* Query information about the node configurations. + +Traditional system configuration automation systems, like [Puppet](https://puppetlabs.com/puppet/puppet-open-source/) or [Chef](http://www.opscode.com/chef/), deploy changes to servers using a pull method. Each server pulls a manifest from a central master server and uses this to alter the state of the server. + +Instead, the `leap` tool uses a masterless push method: The sysadmin runs `leap deploy` from the provider instance directory on their desktop machine to push the changes out to every server (or a subset of servers). LEAP still uses Puppet, but there is no central master server that each node must pull from. + +One other significant difference between LEAP and typical system automation is how interactions among servers are handled. Rather than store a central database of information about each server that can be queried when a recipe is applied, the `leap` command compiles static representation of all the information a particular server will need in order to apply the recipes. In compiling this static representation, `leap` can use arbitrary programming logic to query and manipulate information about other servers. + +These two approaches, masterless push and pre-compiled static configuration, allow the sysadmin to manage a set of LEAP servers using traditional software development techniques of branching and merging, to more easily create local testing environments using virtual servers, and to deploy without the added complexity and failure potential of a master server. + +The `leap` command line tool is distributed as a git repository: `git://leap.se/leap_cli`. It can be installed with `sudo gem install leap_cli`. + +Getting started +---------------------------------- + +We recommend reading the platform documentation in the following order: + +1. [Quick start tutorial](platform/quick-start). +2. [Platform Guide](platform/guide). +3. [Configuration format](platform/config). +4. The `leap` [command reference](platform/commands). diff --git a/doc/guide.md b/doc/guide.md new file mode 100644 index 00000000..dae392e5 --- /dev/null +++ b/doc/guide.md @@ -0,0 +1,257 @@ +@title = "LEAP Platform Guide" +@nav_title = "Guide" + +Services +================================ + +Every node has one or more services that determines the node's function within your provider's infrastructure. + +When adding a new node to your provider, you should ask yourself four questions: + +* **many or few?** Some services benefit from having many nodes, while some services are best run on only one or two nodes. +* **required or optional?** Some services are required, while others can be left out. +* **who does the node communicate with?** Some services communicate very heavily with other particular services. Nodes running these services should be close together. +* **public or private?** Some services communicate with the public internet, while others only need to communicate with other nodes in the infrastructure. + +Brief overview of the services: + + + +* **webapp**: The web application. Runs both webapp control panel for users and admins as well as the REST API that the client uses. Needs to communicate heavily with `couchdb` nodes. You need at least one, good to have two for redundancy. The webapp does not get a lot of traffic, so you will not need many. +* **couchdb**: The database for users and user data. You can get away with just one, but for proper redundancy you should have at least three. Communicates heavily with `webapp` and `mx` nodes. +* **soledad**: Handles the data syncing with clients. Typically combined with `couchdb` service, since it communicates heavily with couchdb. (not currently in stable release) +* **mx**: Incoming and outgoing MX servers. Communicates with the public internet, clients, and `couchdb` nodes. (not currently in stable release) +* **openvpn**: OpenVPN gateway for clients. You need at least one, but want as many as needed to support the bandwidth your users are doing. The `openvpn` nodes are autonomous and don't need to communicate with any other nodes. Often combined with `tor` service. + +Not pictured: + +* **monitor**: Internal service to monitor all the other nodes. Currently, you can have zero or one `monitor` nodes. +* **tor**: Sets up a tor exit node, unconnected to any other service. +* **dns**: Not yet implemented. + +Locations +================================ + +All nodes should have a `location.name` specified, and optionally additional information about the location, like the time zone. This location information is used for two things: + +* Determine which nodes can, or must, communicate with one another via a local network. The way some virtualization environments work, like OpenStack, requires that nodes communicate via the local network if they are on the same network. +* Allows the client to prefer connections to nodes that are closer in physical proximity to the user. This is particularly important for OpenVPN nodes. + +The location stanza in a node's config file looks like this: + + { + "location": { + "id": "ankara", + "name": "Ankara", + "country_code": "TR", + "timezone": "+2", + "hemisphere": "N" + } + } + +The fields: + +* `id`: An internal handle to use for this location. If two nodes have match `location.id`, then they are treated as being on a local network with one another. This value defaults to downcase and underscore of `location.name`. +* `name`: Can be anything, might be displayed to the user in the client if they choose to manually select a gateway. +* `country_code`: The [ISO 3166-1](https://en.wikipedia.org/wiki/ISO_3166-1) two letter country code. +* `timezone`: The timezone expressed as an offset from UTC (in standard time, not daylight savings). You can look up the timezone using this [handy map](http://www.timeanddate.com/time/map/). +* `hemisphere`: This should be "S" for all servers in South America, Africa, or Australia. Otherwise, this should be "N". + +These location options are very imprecise, but good enough for most usage. The client often does not know its own location precisely either. Instead, the client makes an educated guess at location based on the OS's timezone and locale. + +If you have multiple nodes in a single location, it is best to use a tag for the location. For example: + +`tags/ankara.json`: + + { + "location": { + "name": "Ankara", + "country_code": "TR", + "timezone": "+2", + "hemisphere": "N" + } + } + +`nodes/vpngateway.json`: + + { + "services": "openvpn", + "tags": ["production", "ankara"], + "ip_address": "1.1.1.1", + "openvpn": { + "gateway_address": "1.1.1.2" + } + } + +Unless you are using OpenStack or AWS, setting `location` for nodes is not required. It is, however, highly recommended. + +Working with SSH +================================ + +Whenever the `leap` command nees to push changes to a node or gather information from a node, it tunnels this command over SSH. Another way to put this: the security of your servers rests entirely on SSH. Because of this, it is important that you understand how `leap` uses SSH. + +SSH related files +------------------------------- + +Assuming your provider directory is called 'provider': + +* `provider/nodes/crow/crow_ssh.pub` -- The public SSH host key for node 'crow'. +* `provider/users/alice/alice_ssh.pub` -- The public SSH user key for user 'alice'. Anyone with the private key that corresponds to this public key will have root access to all nodes. +* `provider/files/ssh/known_hosts` -- An autogenerated known_hosts, built from combining `provider/nodes/*/*_ssh.pub`. You must not edit this file directly. If you need to change it, remove or change one of the files that is used to generate `known_hosts` and then run `leap compile`. +* `provider/files/ssh/authorized_keys` -- An autogenerated list of all the user SSH keys with root access to the notes. It is created from `provider/users/*/*_ssh.pub`. You must not edit this file directly. If you need to change it, remove or change one of the files that is used to generate `authorized_keys` and then run `leap compile`. + +All of these files should be committed to source control. + +If you rename, remove, or add a node with `leap node [mv|add|rm]` the SSH key files and the `known_hosts` file will get properly updated. + +SSH and local nodes +----------------------------- + +Local nodes are run as Vagrant virtual machines. The `leap` command handles SSH slightly differently for these nodes. + +Basically, all the SSH security is turned off for local nodes. Since local nodes only exist for a short time on your computer and can't be reached from the internet, this is not a problem. + +Specifically, for local nodes: + +1. `known_hosts` is never updated with local node keys, since the SSH public key of a local node is different for each user. +2. `leap` entirely skips the checking of host keys when connecting with a local node. +3. `leap` adds the public Vagrant SSH key to the list of SSH keys for a user. The public Vagrant SSH key is a shared and insecure key that has root access to most Vagrant virtual machines. + +When SSH host key changes +------------------------------- + +If the host key for a node has changed, you will get an error "WARNING: REMOTE HOST IDENTIFICATION HAS CHANGED". + +To fix this, you need to remove the file `files/nodes/stompy/stompy_ssh.pub` and run `leap node init stompy`, where the node's name is 'stompy'. **Only do this if you are ABSOLUTELY CERTAIN that the node's SSH host key has changed**. + +Changing the SSH port +-------------------------------- + +Suppose you have a node `blinky` that has SSH listening on port 22 and you want to make it port 2200. + +First, modify the configuration for `blinky` to specify the variable `ssh.port` as 2200. Usually, this is done in `common.json` or in a tag file. + +For example, you could put this in `tags/production.json`: + + { + "ssh": { + "port": 2200 + } + } + +Run `leap compile` and open `hiera/blinky.yaml` to confirm that `ssh.port` is set to 2200. The port number must be specified as a number, not a string (no quotes). + +Then, you need to deploy this change so that SSH will bind to 2200. You cannot simply run `leap deploy blinky` because this command will default to using the variable `ssh.port` which is now `2200` but SSH on the node is still bound to 22. + +So, you manually override the port in the deploy command, using the old port: + + leap deploy --port 22 blinky + +Afterwards, SSH on `blinky` should be listening on port 2200 and you can just run `leap deploy blinky` from then on. + +X.509 Certificates +================================ + +Configuration options +------------------------------------------- + +The `ca` option in provider.json provides settings used when generating CAs and certificates. The defaults are as follows: + + "ca": { + "name": "= global.provider.ca.organization + ' Root CA'", + "organization": "= global.provider.name", + "organizational_unit": "= 'https://' + global.provider.name", + "bit_size": 4096, + "digest": "SHA256", + "life_span": "10y", + "server_certificates": { + "bit_size": 2024, + "digest": "SHA256", + "life_span": "1y" + }, + "client_certificates": { + "bit_size": 2024, + "digest": "SHA256", + "life_span": "2m", + "limited_prefix": "LIMITED", + "unlimited_prefix": "UNLIMITED" + } + } + +To see what values are used for your provider, run `leap inspect provider.json`. You can modify the defaults as you wish by adding the values to provider.json. + +NOTE: A certificate `bit_size` greater than 2024 will probably not be recognized by most commercial CAs. + +Certificate Authorities +----------------------------------------- + +There are three x.509 certificate authorities (CA) associated with your provider: + +1. **Commercial CA:** It is strongly recommended that you purchase a commercial cert for your primary domain. The goal of platform is to not depend on the commercial CA system, but it does increase security and usability if you purchase a certificate. The cert for the commercial CA must live at `files/cert/commercial_ca.crt`. +2. **Server CA:** This is a self-signed CA responsible for signing all the **server** certificates. The private key lives at `files/ca/ca.key` and the public cert lives at `files/ca/ca.crt`. The key is very sensitive information and must be kept private. The public cert is distributed publicly. +3. **Client CA:** This is a self-signed CA responsible for signing all the **client** certificates. The private key lives at `files/ca/client_ca.key` and the public cert lives at `files/ca/client_ca.crt`. Neither file is distribute publicly. It is not a big deal if the private key for the client CA is compromised, you can just generate a new one and re-deploy. + +To generate both the Server CA and the Client CA, run the command: + + leap cert ca + +Server certificates +----------------------------------- + +Most every server in your service provider will have a x.509 certificate, generated by the `leap` command using the Server CA. Whenever you modify any settings of a node that might affect it's certificate (like changing the IP address, hostname, or settings in provider.json), you can magically regenerate all the certs that need to be regenerated with this command: + + leap cert update + +Run `leap help cert update` for notes on usage options. + +Because the server certificates are generated locally on your personal machine, the private key for the Server CA need never be put on any server. It is up to you to keep this file secure. + +Client certificates +-------------------------------- + +Every leap client gets its own time-limited client certificate. This cert is use to connect to the OpenVPN gateway (and probably other things in the future). It is generated on the fly by the webapp using the Client CA. + +To make this work, the private key of the Client CA is made available to the webapp. This might seem bad, but compromise of the Client CA simply allows the attacker to use the OpenVPN gateways without paying. In the future, we plan to add a command to automatically regenerate the Client CA periodically. + +There are two types of client certificates: limited and unlimited. A client using a limited cert will have its bandwidth limited to the rate specified by `provider.service.bandwidth_limit` (in Bytes per second). An unlimited cert is given to the user if they authenticate and the user's service level matches one configured in `provider.service.levels` without bandwidth limits. Otherwise, the user is given a limited client cert. + +Commercial certificates +----------------------------------- + +We strongly recommend that you use a commercial signed server certificate for your primary domain (in other words, a certificate with a common name matching whatever you have configured for `provider.domain`). This provides several benefits: + +1. When users visit your website, they don't get a scary notice that something is wrong. +2. When a user runs the LEAP client, selecting your service provider will not cause a warning message. +3. When other providers first discover your provider, they are more likely to trust your provider key if it is fetched over a commercially verified link. + +The LEAP platform is designed so that it assumes you are using a commercial cert for the primary domain of your provider, but all other servers are assumed to use non-commercial certs signed by the Server CA you create. + +To generate a CSR, run: + + leap cert csr + +This command will generate the CSR and private key matching `provider.domain` (you can change the domain with `--domain=DOMAIN` switch). It also generates a server certificate signed with the Server CA. You should delete this certificate and replace it with a real one once it is created by your commercial CA. + +The related commercial cert files are: + + files/ + certs/ + domain.org.crt # Server certificate for domain.org, obtained by commercial CA. + domain.org.csr # Certificate signing request + domain.org.key # Private key for you certificate + commercial_ca.crt # The CA cert obtained from the commercial CA. + +The private key file is extremely sensitive and care should be taken with its provenance. + +If your commercial CA has a chained CA cert, you should be OK if you just put the **last** cert in the chain into the `commercial_ca.crt` file. This only works if the other CAs in the chain have certs in the debian package `ca-certificates`, which is the case for almost all CAs. + +Facts +============================== + +There are a few cases when we must gather internal data from a node before we can successfully deploy to other nodes. This is what `facts.json` is for. It stores a snapshot of certain facts about each node, as needed. Entries in `facts.json` are updated automatically when you initialize, rename, or remove a node. To manually force a full update of `facts.json`, run: + + leap facts update FILTER + +Run `leap help facts update` for more information. + +The file `facts.json` should be committed to source control. You might not have a `facts.json` if one is not required for your provider. diff --git a/doc/known-issues.md b/doc/known-issues.md new file mode 100644 index 00000000..abd28084 --- /dev/null +++ b/doc/known-issues.md @@ -0,0 +1,64 @@ +@title = 'Leap Platform Release Notes' +@nav_title = 'Known issues' +@summary = 'Known issues in the Leap Platform.' +@toc = true + +Here you can find documentation about known issues and potential work-arounds in the current Leap Platform release. + +0.2.2 +===== + +In this release the following issues are known, work-arounds are noted when available. + +General Issues +-------------- + +. This release does *not* anonymize your logs (see: https://leap.se/code/issues/1897) + +. This release does *not* setup email relaying, so admins will not receive important email notifications. Email service will be part of the next release (see: https://leap.se/code/issues/1683 https://leap.se/code/issues/1905) + +. Your openvpn gateway address will be added on the /24 network, and is not configurable in this release (see: https://leap.se/code/issues/1863) + +. You must not add a node with an underscore in the name, you also cannot use a hyphen for a vagrant node (see: https://leap.se/code/issues/3087) + +. The nagios website check reports success when the webapp is not functioning but apache is up (see: https://leap.se/code/issues/1629) + +User setup and ssh +------------------ + +. if you aren't using a single ssh key, but have different ones, you will need to define the following at the top of your ~/.ssh/config: + HostName <ip address> + IdentityFile <path to identity file> + + (see: https://leap.se/code/issues/2946 and https://leap.se/code/issues/3002) + +. If the ssh host key changes, you need to run node init again (see: https://leap.se/en/docs/platform/guide#Working.with.SSH) + +. At the moment, only ECDSA ssh host keys are supported. If you get the following error: `= FAILED ssh-keyscan: no hostkey alg (must be missing an ecdsa public host key)` then you should confirm that you have the following line defined in your server's /etc/ssh/sshd_config: +HostKey /etc/ssh/ssh_host_ecdsa_key and that file exists. If you made a change to your sshd_config, then you need to run `/etc/init.d/ssh restart` (see: https://leap.se/code/issues/2373) + +. To remove an admin's access to your servers, please remove the directory for that user under the `users/` subdirectory in your provider directory and then remove that user's ssh keys from files/ssh/authorized_keys. When finished you *must* run a `leap deploy` to update that information on the servers (see: https://leap.se/code/issues/1863) + +. At the moment, it is only possible to add an admin who will have access to all LEAP servers (see: https://leap.se/code/issues/2280) + +. leap add-user --self allows only one key - if you run that command twice with different keys, you will just replace the key with the second key. To add a second key, add it manually to files/ssh/authorized_keys (see: https://leap.se/code/issues/866) + +Deploying +--------- + +. If you have any errors during a run, please try to deploy again as this often solves non-deterministic issues that were not uncovered in our testing. Please re-deploy with `leap -v2 deploy` to get more verbose logs and capture the complete output to provide to us for debugging. + +. If when deploying your debian mirror fails for some reason, network anomoly or the mirror itself is out of date, then platform deployment will not succeed properly. Check the mirror is up and try to deploy again when it is resolved (see: https://leap.se/code/issues/1091) + +. Deployment gives 'error: in `%`: too few arguments (ArgumentError)' - this is because you attempted to do a deploy before initializing a node, please initialize the node first and then do a deploy afterwards (see: https://leap.se/code/issues/2550) + +. This release has no ability to custom configure apt sources or proxies (see: https://leap.se/code/issues/1971) + +. When running a deploy at a verbosity level of 2 and above, you will notice puppet deprecation warnings, these are known and we are working on fixing them + +Special Environments +-------------------- + +. When deploying to OpenStack release "nova" or newer, you will need to do an initial deploy, then when it has finished run `leap facts update` and then deploy again (see: https://leap.se/code/issues/3020) + +. It is not possible to actually use the EIP openvpn server on vagrant nodes (see: https://leap.se/code/issues/2401) diff --git a/doc/quick-start.md b/doc/quick-start.md new file mode 100644 index 00000000..5ba28f8d --- /dev/null +++ b/doc/quick-start.md @@ -0,0 +1,245 @@ +@title = 'LEAP Platform Quick Start' +@nav_title = 'Quick Start' + +This tutorial walks you through the initial process of creating and deploying a service provider running the [LEAP platform](platform). First examples aim to build a provider in a virtual environment, and in the end running in real hardware is targeted. + +First, a few definitions: + +* **node:** A server that is part of the service provider's infrastructure. All nodes are running the Debian GNU/Linux operating system. +* **sysadmin:** This is you. +* **sysadmin machine:** Your desktop or laptop computer that you use to control the nodes. This machine can be running any variant of Unix, Linux, or Mac OS (however, only Debian derivatives are supported at the moment). + +All the commands in this tutorial are run on your sysadmin machine. In order to complete the tutorial, the sysadmin machine must: + +* Be a real machine with virtualization support in the CPU (VT-x or AMD-V). In other words, not a virtual machine. +* Have at least 4gb of RAM. +* Have a fast internet connection (because you will be downloading a lot of big files, like virtual machine images). + +Install prerequisites +-------------------------------- + +*Debian & Ubuntu* + +Install core prerequisites: + + sudo apt-get install git ruby ruby-dev rsync openssh-client openssl rake make + +Install Vagrant in order to be able to test with local virtual machines (typically optional, but required for this tutorial): + + sudo apt-get install vagrant virtualbox + +<!-- +*Mac OS* + +1. Install rubygems from https://rubygems.org/pages/download (unless the `gem` command is already installed). +2. Install Vagrant.dmg from http://downloads.vagrantup.com/ +--> + +Install leap +--------------------- + +<!--Install the `leap` command as a gem: + + sudo gem install leap_cli + +Alternately, you can install `leap` from source: + + git clone git://leap.se/leap_cli.git + cd leap_cli + rake build +--> + +Install `leap` command from source: + + git clone git://leap.se/leap_cli.git + cd leap_cli + rake build + +Then, install as root user (recommended): + + sudo rake install + +Or, install as unprivileged user: + + rake install + # watch out for the directory leap is installed to, then i.e. + sudo ln -s ~/.gem/ruby/1.9.1/bin/leap /usr/local/bin/leap + +With both methods, you can use now /usr/local/bin/leap, which in most cases will be in your $PATH. + + +Create a provider instance +--------------------------------------- + +A provider instance is a directory tree, usually stored in git, that contains everything you need to manage an infrastructure for a service provider. In this case, we create one for bitmask.net and call the instance directory 'bitmask'. + + mkdir -p ~/leap/bitmask + +Now, we will initialize this directory to make it a provider instance. Your provider instance will need to know where it can find local copy of the git repository leap_platform, which holds the puppet recipes you will need to manage your servers. Typically, you will not need to modify leap_platform. + + cd ~/leap/bitmask + leap new . + +The `leap new` command will ask you for several required values: + +* domain: The primary domain name of your service provider. In this tutorial, we will be using "bitmask.net". +* name: The name of your service provider. +* contact emails: A comma separated list of email addresses that should be used for important service provider contacts (for things like postmaster aliases, Tor contact emails, etc). +* platform: The directory where you have a copy of the `leap_platform` git repository checked out. If it doesn't exist, it will be downloaded for you. + +You may want to poke around and see what is in the files we just created. For example: + + cat provider.json + +Optionally, commit your provider directory using the version control software you fancy. For example: + + git init + git add . + git commit -m "initial commit" + +Now add yourself as a privileged sysadmin who will have access to deploy to servers: + + leap add-user --self + +NOTE: in most cases, `leap` must be run from within a provider instance directory tree (e.g. ~/leap/bitmask). + +Now generate required X509 certificates and keys: + + leap cert ca + leap cert csr + +To see details about the keys and certs that the prior two commands created, you can use `leap inspect` like so: + + leap inspect files/ca/ca.crt + + +Edit provider.json configuration +-------------------------------------- + +There are a few required settings in provider.json. At a minimum, you must have: + + { + "domain": "bitmask.net", + "name": "Bitmask", + "contacts": { + "default": "email1@domain.org, email2@domain.org" + } + } + +For a full list of possible settings, you can use `leap inspect` to see how provider.json is evaluated after including the inherited defaults: + + leap inspect provider.json + +Create nodes +--------------------- + +A "node" is a server that is part of your infrastructure. Every node can have one or more services associated with it. Some nodes are "local" and used only for testing. These local nodes exist only as virtual machines on your computer and cannot be accessed from outside (see `leap help local` for more information). + +Create a local node, with the service "webapp": + + leap node add --local web1 services:webapp + +This created a node configuration file in `nodes/web1.json`, but it did not create the virtual machine. In order to test our node "web1", we need to first spin up a virtual machine. The next command will probably take a very long time, because it will need to download a VM image (about 700mb). + + leap local start + +Now that the virtual machine for web1 is running, you need to initialize it and then deploy the recipes to it. You only need to initialize a node once, but there is no harm in doing it multiple times. These commands will take a while to run the first time, as it needs to update the package cache on the new virtual machine. + + leap node init web1 + leap deploy web1 + +That is it, you should now have your first running node. However, the LEAP web application requires a database to run, so let's add a "couchdb" node: + + leap node add --local db1 services:couchdb + leap local start + leap node init db1 + leap deploy db1 + +Access the web application +-------------------------------------------- + +You should now have two local virtual machines running, one for the web application and one for the database. In order to connect to the web application in your browser, you need to point your domain at the IP address of the web application node (named web1 in this example). + +There are a lot of different ways to do this, but one easy way is to modify your `/etc/hosts` file. First, find the IP address of the webapp node: + + leap list webapp --print ip_address + +Then modify `/etc/hosts` like so: + + 10.5.5.47 DOMAIN + +Replacing 'DOMAIN' with whatever you specified as the `domain` in the `leap new` command. + +Next, you can connect to the web application either using a web browser or via the API using the LEAP client. To use a browser, connect to https://DOMAIN. Your browser will complain about an untrusted cert, but for now just bypass this. From there, you should be able to register a new user and login. + +What is going on here? +-------------------------------------------- + +First, some background terminology: + +* **puppet**: Puppet is a system for automating deployment and management of servers (called nodes). +* **hiera files**: In puppet, you can use something called a 'hiera file' to seed a node with a few configuration values. In LEAP, we go all out and put *every* configuration value needed for a node in the hiera file, and automatically compile a custom hiera file for each node. + +When you run `leap deploy`, a bunch of things happen, in this order: + +1. **Compile hiera files**: The hiera configuration file for each node is compiled in YAML format and saved in the directory `hiera`. The source material for this hiera file consists of all the JSON configuration files imported or inherited by the node's JSON config file. +* **Copy required files to node**: All the files needed for puppet to run are rsync'ed to each node. This includes the entire leap_platform directory, as well as the node's hiera file and other files needed by puppet to set up the node (keys, binary files, etc). +* **Puppet is run**: Once the node is ready, leap connects to the node via ssh and runs `puppet apply`. Puppet is applied locally on the node, without a daemon or puppetmaster. + +You can run `leap -v2 deploy` to see exactly what commands are being executed. + +<!-- See [under the hood](under-the-hood) for more details. --> + +Additional commands +------------------------------------------- + +Here are a few useful commands you can run on your new local nodes: + +* `leap ssh web1` -- SSH into node web1 (requires `leap node init web1` first). +* `leap list` -- list all nodes. +* `leap list --print ip_address` -- list a particular attribute of all nodes. +* `leap local reset web1` -- return web1 to a pristine state. +* `leap local stop` -- stop all local virtual machines. +* `leap local status` -- get the running state of all the local virtual machines. +* `leap cert update` -- generate new certificates if needed. + +See the full command reference for more information. + +Node filters +------------------------------------------- + +Many of the `leap` commands take a "node filter". You can use a node filter to target a command at one or more nodes. + +A node filter consists of one or more keywords, with an optional "+" before each keyword. + +* keywords can be a node name, a service type, or a tag. +* the "+" before the keyword constructs an AND condition +* otherwise, multiple keywords together construct an OR condition + +Examples: + +* `leap list openvpn` -- list all nodes with service openvpn. +* `leap list openvpn +production` -- only nodes of service type openvpn AND tag production. +* `leap deploy webapp openvpn` -- deploy to all webapp OR openvpn nodes. +* `leap node init vpn1` -- just init the node named vpn1. + +Running on real hardware +----------------------------------- + +The steps required to initialize and deploy to nodes on the public internet are basically the same as we have seen so far for local testing nodes. There are a few key differences: + +* Obviously, you will need to acquire a real or virtual machine that you can SSH into remotely. +* When creating the node configuration, you should give it the tag "production" if the node is to be used in your production infrastructure. +* When creating the node configuration, you need to specify the IP address of the node. + +For example: + + leap node add db1 tags:production services:couchdb ip_address:4.4.4.4 + +Also, running `leap node init NODE_NAME` on a real server will prompt you to verify the fingerprint of the SSH host key and to provide the root password of the server NODE_NAME. You should only need to do this once. + +What's next +----------------------------------- + +Read the [LEAP platform guide](guide) to learn about planning and securing your infrastructure. + diff --git a/doc/service-diagram.odg b/doc/service-diagram.odg Binary files differnew file mode 100644 index 00000000..09265c2d --- /dev/null +++ b/doc/service-diagram.odg diff --git a/doc/service-diagram.png b/doc/service-diagram.png Binary files differnew file mode 100644 index 00000000..85e62436 --- /dev/null +++ b/doc/service-diagram.png diff --git a/doc/under-the-hood.md b/doc/under-the-hood.md new file mode 100644 index 00000000..080a153e --- /dev/null +++ b/doc/under-the-hood.md @@ -0,0 +1,37 @@ +@title = "Under the hood" + +This page contains various details on the how the platform is implemented. You can safely ignore this page, although it may be useful if you plan to make modifications to the platform. + +Puppet Details +====================================== + +Run stages +---------- + +We use two run stages for resource ordering: + +* initial: configure hostname, apt-get update + apt-get dist-upgrade +* main: everything else + +Stage initial is run before stage main. + +see http://docs.puppetlabs.com/puppet/2.7/reference/lang_run_stages.html for run stage documentation. + +Tags +---- + +Tags are beeing used to deploy different classes. + +* leap_base: site_config::default (configure hostname + resolver, sshd, ) +* leap_slow: site_config::slow (slow: apt-get update, apt-get dist-upgrade) +* leap_service: cofigure platform service (openvpn, couchdb, etc.) + +You can pass any combination of tags, i.e. use + +* "--tags leap_base,leap_slow,leap_service" (DEFAULT): Deploy all +* "--tags leap_service": Only deploy service(s) (useful for debugging/development) +* "--tags leap_base": Only deploy basic configuration (again, useful for debugging/development) + +See http://docs.puppetlabs.com/puppet/2.7/reference/lang_tags.html for puppet tag usage. + + diff --git a/platform.rb b/platform.rb new file mode 100644 index 00000000..9f63b4ca --- /dev/null +++ b/platform.rb @@ -0,0 +1,77 @@ +# +# These are variables defined by this leap_platform and used by leap_cli. +# + +Leap::Platform.define do + self.version = "1.1.2" + self.compatible_cli = "1.1.2".."1.99" + + # + # the facter facts that should be gathered + # + self.facts = ["ec2_local_ipv4"] + + # + # the named paths for this platform + # + self.paths = { + # directories + :hiera_dir => 'hiera', + :files_dir => 'files', + :nodes_dir => 'nodes', + :services_dir => 'services', + :tags_dir => 'tags', + :node_files_dir => 'files/nodes/#{arg}', + + # input config files + :common_config => 'common.json', + :provider_config => 'provider.json', + :secrets_config => 'secrets.json', + :node_config => 'nodes/#{arg}.json', + :service_config => 'services/#{arg}.json', + :tag_config => 'tags/#{arg}.json', + + # input templates + :provider_json_template => 'files/service-definitions/provider.json.erb', + :eip_service_json_template => 'files/service-definitions/#{arg}/eip-service.json.erb', + :soledad_service_json_template => 'files/service-definitions/#{arg}/soledad-service.json.erb', + :smtp_service_json_template => 'files/service-definitions/#{arg}/smtp-service.json.erb', + + # output files + :facts => 'facts.json', + :user_ssh => 'users/#{arg}/#{arg}_ssh.pub', + :user_pgp => 'users/#{arg}/#{arg}_pgp.pub', + :known_hosts => 'files/ssh/known_hosts', + :authorized_keys => 'files/ssh/authorized_keys', + :ca_key => 'files/ca/ca.key', + :ca_cert => 'files/ca/ca.crt', + :client_ca_key => 'files/ca/client_ca.key', + :client_ca_cert => 'files/ca/client_ca.crt', + :dh_params => 'files/ca/dh.pem', + :commercial_key => 'files/cert/#{arg}.key', + :commercial_csr => 'files/cert/#{arg}.csr', + :commercial_cert => 'files/cert/#{arg}.crt', + :commercial_ca_cert => 'files/cert/commercial_ca.crt', + :vagrantfile => 'test/Vagrantfile', + + # node output files + :hiera => 'hiera/#{arg}.yaml', + :node_ssh_pub_key => 'files/nodes/#{arg}/#{arg}_ssh.pub', + :node_x509_key => 'files/nodes/#{arg}/#{arg}.key', + :node_x509_cert => 'files/nodes/#{arg}/#{arg}.crt', + + # testing files + :test_client_key => 'test/cert/client.key', + :test_client_cert => 'test/cert/client.crt', + :test_openvpn_config => 'test/openvpn/#{arg}.ovpn', + :test_client_openvpn_template => 'test/openvpn/client.ovpn.erb' + } + + # + # the files that need to get renamed when a node is renamed + # + self.node_files = [ + :node_config, :hiera, :node_x509_cert, :node_x509_key, :node_ssh_pub_key + ] +end + diff --git a/provider_base/common.json b/provider_base/common.json index e674edb6..2313bd8b 100644 --- a/provider_base/common.json +++ b/provider_base/common.json @@ -1,5 +1,6 @@ { "ip_address": null, + "environment": null, "services": [], "tags": [], "domain": { @@ -13,9 +14,13 @@ "public": "= service_type != 'internal_service'" }, "ssh": { - "authorized_keys": "= file :authorized_keys", + "authorized_keys": "= authorized_keys", "known_hosts": "=> known_hosts_file", - "port": 22 + "port": 22, + "mosh": { + "ports": "60000:61000", + "enabled": false + } }, "hosts": "=> hosts_file", "x509": { @@ -24,11 +29,11 @@ "key": "= x509.use ? file(:node_x509_key, :missing => 'x509 key for node $node. Run `leap cert update`') : nil", "ca_cert": "= try_file :ca_cert" }, - "local": false, - "production": false, "service_type": "internal_service", "development": { "site_config": true }, - "name": "common" + "name": "common", + "location": null, + "enabled": true } diff --git a/provider_base/files/service-definitions/eip-service.json.erb b/provider_base/files/service-definitions/eip-service.json.erb deleted file mode 100644 index 8dc7211d..00000000 --- a/provider_base/files/service-definitions/eip-service.json.erb +++ /dev/null @@ -1,37 +0,0 @@ -<%= - def underscore(words) - words = words.to_s.dup - words.downcase! - words.gsub! /[^a-z]/, '_' - words - end - - hsh = {} - hsh["serial"] = 1 - hsh["version"] = 1 - clusters = {} - gateways = [] - global.services['openvpn'].node_list.each_node do |node| - next if node.vagrant? - gateway = {} - gateway["capabilities"] = node.openvpn.pick( - :ports, :protocols, :user_ips, :adblock, :filter_dns) - gateway["capabilities"]["transport"] = ["openvpn"] - gateway["ip_address"] = node.openvpn.gateway_address - gateway["host"] = node.domain.full - gateway["cluster"] = underscore(node.openvpn.location) - gateways << gateway - clusters[gateway["cluster"]] ||= { - "name" => gateway["cluster"], - "label" => {"en" => node.openvpn.location} - } - end - hsh["gateways"] = gateways - hsh["clusters"] = clusters.values - hsh["openvpn_configuration"] = { - "tls-cipher" => "DHE-RSA-AES128-SHA", - "auth" => "SHA1", - "cipher" => "AES-128-CBC" - } - generate_json hsh -%>
\ No newline at end of file diff --git a/provider_base/files/service-definitions/provider.json.erb b/provider_base/files/service-definitions/provider.json.erb index f26f25a2..5d4c63a0 100644 --- a/provider_base/files/service-definitions/provider.json.erb +++ b/provider_base/files/service-definitions/provider.json.erb @@ -1,20 +1,21 @@ <%= - hsh = {} - # grab some fields from provider.json hsh = global.provider.pick( :languages, :description, :name, - :enrollment_policy, :default_language, :domain + :enrollment_policy, :default_language, :service ) + hsh['domain'] = domain.full_suffix - # advertise services that are 'user services' - hsh['services'] = global.services[:service_type => :user_service].field(:name) + # advertise services that are 'user services' and for which there are actually nodes + hsh['services'] = global.services[:service_type => :user_service].field(:name).select do |service| + nodes_like_me[:services => service].any? + end hsh['api_version'] = "1" - hsh['api_uri'] = "https://" + api.domain + ':' + api.port + hsh['api_uri'] = ["https://", api.domain, ':', api.port].join - hsh['ca_cert_uri'] = 'https://' + global.provider.domain + '/ca.crt' + hsh['ca_cert_uri'] = 'https://' + domain.full_suffix + '/ca.crt' hsh['ca_cert_fingerprint'] = fingerprint(:ca_cert) - generate_json hsh + hsh.dump_json %>
\ No newline at end of file diff --git a/provider_base/files/service-definitions/v1/eip-service.json.erb b/provider_base/files/service-definitions/v1/eip-service.json.erb new file mode 100644 index 00000000..feaea25b --- /dev/null +++ b/provider_base/files/service-definitions/v1/eip-service.json.erb @@ -0,0 +1,48 @@ +<%= + def underscore(words) + words = words.to_s.dup + words.downcase! + words.gsub! /[^a-z]/, '_' + words + end + + def add_gateway(node, locations, options={}) + return nil if options[:ip] == 'REQUIRED' + gateway = {} + gateway["capabilities"] = node.openvpn.pick(:ports, :protocols, :user_ips, :adblock, :filter_dns) + gateway["capabilities"]["transport"] = ["openvpn"] + gateway["host"] = node.domain.full + gateway["ip_address"] = options[:ip] + gateway["capabilities"]["limited"] = options[:limited] + if node['location'] + location_name = underscore(node.location.name) + gateway["location"] = location_name + locations[location_name] ||= node.location + end + gateway + end + + hsh = {} + hsh["serial"] = 1 + hsh["version"] = 1 + locations = {} + gateways = [] + nodes_like_me[:services => 'openvpn'].each_node do |node| + if node.openvpn.allow_limited && node.openvpn.allow_unlimited + gateways << add_gateway(node, locations, :ip => node.openvpn.gateway_address, :limited => false) + gateways << add_gateway(node, locations, :ip => node.openvpn.second_gateway_address, :limited => true) + elsif node.openvpn.allow_unlimited + gateways << add_gateway(node, locations, :ip => node.openvpn.gateway_address, :limited => false) + elsif node.openvpn.allow_limited + gateways << add_gateway(node, locations, :ip => node.openvpn.gateway_address, :limited => true) + end + end + hsh["gateways"] = gateways.compact + hsh["locations"] = locations + hsh["openvpn_configuration"] = { + "tls-cipher" => "DHE-RSA-AES128-SHA", + "auth" => "SHA1", + "cipher" => "AES-128-CBC" + } + JSON.sorted_generate hsh +%>
\ No newline at end of file diff --git a/provider_base/files/service-definitions/v1/smtp-service.json.erb b/provider_base/files/service-definitions/v1/smtp-service.json.erb new file mode 100644 index 00000000..60129f5f --- /dev/null +++ b/provider_base/files/service-definitions/v1/smtp-service.json.erb @@ -0,0 +1,29 @@ +<%= + def underscore(words) + words = words.to_s.dup + words.downcase! + words.gsub! /[^a-z]/, '_' + words + end + + hsh = {} + hsh["serial"] = 1 + hsh["version"] = 1 + locations = {} + hosts = {} + nodes_like_me[:services => 'mx'].each_node do |node| + host = {} + host["hostname"] = node.domain.full + host["ip_address"] = node.ip_address + host["port"] = 25 # hard coded for now, later node.smtp.port + if node['location'] + location_name = underscore(node.location.name) + host["location"] = location_name + locations[location_name] ||= node.location + end + hosts[node.name] = host + end + hsh["hosts"] = hosts + hsh["locations"] = locations + JSON.sorted_generate hsh +%>
\ No newline at end of file diff --git a/provider_base/files/service-definitions/v1/soledad-service.json.erb b/provider_base/files/service-definitions/v1/soledad-service.json.erb new file mode 100644 index 00000000..0cd1c927 --- /dev/null +++ b/provider_base/files/service-definitions/v1/soledad-service.json.erb @@ -0,0 +1,29 @@ +<%= + def underscore(words) + words = words.to_s.dup + words.downcase! + words.gsub! /[^a-z]/, '_' + words + end + + hsh = {} + hsh["serial"] = 1 + hsh["version"] = 1 + locations = {} + hosts = {} + nodes_like_me[:services => 'soledad'].each_node do |node| + host = {} + host["hostname"] = node.domain.full + host["ip_address"] = node.ip_address + host["port"] = node.soledad.port + if node['location'] + location_name = underscore(node.location.name) + host["location"] = location_name + locations[location_name] ||= node.location + end + hosts[node.name] = host + end + hsh["hosts"] = hosts + hsh["locations"] = locations + JSON.sorted_generate hsh +%>
\ No newline at end of file diff --git a/provider_base/provider.json b/provider_base/provider.json index 8ce848f3..b6a7af21 100644 --- a/provider_base/provider.json +++ b/provider_base/provider.json @@ -8,22 +8,46 @@ "en": "REQUIRED" }, "contacts": { - "default": "REQUIRED" + "default": "REQUIRED", + "english": "= contacts.default.split('@').join(' at the domain ')" }, "languages": ["en"], "default_language": "en", "enrollment_policy": "open", + "service": { + "levels": [ + // bandwidth limit is in Bytes, storage limit is in MB. + {"id": 1, "name": "free", "storage":50}, + {"id": 2, "name": "basic", "storage":1000, "rate": ["US$10", "€10"]}, + {"id": 3, "name": "pro", "storage":10000, "rate": ["US$20", "€20"]} + ], + "default_service_level": 1, + "bandwidth_limit": 102400, + "allow_free": "= global.provider.service.levels.select {|l| l['rate'].nil?}.any?", + "allow_paid": "= global.provider.service.levels.select {|l| !l['rate'].nil?}.any?", + "allow_anonymous": "= global.provider.service.levels.select {|l| l['name'] == 'anonymous'}.any?", + "allow_registration": "= global.provider.service.levels.select {|l| l['name'] != 'anonymous'}.any?", + "allow_limited_bandwidth": "= global.provider.service.levels.select {|l| l['bandwidth'] == 'limited'}.any?", + "allow_unlimited_bandwidth": "= global.provider.service.levels.select {|l| l['bandwidth'].nil?}.any?" + }, "ca": { "name": "= global.provider.ca.organization + ' Root CA'", "organization": "= global.provider.name[global.provider.default_language]", - "organizational_unit": "= 'https://' + global.common.domain.full_suffix", + "organizational_unit": "= 'https://' + global.provider.domain", "bit_size": 4096, "digest": "SHA256", "life_span": "10y", "server_certificates": { - "bit_size": 3248, + "bit_size": 2024, "digest": "SHA256", "life_span": "1y" + }, + "client_certificates": { + "bit_size": 2024, + "digest": "SHA256", + "life_span": "2m", + "limited_prefix": "LIMITED", + "unlimited_prefix": "UNLIMITED" } }, "hiera_sync_destination": "/etc/leap" diff --git a/provider_base/services/ca.json b/provider_base/services/ca.json deleted file mode 100644 index 3fb8bf6c..00000000 --- a/provider_base/services/ca.json +++ /dev/null @@ -1,11 +0,0 @@ -{ - "ca_daemon": { - "couchdb_hosts": "= hostnames nodes[:services => :couchdb][:local => local]", - "couchdb_user": "= global.services[:couchdb].couch.users[:ca_daemon]" - }, - "service_type": "internal_service", - "x509": { - "use": true, - "ca_key": "= file(:ca_key, :missing => 'CA key. Run `leap cert ca` to create the Certificate Authority.')" - } -} diff --git a/provider_base/services/couchdb.json b/provider_base/services/couchdb.json index 1c8005c2..a26579c8 100644 --- a/provider_base/services/couchdb.json +++ b/provider_base/services/couchdb.json @@ -1,21 +1,37 @@ { - "service_type": "internal_service", "x509": { "use": true }, + "stunnel": { + "couch_server": "= stunnel_server(couch.port)", + "epmd_server": "= stunnel_server(couch.bigcouch.epmd_port)", + "epmd_clients": "= stunnel_client(nodes_like_me[:services => :couchdb], global.services[:couchdb].couch.bigcouch.epmd_port)", + "ednp_server": "= stunnel_server(couch.bigcouch.ednp_port)", + "ednp_clients": "= stunnel_client(nodes_like_me[:services => :couchdb], global.services[:couchdb].couch.bigcouch.ednp_port)" + }, "couch": { + "port": 5984, + "bigcouch": { + "epmd_port": 4369, + "ednp_port": 9002, + "cookie": "= secret :bigcouch_cookie", + "neighbors": "= nodes_like_me[:services => :couchdb].exclude(self).field('domain.full')" + }, "users": { "admin": { "username": "admin", - "password": "= secret :couch_admin_password" + "password": "= secret :couch_admin_password", + "salt": "= hex_secret :couch_admin_password_salt, 128" }, "webapp": { "username": "webapp", - "password": "= secret :couch_webapp_password" + "password": "= secret :couch_webapp_password", + "salt": "= hex_secret :couch_webapp_password_salt, 128" }, - "ca_daemon": { - "username": "ca_daemon", - "password": "= secret :couch_ca_daemon_password" + "soledad": { + "username": "soledad", + "password": "= secret :couch_soledad_password", + "salt": "= hex_secret :couch_soledad_password_salt, 128" } } } diff --git a/provider_base/services/openvpn.json b/provider_base/services/openvpn.json index 7b67ccb3..5d77f946 100644 --- a/provider_base/services/openvpn.json +++ b/provider_base/services/openvpn.json @@ -5,12 +5,19 @@ "client_ca_cert": "= file :client_ca_cert, :missing => 'Certificate Authority. Run `leap cert ca`'", "dh": "= file :dh_params, :missing => 'Diffie-Hellman parameters. Run `leap cert dh`'" }, + "location": null, "openvpn": { - "location": "Location Unknown", + "gateway_address": "REQUIRED", + "second_gateway_address": "= openvpn.allow_limited && openvpn.allow_unlimited ? 'REQUIRED' : nil", "ports": ["80", "443", "53", "1194"], "protocols": ["tcp", "udp"], "filter_dns": false, "adblock": false, - "user_ips": false + "user_ips": false, + "allow_limited": "= global.provider.service.allow_limited_bandwidth", + "allow_unlimited": "= global.provider.service.allow_unlimited_bandwidth", + "limited_prefix": "= global.provider.ca.client_certificates.limited_prefix", + "unlimited_prefix": "= global.provider.ca.client_certificates.unlimited_prefix", + "rate_limit": "= openvpn.allow_limited ? global.provider.service.bandwidth_limit : nil" } } diff --git a/provider_base/services/soledad.json b/provider_base/services/soledad.json new file mode 100644 index 00000000..10657563 --- /dev/null +++ b/provider_base/services/soledad.json @@ -0,0 +1,6 @@ +{ + "service_type": "public_service", + "soledad": { + "port": 1111 + } +}
\ No newline at end of file diff --git a/provider_base/services/webapp.json b/provider_base/services/webapp.json index 8ccd3e3e..93396ec7 100644 --- a/provider_base/services/webapp.json +++ b/provider_base/services/webapp.json @@ -1,34 +1,52 @@ { "webapp": { "modules": ["user", "billing", "help"], - "couchdb_hosts": "= hostnames nodes[:services => :couchdb][:local => local]", - // NOTE: this is bad, but pending a fix to https://leap.se/code/issues/1163 - // before we can use user "webapp" - "couchdb_user": "= global.services[:couchdb].couch.users[:admin]", + "couchdb_admin_user": "= global.services[:couchdb].couch.users[:admin]", +// "couchdb_webapp_user": "= global.services[:couchdb].couch.users[:webapp]", + "couchdb_webapp_user": "= global.services[:couchdb].couch.users[:admin]", "favicon": "= file_path 'branding/favicon.ico'", "tail_scss": "= file_path 'branding/tail.scss'", "head_scss": "= file_path 'branding/head.scss'", - "img_dir": "= file_path 'branding/img'" + "img_dir": "= file_path 'branding/img'", + "client_certificates": "= global.provider.ca.client_certificates", + "allow_limited_certs": "= global.provider.service.allow_limited_bandwidth", + "allow_unlimited_certs": "= global.provider.service.allow_unlimited_bandwidth", + "allow_anonymous_certs": "= global.provider.service.allow_anonymous", + "secret_token": "= secret :webapp_secret_token", + "api_version": 1 + }, + "stunnel": { + "couch_client": "= stunnel_client(nodes_like_me[:services => :couchdb], global.services[:couchdb].couch.port)" + }, + "haproxy": { + "local_ports": "= stunnel.couch_client.field(:accept_port)" }, "definition_files": { "provider": "= file :provider_json_template", - "eip_service": "= file :eip_service_json_template" + "eip_service": "= file [:eip_service_json_template, 'v'+webapp.api_version.to_s]", + "soledad_service": "= file [:soledad_service_json_template, 'v'+webapp.api_version.to_s]", + "smtp_service": "= file [:smtp_service_json_template, 'v'+webapp.api_version.to_s]" }, "service_type": "public_service", "api": { "domain": "= 'api.' + domain.full_suffix", - "port": "4430" + "port": 4430 + }, + "nickserver": { + "domain": "= 'nicknym.' + domain.full_suffix", + "port": 6425, + "couchdb_user": "= global.services[:couchdb].couch.users[:admin]" }, "dns": { - "aliases": "= [domain.full, api.domain]" + "aliases": "= [domain.full_suffix, domain.full, api.domain, nickserver.domain]" }, "x509": { "use": true, "ca_cert": "= file :ca_cert, :missing => 'provider CA. Run `leap cert ca`'", "client_ca_cert": "= file_path :client_ca_cert", "client_ca_key": "= file_path :client_ca_key", - "commercial_cert": "= file [:commercial_cert, global.provider.domain]", - "commercial_key": "= file [:commercial_key, global.provider.domain]", + "commercial_cert": "= file [:commercial_cert, domain.full_suffix]", + "commercial_key": "= file [:commercial_key, domain.full_suffix]", "commercial_ca_cert": "= try_file :commercial_ca_cert" } -}
\ No newline at end of file +} diff --git a/provider_base/tags/development.json b/provider_base/tags/development.json new file mode 100644 index 00000000..6d4f9e25 --- /dev/null +++ b/provider_base/tags/development.json @@ -0,0 +1,7 @@ +{ + "environment": "development", + "domain": { + "full_suffix": "= 'dev.' + global.provider.domain", + "internal_suffix": "= 'dev.' + global.provider.domain_internal" + } +}
\ No newline at end of file diff --git a/provider_base/tags/local.json b/provider_base/tags/local.json index 9cb16602..48312b33 100644 --- a/provider_base/tags/local.json +++ b/provider_base/tags/local.json @@ -1,3 +1,3 @@ { - "local": true + "environment": "local" }
\ No newline at end of file diff --git a/provider_base/tags/production.json b/provider_base/tags/production.json index b35c0650..ea17498f 100644 --- a/provider_base/tags/production.json +++ b/provider_base/tags/production.json @@ -1,3 +1,3 @@ { - "production": true + "environment": "production" }
\ No newline at end of file diff --git a/provider_base/test/openvpn/client.ovpn.erb b/provider_base/test/openvpn/client.ovpn.erb index a0bdd307..af183ef4 100644 --- a/provider_base/test/openvpn/client.ovpn.erb +++ b/provider_base/test/openvpn/client.ovpn.erb @@ -18,9 +18,11 @@ tls-cipher DHE-RSA-AES128-SHA </ca> <cert> -<%= read_file! :test_client_cert -%> +<%# read_file! :test_client_cert -%> +<%= cert -%> </cert> <key> -<%= read_file! :test_client_key -%> +<%# read_file! :test_client_key -%> +<%= key -%> </key> diff --git a/puppet/manifests/setup.pp b/puppet/manifests/setup.pp new file mode 100644 index 00000000..80e7ffc2 --- /dev/null +++ b/puppet/manifests/setup.pp @@ -0,0 +1,16 @@ +# +# this is applied before each run of site.pp +# +$services = '' + +Exec { path => '/usr/bin:/usr/sbin/:/bin:/sbin:/usr/local/bin:/usr/local/sbin' } + +include site_config::hosts + +include site_apt + +package { 'facter': + ensure => latest, + require => Exec['refresh_apt'] +} + diff --git a/puppet/manifests/site.pp b/puppet/manifests/site.pp index 1ec806d9..08cbbb9e 100644 --- a/puppet/manifests/site.pp +++ b/puppet/manifests/site.pp @@ -1,39 +1,39 @@ # set a default exec path Exec { path => '/usr/bin:/usr/sbin/:/bin:/sbin:/usr/local/bin:/usr/local/sbin' } -stage { 'initial': - before => Stage['main'], -} +# parse services for host +$services=join(hiera_array('services'), ' ') +notice("Services for ${fqdn}: ${services}") + +# make sure apt is updated before any packages are installed +include apt::update +Package { require => Exec['apt_updated'] } + +include stdlib import 'common' include site_config::default include site_config::slow -# parse services for host -$services=hiera_array('services') -notice("Services for ${fqdn}: ${services}") # configure eip -if 'openvpn' in $services { +if $services =~ /\bopenvpn\b/ { include site_openvpn } -if 'couchdb' in $services { +if $services =~ /\bcouchdb\b/ { include site_couchdb } -if 'webapp' in $services { +if $services =~ /\bwebapp\b/ { include site_webapp + include site_nickserver } -if 'ca' in $services { - include site_ca_daemon -} - -if 'monitor' in $services { +if $services =~ /\bmonitor\b/ { include site_nagios } -if 'tor' in $services { +if $services =~ /\btor\b/ { include site_tor } diff --git a/puppet/modules/apache b/puppet/modules/apache -Subproject 077d4d1508b9ff3355f73ff8597991043b3ba5d +Subproject c3e92a9b3cb02f1546b6b1570f10a968d380005 diff --git a/puppet/modules/apt b/puppet/modules/apt -Subproject f16a0727dce187d07389388da8b816f7b520205 +Subproject 1a72a99693c1d77bfe891546408f88264fca98e diff --git a/puppet/modules/couchdb b/puppet/modules/couchdb -Subproject b915a67c6e7e3b1b75400dbbd4a9ac961c8eb03 +Subproject 20deb0652ccfe105eddec6ba2ad32b8d633705f diff --git a/puppet/modules/haproxy b/puppet/modules/haproxy new file mode 160000 +Subproject b398f3cb0a67d1170d0564a3f03977f9a08c2b6 diff --git a/puppet/modules/site_apache/files/vhosts.d/couchdb_proxy.conf b/puppet/modules/site_apache/files/vhosts.d/couchdb_proxy.conf deleted file mode 100644 index 0dff2cd6..00000000 --- a/puppet/modules/site_apache/files/vhosts.d/couchdb_proxy.conf +++ /dev/null @@ -1,10 +0,0 @@ -Listen 0.0.0.0:6984 - -<VirtualHost *:6984> - SSLEngine On - SSLProxyEngine On - SSLCertificateKeyFile /etc/x509/keys/leap_couchdb.key - SSLCertificateFile /etc/x509/certs/leap_couchdb.crt - ProxyPass / http://127.0.0.1:5984/ - ProxyPassReverse / http://127.0.0.1:5984/ -</VirtualHost> diff --git a/puppet/modules/site_apache/templates/vhosts.d/api.conf.erb b/puppet/modules/site_apache/templates/vhosts.d/api.conf.erb index cdfcbd68..ae894cd4 100644 --- a/puppet/modules/site_apache/templates/vhosts.d/api.conf.erb +++ b/puppet/modules/site_apache/templates/vhosts.d/api.conf.erb @@ -21,8 +21,7 @@ Listen 0.0.0.0:<%= api_port %> RequestHeader set X_FORWARDED_PROTO 'https' - DocumentRoot /srv/leap-webapp/public - Alias /1 /srv/leap-webapp/public + DocumentRoot /srv/leap/webapp/public # Check for maintenance file and redirect all requests RewriteEngine On diff --git a/puppet/modules/site_apache/templates/vhosts.d/leap_webapp.conf.erb b/puppet/modules/site_apache/templates/vhosts.d/leap_webapp.conf.erb index 4928cdd6..4b051699 100644 --- a/puppet/modules/site_apache/templates/vhosts.d/leap_webapp.conf.erb +++ b/puppet/modules/site_apache/templates/vhosts.d/leap_webapp.conf.erb @@ -21,8 +21,7 @@ RequestHeader set X_FORWARDED_PROTO 'https' - DocumentRoot /srv/leap-webapp/public - Alias /1 /srv/leap-webapp/public + DocumentRoot /srv/leap/webapp/public RewriteEngine On # Check for maintenance file and redirect all requests @@ -37,10 +36,10 @@ PassengerFriendlyErrorPages off SetEnv TMPDIR /var/tmp - <% if (defined? @services) and (services.is_a? Array) and (@services.include? 'monitor') -%> + <% if (defined? @services) and (@services.include? 'monitor') -%> <DirectoryMatch (/usr/share/nagios3/htdocs|/usr/lib/cgi-bin/nagios3|/etc/nagios3/stylesheets)> PassengerEnabled off - AllowOverride all + AllowOverride all </DirectoryMatch> <% end -%> </VirtualHost> diff --git a/puppet/modules/site_apt/files/keys/cloudant-key.asc b/puppet/modules/site_apt/files/keys/cloudant-key.asc new file mode 100644 index 00000000..99716a3c --- /dev/null +++ b/puppet/modules/site_apt/files/keys/cloudant-key.asc @@ -0,0 +1,52 @@ +-----BEGIN PGP PUBLIC KEY BLOCK----- +Version: GnuPG v1.4.11 (GNU/Linux) + +mQINBFE7fhIBEACrDREcODnhdugNozMeBawOm2irpNCP54yMljST/DOXx1uo3gQw +HnVcQ4lL7lXhbfL6Tp0WhrNYTWbbWHO0DaQbW0GQMHa2BGG0Xm0HPrjr3j55tAcM +NPr0ArDuplq4Py2pwviZiEtQkkn+biH9oV+N3jNO+8+zVHLVU7pHaX6Yd7HAxFM8 +XX+7SeVtplZ7nvSxUREiMNxQb9o0kYNRPS+b0UjiIXHrFO9afl7lTdg/I8AhKWa0 +3jJoY/IRvVopJblISQNGFipR11Lpu5sOHghgz4V8mk/in7JLMmoqSl5DP5VhRII8 +OyADBjaUJD2mkv5cGaevqpB4AId78X9+Y62gFJrGkIHY9uBxIUkRe+leYI4Zz4Bm +D9qBIbEY/kKkblTlC1G7u3qbGQcsbCRVIOnhruCih7vifcP40YwGUk5NmDA5AE78 +OovCGYGp4zMepDTSJxGT3sJOTEbzN09so6C7fQWBeQiiG5Uepp1q+VnaGpT1L4rc +Y6yRbu9dOFj6WzY4W5HtnbalzTIEYy+SIGZqRkJt6jREYLiFfyrpSFIgGoJAs0yx +9M0McXfeOod69TPufB1PeppnBwFcTmYNYxakusQxAebRDPEBZqoEgl0gMmxWbAdI +nxGMWWnSsN/Dj0dXRf1MG/5akOhX2zQcUzBOE2m/Xr5kjDPYFtFxVJDGzQARAQAB +tDNDbG91ZGFudCBQYWNrYWdlIFNpZ25pbmcgS2V5IDxzdXBwb3J0QGNsb3VkYW50 +LmNvbT6JAj4EEwECACgFAlE7fhICGwMFCQHhM4AGCwkIBwMCBhUIAgkKCwQWAgMB +Ah4BAheAAAoJEFngH70Vvo4mciIP/AlqHA/LDtSYfrFwdXifY2ImCMyzYvH40Ko2 +DHCw2qDjvK5UXn1iWuzXidT7DrxOfYoZpzySRP7VGyHxa3VPhOtzLDZSvTpk9ELo +2x2IczUwLC17M0Iis4CpqlxSFIBYGX78pMzvsEyC4TFqUDfXRlye3apjD0iwK0hE +kdP1+TPdJjhWImJm+3TLu45zTw3Ph5dnf5pLQPNhKfBSdku+vRrd35N5hHso9S1y +Z3NrxcQlWnXuqkLIA14gM7qbBFD+el9Y+tZ7ERGYg3s5uNDQRTb0QC8zg/um2+zW +4hHmuRcWY3n8IgHcYUruC1VyrrsFIWWMyLv7SZkAAoSY+jKyESDfYpJQ8jtZ4EF9 +2/gYm4FgZR8j4gWkzHSLGVt/4EIykJZb0yIg/QEovmmHqpy8xYri3goMSl4h7tfF +TOCZLTzTyQ7xONdyEsrvQPhmdtXEgvSo5S7ZU9kkx32OjCoshLLjhtqAipBgEXqb +hElFo1oSyOVoGc7UNh7KNBjWfeP8dNdCbIbIYPMeM0/CVjD60kW5ZEVDuYglT+Rz +enJJvS4Hs+fq8cFNxMB+l64qE7iS+I6RP2bPeQM2aBa2UZNWxUIbXF7bb3zLrCGn +GT8GF1AFRoW3GiDzB7QnLVp8BhIaqFUzbDim+5mFFG8wguxHTiz4snDdQXq2Es6V +UETFsNsluQINBFE7fhIBEADIyLHyBh8AKJKQHksFAPHOyA48ocxgQDpQnqYlQcAK +D8eUbRXciIz4ePBmvjaQmz8wJgWULc04u4i9jK8Jd/Ks+VhEz3AjRBfjvkBaVMog +FMPKaoDn9LVMBSZJ3fcC1DVck1oO8LnFIdktt0zhvzG+pV5b/UTRsVZmwNh1p2dM +4cJswxlksJXYnI9tFA74qiomDCPYM0zpv7TEjX23PZTLqTSHP5aWctx+MIEtdoqp +EsEDL6npvYBRz/tuL41cUWs7CItH131Hyuizo4vGrxgWPnoXIxLmLOOZCMk/kbx0 +XCSvengqYwNgAOlIjewtTw+WJm1gtNQQeKmaXBX7njf2Wz7LI/0KVxttEpKT5/5y +embOGn7My9i7zOc1frMCDivIOTQDBZTzR9o7/6wUJ69DIoFLMlO8UcCK3R7o5VUI +ezx+XYsOAD7D2vKoiD8Se65Vnax2rfFlLP7OQqdem5l2lkHpJzP3lA8qmA2MfJ7V +jsk7eDSyJQjG5c6KBoaFlYGhp/E2kR82cAKVaFIbW3euMM4XK6Mgzy3+DVKfk8mu +AEuHub7plfxM+65yjLNAK6l6IKtY1HfM7F4GFyNSd3mNNcWN7ceIHh8Ur4DeD2Tp +7r3XcWd6/czLYNsw2BAHeVUxnMTCeGN99UZTtHgVq9IJMOCDOPwMSzHFfZ6sNaYL +qQARAQABiQIlBBgBAgAPBQJRO34SAhsMBQkB4TOAAAoJEFngH70Vvo4mpokP/jJJ +2mXdhMVqZCtZhwphJfdxg8nBERzrd6ebXxKbTq1MmSN/fDwLknPabFHUpzk1ADCf +6mh2o0HB+67yMzo1UVtyfPOaHgCE/pWer5ultJM8gOdpBfSWL8jRwU8ZQ4fDu3z8 +AC6zTNq7znOVLEzZPy8U7q5Rt5/6QdQYoTLe6DwlLmkflzWP5VWi/mTGvtu/t5OV +tGZkzBYQ5QAXRXXkKswqkJpQFuW6d1vlYm9+x/+Q1+2kGT+CKbRAkqkf77qVcyJR +1M2JQSs4ko+rLMZzr01sYA+EBD17nxqV8vUdYebNc9Qnk8Aphid1zarUbySgAdnJ +5SLAjLe/6N6IEE9F3uKsPEs87gJrnwrYHRrmu0wAPwA0cMmtgD4Bz7Iiz4CLYPFW +rHpQCA313K+rS/LLfLBL66wIRKcPuYIFR9N03jX9eGR6qtk0b5Zb3YjWOo4V9Q1r +o+g6IB0Us5vH6ISuokq7Bv+8cXhEMVoctL9A8xWN1KDkweZ+7dNWCGV8lUWKy3Hw +ig6hENH6H7J57U8H2v2aZTeUo6e7VDP9gddNKPSEEeoBKfVnWYGoG8mVPQ2PzTgZ +ZO2vwp4c3Ix/kIV3xe+/Opcq1lxYhD7HSre1MB7HOeFmis6tBBjMJPaatZVfzj1v +6Uhz5oUCwcPol8rsp69DvGVUPSHfDwBxurDX71oG +=lEm7 +-----END PGP PUBLIC KEY BLOCK----- diff --git a/puppet/modules/site_apt/files/keys/leap_key.asc b/puppet/modules/site_apt/files/keys/leap_key.asc new file mode 100644 index 00000000..b69251f0 --- /dev/null +++ b/puppet/modules/site_apt/files/keys/leap_key.asc @@ -0,0 +1,63 @@ +-----BEGIN PGP PUBLIC KEY BLOCK----- +Version: GnuPG v1.4.11 (GNU/Linux) + +mQINBFESwt0BEAC2CR+XgW04DVwT427v2T4+qz+O/xGOwQcalVaSOUuguYgf29en +Apb6mUqROOTuJWN1nw1lvXiA6iFxg6DjDUhsp6j54X7GAAAjZ9QuavPgcsractsJ +LRz9WSWqDjOAYsb4B5pwmSPAKYtmRAxLVzdxUsuHs2HxRO4VWnaNJQEBj7j7zuGs +gvSJBSq9Vici6cGI9c1fsWyKsnp7R6M54mmQRbsCg2+G/N0hqOz0HE6ZlJKVKaZq +uTrPxGWFuU3mAUpzFLa6Wj8DSUYiWZ/xrqiFdbB4t1HM3vlKB9LEg93DEuG/8Q0T +g2KS0lEWxequBXyE6+jklDNqJeyHmfgkuAfFlkNYa5870XT87MzGE/hS40lbmhQV +HHlwxMkAiERMc0Ys+OfgUJMbIDQBNRFg3Q/bjajFoVBgBoKFp7C22zgoJkUNT+7H +Yv/t6zeDlIzNhgYms5d0gEiAeLauwju36BmwUsbQHwejWKP8pADRZL1bTj0E+rRU +M4FFNh9D2XTFFKaaNubub8tUmo+ZUIEEKfPhNHK9wS/bsFyPv9y3HLe2b3NYGFK5 ++Hznqg8N0H+29I7zLx7VpOh3iRN3Lbxv9dMmukVJtw8Rq/Udprd3Z5p8oCisFo+k +nY+J+IgNjC0eniN8rkkl/4rIN5fvvOR8YCts50hL1fAy3dd/MKExz+QTXQARAQAB +tClMRUFQIGFyY2hpdmUgc2lnbmluZyBrZXkgPHN5c2RldkBsZWFwLnNlPokCHAQQ +AQoABgUCURPzwAAKCRBIWxL6IY6B65FzEACn1Q+9dcLig6yCRPGF8d5qdnWYquts +fLc/W8P9uFCo4bLFhy+BlalZVhOSPt2KMBCApoW0fAc5aXOWjxEmtFOvziPtJ0N7 +uJj7y8XLk1//v7QXDJNYotiO82b9XTmF2G9URhxe/YU7mgx1cRW9X2h6LOG4VCIw +Bd00wM9vV984f50hpftdyjCcWTO9WoSus7dOL457DhcX7uX89AGUJLC9RTiaDtIL +/G/VEM8pIx5zW6Q2TwUXndVsNqyG5s0J0908KNyp5IPI66M07rR939JVAL8HXMxY +KdA9pxkKzPSThx8yWZknJoINsUhrd5ijfiA6kM7HJlJF1SnwyHSSs3KydKHj5zN2 +n3oGGT0bjZiXZHShsWa5mjEvCJ7oqwtcCdo8thW128LY2/0h3JkSsYdgdsJjGJbG +76nYjCIZYa6the4+QI8HM2WG5nrZL4B/EnYHK2lDdeVy/ynu96YhC4mdk566Vcqs +RrWJgRxImkSbxp3f6SAOsLwOdmrs52wCoEpAYPMbu79jb2G7JbR4uDB0i/pXCp+c +aleyKb4ve2EjHAY/VPF5BXKaQh3JIvGKVEZIv5ospoosr78UHBk60RMMzDSlOFso +BcB6Plpqoq4lI/4Zh8M1+eDjAOnOKwQanS4Hv7O2PqldGBUAXS3m6OI2Kvv3VqnM +X0GOB2sX4Ox8UYkCPQQTAQoAJwUCURLC3QIbAwUJAeEzgAULCQgHAwUVCgkICwUW +AgMBAAIeAQIXgAAKCRAeNKGCjiB5AZBiD/wJwUVZjfNeWdpKrYy9HtZExtTcU/94 +3lgRUNinUuLPFU4i2s+hR3h5fzXR15nUD+IBJlXlzLV2G/IjXYPTp4a0gqHpWULa +b5Stu7AzFiO42/RWUAzWD1Fyh6SuZ3FDERvheid8s4SXoe6y4cJ5ErfSlJS6qqku +8ss8mS8lM1Mp+lc9wYTWQ+8hmSUivAZb9WLEljFxhvEnvAKPwD18o7+S9GABFwYs +xflQvKZHguaOVqBEksry+vu8okWNrg3Ll3dDQEeahr7nrLrHe8gqONJgOE9jjxRv +bJmGtIUTyGqgWZZzBfQXL/6uXL23bWkYZDkQNhfsm+colAV8gpj+/E3q/uMXwqz1 +bv06K/LsK3NHzBNE57kJHEhg9K3Uw2Wx5qwFMU1GDxsB3P9p+TyqAboEZAB2irTR +y9k8peFB7wwf0sW3Eg78XFsfy4gyV619VnBR+PbfOpKqFFXAodF1mFiIrPeefaVp +F9fiQ5Owt0sJjDaJnYT83ksAO2Aj+VsY3UjnDrGFaiV8Neit9y/8W8DqmZ3EZEF/ +M3iS0yDjqqt9ACFD+jkGlKYsyHv7gbpTq0yi6u/kRXHUTIvVwFL9M6Z6AUcG8gzo +qbKhXGfWKEq0lN5HAjJ//V9ro3DekFd0A+NQOlFV6XtspZwphVdtW1WS078HmVlw +F5dbD8pcfT/RjbkCDQRREsLdARAA3Frw+j6H9McEIi/gjiGwvxnIdGc8McWchnFp +OWvdhTW9056v+y22DoKbULjT8k+8GzuRQ0xp4VwCC1rX3UExwceczzGs+tSKuIGm +g1ELygsaOZHdQBNLGPvn+TZNGlaYXPlQo7m8YhXGHwgQrdKyjcFD5xnOHxe981LT +q+IQ6jVYhho7/Qik9rVE1XHxoOfYvnNZJD0cFdf9OcX47YoqmM4sZYPMoOmKoVQT +sAAQ527wz742Bd6SpuhqBpdEw6YiCYxEoo5kBY3IhP3L5OTS4tzhOkdf1xlhWSnC +FE7NkPcK6o+r6qCcUqRGV9jRwI97JlPKegEHYWvLD4Sk31pWi8NZ0toU/nqRvxbh +htHxuNf3jeAAzxQBhGVi0C/IBr4vqyFqmEHr9JxIa3DTV8w/a0Y4hX2bczL9Y1cB +6n8qOA68aAn+xerJcSOroTIJh83D/7OguexGGYoZBDvX6dWguf8udFPeYpJvkT6T +SYF9U0JpVTtlCNutjScUO2uaV9+uDqACngwqbzBTjL8UucAleVcFfOi48yepnOd1 +1YFYxbw+/BcqLNhi1eP2AaGxIgXbR88tF9OC0SXaCH+1Z1bbalOmQNYstOv9BbsH +vW7mPgX2xhyoDkVRWaNAQoDLbnJr4gi9cD8/kQMzdlGOzt2ist/+xueblXJs5TOO +80Rw+AEAEQEAAYkCJQQYAQoADwUCURLC3QIbDAUJAeEzgAAKCRAeNKGCjiB5AdMq +D/9SXulJq6Q4U7aN6o7TLMU2MgqeWqtBqwTNIisBoSJjXq9Od4iN2S5Akwo/ZQO0 +1nRNPPc9yjwidgb7wCUFDNglUDuGS2nXaQ0XAO83qHMOsORN2S93dO6xVRX2Chhz +l9bUr1WIQcM+lIs/LZCX2rvKlsFYmZQHX/ibhQs7T01RXajwJqwxyXyVPL+kPNeo +wva4ZUf6rzdqKZLfFgyJyGdHI18bF6lahgHdN2OOawEeU2K+MlluR3ZahoyN4u1M +qijf6snmfd0++EIqDHwYPn70F4JPdMhyuVpYBVyVtsgHy9W5fS+zSj+vX+qj6MBX +dFBs+a9nr8GZJO4BUP2mtyNgmEfUVQefSHnq+0OlGPZG4raxTEqJfp2KTRCGB4hI +zYWO1g1cOBeXxFfXJdkX8LoKbP5s2Kzn9sAK6BxmazOvSNpuimCDNvKjR00iKNS4 +Dxix2FBXQU/4pVpGHjXTQP6RqeTrAedXvpgCHWP1UIlswIQecGmQcJ/hRZjd+0vl +cjfCYhZHr7N96Da6Cy8v2fZiZHaSAt7T2oIZ9X3gEh/kOlLDcuIdvMHUfojn0MrP +Ce1AqOHyQQqhkVylvZpS0PdE0VW3PmJ98uKfX2FVAOTUD4Rw3n9Ew7bfM249HuP4 +JOXi/Skp4sBB/xgrtV1u+E+BW0SS/BOiwfrI4xUy+MrWuw== +=4STg +-----END PGP PUBLIC KEY BLOCK----- diff --git a/puppet/modules/site_apt/manifests/dist_upgrade.pp b/puppet/modules/site_apt/manifests/dist_upgrade.pp index f129dd73..08de31bb 100644 --- a/puppet/modules/site_apt/manifests/dist_upgrade.pp +++ b/puppet/modules/site_apt/manifests/dist_upgrade.pp @@ -1,15 +1,17 @@ class site_apt::dist_upgrade { - if $::apt_running == 'true' { + if $::apt_running == 'true' { fail ('apt-get is running in background - Please wait until it finishes. Exiting.') } else { exec{'initial_apt_update': - command => '/usr/bin/apt-get update && /usr/bin/apt-get autoclean', + command => '/usr/bin/apt-get update', refreshonly => false, + timeout => 360, } exec{'initial_apt_dist_upgrade': command => "/usr/bin/apt-get -q -y -o 'DPkg::Options::=--force-confold' dist-upgrade", refreshonly => false, + timeout => 1200, } } } diff --git a/puppet/modules/site_apt/manifests/init.pp b/puppet/modules/site_apt/manifests/init.pp index 80c6fbde..8821c110 100644 --- a/puppet/modules/site_apt/manifests/init.pp +++ b/puppet/modules/site_apt/manifests/init.pp @@ -1,6 +1,20 @@ -class site_apt { +class site_apt { - include ::apt + # on couchdb we need to include squeeze in apt preferences, + # so the cloudant package can pull some packages from squeeze + # template() must be unquoted ! + if 'couchdb' in $::services { + $custom_preferences = template("site_apt/preferences.include_squeeze") + } else { + $custom_preferences = '' + } + class { 'apt': + custom_preferences => $custom_preferences, + custom_key_dir => 'puppet:///modules/site_apt/keys' + } + + # enable http://deb.leap.se debian package repository + include site_apt::leap_repo apt::apt_conf { '90disable-pdiffs': content => 'Acquire::PDiffs "false";'; @@ -8,8 +22,21 @@ class site_apt { include ::apt::unattended_upgrades - apt::sources_list { 'fallback.list.disabled': - content => template('site_apt/fallback.list'); + apt::sources_list { 'secondary.list.disabled': + content => template('site_apt/secondary.list'); } + apt::preferences_snippet { 'facter': + release => "${::lsbdistcodename}-backports", + priority => 999 + } + + # All packages should be installed _after_ refresh_apt is called, + # which does an apt-get update. + # There is one exception: + # The creation of sources.list depends on the lsb package + + File['/etc/apt/preferences'] -> + Exec['refresh_apt'] + Package <| ( title != 'lsb' ) |> } diff --git a/puppet/modules/site_apt/manifests/leap_repo.pp b/puppet/modules/site_apt/manifests/leap_repo.pp new file mode 100644 index 00000000..6b3d9919 --- /dev/null +++ b/puppet/modules/site_apt/manifests/leap_repo.pp @@ -0,0 +1,14 @@ +class site_apt::leap_repo { + apt::sources_list { 'leap.list': + content => 'deb http://deb.leap.se/debian stable main', + before => Exec[refresh_apt] + } + + package { 'leap-keyring': + ensure => latest + } + + # We wont be able to install the leap-keyring package unless the leap apt + # source has been added and apt has been refreshed + Exec['refresh_apt'] -> Package['leap-keyring'] +} diff --git a/puppet/modules/site_apt/templates/preferences.include_squeeze b/puppet/modules/site_apt/templates/preferences.include_squeeze new file mode 100644 index 00000000..d6d36b60 --- /dev/null +++ b/puppet/modules/site_apt/templates/preferences.include_squeeze @@ -0,0 +1,25 @@ +Explanation: Debian wheezy +Package: * +Pin: release o=Debian,n=wheezy +Pin-Priority: 990 + +Explanation: Debian wheezy-updates +Package: * +Pin: release o=Debian,n=wheezy-updates +Pin-Priority: 990 + +Explanation: Debian sid +Package: * +Pin: release o=Debian,n=sid +Pin-Priority: 1 + +Explanation: Debian squeeze +Package: * +Pin: release o=Debian,n=squeeze +Pin-Priority: 980 + +Explanation: Debian fallback +Package: * +Pin: release o=Debian +Pin-Priority: -10 + diff --git a/puppet/modules/site_apt/templates/fallback.list b/puppet/modules/site_apt/templates/secondary.list index 41334b0b..41334b0b 100644 --- a/puppet/modules/site_apt/templates/fallback.list +++ b/puppet/modules/site_apt/templates/secondary.list diff --git a/puppet/modules/site_ca_daemon/manifests/apache.pp b/puppet/modules/site_ca_daemon/manifests/apache.pp deleted file mode 100644 index ab6b08fd..00000000 --- a/puppet/modules/site_ca_daemon/manifests/apache.pp +++ /dev/null @@ -1,62 +0,0 @@ -class site_ca_daemon::apache { - - $api_domain = hiera('api_domain') - $x509 = hiera('x509') - $commercial_key = $x509['commercial_key'] - $commercial_cert = $x509['commercial_cert'] - $commercial_root = $x509['commercial_ca_cert'] - $api_key = $x509['key'] - $api_cert = $x509['cert'] - $api_root = $x509['ca_cert'] - - $apache_no_default_site = true - include apache::ssl - - apache::module { - 'alias': ensure => present; - 'rewrite': ensure => present; - 'headers': ensure => present; - } - - class { 'passenger': use_munin => false } - - apache::vhost::file { - 'leap_ca_daemon': - content => template('site_apache/vhosts.d/leap_ca_daemon.conf.erb') - } - - apache::vhost::file { - 'api': - content => template('site_apache/vhosts.d/api.conf.erb') - } - - x509::key { - 'leap_ca_daemon': - content => $commercial_key, - notify => Service[apache]; - - 'leap_api': - content => $api_key, - notify => Service[apache]; - } - - x509::cert { - 'leap_ca_daemon': - content => $commercial_cert, - notify => Service[apache]; - - 'leap_api': - content => $api_cert, - notify => Service[apache]; - } - - x509::ca { - 'leap_ca_daemon': - content => $commercial_root, - notify => Service[apache]; - - 'leap_api': - content => $api_root, - notify => Service[apache]; - } -} diff --git a/puppet/modules/site_ca_daemon/manifests/couchdb.pp b/puppet/modules/site_ca_daemon/manifests/couchdb.pp deleted file mode 100644 index f446a05b..00000000 --- a/puppet/modules/site_ca_daemon/manifests/couchdb.pp +++ /dev/null @@ -1,16 +0,0 @@ -class site_ca_daemon::couchdb { - - $ca = hiera('ca_daemon') - $couchdb_host = $ca['couchdb_hosts'] - $couchdb_user = $ca['couchdb_user']['username'] - $couchdb_password = $ca['couchdb_user']['password'] - - file { - '/etc/leap/leap_ca.yaml': - content => template('site_ca_daemon/leap_ca.yaml.erb'), - owner => leap_ca_daemon, - group => leap_ca_daemon, - mode => '0600'; - } - -} diff --git a/puppet/modules/site_ca_daemon/manifests/init.pp b/puppet/modules/site_ca_daemon/manifests/init.pp deleted file mode 100644 index 8ba9c506..00000000 --- a/puppet/modules/site_ca_daemon/manifests/init.pp +++ /dev/null @@ -1,103 +0,0 @@ -class site_ca_daemon { - tag 'leap_service' - #$definition_files = hiera('definition_files') - #$provider = $definition_files['provider'] - #$eip_service = $definition_files['eip_service'] - $x509 = hiera('x509') - - Class[Ruby] -> Class[rubygems] -> Class[bundler::install] - - class { 'ruby': ruby_version => '1.9.3' } - - class { 'bundler::install': install_method => 'package' } - - include rubygems - #include site_ca_daemon::apache - include site_ca_daemon::couchdb - - group { 'leap_ca_daemon': - ensure => present, - allowdupe => false; - } - - user { 'leap_ca_daemon': - ensure => present, - allowdupe => false, - gid => 'leap_ca_daemon', - home => '/srv/leap_ca_daemon', - require => [ Group['leap_ca_daemon'] ]; - } - - - x509::key { - 'leap_ca_daemon': - content => $x509['ca_key']; - #notify => Service['leap_ca_daemon']; <== no service yet for leap_ca_daemon - } - - x509::cert { - 'leap_ca_daemon': - content => $x509['ca_cert']; - #notify => Service['leap_ca_daemon']; <== no service yet for leap_ca_daemon - } - - # - # Does CA need a server key/cert? I think not now. - # - # x509::key { - # 'server': - # content => $x509['key']; - # } - # - # x509::cert { - # 'server': - # content => $x509['cert']; - # } - - # x509::ca { - # 'leap_ca_daemon': - # content => $x509['ca_cert']; - # } - - - file { '/srv/leap_ca_daemon': - ensure => directory, - owner => 'leap_ca_daemon', - group => 'leap_ca_daemon', - require => User['leap_ca_daemon']; - } - - vcsrepo { '/srv/leap_ca_daemon': - ensure => present, - revision => 'origin/master', - provider => git, - source => 'git://code.leap.se/leap_ca', - owner => 'leap_ca_daemon', - group => 'leap_ca_daemon', - require => [ User['leap_ca_daemon'], Group['leap_ca_daemon'] ], - notify => Exec['bundler_update'] - } - - exec { 'bundler_update': - cwd => '/srv/leap_ca_daemon', - command => '/bin/bash -c "/usr/bin/bundle check || /usr/bin/bundle install"', - unless => '/usr/bin/bundle check', - timeout => 600, - require => [ Class['bundler::install'], Vcsrepo['/srv/leap_ca_daemon'] ]; - } - - file { '/usr/local/bin/leap_ca_daemon': - ensure => link, - target => '/srv/leap_ca_daemon/bin/leap_ca_daemon', - } - - file { '/etc/cron.hourly/leap_ca': - ensure => present, - content => "#/bin/sh\n/srv/leap_ca_daemon/bin/leap_ca_daemon --run-once > /dev/null", - owner => 'root', - group => 0, - mode => '0755', - } - - -} diff --git a/puppet/modules/site_ca_daemon/templates/leap_ca.yaml.erb b/puppet/modules/site_ca_daemon/templates/leap_ca.yaml.erb deleted file mode 100644 index e0b95278..00000000 --- a/puppet/modules/site_ca_daemon/templates/leap_ca.yaml.erb +++ /dev/null @@ -1,31 +0,0 @@ -# -# Default configuration options for LEAP Certificate Authority Daemon -# - -# -# Certificate Authority -# -ca_key_path: "/etc/x509/keys/leap_ca_daemon.key" -ca_key_password: nil -ca_cert_path: "/etc/x509/certs/leap_ca_daemon.crt" - -# -# Certificate pool -# -max_pool_size: 100 -client_cert_lifespan: 2 -client_cert_bit_size: 2024 -client_cert_hash: "SHA256" - -# -# Database -# -db_name: "client_certificates" -couch_connection: - protocol: "https" - host: <%= couchdb_host %> - port: 6984 - username: <%= couchdb_user %> - password: <%= couchdb_password %> - prefix: "" - suffix: "" diff --git a/puppet/modules/site_config/files/xterm-title.sh b/puppet/modules/site_config/files/xterm-title.sh new file mode 100644 index 00000000..3cff0e3a --- /dev/null +++ b/puppet/modules/site_config/files/xterm-title.sh @@ -0,0 +1,8 @@ +# If this is an xterm set the title to user@host:dir +case "$TERM" in +xterm*|rxvt*) + PROMPT_COMMAND='echo -ne "\033]0;${USER}@${HOSTNAME}: ${PWD}\007"' + ;; +*) + ;; +esac diff --git a/puppet/modules/site_config/manifests/base_packages.pp b/puppet/modules/site_config/manifests/base_packages.pp new file mode 100644 index 00000000..3d40f7a2 --- /dev/null +++ b/puppet/modules/site_config/manifests/base_packages.pp @@ -0,0 +1,28 @@ +class site_config::base_packages { + + # base set of packages that we want to have installed everywhere + package { [ 'etckeeper', 'screen', 'less' ]: + ensure => installed, + } + + # base set of packages that we want to remove everywhere + package { [ 'acpi', 'acpid', 'acpi-support-base', 'eject', 'ftp', + 'laptop-detect', 'lpr', 'nfs-common', 'nfs-kernel-server', + 'portmap', 'pppconfig', 'pppoe', 'pump', 'qstat', 'rpcbind', + 'samba-common', 'samba-common-bin', 'smbclient', 'tcl8.5', + 'tk8.5', 'os-prober', 'unzip', 'xauth', 'x11-common', + 'x11-utils', 'xterm' ]: + ensure => absent; + } + + if $::virtual == 'virtualbox' { + $virtualbox_ensure = present + } else { + $virtualbox_ensure = absent + } + + package { [ 'build-essential', 'fontconfig-config', 'g++', 'g++-4.7', 'gcc', + 'gcc-4.6', 'gcc-4.7', 'cpp', 'cpp-4.6', 'cpp-4.7', 'libc6-dev' ]: + ensure => $virtualbox_ensure + } +} diff --git a/puppet/modules/site_config/manifests/default.pp b/puppet/modules/site_config/manifests/default.pp index 2191e9a1..00eee9d0 100644 --- a/puppet/modules/site_config/manifests/default.pp +++ b/puppet/modules/site_config/manifests/default.pp @@ -12,10 +12,14 @@ class site_config::default { # configure apt include site_apt - # configure ssh and include ssh-keys include site_config::sshd + # fix dhclient from changing resolver information + if $::ec2_instance_id { + include site_config::dhclient + } + # configure /etc/resolv.conf include site_config::resolvconf @@ -24,13 +28,17 @@ class site_config::default { # configure /etc/hosts class { 'site_config::hosts': - stage => initial, + stage => setup, } - package { [ 'etckeeper' ]: - ensure => installed, - } + # install/remove base packages + include site_config::base_packages # include basic shorewall config include site_shorewall::defaults + + Class['git'] -> Vcsrepo<||> + + # include basic shell config + include site_config::shell } diff --git a/puppet/modules/site_config/manifests/dhclient.pp b/puppet/modules/site_config/manifests/dhclient.pp new file mode 100644 index 00000000..7ac0caf3 --- /dev/null +++ b/puppet/modules/site_config/manifests/dhclient.pp @@ -0,0 +1,30 @@ +class site_config::dhclient { + + # Unfortunately, there does not seem to be a way to reload the dhclient.conf + # config file, or a convenient way to disable the modifications to + # /etc/resolv.conf. So the following makes the functions involved noops and + # ships a script to kill and restart dhclient. See the debian bugs: + # #681698, #712796 + + include site_config::params + + file { '/usr/local/sbin/reload_dhclient': + owner => 0, + group => 0, + mode => '0755', + content => template('site_config/reload_dhclient.erb'); + } + + exec { 'reload_dhclient': + refreshonly => true, + command => '/usr/local/sbin/reload_dhclient'; + } + + file { '/etc/dhcp/dhclient-enter-hooks.d/disable_resolvconf': + content => 'make_resolv_conf() { : ; } ; set_hostname() { : ; }', + mode => '0644', + owner => 'root', + group => 'root', + notify => Exec['reload_dhclient']; + } +} diff --git a/puppet/modules/site_config/manifests/hosts.pp b/puppet/modules/site_config/manifests/hosts.pp index 6c00f3b6..ccedf036 100644 --- a/puppet/modules/site_config/manifests/hosts.pp +++ b/puppet/modules/site_config/manifests/hosts.pp @@ -1,22 +1,34 @@ class site_config::hosts() { + $hosts = hiera('hosts','') + $hostname = hiera('name') + $domain_hash = hiera('domain') + $domain_public = $domain_hash['full_suffix'] - $hosts = hiera('hosts','') - $hostname = hiera('name') - - $domain_public = $site_config::default::domain_hash['full_suffix'] - - file { "/etc/hostname": - ensure => present, + file { '/etc/hostname': + ensure => present, content => $hostname } - exec { "/bin/hostname $hostname": + exec { "/bin/hostname ${hostname}": subscribe => [ File['/etc/hostname'], File['/etc/hosts'] ], refreshonly => true; } + # we depend on reliable hostnames from /etc/hosts for the stunnel services + # so restart stunnel service when /etc/hosts is modified + # because this is done in an early stage, the stunnel module may not + # have been deployed and will not be available for overriding, so + # this is handled in an unorthodox manner + exec { '/etc/init.d/stunnel4 restart': + subscribe => File['/etc/hosts'], + refreshonly => true, + onlyif => 'test -f /etc/init.d/stunnel4'; + } + file { '/etc/hosts': content => template('site_config/hosts'), - mode => '0644', owner => root, group => root; + mode => '0644', + owner => root, + group => root; } } diff --git a/puppet/modules/site_config/manifests/params.pp b/puppet/modules/site_config/manifests/params.pp new file mode 100644 index 00000000..237ee454 --- /dev/null +++ b/puppet/modules/site_config/manifests/params.pp @@ -0,0 +1,25 @@ +class site_config::params { + + $ip_address = hiera('ip_address') + $ip_address_interface = getvar("interface_${ip_address}") + $ec2_local_ipv4_interface = getvar("interface_${::ec2_local_ipv4}") + + if $::virtual == 'virtualbox' { + $interface = [ 'eth0', 'eth1' ] + } + elsif hiera('interface','') != '' { + $interface = hiera('interface') + } + elsif $ip_address_interface != '' { + $interface = $ip_address_interface + } + elsif $ec2_local_ipv4_interface != '' { + $interface = $ec2_local_ipv4_interface + } + elsif $::interfaces =~ /eth0/ { + $interface = eth0 + } + else { + fail("unable to determine a valid interface, please set a valid interface for this node in nodes/${::hostname}.json") + } +} diff --git a/puppet/modules/site_config/manifests/resolvconf.pp b/puppet/modules/site_config/manifests/resolvconf.pp index d73f0b78..271c5043 100644 --- a/puppet/modules/site_config/manifests/resolvconf.pp +++ b/puppet/modules/site_config/manifests/resolvconf.pp @@ -1,16 +1,5 @@ class site_config::resolvconf { - # bind9 purging can be taken out after some time - package { 'bind9': - ensure => absent, - } - file { '/etc/default/bind9': - ensure => absent; - } - file { '/etc/bind/named.conf.options': - ensure => absent; - } - $domain_public = $site_config::default::domain_hash['full_suffix'] # 127.0.0.1: caching-only local bind diff --git a/puppet/modules/site_config/manifests/ruby.pp b/puppet/modules/site_config/manifests/ruby.pp new file mode 100644 index 00000000..2a720114 --- /dev/null +++ b/puppet/modules/site_config/manifests/ruby.pp @@ -0,0 +1,14 @@ +class site_config::ruby { + Class[Ruby] -> Class[rubygems] -> Class[bundler::install] + class { '::ruby': ruby_version => '1.9.3' } + class { 'bundler::install': install_method => 'package' } + include rubygems +} + + +# +# Ruby settings common to all servers +# +# Why this way? So that other classes can do 'include site_ruby' without creating redeclaration errors. +# See https://puppetlabs.com/blog/modeling-class-composition-with-parameterized-classes/ +# diff --git a/puppet/modules/site_config/manifests/shell.pp b/puppet/modules/site_config/manifests/shell.pp new file mode 100644 index 00000000..5b8c025d --- /dev/null +++ b/puppet/modules/site_config/manifests/shell.pp @@ -0,0 +1,22 @@ +class site_config::shell { + + file { + '/etc/profile.d/leap_path.sh': + content => 'PATH=$PATH:/srv/leap/bin', + mode => '0644', + owner => root, + group => root; + } + + ## + ## XTERM TITLE + ## + + file { '/etc/profile.d/xterm-title.sh': + source => 'puppet:///modules/site_config/xterm-title.sh', + owner => root, + group => 0, + mode => '0644'; + } + +} diff --git a/puppet/modules/site_config/manifests/slow.pp b/puppet/modules/site_config/manifests/slow.pp index 18b22a9c..94bac88d 100644 --- a/puppet/modules/site_config/manifests/slow.pp +++ b/puppet/modules/site_config/manifests/slow.pp @@ -1,6 +1,6 @@ class site_config::slow { tag 'leap_slow' class { 'site_apt::dist_upgrade': - stage => initial, + stage => setup, } } diff --git a/puppet/modules/site_config/manifests/sshd.pp b/puppet/modules/site_config/manifests/sshd.pp index 944dbce2..8ff337a0 100644 --- a/puppet/modules/site_config/manifests/sshd.pp +++ b/puppet/modules/site_config/manifests/sshd.pp @@ -2,7 +2,7 @@ class site_config::sshd { # configure sshd include sshd include site_sshd - # no need for configuring authorized_keys as leap_cli cares for that + # no need for configuring authorized_keys as leap_cli cares for that #$ssh_pubkeys=hiera_hash('ssh_pubkeys') #notice($ssh_pubkeys) #create_resources('site_sshd::ssh_key', $ssh_pubkeys) diff --git a/puppet/modules/site_config/templates/hosts b/puppet/modules/site_config/templates/hosts index 00cc6a79..2c784b05 100644 --- a/puppet/modules/site_config/templates/hosts +++ b/puppet/modules/site_config/templates/hosts @@ -1,10 +1,12 @@ # This file is managed by puppet, any changes will be overwritten! 127.0.0.1 localhost -127.0.1.1 <%= hostname %>.<%= @domain_public %> <%= hostname %> +127.0.1.1 <%= @hostname %>.<%= @domain_public %> <%= @hostname %> -<%- if hosts.to_s != '' then -%> -<%= hosts %> +<%- if @hosts then -%> +<% @hosts.each do |name, props| -%> +<%= props["ip_address"] %> <%= props["domain_full"] %> <%= props["domain_internal"] %> <%= name %> +<% end -%> <% end -%> # The following lines are desirable for IPv6 capable hosts diff --git a/puppet/modules/site_config/templates/reload_dhclient.erb b/puppet/modules/site_config/templates/reload_dhclient.erb new file mode 100644 index 00000000..075828b7 --- /dev/null +++ b/puppet/modules/site_config/templates/reload_dhclient.erb @@ -0,0 +1,13 @@ +#!/bin/sh + +# Get the PID +PIDFILE='/var/run/dhclient.<%= scope.lookupvar('site_config::params::interface') %>.pid' + +# Capture how dhclient is currently running so we can relaunch it +dhclient=`/bin/ps --no-headers --pid $(cat $PIDFILE) -f | /usr/bin/awk '{for(i=8;i<=NF;++i) printf("%s ", $i) }'` + +# Kill the current dhclient +/usr/bin/pkill -F $PIDFILE + +# Restart dhclient with the arguments it had previously +$dhclient diff --git a/puppet/modules/site_couchdb/files/couchdb b/puppet/modules/site_couchdb/files/couchdb deleted file mode 100755 index ccdfe716..00000000 --- a/puppet/modules/site_couchdb/files/couchdb +++ /dev/null @@ -1,160 +0,0 @@ -#!/bin/sh -e - -# Licensed under the Apache License, Version 2.0 (the "License"); you may not -# use this file except in compliance with the License. You may obtain a copy of -# the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT -# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the -# License for the specific language governing permissions and limitations under -# the License. - -### BEGIN INIT INFO -# Provides: couchdb -# Required-Start: $local_fs $remote_fs -# Required-Stop: $local_fs $remote_fs -# Default-Start: 2 3 4 5 -# Default-Stop: 0 1 6 -# Short-Description: Apache CouchDB init script -# Description: Apache CouchDB init script for the database server. -### END INIT INFO - -SCRIPT_OK=0 -SCRIPT_ERROR=1 - -DESCRIPTION="database server" -NAME=couchdb -SCRIPT_NAME=`basename $0` -COUCHDB=/usr/bin/couchdb -CONFIGURATION_FILE=/etc/default/couchdb -RUN_DIR=/var/run/couchdb -LSB_LIBRARY=/lib/lsb/init-functions - -if test ! -x $COUCHDB; then - exit $SCRIPT_ERROR -fi - -if test -r $CONFIGURATION_FILE; then - . $CONFIGURATION_FILE -fi - -log_daemon_msg () { - # Dummy function to be replaced by LSB library. - - echo $@ -} - -log_end_msg () { - # Dummy function to be replaced by LSB library. - - if test "$1" != "0"; then - echo "Error with $DESCRIPTION: $NAME" - fi - return $1 -} - -if test -r $LSB_LIBRARY; then - . $LSB_LIBRARY -fi - -run_command () { - command="$1" - if test -n "$COUCHDB_OPTIONS"; then - command="$command $COUCHDB_OPTIONS" - fi - if test -n "$COUCHDB_USER"; then - if su $COUCHDB_USER -c "$command"; then - return $SCRIPT_OK - else - return $SCRIPT_ERROR - fi - else - if $command; then - return $SCRIPT_OK - else - return $SCRIPT_ERROR - fi - fi -} - -start_couchdb () { - # Start Apache CouchDB as a background process. - - mkdir -p "$RUN_DIR" - chown -R "$COUCHDB_USER" "$RUN_DIR" - command="$COUCHDB -b" - if test -n "$COUCHDB_STDOUT_FILE"; then - command="$command -o $COUCHDB_STDOUT_FILE" - fi - if test -n "$COUCHDB_STDERR_FILE"; then - command="$command -e $COUCHDB_STDERR_FILE" - fi - if test -n "$COUCHDB_RESPAWN_TIMEOUT"; then - command="$command -r $COUCHDB_RESPAWN_TIMEOUT" - fi - run_command "$command" > /dev/null -} - -stop_couchdb () { - # Stop the running Apache CouchDB process. - - run_command "$COUCHDB -d" > /dev/null - pkill -u couchdb - # always return true even if no remaining couchdb procs got killed - /bin/true -} - -display_status () { - # Display the status of the running Apache CouchDB process. - - run_command "$COUCHDB -s" -} - -parse_script_option_list () { - # Parse arguments passed to the script and take appropriate action. - - case "$1" in - start) - log_daemon_msg "Starting $DESCRIPTION" $NAME - if start_couchdb; then - log_end_msg $SCRIPT_OK - else - log_end_msg $SCRIPT_ERROR - fi - ;; - stop) - log_daemon_msg "Stopping $DESCRIPTION" $NAME - if stop_couchdb; then - log_end_msg $SCRIPT_OK - else - log_end_msg $SCRIPT_ERROR - fi - ;; - restart|force-reload) - log_daemon_msg "Restarting $DESCRIPTION" $NAME - if stop_couchdb; then - if start_couchdb; then - log_end_msg $SCRIPT_OK - else - log_end_msg $SCRIPT_ERROR - fi - else - log_end_msg $SCRIPT_ERROR - fi - ;; - status) - display_status - ;; - *) - cat << EOF >&2 -Usage: $SCRIPT_NAME {start|stop|restart|force-reload|status} -EOF - exit $SCRIPT_ERROR - ;; - esac -} - -parse_script_option_list $@ diff --git a/puppet/modules/site_couchdb/files/local.ini b/puppet/modules/site_couchdb/files/local.ini index b3376cbb..22aa0177 100644 --- a/puppet/modules/site_couchdb/files/local.ini +++ b/puppet/modules/site_couchdb/files/local.ini @@ -28,8 +28,10 @@ [httpd_global_handlers] ;_google = {couch_httpd_proxy, handle_proxy_req, <<"http://www.google.com">>} +# futon is enabled by default on bigcouch in default.ini +# we need to find another way to disable futon, it won't work disabling it here # enable futon -_utils = {couch_httpd_misc_handlers, handle_utils_dir_req, "/usr/share/couchdb/www"} +#_utils = {couch_httpd_misc_handlers, handle_utils_dir_req, "/usr/share/couchdb/www"} # disable futon #_utils = {couch_httpd_misc_handlers, handle_welcome_req, <<"Welcome, Futon is disabled!">>} diff --git a/puppet/modules/site_couchdb/manifests/apache_ssl_proxy.pp b/puppet/modules/site_couchdb/manifests/apache_ssl_proxy.pp deleted file mode 100644 index 7739473e..00000000 --- a/puppet/modules/site_couchdb/manifests/apache_ssl_proxy.pp +++ /dev/null @@ -1,25 +0,0 @@ -define site_couchdb::apache_ssl_proxy ($key, $cert) { - - $apache_no_default_site = true - include apache - apache::module { - 'proxy': ensure => present; - 'proxy_http': ensure => present; - 'rewrite': ensure => present; - 'ssl': ensure => present; - } - apache::vhost::file { 'couchdb_proxy': } - - x509::key { - 'leap_couchdb': - content => $key, - notify => Service[apache]; - } - - x509::cert { - 'leap_couchdb': - content => $cert, - notify => Service[apache]; - } - -} diff --git a/puppet/modules/site_couchdb/manifests/bigcouch/add_nodes.pp b/puppet/modules/site_couchdb/manifests/bigcouch/add_nodes.pp new file mode 100644 index 00000000..241a4914 --- /dev/null +++ b/puppet/modules/site_couchdb/manifests/bigcouch/add_nodes.pp @@ -0,0 +1,5 @@ +class site_couchdb::bigcouch::add_nodes { + # loop through neighbors array and add nodes + $nodes = $::site_couchdb::bigcouch_config['neighbors'] + couchdb::bigcouch::add_node { $nodes: } +} diff --git a/puppet/modules/site_couchdb/manifests/configure.pp b/puppet/modules/site_couchdb/manifests/configure.pp deleted file mode 100644 index 333511b5..00000000 --- a/puppet/modules/site_couchdb/manifests/configure.pp +++ /dev/null @@ -1,27 +0,0 @@ -class site_couchdb::configure { - - file { '/etc/init.d/couchdb': - source => 'puppet:///modules/site_couchdb/couchdb', - mode => '0755', - owner => 'root', - group => 'root', - } - - file { '/etc/couchdb/local.d/admin.ini': - content => "[admins] -admin = $site_couchdb::couchdb_admin_pw -", - mode => '0600', - owner => 'couchdb', - group => 'couchdb', - notify => Service[couchdb] - } - - - exec { '/etc/init.d/couchdb restart; sleep 6': - path => ['/bin', '/usr/bin',], - subscribe => File['/etc/couchdb/local.d/admin.ini', - '/etc/couchdb/local.ini'], - refreshonly => true - } -} diff --git a/puppet/modules/site_couchdb/manifests/init.pp b/puppet/modules/site_couchdb/manifests/init.pp index 9ecde5e6..802f3224 100644 --- a/puppet/modules/site_couchdb/manifests/init.pp +++ b/puppet/modules/site_couchdb/manifests/init.pp @@ -1,64 +1,83 @@ class site_couchdb { tag 'leap_service' - include couchdb $x509 = hiera('x509') $key = $x509['key'] $cert = $x509['cert'] + $ca = $x509['ca_cert'] + $couchdb_config = hiera('couch') $couchdb_users = $couchdb_config['users'] $couchdb_admin = $couchdb_users['admin'] $couchdb_admin_user = $couchdb_admin['username'] $couchdb_admin_pw = $couchdb_admin['password'] + $couchdb_admin_salt = $couchdb_admin['salt'] $couchdb_webapp = $couchdb_users['webapp'] $couchdb_webapp_user = $couchdb_webapp['username'] $couchdb_webapp_pw = $couchdb_webapp['password'] - $couchdb_ca_daemon = $couchdb_users['ca_daemon'] - $couchdb_ca_daemon_user = $couchdb_ca_daemon['username'] - $couchdb_ca_daemon_pw = $couchdb_ca_daemon['password'] + $couchdb_webapp_salt = $couchdb_webapp['salt'] + $couchdb_soledad = $couchdb_users['soledad'] + $couchdb_soledad_user = $couchdb_soledad['username'] + $couchdb_soledad_pw = $couchdb_soledad['password'] + $couchdb_soledad_salt = $couchdb_soledad['salt'] + + $bigcouch_config = $couchdb_config['bigcouch'] + $bigcouch_cookie = $bigcouch_config['cookie'] + + $ednp_port = $bigcouch_config['ednp_port'] + + class { 'couchdb': + bigcouch => true, + admin_pw => $couchdb_admin_pw, + admin_salt => $couchdb_admin_salt, + bigcouch_cookie => $bigcouch_cookie, + ednp_port => $ednp_port + } + + class { 'couchdb::bigcouch::package::cloudant': } - Package ['couchdb'] - -> File['/etc/init.d/couchdb'] - -> File['/etc/couchdb/local.ini'] - -> File['/etc/couchdb/local.d/admin.ini'] - -> File['/etc/couchdb/couchdb.netrc'] + Class ['couchdb::bigcouch::package::cloudant'] + -> Service ['couchdb'] + -> Class ['site_couchdb::bigcouch::add_nodes'] -> Couchdb::Create_db['users'] - -> Couchdb::Create_db['client_certificates'] + -> Couchdb::Create_db['tokens'] -> Couchdb::Add_user[$couchdb_webapp_user] - -> Couchdb::Add_user[$couchdb_ca_daemon_user] - -> Site_couchdb::Apache_ssl_proxy['apache_ssl_proxy'] + -> Couchdb::Add_user[$couchdb_soledad_user] - include site_couchdb::configure - include couchdb::deploy_config - - site_couchdb::apache_ssl_proxy { 'apache_ssl_proxy': - key => $key, - cert => $cert + class { 'site_couchdb::stunnel': + key => $key, + cert => $cert, + ca => $ca } + class { 'site_couchdb::bigcouch::add_nodes': } + couchdb::query::setup { 'localhost': user => $couchdb_admin_user, - pw => $couchdb_admin_pw + pw => $couchdb_admin_pw, } # Populate couchdb couchdb::add_user { $couchdb_webapp_user: - roles => '["certs"]', - pw => $couchdb_webapp_pw + roles => '["auth"]', + pw => $couchdb_webapp_pw, + salt => $couchdb_webapp_salt } - couchdb::add_user { $couchdb_ca_daemon_user: - roles => '["certs"]', - pw => $couchdb_ca_daemon_pw + couchdb::add_user { $couchdb_soledad_user: + roles => '["auth"]', + pw => $couchdb_soledad_pw, + salt => $couchdb_soledad_salt } couchdb::create_db { 'users': readers => "{ \"names\": [\"$couchdb_webapp_user\"], \"roles\": [] }" } - couchdb::create_db { 'client_certificates': - readers => "{ \"names\": [], \"roles\": [\"certs\"] }" + couchdb::create_db { 'tokens': + readers => "{ \"names\": [], \"roles\": [\"auth\"] }" } include site_shorewall::couchdb + include site_shorewall::couchdb::bigcouch } diff --git a/puppet/modules/site_couchdb/manifests/stunnel.pp b/puppet/modules/site_couchdb/manifests/stunnel.pp new file mode 100644 index 00000000..d982013e --- /dev/null +++ b/puppet/modules/site_couchdb/manifests/stunnel.pp @@ -0,0 +1,104 @@ +class site_couchdb::stunnel ($key, $cert, $ca) { + + $stunnel = hiera('stunnel') + + $couch_server = $stunnel['couch_server'] + $couch_server_accept = $couch_server['accept'] + $couch_server_connect = $couch_server['connect'] + + # Erlang Port Mapper Daemon (epmd) stunnel server/clients + $epmd_server = $stunnel['epmd_server'] + $epmd_server_accept = $epmd_server['accept'] + $epmd_server_connect = $epmd_server['connect'] + $epmd_clients = $stunnel['epmd_clients'] + + # Erlang Distributed Node Protocol (ednp) stunnel server/clients + $ednp_server = $stunnel['ednp_server'] + $ednp_server_accept = $ednp_server['accept'] + $ednp_server_connect = $ednp_server['connect'] + $ednp_clients = $stunnel['ednp_clients'] + + include x509::variables + $cert_name = 'leap_couchdb' + $ca_name = 'leap_ca' + $ca_path = "${x509::variables::local_CAs}/${ca_name}.crt" + $cert_path = "${x509::variables::certs}/${cert_name}.crt" + $key_path = "${x509::variables::keys}/${cert_name}.key" + + # basic setup: ensure cert, key, ca files are in place, and some generic + # stunnel things are done + class { 'site_stunnel::setup': + cert_name => $cert_name, + key => $key, + cert => $cert, + ca_name => $ca_name, + ca => $ca + } + + # setup a stunnel server for the webapp to connect to couchdb + stunnel::service { 'couch_server': + accept => $couch_server_accept, + connect => $couch_server_connect, + client => false, + cafile => $ca_path, + key => $key_path, + cert => $cert_path, + verify => '2', + pid => '/var/run/stunnel4/couchserver.pid', + rndfile => '/var/lib/stunnel4/.rnd', + debuglevel => '4' + } + + + # setup stunnel server for Erlang Port Mapper Daemon (epmd), necessary for + # bigcouch clustering between each bigcouchdb node + stunnel::service { 'epmd_server': + accept => $epmd_server_accept, + connect => $epmd_server_connect, + client => false, + cafile => $ca_path, + key => $key_path, + cert => $cert_path, + verify => '2', + pid => '/var/run/stunnel4/epmd_server.pid', + rndfile => '/var/lib/stunnel4/.rnd', + debuglevel => '4' + } + + # setup stunnel clients for Erlang Port Mapper Daemon (epmd) to connect + # to the above epmd stunnel server. + $epmd_client_defaults = { + 'client' => true, + 'cafile' => $ca_path, + 'key' => $key_path, + 'cert' => $cert_path, + } + + create_resources(site_stunnel::clients, $epmd_clients, $epmd_client_defaults) + + # setup stunnel server for Erlang Distributed Node Protocol (ednp), necessary + # for bigcouch clustering between each bigcouchdb node + stunnel::service { 'ednp_server': + accept => $ednp_server_accept, + connect => $ednp_server_connect, + client => false, + cafile => $ca_path, + key => $key_path, + cert => $cert_path, + verify => '2', + pid => '/var/run/stunnel4/ednp_server.pid', + rndfile => '/var/lib/stunnel4/.rnd', + debuglevel => '4' + } + + # setup stunnel clients for Erlang Distributed Node Protocol (ednp) to connect + # to the above ednp stunnel server. + $ednp_client_defaults = { + 'client' => true, + 'cafile' => $ca_path, + 'key' => $key_path, + 'cert' => $cert_path, + } + + create_resources(site_stunnel::clients, $ednp_clients, $ednp_client_defaults) +} diff --git a/puppet/modules/site_haproxy/manifests/init.pp b/puppet/modules/site_haproxy/manifests/init.pp new file mode 100644 index 00000000..ace88a7b --- /dev/null +++ b/puppet/modules/site_haproxy/manifests/init.pp @@ -0,0 +1,26 @@ +class site_haproxy { + + class { 'haproxy': + enable => true, + version => '1.4.23-0.1~leap60+1', + manage_service => true, + global_options => { + 'log' => '127.0.0.1 local0', + 'maxconn' => '4096', + 'stats' => 'socket /var/run/haproxy.sock user haproxy group haproxy', + 'chroot' => '/usr/share/haproxy', + 'user' => 'haproxy', + 'group' => 'haproxy', + 'daemon' => '' + }, + defaults_options => { + 'log' => 'global', + 'retries' => '3', + 'option' => 'redispatch', + 'timeout connect' => '4000', + 'timeout client' => '20000', + 'timeout server' => '20000' + } + } + +} diff --git a/puppet/modules/site_nagios/manifests/server.pp b/puppet/modules/site_nagios/manifests/server.pp index c98a8a1f..c114a39a 100644 --- a/puppet/modules/site_nagios/manifests/server.pp +++ b/puppet/modules/site_nagios/manifests/server.pp @@ -2,7 +2,7 @@ class site_nagios::server inherits nagios::base { # First, purge old nagios config (see #1467) class { 'site_nagios::server::purge': - stage => initial + stage => setup } $nagios_hiera=hiera('nagios') diff --git a/puppet/modules/site_nickserver/manifests/init.pp b/puppet/modules/site_nickserver/manifests/init.pp new file mode 100644 index 00000000..7dfa2603 --- /dev/null +++ b/puppet/modules/site_nickserver/manifests/init.pp @@ -0,0 +1,162 @@ +# +# TODO: currently, this is dependent on some things that are set up in site_webapp +# +# (1) HAProxy -> couchdb +# (2) Apache +# +# It would be good in the future to make nickserver installable independently of site_webapp. +# + +class site_nickserver { + tag 'leap_service' + include site_config::ruby + + # + # VARIABLES + # + + $nickserver = hiera('nickserver') + $nickserver_port = $nickserver['port'] # the port that public connects to (should be 6425) + $nickserver_local_port = '64250' # the port that nickserver is actually running on + $nickserver_domain = $nickserver['domain'] + + $couchdb_user = $nickserver['couchdb_user']['username'] + $couchdb_password = $nickserver['couchdb_user']['password'] + $couchdb_host = 'localhost' # couchdb is available on localhost via haproxy, which is bound to 4096. + $couchdb_port = '4096' # See site_webapp/templates/haproxy_couchdb.cfg.erg + + # temporarily for now: + $domain = hiera('domain') + $address_domain = $domain['full_suffix'] + $x509 = hiera('x509') + $x509_key = $x509['key'] + $x509_cert = $x509['cert'] + $x509_ca = $x509['ca_cert'] + + # + # USER AND GROUP + # + + group { 'nickserver': + ensure => present, + allowdupe => false; + } + user { 'nickserver': + ensure => present, + allowdupe => false, + gid => 'nickserver', + home => '/srv/leap/nickserver', + require => Group['nickserver']; + } + + # + # NICKSERVER CODE + # NOTE: in order to support TLS, libssl-dev must be installed before EventMachine gem + # is built/installed. + # + + package { + 'libssl-dev': ensure => installed; + } + vcsrepo { '/srv/leap/nickserver': + ensure => present, + revision => 'origin/master', + provider => git, + source => 'git://code.leap.se/nickserver', + owner => 'nickserver', + group => 'nickserver', + require => [ User['nickserver'], Group['nickserver'] ], + notify => Exec['nickserver_bundler_update']; + } + exec { 'nickserver_bundler_update': + cwd => '/srv/leap/nickserver', + command => '/bin/bash -c "/usr/bin/bundle check || /usr/bin/bundle install --path vendor/bundle"', + unless => '/usr/bin/bundle check', + user => 'nickserver', + timeout => 600, + require => [ Class['bundler::install'], Vcsrepo['/srv/leap/nickserver'], Package['libssl-dev'] ], + notify => Service['nickserver']; + } + + # + # NICKSERVER CONFIG + # + + file { '/etc/leap/nickserver.yml': + content => template('site_nickserver/nickserver.yml.erb'), + owner => nickserver, + group => nickserver, + mode => '0600', + notify => Service['nickserver']; + } + + # + # NICKSERVER DAEMON + # + + file { + '/usr/bin/nickserver': + ensure => link, + target => '/srv/leap/nickserver/bin/nickserver', + require => Vcsrepo['/srv/leap/nickserver']; + '/etc/init.d/nickserver': + owner => root, group => 0, mode => '0755', + source => '/srv/leap/nickserver/dist/debian-init-script', + require => Vcsrepo['/srv/leap/nickserver']; + } + + service { 'nickserver': + ensure => running, + enable => true, + hasrestart => true, + hasstatus => true, + require => File['/etc/init.d/nickserver']; + } + + # + # FIREWALL + # poke a hole in the firewall to allow nickserver requests + # + + file { '/etc/shorewall/macro.nickserver': + content => "PARAM - - tcp $nickserver_port", + notify => Service['shorewall'], + require => Package['shorewall']; + } + + shorewall::rule { 'net2fw-nickserver': + source => 'net', + destination => '$FW', + action => 'nickserver(ACCEPT)', + order => 200; + } + + # + # APACHE REVERSE PROXY + # nickserver doesn't speak TLS natively, let Apache handle that. + # + + apache::module { + 'proxy': ensure => present; + 'proxy_http': ensure => present + } + + apache::vhost::file { + 'nickserver': content => template('site_nickserver/nickserver-proxy.conf.erb') + } + + x509::key { 'nickserver': + content => $x509_key, + notify => Service[apache]; + } + + x509::cert { 'nickserver': + content => $x509_cert, + notify => Service[apache]; + } + + x509::ca { 'nickserver': + content => $x509_ca, + notify => Service[apache]; + } +}
\ No newline at end of file diff --git a/puppet/modules/site_nickserver/templates/nickserver-proxy.conf.erb b/puppet/modules/site_nickserver/templates/nickserver-proxy.conf.erb new file mode 100644 index 00000000..67896cd3 --- /dev/null +++ b/puppet/modules/site_nickserver/templates/nickserver-proxy.conf.erb @@ -0,0 +1,23 @@ +# +# Apache reverse proxy configuration for the Nickserver +# + +Listen 0.0.0.0:<%= @nickserver_port -%> + +<VirtualHost *:<%= @nickserver_port -%>> + ServerName <%= @nickserver_domain %> + ServerAlias <%= @address_domain %> + + SSLEngine on + SSLProtocol -all +SSLv3 +TLSv1 + SSLCipherSuite HIGH:MEDIUM:!aNULL:!SSLv2:!MD5:@STRENGTH + SSLHonorCipherOrder on + + SSLCACertificatePath /etc/ssl/certs + SSLCertificateChainFile /etc/ssl/certs/nickserver.pem + SSLCertificateKeyFile /etc/x509/keys/nickserver.key + SSLCertificateFile /etc/x509/certs/nickserver.crt + + ProxyPass / http://localhost:<%= @nickserver_local_port %>/ + ProxyPreserveHost On # preserve Host header in HTTP request +</VirtualHost> diff --git a/puppet/modules/site_nickserver/templates/nickserver.yml.erb b/puppet/modules/site_nickserver/templates/nickserver.yml.erb new file mode 100644 index 00000000..7aab5605 --- /dev/null +++ b/puppet/modules/site_nickserver/templates/nickserver.yml.erb @@ -0,0 +1,19 @@ +# +# configuration for nickserver. +# + +domain: "<%= @address_domain %>" + +couch_host: "<%= @couchdb_host %>" +couch_port: <%= @couchdb_port %> +couch_database: "users" +couch_user: "<%= @couchdb_user %>" +couch_password: "<%= @couchdb_password %>" + +hkp_url: "https://hkps.pool.sks-keyservers.net:/pks/lookup" + +user: "nickserver" +port: <%= @nickserver_local_port %> +pid_file: "/var/run/nickserver" +log_file: "/var/log/nickserver.log" + diff --git a/puppet/modules/site_openvpn/README b/puppet/modules/site_openvpn/README new file mode 100644 index 00000000..cef5be23 --- /dev/null +++ b/puppet/modules/site_openvpn/README @@ -0,0 +1,20 @@ +Place to look when debugging problems +======================================== + +Log files: + + openvpn: /var/log/syslog + shorewall: /var/log/syslog + shorewall startup: /var/log/shorewall-init.log + +Check NAT masq: + + iptables -t nat --list-rules + +Check interfaces: + + ip addr ls + +Scripts: + + /usr/local/bin/add_gateway_ips.sh
\ No newline at end of file diff --git a/puppet/modules/site_openvpn/manifests/init.pp b/puppet/modules/site_openvpn/manifests/init.pp index e3d2a9af..4f900623 100644 --- a/puppet/modules/site_openvpn/manifests/init.pp +++ b/puppet/modules/site_openvpn/manifests/init.pp @@ -1,55 +1,141 @@ +# +# An openvpn gateway can support three modes: +# +# (1) limited and unlimited +# (2) unlimited only +# (3) limited only +# +# The difference is that 'unlimited' gateways only allow client certs that match the 'unlimited_prefix', +# and 'limited' gateways only allow certs that match the 'limited_prefix'. +# +# We potentially create four openvpn config files (thus four daemons): +# +# (1) unlimited + tcp => tcp_config.conf +# (2) unlimited + udp => udp_config.conf +# (3) limited + tcp => limited_tcp_config.conf +# (4) limited + udp => limited_udp_config.conf +# + class site_openvpn { tag 'leap_service' - # parse hiera config - $ip_address = hiera('ip_address') - $interface = getvar("interface_${ip_address}") - #$gateway_address = hiera('gateway_address') - $openvpn_config = hiera('openvpn') - $openvpn_gateway_address = $openvpn_config['gateway_address'] - $openvpn_tcp_network_prefix = '10.1.0' - $openvpn_tcp_netmask = '255.255.248.0' - $openvpn_tcp_cidr = '21' - $openvpn_udp_network_prefix = '10.2.0' - $openvpn_udp_netmask = '255.255.248.0' - $openvpn_udp_cidr = '21' - $x509_config = hiera('x509') + + $openvpn_config = hiera('openvpn') + $x509_config = hiera('x509') + $openvpn_ports = $openvpn_config['ports'] + + if $::ec2_instance_id { + $openvpn_gateway_address = $::ipaddress + } else { + $openvpn_gateway_address = $openvpn_config['gateway_address'] + if $openvpn_config['second_gateway_address'] { + $openvpn_second_gateway_address = $openvpn_config['second_gateway_address'] + } else { + $openvpn_second_gateway_address = undef + } + } + + $openvpn_allow_unlimited = $openvpn_config['allow_unlimited'] + $openvpn_unlimited_prefix = $openvpn_config['unlimited_prefix'] + $openvpn_unlimited_tcp_network_prefix = '10.41.0' + $openvpn_unlimited_tcp_netmask = '255.255.248.0' + $openvpn_unlimited_tcp_cidr = '21' + $openvpn_unlimited_udp_network_prefix = '10.42.0' + $openvpn_unlimited_udp_netmask = '255.255.248.0' + $openvpn_unlimited_udp_cidr = '21' + + if !$::ec2_instance_id { + $openvpn_allow_limited = $openvpn_config['allow_limited'] + $openvpn_limited_prefix = $openvpn_config['limited_prefix'] + $openvpn_rate_limit = $openvpn_config['rate_limit'] + $openvpn_limited_tcp_network_prefix = '10.43.0' + $openvpn_limited_tcp_netmask = '255.255.248.0' + $openvpn_limited_tcp_cidr = '21' + $openvpn_limited_udp_network_prefix = '10.44.0' + $openvpn_limited_udp_netmask = '255.255.248.0' + $openvpn_limited_udp_cidr = '21' + } # deploy ca + server keys include site_openvpn::keys - # create 2 openvpn config files, one for tcp, one for udp - site_openvpn::server_config { 'tcp_config': - port => '1194', - proto => 'tcp', - local => $openvpn_gateway_address, - server => "${openvpn_tcp_network_prefix}.0 ${openvpn_tcp_netmask}", - push => "\"dhcp-option DNS ${openvpn_tcp_network_prefix}.1\"", - management => '127.0.0.1 1000' + if $openvpn_allow_unlimited and $openvpn_allow_limited { + $unlimited_gateway_address = $openvpn_gateway_address + $limited_gateway_address = $openvpn_second_gateway_address + } elsif $openvpn_allow_unlimited { + $unlimited_gateway_address = $openvpn_gateway_address + $limited_gateway_address = undef + } elsif $openvpn_allow_limited { + $unlimited_gateway_address = undef + $limited_gateway_address = $openvpn_gateway_address } - site_openvpn::server_config { 'udp_config': - port => '1194', - proto => 'udp', - server => "${openvpn_udp_network_prefix}.0 ${openvpn_udp_netmask}", - push => "\"dhcp-option DNS ${openvpn_udp_network_prefix}.1\"", - local => $openvpn_gateway_address, - management => '127.0.0.1 1001' + + if $openvpn_allow_unlimited { + site_openvpn::server_config { 'tcp_config': + port => '1194', + proto => 'tcp', + local => $unlimited_gateway_address, + tls_remote => "\"${openvpn_unlimited_prefix}\"", + server => "${openvpn_unlimited_tcp_network_prefix}.0 ${openvpn_unlimited_tcp_netmask}", + push => "\"dhcp-option DNS ${openvpn_unlimited_tcp_network_prefix}.1\"", + management => '127.0.0.1 1000' + } + site_openvpn::server_config { 'udp_config': + port => '1194', + proto => 'udp', + local => $unlimited_gateway_address, + tls_remote => "\"${openvpn_unlimited_prefix}\"", + server => "${openvpn_unlimited_udp_network_prefix}.0 ${openvpn_unlimited_udp_netmask}", + push => "\"dhcp-option DNS ${openvpn_unlimited_udp_network_prefix}.1\"", + management => '127.0.0.1 1001' + } + } else { + tidy { "/etc/openvpn/tcp_config.conf": } + tidy { "/etc/openvpn/udp_config.conf": } + } + + if $openvpn_allow_limited { + site_openvpn::server_config { 'limited_tcp_config': + port => '1194', + proto => 'tcp', + local => $limited_gateway_address, + tls_remote => "\"${openvpn_limited_prefix}\"", + server => "${openvpn_limited_tcp_network_prefix}.0 ${openvpn_limited_tcp_netmask}", + push => "\"dhcp-option DNS ${openvpn_limited_tcp_network_prefix}.1\"", + management => '127.0.0.1 1002' + } + site_openvpn::server_config { 'limited_udp_config': + port => '1194', + proto => 'udp', + local => $limited_gateway_address, + tls_remote => "\"${openvpn_limited_prefix}\"", + server => "${openvpn_limited_udp_network_prefix}.0 ${openvpn_limited_udp_netmask}", + push => "\"dhcp-option DNS ${openvpn_limited_udp_network_prefix}.1\"", + management => '127.0.0.1 1003' + } + } else { + tidy { "/etc/openvpn/limited_tcp_config.conf": } + tidy { "/etc/openvpn/limited_udp_config.conf": } } - # add second IP on given interface - file { '/usr/local/bin/leap_add_second_ip.sh': - content => "#!/bin/sh -ip addr show dev $interface | grep -q ${openvpn_gateway_address}/24 || ip addr add ${openvpn_gateway_address}/24 dev $interface -/bin/echo 1 > /proc/sys/net/ipv4/ip_forward -", - mode => '0755', + file { + '/usr/local/bin/add_gateway_ips.sh': + content => template('site_openvpn/add_gateway_ips.sh.erb'), + mode => '0755'; } - exec { '/usr/local/bin/leap_add_second_ip.sh': - subscribe => File['/usr/local/bin/leap_add_second_ip.sh'], + exec { '/usr/local/bin/add_gateway_ips.sh': + subscribe => File['/usr/local/bin/add_gateway_ips.sh'], } - cron { 'leap_add_second_ip.sh': - command => "/usr/local/bin/leap_add_second_ip.sh", + exec { 'restart_openvpn': + command => '/etc/init.d/openvpn restart', + refreshonly => true, + subscribe => File['/etc/openvpn'], + require => [ Package['openvpn'], File['/etc/openvpn'] ]; + } + + cron { 'add_gateway_ips.sh': + command => '/usr/local/bin/add_gateway_ips.sh', user => 'root', special => 'reboot', } @@ -63,6 +149,7 @@ ip addr show dev $interface | grep -q ${openvpn_gateway_address}/24 || ip addr a 'openvpn': ensure => installed; } + service { 'openvpn': ensure => running, @@ -74,6 +161,7 @@ ip addr show dev $interface | grep -q ${openvpn_gateway_address}/24 || ip addr a file { '/etc/openvpn': ensure => directory, + notify => Exec['restart_openvpn'], require => Package['openvpn']; } diff --git a/puppet/modules/site_openvpn/manifests/resolver.pp b/puppet/modules/site_openvpn/manifests/resolver.pp index d3963c95..dc31767c 100644 --- a/puppet/modules/site_openvpn/manifests/resolver.pp +++ b/puppet/modules/site_openvpn/manifests/resolver.pp @@ -1,5 +1,53 @@ class site_openvpn::resolver { + if $site_openvpn::openvpn_allow_unlimited { + $ensure_unlimited = 'present' + file { + '/etc/unbound/conf.d/vpn_unlimited_udp_resolver': + content => "interface: ${site_openvpn::openvpn_unlimited_udp_network_prefix}.1\naccess-control: ${site_openvpn::openvpn_unlimited_udp_network_prefix}.0/${site_openvpn::openvpn_unlimited_udp_cidr} allow\n", + owner => root, + group => root, + mode => '0644', + require => Service['openvpn'], + notify => Service['unbound']; + '/etc/unbound/conf.d/vpn_unlimited_tcp_resolver': + content => "interface: ${site_openvpn::openvpn_unlimited_tcp_network_prefix}.1\naccess-control: ${site_openvpn::openvpn_unlimited_tcp_network_prefix}.0/${site_openvpn::openvpn_unlimited_tcp_cidr} allow\n", + owner => root, + group => root, + mode => '0644', + require => Service['openvpn'], + notify => Service['unbound']; + } + } else { + $ensure_unlimited = 'absent' + tidy { '/etc/unbound/conf.d/vpn_unlimited_udp_resolver': } + tidy { '/etc/unbound/conf.d/vpn_unlimited_tcp_resolver': } + } + + if $site_openvpn::openvpn_allow_limited { + $ensure_limited = 'present' + file { + '/etc/unbound/conf.d/vpn_limited_udp_resolver': + content => "interface: ${site_openvpn::openvpn_limited_udp_network_prefix}.1\naccess-control: ${site_openvpn::openvpn_limited_udp_network_prefix}.0/${site_openvpn::openvpn_limited_udp_cidr} allow\n", + owner => root, + group => root, + mode => '0644', + require => Service['openvpn'], + notify => Service['unbound']; + '/etc/unbound/conf.d/vpn_limited_tcp_resolver': + content => "interface: ${site_openvpn::openvpn_limited_tcp_network_prefix}.1\naccess-control: ${site_openvpn::openvpn_limited_tcp_network_prefix}.0/${site_openvpn::openvpn_limited_tcp_cidr} allow\n", + owner => root, + group => root, + mode => '0644', + require => Service['openvpn'], + notify => Service['unbound']; + } + } else { + $ensure_limited = 'absent' + tidy { '/etc/unbound/conf.d/vpn_limited_udp_resolver': } + tidy { '/etc/unbound/conf.d/vpn_limited_tcp_resolver': } + } + # this is an unfortunate way to get around the fact that the version of # unbound we are working with does not accept a wildcard include directive # (/etc/unbound/conf.d/*), when it does, these line definitions should @@ -7,30 +55,30 @@ class site_openvpn::resolver { # include: /etc/unbound/conf.d/* line { - 'add_tcp_resolver': - ensure => present, - file => '/etc/unbound/unbound.conf', - line => 'server: include: /etc/unbound/conf.d/vpn_tcp_resolver', - notify => Service['unbound']; - - 'add_udp_resolver': - ensure => present, - file => '/etc/unbound/unbound.conf', - line => 'server: include: /etc/unbound/conf.d/vpn_udp_resolver', - notify => Service['unbound']; + 'add_unlimited_tcp_resolver': + ensure => $ensure_unlimited, + file => '/etc/unbound/unbound.conf', + line => 'server: include: /etc/unbound/conf.d/vpn_unlimited_tcp_resolver', + notify => Service['unbound'], + require => Package['unbound']; + 'add_unlimited_udp_resolver': + ensure => $ensure_unlimited, + file => '/etc/unbound/unbound.conf', + line => 'server: include: /etc/unbound/conf.d/vpn_unlimited_udp_resolver', + notify => Service['unbound'], + require => Package['unbound']; + 'add_limited_tcp_resolver': + ensure => $ensure_limited, + file => '/etc/unbound/unbound.conf', + line => 'server: include: /etc/unbound/conf.d/vpn_limited_tcp_resolver', + notify => Service['unbound'], + require => Package['unbound']; + 'add_limited_udp_resolver': + ensure => $ensure_limited, + file => '/etc/unbound/unbound.conf', + line => 'server: include: /etc/unbound/conf.d/vpn_limited_udp_resolver', + notify => Service['unbound'], + require => Package['unbound'] } - file { - '/etc/unbound/conf.d/vpn_udp_resolver': - content => "interface: ${site_openvpn::openvpn_udp_network_prefix}.1\naccess-control: ${site_openvpn::openvpn_udp_network_prefix}.0/${site_openvpn::openvpn_udp_cidr} allow\n", - owner => root, group => root, mode => '0644', - require => Service['openvpn'], - notify => Service['unbound']; - - '/etc/unbound/conf.d/vpn_tcp_resolver': - content => "interface: ${site_openvpn::openvpn_tcp_network_prefix}.1\naccess-control: ${site_openvpn::openvpn_tcp_network_prefix}.0/${site_openvpn::openvpn_tcp_cidr} allow\n", - owner => root, group => root, mode => '0644', - require => Service['openvpn'], - notify => Service['unbound']; - } } diff --git a/puppet/modules/site_openvpn/manifests/server_config.pp b/puppet/modules/site_openvpn/manifests/server_config.pp index de273b46..6106cfbb 100644 --- a/puppet/modules/site_openvpn/manifests/server_config.pp +++ b/puppet/modules/site_openvpn/manifests/server_config.pp @@ -52,18 +52,29 @@ # note: the default is BF-CBC (blowfish) # -define site_openvpn::server_config ($port, $proto, $local, $server, $push, $management ) { +define site_openvpn::server_config( + $port, $proto, $local, $server, $push, + $management, $tls_remote = undef) { $openvpn_configname = $name concat { - "/etc/openvpn/$openvpn_configname.conf": + "/etc/openvpn/${openvpn_configname}.conf": owner => root, group => root, mode => 644, warn => true, require => File['/etc/openvpn'], - notify => Service['openvpn']; + notify => Exec['restart_openvpn']; + } + + if $tls_remote != undef { + openvpn::option { + "tls-remote $openvpn_configname": + key => 'tls-remote', + value => $tls_remote, + server => $openvpn_configname; + } } openvpn::option { diff --git a/puppet/modules/site_openvpn/templates/add_gateway_ips.sh.erb b/puppet/modules/site_openvpn/templates/add_gateway_ips.sh.erb new file mode 100644 index 00000000..05f3d16b --- /dev/null +++ b/puppet/modules/site_openvpn/templates/add_gateway_ips.sh.erb @@ -0,0 +1,11 @@ +#!/bin/sh + +ip addr show dev <%= scope.lookupvar('site_config::params::interface') %> | grep -q <%= @openvpn_gateway_address %>/24 || + ip addr add <%= @openvpn_gateway_address %>/24 dev <%= scope.lookupvar('site_config::params::interface') %> + +<% if @openvpn_second_gateway_address %> +ip addr show dev <%= scope.lookupvar('site_config::params::interface') %> | grep -q <%= @openvpn_second_gateway_address %>/24 || + ip addr add <%= @openvpn_second_gateway_address %>/24 dev <%= scope.lookupvar('site_config::params::interface') %> +<% end %> + +/bin/echo 1 > /proc/sys/net/ipv4/ip_forward diff --git a/puppet/modules/site_shorewall/manifests/couchdb.pp b/puppet/modules/site_shorewall/manifests/couchdb.pp index 9fa59569..73bed62b 100644 --- a/puppet/modules/site_shorewall/manifests/couchdb.pp +++ b/puppet/modules/site_shorewall/manifests/couchdb.pp @@ -2,16 +2,17 @@ class site_shorewall::couchdb { include site_shorewall::defaults - $couchdb_port = '6984' + $stunnel = hiera('stunnel') + $couch_server = $stunnel['couch_server'] + $couch_stunnel_port = $couch_server['accept'] # define macro for incoming services file { '/etc/shorewall/macro.leap_couchdb': - content => "PARAM - - tcp $couchdb_port", + content => "PARAM - - tcp ${couch_stunnel_port}", notify => Service['shorewall'], require => Package['shorewall'] } - shorewall::rule { 'net2fw-couchdb': source => 'net', diff --git a/puppet/modules/site_shorewall/manifests/couchdb/bigcouch.pp b/puppet/modules/site_shorewall/manifests/couchdb/bigcouch.pp new file mode 100644 index 00000000..20740650 --- /dev/null +++ b/puppet/modules/site_shorewall/manifests/couchdb/bigcouch.pp @@ -0,0 +1,51 @@ +class site_shorewall::couchdb::bigcouch { + + include site_shorewall::defaults + + $stunnel = hiera('stunnel') + + # Erlang Port Mapper Daemon (epmd) stunnel server/clients + $epmd_clients = $stunnel['epmd_clients'] + $epmd_server = $stunnel['epmd_server'] + $epmd_server_port = $epmd_server['accept'] + $epmd_server_connect = $epmd_server['connect'] + + # Erlang Distributed Node Protocol (ednp) stunnel server/clients + $ednp_clients = $stunnel['ednp_clients'] + $ednp_server = $stunnel['ednp_server'] + $ednp_server_port = $ednp_server['accept'] + $ednp_server_connect = $ednp_server['connect'] + + # define macro for incoming services + file { '/etc/shorewall/macro.leap_bigcouch': + content => "PARAM - - tcp ${epmd_server_port},${ednp_server_port}", + notify => Service['shorewall'], + require => Package['shorewall'] + } + + shorewall::rule { + 'net2fw-bigcouch': + source => 'net', + destination => '$FW', + action => 'leap_bigcouch(ACCEPT)', + order => 300; + } + + # setup DNAT rules for each epmd + $epmd_shorewall_dnat_defaults = { + 'source' => '$FW', + 'proto' => 'tcp', + 'destinationport' => regsubst($epmd_server_connect, '^([0-9.]+:)([0-9]+)$', '\2') + } + create_resources(site_shorewall::couchdb::dnat, $epmd_clients, $epmd_shorewall_dnat_defaults) + + # setup DNAT rules for each ednp + $ednp_shorewall_dnat_defaults = { + 'source' => '$FW', + 'proto' => 'tcp', + 'destinationport' => regsubst($ednp_server_connect, '^([0-9.]+:)([0-9]+)$', '\2') + } + create_resources(site_shorewall::couchdb::dnat, $ednp_clients, $ednp_shorewall_dnat_defaults) + +} + diff --git a/puppet/modules/site_shorewall/manifests/couchdb/dnat.pp b/puppet/modules/site_shorewall/manifests/couchdb/dnat.pp new file mode 100644 index 00000000..f1bc9acf --- /dev/null +++ b/puppet/modules/site_shorewall/manifests/couchdb/dnat.pp @@ -0,0 +1,21 @@ +define site_shorewall::couchdb::dnat ( + $source, + $connect, + $connect_port, + $accept_port, + $proto, + $destinationport ) +{ + + + shorewall::rule { + "dnat_${name}_${destinationport}": + action => 'DNAT', + source => $source, + destination => "\$FW:127.0.0.1:${accept_port}", + proto => $proto, + destinationport => $destinationport, + originaldest => $connect, + order => 200 + } +} diff --git a/puppet/modules/site_shorewall/manifests/defaults.pp b/puppet/modules/site_shorewall/manifests/defaults.pp index d5639a90..c62c9307 100644 --- a/puppet/modules/site_shorewall/manifests/defaults.pp +++ b/puppet/modules/site_shorewall/manifests/defaults.pp @@ -1,17 +1,10 @@ class site_shorewall::defaults { include shorewall + include site_config::params # be safe for development #if ( $::virtual == 'virtualbox') { $shorewall_startup='0' } - $ip_address = hiera('ip_address') - # a special case for vagrant interfaces - $interface = $::virtual ? { - virtualbox => [ 'eth0', 'eth1' ], - default => getvar("interface_${ip_address}") - } - - # If you want logging: shorewall::params { 'LOG': value => 'debug'; @@ -19,14 +12,13 @@ class site_shorewall::defaults { shorewall::zone {'net': type => 'ipv4'; } - # define interfaces - shorewall::interface { $interface: + shorewall::interface { $site_config::params::interface: zone => 'net', options => 'tcpflags,blacklist,nosmurfs'; } - shorewall::routestopped { $interface: } + shorewall::routestopped { $site_config::params::interface: } shorewall::policy { 'fw-to-all': diff --git a/puppet/modules/site_shorewall/manifests/dnat.pp b/puppet/modules/site_shorewall/manifests/dnat.pp new file mode 100644 index 00000000..a73294cc --- /dev/null +++ b/puppet/modules/site_shorewall/manifests/dnat.pp @@ -0,0 +1,19 @@ +define site_shorewall::dnat ( + $source, + $destination, + $proto, + $destinationport, + $originaldest ) { + + + shorewall::rule { + "dnat_${name}_${destinationport}": + action => 'DNAT', + source => $source, + destination => $destination, + proto => $proto, + destinationport => $destinationport, + originaldest => $originaldest, + order => 200 + } +} diff --git a/puppet/modules/site_shorewall/manifests/dnat_rule.pp b/puppet/modules/site_shorewall/manifests/dnat_rule.pp index 68f480d8..aa298408 100644 --- a/puppet/modules/site_shorewall/manifests/dnat_rule.pp +++ b/puppet/modules/site_shorewall/manifests/dnat_rule.pp @@ -2,24 +2,45 @@ define site_shorewall::dnat_rule { $port = $name if $port != 1194 { - shorewall::rule { - "dnat_tcp_port_$port": - action => 'DNAT', - source => 'net', - destination => "\$FW:${site_openvpn::openvpn_gateway_address}:1194", - proto => 'tcp', - destinationport => $port, - order => 100; + if $site_openvpn::openvpn_allow_unlimited { + shorewall::rule { + "dnat_tcp_port_$port": + action => 'DNAT', + source => 'net', + destination => "\$FW:${site_openvpn::unlimited_gateway_address}:1194", + proto => 'tcp', + destinationport => $port, + order => 100; + } + shorewall::rule { + "dnat_udp_port_$port": + action => 'DNAT', + source => 'net', + destination => "\$FW:${site_openvpn::unlimited_gateway_address}:1194", + proto => 'udp', + destinationport => $port, + order => 100; + } } - - shorewall::rule { - "dnat_udp_port_$port": - action => 'DNAT', - source => 'net', - destination => "\$FW:${site_openvpn::openvpn_gateway_address}:1194", - proto => 'udp', - destinationport => $port, - order => 100; + if $site_openvpn::openvpn_allow_limited { + shorewall::rule { + "dnat_free_tcp_port_$port": + action => 'DNAT', + source => 'net', + destination => "\$FW:${site_openvpn::limited_gateway_address}:1194", + proto => 'tcp', + destinationport => $port, + order => 100; + } + shorewall::rule { + "dnat_free_udp_port_$port": + action => 'DNAT', + source => 'net', + destination => "\$FW:${site_openvpn::limited_gateway_address}:1194", + proto => 'udp', + destinationport => $port, + order => 100; + } } } } diff --git a/puppet/modules/site_shorewall/manifests/eip.pp b/puppet/modules/site_shorewall/manifests/eip.pp index 4e5a5d48..7109b770 100644 --- a/puppet/modules/site_shorewall/manifests/eip.pp +++ b/puppet/modules/site_shorewall/manifests/eip.pp @@ -1,54 +1,56 @@ class site_shorewall::eip { include site_shorewall::defaults + include site_config::params include site_shorewall::ip_forward - $openvpn_config = hiera('openvpn') - $openvpn_ports = $openvpn_config['ports'] - $openvpn_gateway_address = $site_openvpn::openvpn_gateway_address - # define macro for incoming services file { '/etc/shorewall/macro.leap_eip': content => "PARAM - - tcp 1194 -PARAM - - udp 1194 -", - notify => Service['shorewall'] + PARAM - - udp 1194 + ", + notify => Service['shorewall'], + require => Package['shorewall'] } - shorewall::interface { 'tun0': zone => 'eip', options => 'tcpflags,blacklist,nosmurfs'; 'tun1': zone => 'eip', - options => 'tcpflags,blacklist,nosmurfs' + options => 'tcpflags,blacklist,nosmurfs'; + 'tun2': + zone => 'eip', + options => 'tcpflags,blacklist,nosmurfs'; + 'tun3': + zone => 'eip', + options => 'tcpflags,blacklist,nosmurfs'; } + shorewall::zone { + 'eip': + type => 'ipv4'; + } - shorewall::zone {'eip': - type => 'ipv4'; } + $interface = $site_config::params::interface - case $::virtual { - 'virtualbox': { - shorewall::masq { - 'eth0_tcp': - interface => 'eth0', - source => "${site_openvpn::openvpn_tcp_network_prefix}.0/${site_openvpn::openvpn_tcp_cidr}"; - 'eth0_udp': - interface => 'eth0', - source => "${site_openvpn::openvpn_udp_network_prefix}.0/${site_openvpn::openvpn_udp_cidr}"; } - } - default: { - $interface = $site_shorewall::defaults::interface - shorewall::masq { - "${interface}_tcp": - interface => $interface, - source => "${site_openvpn::openvpn_tcp_network_prefix}.0/${site_openvpn::openvpn_tcp_cidr}"; - - "${interface}_udp": - interface => $interface, - source => "${site_openvpn::openvpn_udp_network_prefix}.0/${site_openvpn::openvpn_udp_cidr}"; } + shorewall::masq { + "${interface}_unlimited_tcp": + interface => $interface, + source => "${site_openvpn::openvpn_unlimited_tcp_network_prefix}.0/${site_openvpn::openvpn_unlimited_tcp_cidr}"; + "${interface}_unlimited_udp": + interface => $interface, + source => "${site_openvpn::openvpn_unlimited_udp_network_prefix}.0/${site_openvpn::openvpn_unlimited_udp_cidr}"; + } + if ! $::ec2_instance_id { + shorewall::masq { + "${interface}_limited_tcp": + interface => $interface, + source => "${site_openvpn::openvpn_limited_tcp_network_prefix}.0/${site_openvpn::openvpn_limited_tcp_cidr}"; + "${interface}_limited_udp": + interface => $interface, + source => "${site_openvpn::openvpn_limited_udp_network_prefix}.0/${site_openvpn::openvpn_limited_udp_cidr}"; } } @@ -61,15 +63,14 @@ PARAM - - udp 1194 } shorewall::rule { - 'net2fw-openvpn': - source => 'net', - destination => '$FW', - action => 'leap_eip(ACCEPT)', - order => 200; + 'net2fw-openvpn': + source => 'net', + destination => '$FW', + action => 'leap_eip(ACCEPT)', + order => 200; } # create dnat rule for each port - #create_resources('site_shorewall::dnat_rule', $openvpn_ports) - site_shorewall::dnat_rule { $openvpn_ports: } + site_shorewall::dnat_rule { $site_openvpn::openvpn_ports: } } diff --git a/puppet/modules/site_shorewall/manifests/webapp.pp b/puppet/modules/site_shorewall/manifests/webapp.pp index d12bbc8f..a8d2aa5b 100644 --- a/puppet/modules/site_shorewall/manifests/webapp.pp +++ b/puppet/modules/site_shorewall/manifests/webapp.pp @@ -2,5 +2,6 @@ class site_shorewall::webapp { include site_shorewall::defaults include site_shorewall::service::https + include site_shorewall::service::http include site_shorewall::service::webapp_api } diff --git a/puppet/modules/site_sshd/manifests/authorized_keys.pp b/puppet/modules/site_sshd/manifests/authorized_keys.pp new file mode 100644 index 00000000..c18f691c --- /dev/null +++ b/puppet/modules/site_sshd/manifests/authorized_keys.pp @@ -0,0 +1,19 @@ +define site_sshd::authorized_keys ($keys, $ensure = 'present', $home = '') { + # This line allows default homedir based on $title variable. + # If $home is empty, the default is used. + $homedir = $home ? {'' => "/home/${title}", default => $home} + file { + "${homedir}/.ssh": + ensure => 'directory', + owner => $title, + group => $title, + mode => '0700'; + "${homedir}/.ssh/authorized_keys": + ensure => $ensure, + owner => $ensure ? {'present' => $title, default => undef }, + group => $ensure ? {'present' => $title, default => undef }, + mode => '0600', + require => File["${homedir}/.ssh"], + content => template('site_sshd/authorized_keys.erb'); + } +} diff --git a/puppet/modules/site_sshd/manifests/deploy_authorized_keys.pp b/puppet/modules/site_sshd/manifests/deploy_authorized_keys.pp new file mode 100644 index 00000000..97ca058f --- /dev/null +++ b/puppet/modules/site_sshd/manifests/deploy_authorized_keys.pp @@ -0,0 +1,9 @@ +class site_sshd::deploy_authorized_keys ( $keys ) { + tag 'leap_authorized_keys' + + site_sshd::authorized_keys {'root': + keys => $keys, + home => '/root' + } + +} diff --git a/puppet/modules/site_sshd/manifests/init.pp b/puppet/modules/site_sshd/manifests/init.pp index 630e9bdf..90dd2d0e 100644 --- a/puppet/modules/site_sshd/manifests/init.pp +++ b/puppet/modules/site_sshd/manifests/init.pp @@ -1 +1,31 @@ -class site_sshd {} +class site_sshd { + $ssh = hiera_hash('ssh') + + ## + ## SETUP AUTHORIZED KEYS + ## + + $authorized_keys = $ssh['authorized_keys'] + + class { 'site_sshd::deploy_authorized_keys': + keys => $authorized_keys + } + + ## + ## OPTIONAL MOSH SUPPORT + ## + + $mosh = $ssh['mosh'] + + if $mosh['enabled'] { + class { 'site_sshd::mosh': + ensure => present, + ports => $mosh['ports'] + } + } + else { + class { 'site_sshd::mosh': + ensure => absent + } + } +} diff --git a/puppet/modules/site_sshd/manifests/mosh.pp b/puppet/modules/site_sshd/manifests/mosh.pp new file mode 100644 index 00000000..49f56ca0 --- /dev/null +++ b/puppet/modules/site_sshd/manifests/mosh.pp @@ -0,0 +1,21 @@ +class site_sshd::mosh ( $ensure = present, $ports = '60000-61000' ) { + + package { 'mosh': + ensure => $ensure + } + + file { '/etc/shorewall/macro.mosh': + ensure => $ensure, + content => "PARAM - - udp ${ports}", + notify => Service['shorewall'], + require => Package['shorewall']; + } + + shorewall::rule { 'net2fw-mosh': + ensure => $ensure, + source => 'net', + destination => '$FW', + action => 'mosh(ACCEPT)', + order => 200; + } +} diff --git a/puppet/modules/site_sshd/manifests/ssh_key.pp b/puppet/modules/site_sshd/manifests/ssh_key.pp deleted file mode 100644 index b47b2ebd..00000000 --- a/puppet/modules/site_sshd/manifests/ssh_key.pp +++ /dev/null @@ -1,3 +0,0 @@ -define site_sshd::ssh_key($key) { - # ... todo: deploy ssh_key -} diff --git a/puppet/modules/site_sshd/templates/authorized_keys.erb b/puppet/modules/site_sshd/templates/authorized_keys.erb new file mode 100644 index 00000000..3c65e8ab --- /dev/null +++ b/puppet/modules/site_sshd/templates/authorized_keys.erb @@ -0,0 +1,6 @@ +# NOTICE: This file is autogenerated by Puppet +# all manually added keys will be overridden + +<% keys.sort.each do |user, hash| -%> +<%=hash['type']-%> <%=hash['key']%> <%=user%> +<% end -%> diff --git a/puppet/modules/site_stunnel/manifests/clients.pp b/puppet/modules/site_stunnel/manifests/clients.pp new file mode 100644 index 00000000..ed766e1a --- /dev/null +++ b/puppet/modules/site_stunnel/manifests/clients.pp @@ -0,0 +1,26 @@ +define site_stunnel::clients ( + $accept_port, + $connect_port, + $connect, + $cafile, + $key, + $cert, + $client = true, + $verify = '2', + $pid = $name, + $rndfile = '/var/lib/stunnel4/.rnd', + $debuglevel = '4' ) { + + stunnel::service { $name: + accept => "127.0.0.1:${accept_port}", + connect => "${connect}:${connect_port}", + client => $client, + cafile => $cafile, + key => $key, + cert => $cert, + verify => $verify, + pid => "/var/run/stunnel4/${pid}.pid", + rndfile => $rndfile, + debuglevel => $debuglevel + } +} diff --git a/puppet/modules/site_stunnel/manifests/init.pp b/puppet/modules/site_stunnel/manifests/init.pp new file mode 100644 index 00000000..c7d6acc6 --- /dev/null +++ b/puppet/modules/site_stunnel/manifests/init.pp @@ -0,0 +1,17 @@ +class site_stunnel { + + # include the generic stunnel module + # increase the number of open files to allow for 800 connections + class { 'stunnel': default_extra => 'ulimit -n 4096' } + + # The stunnel.conf provided by the Debian package is broken by default + # so we get rid of it and just define our own. See #549384 + if !defined(File['/etc/stunnel/stunnel.conf']) { + file { + # this file is a broken config installed by the package + '/etc/stunnel/stunnel.conf': + ensure => absent; + } + } +} + diff --git a/puppet/modules/site_stunnel/manifests/setup.pp b/puppet/modules/site_stunnel/manifests/setup.pp new file mode 100644 index 00000000..92eeb425 --- /dev/null +++ b/puppet/modules/site_stunnel/manifests/setup.pp @@ -0,0 +1,24 @@ +class site_stunnel::setup ($cert_name, $key, $cert, $ca_name, $ca) { + + include site_stunnel + + x509::key { + $cert_name: + content => $key, + notify => Service['stunnel']; + } + + x509::cert { + $cert_name: + content => $cert, + notify => Service['stunnel']; + } + + x509::ca { + $ca_name: + content => $ca, + notify => Service['stunnel']; + } + +} + diff --git a/puppet/modules/site_tor/manifests/init.pp b/puppet/modules/site_tor/manifests/init.pp index ceb6fb13..50ab636b 100644 --- a/puppet/modules/site_tor/manifests/init.pp +++ b/puppet/modules/site_tor/manifests/init.pp @@ -15,6 +15,7 @@ class site_tor { address => $address, contact_info => $contact_email, bandwidth_rate => $bandwidth_rate, + my_family => '$2A431444756B0E7228A7918C85A8DACFF7E3B050', } tor::daemon::directory { $::hostname: port => 80 } diff --git a/puppet/modules/site_webapp/files/migrate_design_documents b/puppet/modules/site_webapp/files/migrate_design_documents new file mode 100644 index 00000000..6e24aa5b --- /dev/null +++ b/puppet/modules/site_webapp/files/migrate_design_documents @@ -0,0 +1,16 @@ +#!/bin/sh + +cd /srv/leap/webapp + +# use admin credentials +cp config/couchdb.yml.admin config/couchdb.yml +chown leap-webapp:leap-webapp config/couchdb.yml + +# needs to be run twice +RAILS_ENV=production /usr/bin/bundle exec rake couchrest:migrate +RAILS_ENV=production /usr/bin/bundle exec rake couchrest:migrate + +# use user credentials and remove admin credentials +cp config/couchdb.yml.webapp config/couchdb.yml +chown leap-webapp:leap-webapp config/couchdb.yml + diff --git a/puppet/modules/site_webapp/manifests/apache.pp b/puppet/modules/site_webapp/manifests/apache.pp index 554b9147..8b340160 100644 --- a/puppet/modules/site_webapp/manifests/apache.pp +++ b/puppet/modules/site_webapp/manifests/apache.pp @@ -12,8 +12,7 @@ class site_webapp::apache { $api_cert = $x509['cert'] $api_root = $x509['ca_cert'] - $apache_no_default_site = true - include apache::ssl + class { '::apache': no_default_site => true, ssl => true } apache::module { 'alias': ensure => present; diff --git a/puppet/modules/site_webapp/manifests/couchdb.pp b/puppet/modules/site_webapp/manifests/couchdb.pp index 6cac666f..b4ef0980 100644 --- a/puppet/modules/site_webapp/manifests/couchdb.pp +++ b/puppet/modules/site_webapp/manifests/couchdb.pp @@ -1,16 +1,79 @@ class site_webapp::couchdb { - $webapp = hiera('webapp') - $couchdb_host = $webapp['couchdb_hosts'] - $couchdb_user = $webapp['couchdb_user']['username'] - $couchdb_password = $webapp['couchdb_user']['password'] + $webapp = hiera('webapp') + # haproxy listener on port localhost:4096, see site_webapp::haproxy + $couchdb_host = 'localhost' + $couchdb_port = '4096' + $couchdb_admin_user = $webapp['couchdb_admin_user']['username'] + $couchdb_admin_password = $webapp['couchdb_admin_user']['password'] + $couchdb_webapp_user = $webapp['couchdb_webapp_user']['username'] + $couchdb_webapp_password = $webapp['couchdb_webapp_user']['password'] + + $stunnel = hiera('stunnel') + $couch_client = $stunnel['couch_client'] + $couch_client_connect = $couch_client['connect'] + + include x509::variables + $x509 = hiera('x509') + $key = $x509['key'] + $cert = $x509['cert'] + $ca = $x509['ca_cert'] + $cert_name = 'leap_couchdb' + $ca_name = 'leap_ca' + $ca_path = "${x509::variables::local_CAs}/${ca_name}.crt" + $cert_path = "${x509::variables::certs}/${cert_name}.crt" + $key_path = "${x509::variables::keys}/${cert_name}.key" file { - '/srv/leap-webapp/config/couchdb.yml': + '/srv/leap/webapp/config/couchdb.yml.admin': + content => template('site_webapp/couchdb.yml.admin.erb'), + owner => leap-webapp, + group => leap-webapp, + mode => '0600', + require => Vcsrepo['/srv/leap/webapp']; + + '/srv/leap/webapp/config/couchdb.yml.webapp': content => template('site_webapp/couchdb.yml.erb'), owner => leap-webapp, group => leap-webapp, - mode => '0600'; + mode => '0600', + require => Vcsrepo['/srv/leap/webapp']; + + '/srv/leap/webapp/logs/production.log': + owner => leap-webapp, + group => leap-webapp, + mode => '0666', + require => Vcsrepo['/srv/leap/webapp']; + + '/usr/local/sbin/migrate_design_documents': + source => 'puppet:///modules/site_webapp/migrate_design_documents', + owner => root, + group => root, + mode => '0744'; + } + + class { 'site_stunnel::setup': + cert_name => $cert_name, + key => $key, + cert => $cert, + ca_name => $ca_name, + ca => $ca + } + + exec { 'migrate_design_documents': + cwd => '/srv/leap/webapp', + command => '/usr/local/sbin/migrate_design_documents', + require => Exec['bundler_update'], + notify => Service['apache']; + } + + $couchdb_stunnel_client_defaults = { + 'connect_port' => $couch_client_connect, + 'client' => true, + 'cafile' => $ca_path, + 'key' => $key_path, + 'cert' => $cert_path, } + create_resources(site_stunnel::clients, $couch_client, $couchdb_stunnel_client_defaults) } diff --git a/puppet/modules/site_webapp/manifests/haproxy.pp b/puppet/modules/site_webapp/manifests/haproxy.pp new file mode 100644 index 00000000..4a7e3c25 --- /dev/null +++ b/puppet/modules/site_webapp/manifests/haproxy.pp @@ -0,0 +1,14 @@ +class site_webapp::haproxy { + + include site_haproxy + + $haproxy = hiera('haproxy') + $local_ports = $haproxy['local_ports'] + + # Template uses $global_options, $defaults_options + concat::fragment { 'leap_haproxy_webapp_couchdb': + target => '/etc/haproxy/haproxy.cfg', + order => '20', + content => template('site_webapp/haproxy_couchdb.cfg.erb'), + } +} diff --git a/puppet/modules/site_webapp/manifests/init.pp b/puppet/modules/site_webapp/manifests/init.pp index e8134521..e743dc07 100644 --- a/puppet/modules/site_webapp/manifests/init.pp +++ b/puppet/modules/site_webapp/manifests/init.pp @@ -3,20 +3,19 @@ class site_webapp { $definition_files = hiera('definition_files') $provider = $definition_files['provider'] $eip_service = $definition_files['eip_service'] + $soledad_service = $definition_files['soledad_service'] + $smtp_service = $definition_files['smtp_service'] $node_domain = hiera('domain') $provider_domain = $node_domain['full_suffix'] $webapp = hiera('webapp') + $api_version = $webapp['api_version'] + $secret_token = $webapp['secret_token'] - Class[Ruby] -> Class[rubygems] -> Class[bundler::install] - - class { 'ruby': ruby_version => '1.9.3' } - - class { 'bundler::install': install_method => 'package' } - - include rubygems + include site_config::ruby include site_webapp::apache include site_webapp::couchdb include site_webapp::client_ca + include site_webapp::haproxy group { 'leap-webapp': ensure => present, @@ -28,19 +27,20 @@ class site_webapp { allowdupe => false, gid => 'leap-webapp', groups => 'ssl-cert', - home => '/srv/leap-webapp', + home => '/srv/leap/webapp', require => [ Group['leap-webapp'] ]; } - file { '/srv/leap-webapp': + file { '/srv/leap/webapp': ensure => directory, owner => 'leap-webapp', group => 'leap-webapp', require => User['leap-webapp']; } - vcsrepo { '/srv/leap-webapp': + vcsrepo { '/srv/leap/webapp': ensure => present, + force => true, revision => 'origin/master', provider => git, source => 'git://code.leap.se/leap_web', @@ -51,17 +51,17 @@ class site_webapp { } exec { 'bundler_update': - cwd => '/srv/leap-webapp', - command => '/bin/bash -c "/usr/bin/bundle check || /usr/bin/bundle install --path vendor/bundle"', + cwd => '/srv/leap/webapp', + command => '/bin/bash -c "/usr/bin/bundle check || /usr/bin/bundle install --path vendor/bundle --without test development"', unless => '/usr/bin/bundle check', user => 'leap-webapp', timeout => 600, - require => [ Class['bundler::install'], Vcsrepo['/srv/leap-webapp'] ], + require => [ Class['bundler::install'], Vcsrepo['/srv/leap/webapp'] ], notify => Service['apache']; } exec { 'compile_assets': - cwd => '/srv/leap-webapp', + cwd => '/srv/leap/webapp', command => '/bin/bash -c "/usr/bin/bundle exec rake assets:precompile"', user => 'leap-webapp', require => Exec['bundler_update'], @@ -69,47 +69,72 @@ class site_webapp { } file { - '/srv/leap-webapp/public/provider.json': + '/srv/leap/webapp/public/provider.json': content => $provider, + require => Vcsrepo['/srv/leap/webapp'], owner => leap-webapp, group => leap-webapp, mode => '0644'; - '/srv/leap-webapp/public/ca.crt': + '/srv/leap/webapp/public/ca.crt': ensure => link, + require => Vcsrepo['/srv/leap/webapp'], target => '/usr/local/share/ca-certificates/leap_api.crt'; - '/srv/leap-webapp/public/config': + "/srv/leap/webapp/public/${api_version}": ensure => directory, + require => Vcsrepo['/srv/leap/webapp'], owner => leap-webapp, group => leap-webapp, mode => '0755'; - '/srv/leap-webapp/public/config/eip-service.json': + "/srv/leap/webapp/public/${api_version}/config/": + ensure => directory, + require => Vcsrepo['/srv/leap/webapp'], + owner => leap-webapp, group => leap-webapp, mode => '0755'; + + "/srv/leap/webapp/public/${api_version}/config/eip-service.json": content => $eip_service, + require => Vcsrepo['/srv/leap/webapp'], owner => leap-webapp, group => leap-webapp, mode => '0644'; - } - try::file { - '/srv/leap-webapp/public/favicon.ico': - ensure => 'link', - target => $webapp['favicon']; - - '/srv/leap-webapp/app/assets/stylesheets/tail.scss': - ensure => 'link', - target => $webapp['tail_scss']; + "/srv/leap/webapp/public/${api_version}/config/soledad-service.json": + content => $soledad_service, + require => Vcsrepo['/srv/leap/webapp'], + owner => leap-webapp, group => leap-webapp, mode => '0644'; - '/srv/leap-webapp/app/assets/stylesheets/head.scss': - ensure => 'link', - target => $webapp['head_scss']; + "/srv/leap/webapp/public/${api_version}/config/smtp-service.json": + content => $smtp_service, + require => Vcsrepo['/srv/leap/webapp'], + owner => leap-webapp, group => leap-webapp, mode => '0644'; + } - '/srv/leap-webapp/public/img': - ensure => 'link', - target => $webapp['img_dir']; + try::file { + '/srv/leap/webapp/public/favicon.ico': + ensure => 'link', + require => Vcsrepo['/srv/leap/webapp'], + target => $webapp['favicon']; + + '/srv/leap/webapp/app/assets/stylesheets/tail.scss': + ensure => 'link', + require => Vcsrepo['/srv/leap/webapp'], + target => $webapp['tail_scss']; + + '/srv/leap/webapp/app/assets/stylesheets/head.scss': + ensure => 'link', + require => Vcsrepo['/srv/leap/webapp'], + target => $webapp['head_scss']; + + '/srv/leap/webapp/public/img': + ensure => 'link', + require => Vcsrepo['/srv/leap/webapp'], + target => $webapp['img_dir']; } file { - '/srv/leap-webapp/config/config.yml': + '/srv/leap/webapp/config/config.yml': content => template('site_webapp/config.yml.erb'), owner => leap-webapp, group => leap-webapp, - mode => '0600'; + mode => '0600', + require => Vcsrepo['/srv/leap/webapp'], + notify => Service['apache']; } include site_shorewall::webapp diff --git a/puppet/modules/site_webapp/templates/config.yml.erb b/puppet/modules/site_webapp/templates/config.yml.erb index 9cf85f0c..df562cd9 100644 --- a/puppet/modules/site_webapp/templates/config.yml.erb +++ b/puppet/modules/site_webapp/templates/config.yml.erb @@ -1,5 +1,15 @@ +<%- cert_options = @webapp['client_certificates'] -%> production: admins: [admin] domain: <%= @provider_domain %> client_ca_key: <%= scope.lookupvar('site_webapp::client_ca::key_path') %> client_ca_cert: <%= scope.lookupvar('site_webapp::client_ca::cert_path') %> + secret_token: "<%= @secret_token %>" + client_cert_lifespan: <%= cert_options['life_span'].to_i %> + client_cert_bit_size: <%= cert_options['bit_size'].to_i %> + client_cert_hash: <%= cert_options['digest'] %> + allow_limited_certs: <%= @webapp['allow_limited_certs'].inspect %> + allow_unlimited_certs: <%= @webapp['allow_unlimited_certs'].inspect %> + allow_anonymous_certs: <%= @webapp['allow_anonymous_certs'].inspect %> + limited_cert_prefix: "<%= cert_options['limited_prefix'] %>" + unlimited_cert_prefix: "<%= cert_options['unlimited_prefix'] %>" diff --git a/puppet/modules/site_webapp/templates/couchdb.yml.admin.erb b/puppet/modules/site_webapp/templates/couchdb.yml.admin.erb new file mode 100644 index 00000000..a0921add --- /dev/null +++ b/puppet/modules/site_webapp/templates/couchdb.yml.admin.erb @@ -0,0 +1,9 @@ +production: + prefix: "" + protocol: 'http' + host: <%= @couchdb_host %> + port: <%= @couchdb_port %> + auto_update_design_doc: false + username: <%= @couchdb_admin_user %> + password: <%= @couchdb_admin_password %> + diff --git a/puppet/modules/site_webapp/templates/couchdb.yml.erb b/puppet/modules/site_webapp/templates/couchdb.yml.erb index ee521713..2bef0af5 100644 --- a/puppet/modules/site_webapp/templates/couchdb.yml.erb +++ b/puppet/modules/site_webapp/templates/couchdb.yml.erb @@ -1,8 +1,9 @@ production: prefix: "" - protocol: 'https' + protocol: 'http' host: <%= @couchdb_host %> - port: 6984 - username: <%= @couchdb_user %> - password: <%= @couchdb_password %> + port: <%= @couchdb_port %> + auto_update_design_doc: false + username: <%= @couchdb_webapp_user %> + password: <%= @couchdb_webapp_password %> diff --git a/puppet/modules/site_webapp/templates/haproxy_couchdb.cfg.erb b/puppet/modules/site_webapp/templates/haproxy_couchdb.cfg.erb new file mode 100644 index 00000000..f08161ee --- /dev/null +++ b/puppet/modules/site_webapp/templates/haproxy_couchdb.cfg.erb @@ -0,0 +1,16 @@ + +listen bigcouch-in + mode http + balance roundrobin + option httplog + option dontlognull + option httpchk GET / + option http-server-close + + bind localhost:4096 +<% for port in @local_ports -%> + server couchdb_<%=port%> localhost:<%=port%> check inter 3000 fastinter 1000 downinter 1000 rise 2 fall 1 +<% end -%> + + + diff --git a/puppet/modules/stdlib b/puppet/modules/stdlib -Subproject 2df66c041109ecca1099bf3977657572cc32ad2 +Subproject 66e0fa8f1bc5062e9d753598ad17602c378a299 diff --git a/puppet/modules/stunnel b/puppet/modules/stunnel new file mode 160000 +Subproject fc1589a5f09d80f58d730d4e1f6a8058483f61f diff --git a/puppet/modules/try/manifests/file.pp b/puppet/modules/try/manifests/file.pp index 406c0b7a..47a8c269 100644 --- a/puppet/modules/try/manifests/file.pp +++ b/puppet/modules/try/manifests/file.pp @@ -18,7 +18,10 @@ define try::file ( file { "$name": ensure => $ensure, target => $target, - require => Exec["check_${name}"], + require => $require ? { + undef => Exec["check_${name}"], + default => [ $require, Exec["check_${name}"] ] + }, loglevel => info; } } @@ -37,6 +40,10 @@ define try::file ( exec { "restore_${name}": command => $command, cwd => $file_dirname, + require => $require ? { + undef => undef, + default => [ $require ] + }, loglevel => info; } } else { @@ -44,6 +51,10 @@ define try::file ( unless => "/usr/bin/test -e '${target}'", command => $command, cwd => $file_dirname, + require => $require ? { + undef => undef, + default => [ $require ] + }, loglevel => info; } } diff --git a/puppet/modules/vcsrepo b/puppet/modules/vcsrepo -Subproject 04851c28b12973c679fc9f234fd0f5a193df9d7 +Subproject 4db1120c78763f5244dc6c9d2e0d064a6ef363e |
