platform docs: updated commands, removed config items that are no longer supported, added list of available configuration files.

4 files changed, 245 insertions, 90 deletions

diff --git a/docs/platform/commands.md b/docs/platform/commands.md
index b176541..d735fef 100644
--- a/docs/platform/commands.md
+++ b/docs/platform/commands.md
@@ -5,69 +5,83 @@ The command "leap" can be used to manage a bevy of servers running the LEAP plat
 
 # Global Options
 
-* `--log FILE`  
-Override default log file  
-Default Value: None  
+* `--log FILE`
+Override default log file
+Default Value: None
 
-* `-v|--verbose LEVEL`  
-Verbosity level 0..2  
-Default Value: 1  
+* `-v|--verbose LEVEL`
+Verbosity level 0..5
+Default Value: 1
 
-* `--help`  
-Show this message  
+* `--[no-]color`
+Disable colors in output
 
-* `--version`  
-Display version number and exit  
+* `--debug`
+Enable debugging library (leap_cli development only)
 
-* `--yes`  
-Skip prompts and assume "yes"  
+* `--help`
+Show this message
+
+* `--version`
+Display version number and exit
+
+* `--yes`
+Skip prompts and assume "yes"
 
 
 # leap add-user  USERNAME
 
-Adds a new trusted sysadmin
+Adds a new trusted sysadmin by adding public keys to the "users" directory.
 
 
 
 **Options**
 
-* `--pgp-pub-key arg`  
-OpenPGP public key file for this new user  
-Default Value: None  
+* `--pgp-pub-key arg`
+OpenPGP public key file for this new user
+Default Value: None
 
-* `--ssh-pub-key arg`  
-SSH public key file for this new user  
-Default Value: None  
+* `--ssh-pub-key arg`
+SSH public key file for this new user
+Default Value: None
 
-* `--self`  
-lets you choose among your public keys  
+* `--self`
+Add yourself as a trusted sysadin by choosing among the public keys available for the current user.
 
 
-# leap cert 
+# leap cert
 
 Manage X.509 certificates
 
 
 
-## leap cert ca 
+## leap cert ca
 
 Creates two Certificate Authorities (one for validating servers and one for validating clients).
 
 See see what values are used in the generation of the certificates (like name and key size), run `leap inspect provider` and look for the "ca" property. To see the details of the created certs, run `leap inspect <file>`.
 
-## leap cert csr 
+## leap cert csr
 
 Creates a CSR for use in buying a commercial X.509 certificate.
 
-The CSR created is for the for the provider's primary domain. The properties used for this CSR come from `provider.ca.server_certificates`.
+Unless specified, the CSR is created for the provider's primary domain. The properties used for this CSR come from `provider.ca.server_certificates`.
+
+**Options**
+
+* `--domain DOMAIN`
+Specify what domain to create the CSR for.
+Unless specified, the CSR is created for the provider's primary domain. The properties used for this CSR come from `provider.ca.server_certificates`.
+Default Value: None
 
-## leap cert dh 
+
+## leap cert dh
 
 Creates a Diffie-Hellman parameter file.
 
 
 
-## leap cert update  <node-filter>
+## leap cert update  FILTER
 
 Creates or renews a X.509 certificate/key pair for a single node or all nodes, but only if needed.
 
@@ -75,22 +89,47 @@ This command will a generate new certificate for a node if some value in the nod
 
 **Options**
 
-* `--force`  
-Always generate new certificates  
+* `--force`
+Always generate new certificates
 
 
-# leap clean 
+# leap clean
 
 Removes all files generated with the "compile" command.
 
 
 
-# leap compile 
+# leap compile
+
+Compile generated files.
+
+
+
+## leap compile all  [ENVIRONMENT]
 
 Compiles node configuration files into hiera files used for deployment.
 
 
 
+## leap compile zone
+
+Compile a DNS zone file for your provider.
+
+
+Default Command: all
+
+# leap db
+
+Database commands.
+
+
+
+## leap db destroy  [FILTER]
+
+Destroy all the databases. If present, limit to FILTER nodes.
+
+
+
 # leap deploy  FILTER
 
 Apply recipes to a node or set of nodes.
@@ -99,13 +138,67 @@ The FILTER can be the name of a node, service, or tag.
 
 **Options**
 
-* `--tags TAG[,TAG]`  
-Specify tags to pass through to puppet (overriding the default).  
-Default Value: leap_base,leap_service  
+* `--ip IPADDRESS`
+Override the default SSH IP address.
+Default Value: None
+
+* `--port PORT`
+Override the default SSH port.
+Default Value: None
+
+* `--tags TAG[,TAG]`
+Specify tags to pass through to puppet (overriding the default).
+Default Value: leap_base,leap_service
+
+* `--dev`
+Development mode: don't run 'git submodule update' before deploy.
+
+* `--fast`
+Makes the deploy command faster by skipping some slow steps. A "fast" deploy can be used safely if you recently completed a normal deploy.
+
+* `--force`
+Deploy even if there is a lockfile.
+
+* `--[no-]sync`
+Sync files, but don't actually apply recipes.
+
+
+# leap env
+
+Manipulate and query environment information.
+
+The 'environment' node property can be used to isolate sets of nodes into entirely separate environments. A node in one environment will never interact with a node from another environment. Environment pinning works by modifying your ~/.leaprc file and is dependent on the absolute file path of your provider directory (pins don't apply if you move the directory)
+
+## leap env ls
+
+List the available environments. The pinned environment, if any, will be marked with '*'.
+
 
-* `--fast`  
-Makes the deploy command faster by skipping some slow steps. A "fast" deploy can be used safely if you recently completed a normal deploy.  
 
+## leap env pin  ENVIRONMENT
+
+Pin the environment to ENVIRONMENT. All subsequent commands will only apply to nodes in this environment.
+
+
+
+## leap env unpin
+
+Unpin the environment. All subsequent commands will apply to all nodes.
+
+
+Default Command: ls
+
+# leap facts
+
+Gather information on nodes.
+
+
+
+## leap facts update  FILTER
+
+Query servers to update facts.json.
+
+Queries every node included in FILTER and saves the important information to facts.json
 
 # leap help  command
 
@@ -115,8 +208,8 @@ Gets help for the application or its commands. Can also list the commands in a w
 
 **Options**
 
-* `-c`  
-List commands one per line, to assist with shell completion  
+* `-c`
+List commands one per line, to assist with shell completion
 
 
 # leap inspect  FILE
@@ -125,6 +218,12 @@ Prints details about a file. Alternately, the argument FILE can be the name of a
 
 
 
+**Options**
+
+* `--base`
+Inspect the FILE from the provider_base (i.e. without local inheritance).
+
+
 # leap list  [FILTER]
 
 List nodes and their classifications
@@ -137,12 +236,15 @@ Prints out a listing of nodes, services, or tags. If present, the FILTER can be
 
 **Options**
 
-* `--print arg`  
-What attributes to print (optional)  
-Default Value: None  
+* `--print arg`
+What attributes to print (optional)
+Default Value: None
+
+* `--disabled`
+Include disabled nodes in the list.
 
 
-# leap local 
+# leap local
 
 Manage local virtual machines.
 
@@ -184,6 +286,12 @@ Shuts down the virtual machine(s)
 
 
 
+# leap mosh  NAME
+
+Log in to the specified node with an interactive shell using mosh (requires node to have mosh.enabled set to true).
+
+
+
 # leap new  DIRECTORY
 
 Creates a new provider instance in the specified directory, creating it if necessary.
@@ -192,24 +300,24 @@ Creates a new provider instance in the specified directory, creating it if neces
 
 **Options**
 
-* `--contacts arg`  
-Default email address contacts.  
-Default Value: None  
+* `--contacts arg`
+Default email address contacts.
+Default Value: None
 
-* `--domain arg`  
-The primary domain of the provider.  
-Default Value: None  
+* `--domain arg`
+The primary domain of the provider.
+Default Value: None
 
-* `--name arg`  
-The name of the provider.  
-Default Value: None  
+* `--name arg`
+The name of the provider.
+Default Value: None
 
-* `--platform arg`  
-File path of the leap_platform directory.  
-Default Value: None  
+* `--platform arg`
+File path of the leap_platform directory.
+Default Value: None
 
 
-# leap node 
+# leap node
 
 Node management
 
@@ -231,20 +339,28 @@ Separeate multiple values for a single property with a comma, like so: `leap nod
 
 **Options**
 
-* `--local`  
-Make a local testing node (by automatically assigning the next available local IP address). Local nodes are run as virtual machines on your computer.  
+* `--local`
+Make a local testing node (by automatically assigning the next available local IP address). Local nodes are run as virtual machines on your computer.
 
 
 ## leap node init  FILTER
 
 Bootstraps a node or nodes, setting up SSH keys and installing prerequisite packages
 
-This command prepares a server to be used with the LEAP Platform by saving the server's SSH host key, copying the authorized_keys file, and installing packages that are required for deploying. Node init must be run before deploying to a server, and the server must be running and available via the network. This command only needs to be run once, but there is no harm in running it multiple times.
+This command prepares a server to be used with the LEAP Platform by saving the server's SSH host key, copying the authorized_keys file, installing packages that are required for deploying, and registering important facts. Node init must be run before deploying to a server, and the server must be running and available via the network. This command only needs to be run once, but there is no harm in running it multiple times.
 
 **Options**
 
-* `--echo`  
-If set, passwords are visible as you type them (default is hidden)  
+* `--ip IPADDRESS`
+Override the default SSH IP address.
+Default Value: None
+
+* `--port PORT`
+Override the default SSH port.
+Default Value: None
+
+* `--echo`
+If set, passwords are visible as you type them (default is hidden)
 
 
 ## leap node mv  OLD_NAME NEW_NAME
@@ -265,21 +381,38 @@ Log in to the specified node with an interactive shell.
 
 
 
-# leap test 
+**Options**
+
+* `--port arg`
+Override ssh port for remote host
+Default Value: None
+
+* `--ssh arg`
+Pass through raw options to ssh (e.g. --ssh '-F ~/sshconfig')
+Default Value: None
+
+
+# leap test
 
 Run tests.
 
 
 
-## leap test init 
+## leap test init
 
 Creates files needed to run tests.
 
 
 
-## leap test run 
+## leap test run
 
 Run tests.
 
 
+
+**Options**
+
+* `--[no-]continue`
+Continue over errors and failures (default is --no-continue).
+
 Default Command: run
diff --git a/docs/platform/config.md b/docs/platform/config.md
index d0b1f6a..3ec2f42 100644
--- a/docs/platform/config.md
+++ b/docs/platform/config.md
@@ -1,5 +1,39 @@
 @title = "Configuration Files"
 
+Files
+-------------------------------------------
+
+Here are a list of some of the common files that make up a provider. Except for Leapfile and provider.json, the files are optional. Unless otherwise specified, all file names are relative to the 'provider directory' root (where the Leapfile is).
+
+`Leapfile` -- If present, this file tells `leap` that the directory is a provider directory. This file is usually empty, but can contain global options.
+
+`~/.leaprc` -- Evaluated the same as Leapfile, but not committed to source control.
+
+`provider.json` -- Global options related to this provider.
+
+`provider.ENVIRONMENT.json` -- Global options for the provider that are applied to only a single environment.
+
+`common.json` -- All nodes inherit from this file.
+
+`secrets.json` -- An automatically generated file that contains any randomly generated strings needed in order to deploy. These strings are often secret and should be protected, although any need for a random string or number that is remembered will produce another entry in this file. This file is automatically generated and refreshed each time you run `leap compile` or `leap deploy`. If an entry is no longer needed, it will get removed. If you want to change a secret, you can remove this file and have it regenerated, or remove the particular line item and just those items will be created anew.
+
+`facts.json` -- If some of your servers are running on AWS or OpenStack, you will need to discover certain properties about how networking is configured on these machines in order for a full deploy to work. In these cases, make sure to run `leap facts update` to periodically regenerate the facts.json file.
+
+`nodes/NAME.json` -- The configuration file for node called NAME.
+
+`services/SERVICE.json` -- The properties in this configuration file are applied to any node that includes SERVICE in its `services` property.
+
+`services/SERVICE.ENVIRONMENT.json` -- The properties in this configuration file are applied to any node that includes SERVICE in its services and has environment equal to ENVIRONMENT.
+
+`services/TAG.json` -- The properties in this configuration file are applied to any node that has includes TAG in its `tags` property.
+
+`services/TAG.ENVIRONMENT.json` -- The properties in this configuration file are applied to any node that has includes TAG in its `tags` property and has `environment` property equal to ENVIRONMENT.
+
+`files/*` -- Various static files used by the platform (e.g. keys, certificates, webapp customization, etc).
+
+`users/USER/` -- A directory that stores the public keys of the sysadmin with name USER. This person will have root access to all the servers.
+
+
 Leapfile
 -------------------------------------------
 
@@ -13,8 +47,6 @@ Additionally, you can create a `~/.leaprc` file that is loaded after `Leapfile`
 Platform options:
 
 * `@platform_directory_path` (required). This must be set to the path where `leap_platform` lives. The path may be relative.
-* `@platform_branch`. If set, a check is preformed before running any command to ensure that the currently checked out branch of `leap_platform` matches the value set for `@platform_branch`. This is useful if you have a stable branch of your provider that you want to ensure runs off the master branch of `leap_platform`.
-* `@allow_production_deploy`. By default, you can only deploy to production nodes if the current branch is 'master' or if the provider directory is not a git repository. This option allows you to override this behavior.
 
 Vagrant options:
 
@@ -26,7 +58,7 @@ Logging options:
 * `@log`. If set, all command invocation and results are logged to the specified file. This is the same as the switch `--log FILE`, except that the command line switch will override the value in the Leapfile.
 
 
-Configuration files
+JSON format
 -------------------------------------------
 
 All configuration files, other than `Leapfile`, are in the JSON format. For example:
@@ -195,7 +227,8 @@ Access an element by name:
 Create a new hash table by applying filters:
 
     nodes[:public_dns => true] # all nodes where public_dns == true
-    nodes[:services => 'openvpn', :services => 'tor'] # openvpn OR tor
+    nodes[:services => 'openvpn', 'location.country_code' => 'US'] # openvpn service OR in the US.
+    nodes[[:services, 'openvpn'], [:services, 'tor']]  # services equal to openvpn OR tor
     nodes[:services => 'openvpn'][:tags => 'production']  # openvpn AND production
     nodes[:name => "!bob"] # all nodes that are NOT named "bob"
 
diff --git a/docs/platform/troubleshooting.md b/docs/platform/troubleshooting.md
index 6869a8e..528248d 100644
--- a/docs/platform/troubleshooting.md
+++ b/docs/platform/troubleshooting.md
@@ -17,8 +17,9 @@ Places to look for errors
 * `/var/log/apache2/error.log`
 * `/srv/leap/webapp/log/production.log`
 * `/var/log/syslog` (watch out for stunnel issues)
+* `/var/log/leap/*`
 
-Is haproxy ok ? 
+Is haproxy ok ?
 ---------------
 
 
@@ -33,7 +34,7 @@ Is couchdb accessible through stunnel ?
 
     curl -s -X  GET "http://127.0.0.1:4000"
     curl -s -X  GET "http://127.0.0.1:4001"
-    ... 
+    ...
 
 
 Check couchdb acl as admin
@@ -47,7 +48,7 @@ Check couchdb acl as admin
     curl -s --netrc-file /etc/couchdb/couchdb-admin.netrc -X GET "http://127.0.0.1:4096"
     curl -s --netrc-file /etc/couchdb/couchdb-admin.netrc -X GET "http://127.0.0.1:4096/_all_dbs"
 
-Check couchdb acl as unpriviledged user 
+Check couchdb acl as unpriviledged user
 ---------------------------------------
 
     cat /srv/leap/webapp/config/couchdb.yml  # see username and password
@@ -57,7 +58,7 @@ Check couchdb acl as unpriviledged user
     curl -s --netrc-file /etc/couchdb/couchdb-webapp.netrc -X GET "http://127.0.0.1:4096"
     curl -s --netrc-file /etc/couchdb/couchdb-webapp.netrc -X GET "http://127.0.0.1:4096/_all_dbs"
 
-    
+
 Check client config files
 -------------------------
 
@@ -106,10 +107,10 @@ Bigcouch membership
 Databases
 ---------
 
-* Following output shows all neccessary DBs that should be present. Note that the `user-0123456....` DBs are the data stores for a particular user. 
+* Following output shows all neccessary DBs that should be present. Note that the `user-0123456....` DBs are the data stores for a particular user.
 
 <pre>
-    curl -s --netrc-file /etc/couchdb/couchdb.netrc -X GET 'http://127.0.0.1:5984/_all_dbs' 
+    curl -s --netrc-file /etc/couchdb/couchdb.netrc -X GET 'http://127.0.0.1:5984/_all_dbs'
     ["customers","identities","sessions","shared","tickets","tokens","user-0","user-9d34680b01074c75c2ec58c7321f540c","user-9d34680b01074c75c2ec58c7325fb7ff","users"]
 </pre>
 
@@ -164,19 +165,19 @@ Is couchdb accessible through stunnel ?
 
     curl -s -X  GET "http://127.0.0.1:4000"
     curl -s -X  GET "http://127.0.0.1:4001"
-    ... 
+    ...
 
 Query leap-mx
 -------------
 
-* for useraccount 
+* for useraccount
 
 
 <pre>
     postmap -v -q  "joe@dev.bitmask.net" tcp:localhost:2244
     ...
     postmap: dict_tcp_lookup: send: get jow@dev.bitmask.net
-    postmap: dict_tcp_lookup: recv: 200 
+    postmap: dict_tcp_lookup: recv: 200
     ...
 </pre>
 
@@ -194,7 +195,7 @@ Query leap-mx
 </pre>
 
 
-Check couchdb acl as unpriviledged user 
+Check couchdb acl as unpriviledged user
 ---------------------------------------
 
 
@@ -232,7 +233,7 @@ Mailspool
 Testing mail delivery
 ---------------------
 
-    swaks -f alice@example.org -t bob@example.net -s mx1.example.net --port 25 
+    swaks -f alice@example.org -t bob@example.net -s mx1.example.net --port 25
     swaks -f varac@cdev.bitmask.net -t varac@cdev.bitmask.net -s chipmonk.cdev.bitmask.net --port 465 --tlsc
     swaks -f alice@example.org -t bob@example.net -s mx1.example.net --port 587 --tls
 
diff --git a/docs/platform/under-the-hood.md b/docs/platform/under-the-hood.md
index 080a153..2c9a9b7 100644
--- a/docs/platform/under-the-hood.md
+++ b/docs/platform/under-the-hood.md
@@ -5,18 +5,6 @@ This page contains various details on the how the platform is implemented. You c
 Puppet Details
 ======================================
 
-Run stages
-----------
-
-We use two run stages for resource ordering:
-
-* initial: configure hostname, apt-get update + apt-get dist-upgrade
-* main: everything else
-
-Stage initial is run before stage main.
-
-see http://docs.puppetlabs.com/puppet/2.7/reference/lang_run_stages.html for run stage documentation.
-
 Tags
 ----